Is temperature the only battery charging speed limitation?
Can I charge as fast as I want, provided that I cool down sufficiently?
No. To charge faster you need higher voltage to overcome internal resistance. If the voltage is too high it may cause damage due to electrolysis or other effects, depending on the battery chemistry.
A lead-acid battery will 'gas' when the voltage reaches ~2.45V per cell, as the water breaks down into hydrogen and oxygen. This causes a 'wet' lead-acid battery to lose electrolyte, and the gas mixture is an explosion risk. Sealed lead-acid batteries can absorb and recombine a certain amount of gas, but if the pressure gets too high they will vent. Having to keep below gassing voltage is the main factor that limits how fast you can charge a lead-acid battery, even at low current which doesn't heat the battery up much. If you ignore the gassing and try to charge even faster then charging efficiency could drop dramatically, due to gas bubbles reducing the effective electrolyte density and plate surface area.
High voltage can also cause increased corrosion, insulation breakdown, arcing between terminals etc. So you can't just increase the voltage 'as much as you want' to get a faster charge rate, even if you could somehow manage to keep all internal parts of the battery cool.
However with some battery chemistries you can increase charge current well beyond the nominal rate, at least for part of the charge cycle.
The chemical reaction when charging a Nicad battery is endothermic, so the battery naturally cools itself down while charging. If it has thick plates and interconnects to keep Ohmic losses low, a Nicad battery can be charged at very high current until just before full charge. However the current must be switched off as soon as it reaches full charge, else it will rapidly heat and build up internal pressure which may cause it to vent and/or short out.
High power Lithium-ion batteries can also be charged very rapidly during the 'constant current' phase. Li-ion batteries have very high charging efficiency, so they don't normally get hot during charging. The main limiting factor is the increased possibility of the (very thin) separator shorting out at higher voltage. Some high power Lipos are rated for up to 15C charge rate - flat to full in 4 minutes! However since maximum voltage is strictly limited, current must be reduced for the last 20% so the time to reach 100% charge is much longer.
You certainly cannot charge as fast as you want, but you can indeed increase the charging rate if you keep cooling the battery in the process. Though the point is not to make the battery colder than normal, rather, you want to prevent it from getting hotter.
For example, check out this teardown of Tesla battery pack, revealing the cooling system which is required for fast charging.
A lot of the chemical limitations to which speeds you can charge or discharge a battery actually get worse with decreasing temperature.
For example: the charge carrier mobility in a cold battery will be lower than in a warm one. That leads to a higher internal resistance. So you can't charge or discharge as fast, because the charges simply don't move as fast.
That's why on the cold days of winter, starter batteries have a harder time starting cars, because they simply can't deliver as much current. For charging, it's all the same.