Due to overcurrent, shouldn't I be extremely cautious when setting an I/O pin as Output?
As others have said, you shouldn't have a problem as long as you double check your code.
If you do get it wrong, by and large the ATMega IO pins will limit themselves to about ~80mA due to internal resistance of the MOSFETs (value found by experiment). This is not good for the chip, but as long as you don't leave it in this condition for an extended period, they tend to recover ok.
If you are worried, it can be a good idea to put a resistor in series with the inputs. Something in the order of 330R for a 5V VDD, or 220R for 3.3V VDD. This will ensure the short circuit current is limited to ~15mA which is comfortably within spec.
The resistor goes between the IO pin and whatever is driving it (e.g. button w/ pull-up, or CMOS). As the ATMega has a pretty limited useful frequency range (IO freq < 10MHz), the extra resistor won't have any noticable impact on operation of the circuit as the inputs have very little in the way of leakage current and capacitance
You should be cautious but not terrified about it. Setting a pin output and driving it to the other direction will cause a lot of current to flow, but it also does not blow up in smoke immediately. You could for example power the device with current limited labotatory supply when bringing up a design so these kind of errors cause even less damage. I'd be more worried about the pushbutton and capacitor getting damage over continuous use as pushing the button effectively shorts out the capacitor and a spike of current is limited only by cap ESR, PCB wiring and button resistance, so it can exceed many amps. And as there will be contact bounce at the pushbutton, the current has high frequency components and that combined with stray inductance of long wires will convert the current spikes into voltage spikes, protected only by IO pin protection diodes. So in general, a resistor of even 330 ohms in series starts to be a pretty good idea to limit current through pushbutton.
Considering this type of circuit is avidly encouraged, how it doesn't seem to bring any troubles?
Because when engineers set up the I/O for their device, they are careful to ensure that they do not drive inputs as outputs. For most I/O, we will set pins as either inputs or outputs at startup and never change them later. This gives us only one line of code we need to double-check carefully, which is not an exceptionally hard task.
There is no need for a protective resistor unless you think it is likely that you will be careless about your coding. In that case, I strongly suggest you work on improving your coding practises, by doing better testing, or by reviewing your code before you run it.
As a similar example, if you connect a wire directly between your +5V and 0V supply lines, you will burn out your power supply. This does not mean you need to protect your power supply - it just means you need to be careful not to make mistakes like that!
Now, back to the original question, shouldn’t I be extremely cautious when setting an I/O pin as Output?
Yes you should. But then I should hope you are equally cautious about all the many other ways you could cause damage to your circuit, like shorting out pins, turning on both sides of an H-bridge, or other similar mistakes.