How to graduate from Arduino to using a microcontroller directly?
Welcome to the wonderful world of Atmel. Let me offer you some answers to your questions based on my hobby and professional experience.
Do not bother with anything BUT in-circuit programming. Unless you are a perfect coder, removing a chip every time you want to program it is a nightmare. I recommend the AVRISPmkII as an entry level programming tool. The disadvantage is there is no hardware debug supported. An alternative is the dragon but I have no experience with that. I can say that the JTAGICEmk3 is a nice capable debug tool.
Atmel studio is good for programming. It is actually my favorite embedded development environment. You can use CLI tools such as AVRDUDE and AVR-GCC but the IDE takes care of that for you.
IN SUMMARY:
Buy an AVRISP for just programming (easier, plug&play) or a DRAGON (I can not offer advice).
On your board, bring out the ICSP pins to a 6-pin dual row 0.100" header -- the connections are described in this datasheet.
Reset switches are not needed.
You don't have to "graduate" all at once. Here's how I converted:
To start, I kept using an Arduino but converted calls to the Arduino library into register reads and writes, one line at a time, seeing if my programs still worked. This way I got used to using registers and saw that they were not radically discontinuous with the Arduino way of doing things. Arduino functions like digitalWrite() simply manipulate the registers -- you are free to read and write to them directly in your sketches.
My next step was to put an ATTiny 85 on a breadboard and program it with Arduino as ISP (http://highlowtech.org/?p=1695). If you buy a breadboard power supply and use the ATTiny's internal oscillator, this requires laughably little wiring. I used the Arduino IDE to compile but kept using registers as much as I could.
Then, I installed the free Crosspack command line toolchain and compiled a blinky program. After some more trial and error, I managed to program the ATTiny with avrdude only, staying completely outside the Arudino ecosystem.
Once you've attained blinky with the command line tools only, you're free from Arduino. Explore the various peripherals and their registers and soon it'll be totally natural.
After all of this, I often find myself using Arduino because it's faster. I've found that poring over register descriptions kinda sucks; no sense in doing that if you don't have to. It's good to know how to, though.
I agree with HL-DSK's comment above. I'd spend the extra money and get a programmer with debug capabilities. I use JTAGICE3, about $110 on digi-key.
Read up on ISP programming here. It will show you how to bring out the SPI connections. The SPI connections on your chip will be in the ATmega datasheet. Look on page 2 for MISO/MOSI/SCK/RESET pins. The programmer needs to be able to control the reset line. Remember to use a pull-up resistor on the reset line so that your chip will run after the programmer is disconnected.
Get yourself a 2x3 header here. Or make your own using breakaway headers.