Purpose of the diode and capacitor in this motor circuit
The diode is to provide a safe path for the inductive kickback of the motor. If you try to switch off the current in an inductor suddenly, it will make whatever voltage is necessary to keep the current flowing in the short term. Put another way, the current thru an inductor can never change instantaneously. There will always be some finite slope.
The motor is partially an inductor. If the transistor shuts off quickly, then the current that must still flow thru the inductor for a little while will flow thru the diode and cause no harm. Without the diode, the voltage across the motor would get as large as necessary to keep the current flowing, which would probably require frying the transistor.
A small capacitor across the motor will reduce the speed of the possibly fast voltage transitions, which causes less radiation and limits the dV/dt the transistor is subjected to. 100 nF is excessive for this, and will prevent efficient operation at all but low PWM frequencies. I'd use 100 pF or so, perhaps to up 1 nF.
The resistor is to limit current the digital output must source and the transistor base must handle. The transistor B-E looks like a diode to the external circuit. The voltage will therefore be limited to 750 mV or so. Holding a digital output at 750 mV when it is trying to drive to 5 V or 3.3 V is out of spec. It could damage the digital output. Or, if the digital output can source a lot of current, then it could damage the transistor.
1 kΩ is again a questionable value. Even with a 5 V digital output, that will put only 4.3 mA or so thru the base: the voltage drop at the B-E junction ("diode") is 0.7 V, leaving the 4.3 V at the resistor. You don't show specs for the transistor, so let's figure it has a minimum guaranteed gain of 50. That means you can only count on the transistor supporting 4.3 mA x 50 = 215 mA of motor current. That sounds low, especially for startup, unless this is a very small motor. I would look at what the digital output can safely source and adjust R1 to draw most of that.
Another issue is that the 1N4004 diode is inappropriate here, especially since you will be turning the motor on and off rapidly, as implied by "PWM". This diode is a power rectifier intended for normal power line frequencies like 50-60 Hz. It has very slow recovery. Use a Schottky diode instead. Any generic 1 A 30 V Schottky diode will do fine and be better than a 1N4004.
I can see how this circuit can appear to work, but it clearly wasn't designed by someone that really knew what they were doing. In general, if you see an Arduino in a circuit you find on the 'net someplace, especially a simple one, assume it was posted because the author considers it a great accomplishment. Those that know what they are doing and draw out a circuit like this in a minute don't consider it worth writing up a web page on. That leaves those that took two weeks to get the motor to spin without the transistor blowing up and they're not really sure what everything does to write these web pages.
When the motor's windings are carrying current, they generate a magnetic field. It takes energy to do this and the energy is stored in the magnetic field. If the current is suddenly cut off, the magnetic field will collapse. This changing magnetic field will induce a current in the winding that is much higher than usual and produce a higher voltage across the windings. It is very short and can be pretty impressive.
The key to induced current is the changing field. You can see the same effect in a home light switch. If you have switches that are not mercury type ("silent switches"), sometimes you can see a spark or flash of light when you turn lights off. If you happen to break the connection as the AC current passes near zero, nothing happens. If you break near the peak of the current, the wiring to the lights has the maximum magnetic field around it and it will collapse with enough of a voltage spike to arc in the light switch.
Notice that your diode points toward the + side of your circuit. The changing field produces a "back EMF" or voltage going the wrong way. The energy goes out the pipe it came in. (Hope I have that right. I'll check and edit if I have it backwards.) The diode will conduct if the potential, or voltage, across the motor winding is more than about 0.6V in the "wrong" direction. For DC, this is straight forward. For PWM this is more like AC and a quality reliable circuit is more complicated.
As @OlinLathrop said, your base resistor might be a bit large. As typical examples, 2N2222 and 2N3904 have a beta or current gain of about 30 at DC that goes up with frequency to 300-400. If you have much of a motor, the transistor will not supply the current or burn up. You can figure power dissipation in the transistor at about 1W per Amp and much higher if things are not tuned up right. (You can not put bipolar transistors in parallel without a lot of extra work. As they heat up resistance goes down and more current flows and the one that heats the fastest hogs the current - usually to destruction). You can see the little motor drivers sold for Arduinos have either a heat sink or a big part with a metal section meant to be used with a heat sink.
The cap smooths out the current spikes. As they get broader in time, they get lower in peak current and therefore voltage the current produces in the circuit is less. If your motor has brushes, you are getting the on/off current flow at the rate the motor is turning. Again we are back to changing currents and changing fields. This is where radio frequency noise comes from. Spreading out these current spikes means the rate of change of current is lower and as a result RFI (Radio Frequency Interference) is lower. I will bet that if you put an AM radio near your circuit and tune it to a spot with no radio station you will be able to tell when the motor is running. Try different size caps and see if you detect any difference.