Can I connect a PC to a PoE switch?
Yes, it is perfectly safe. The switch performs a check before it applies power.
The following is drawn from this site; the last couple of paragraphs (emphasis added) have the relevant bits.
Network cables, such as Cat 5e and Cat 6, comprise eight wires arranged as four twisted pairs. In 10 and 100BASE-T Ethernet, two of these pairs are used for sending information, and these are known as the data pairs. The other two pairs are unused and are referred to as the spare pairs (Gigabit Ethernet uses all four pairs).
Because electrical currents flow in a loop, two conductors are required to deliver power over a cable. PoE treats each pair as a single conductor, and can use either the two data pairs or the two spare pairs to carry electrical current.
Power over Ethernet is injected onto the cable at a voltage between 44 and 57 volts DC, and typically 48 volts is used. This relatively high voltage allows efficient power transfer along the cable, while still being low enough to be regarded as safe.
This voltage is safe for users, but it can still damage equipment that has not been designed to receive PoE. Therefore, before a PoE switch or midspan (known as a PSE, for power sourcing equipment) can enable power to a connected IP camera or other equipment (known as a PD, for powered device), it must perform a signature detection process.
Signature detection uses a lower voltage to detect a characteristic signature of IEEE-compatible PDs (a 25kOhm resistance). Once this signature has been detected, the PSE knows that higher voltages can be safely applied.
Updated answer for 2016.
The good news is: if you have a switch or injector that doesn't claim to be "passive" and conforms to 802.3af (the PoE standard) or 802.3at (the PoE Plus standard), it will detect PoE before applying power as mentioned by @Richard. This renders the connection perfectly safe to use for data only or for data + power.
However, there exists a class of PoE switches or injectors, usually referred to as "passive" or "always on", which supply power without first performing signature detection. Why would anyone do this? Because it's significantly cheaper. Whether it damages your device depends on the type of passive injector and your device. Many passive devices expect power on the unused pins (4,5,7,8) of a 10/100 RJ45 connection. If your non-PoE 10/100 device leaves these pins disconnected from circuitry (since they aren't used), then probably nothing will happen. However, gigabit connections usually must share power and data on the same wires. And I wouldn't risk it there... maybe the NIC will handle it gracefully... maybe it will burn out.
Yes, of course. The PC won't use the power.