Why are ceramics good electric insulators?
For electricity to flow, electrons need to be moving. So in a conductor, there need to be free or loose electrons so they can carry the flow of electricity. Most metals fulfill this requirement, which is why most metals are conductors. Insulators, therefore, must have their electrons bound tightly, so they cannot carry the flow of electricity. The type of ceramic used as an insulator doesn't have any loose electrons (it is not, however, a perfect insulator) so it is a good insulator. In fact, this is what makes ceramic a good thermal insulator as well: it doesn't have loose electrons!
Solids can be classified according to their band gaps. The band gap determines how much energy you need to supply in order to free or promote an electron from the valence band to the conduction band. Once in the conduction band the electron can move nearly freely and conduct electricity. A solid with a large band gap (> 2.5 eV) is considered to be a good insulator and many ceramics have such large band gaps. Metals don't have such a gap, and hence, electrons flow under the slightest applied voltage (in principle).
Most ceramics are ionic compounds, in which electrons are immobile. This is different to metal, in which the atoms are in a "sea of electrons" that are free to move.
Note that ceramics have some kind of conductivity, it's just extremely low. The conductivity of copper, for example, is ~6×107 S/m. Most ceramics have conductivities in the range of 10-3 S/m to 10-5 S/m. That's around 10 orders of magnitude less conductive than copper, but it is 10 orders of magnitude more conductive than air, for example (10-15 S/m).