Why is the electric field inside a conductor zero in equilibrium?
As Mostafa says, it is macroscopically at equilibrium, not necessarily microscopically.
There may be one misunderstanding you have, which is about "surface". I will talk about it later.
In my opinion, equilibrium should be understood as no electron moving. It is easily to show that the electric field in conductor is zero. If the electric field is non-zero, then electrons in the conductor will feel it and move, until go to the boundary of the conductor, and then stop there. Hence, the surface will accumulate charge, and finally, the distribution of charge on the surface will make the field zero in the conductor.
Now, let us talk about the surface. If it is possible, I would like to say that the charge(electrons) are on the outside surface, mathematically. The field is zero in the conductor, as well as on the inside surface. But the field on the outside surface is not zero.
However, actually, in physics, this statement is not appropriate in microcosm, "surface" is many atoms layers. The electric field changes continuously in space, and external field is not zero but internal is zero.
In fact, when we talk about macroscopic description, we can treat the surface as a surface in mathematics. Therefore, we should distinguish two sides of surface. It helps us discuss clearly.
Review the proof that you post before.
1) Place a gaussian surface inside the conductor. Since the system is at equilibrium, all points on the surface must have an electric field of zero.
2) Therefore the net flux is zero, implying the charge inside is zero.
3) If there is no charge inside, all excess charge must lie on the surface.
In 1), all points on the inside surface must have an electric field of zero.
In 3), all excess charge must lie on the outside surface.
Here, I take a short summary. You have to distinguish two sides of the surface in mathematics.
In addition, the Gaussian surface is a conception totally in mathematics, it has no thickness, and never goes through an electron or a charge. And, "surface" is different in micro and macro. In micro, surface is alway means a very thin layer including a lot of atoms, unless in macro. Only in macro, you can say inside, outside or on the surface.
Update for your another confusion.
Before we discuss deeper, we have to talk about "model". Electromagnetism is a theory to describe macroscopic phenomenon, or a model to show essential properties of electromagnetic macroscopic phenomenon. Actually, in this model, there is no electron, no proton or any other elemental particles, but one thing we have is charge. When we say "an electric particle", we would say "a particle with charge". This is a phenomenological model, we do not care about what charge is and why matter have charge, just care about that charge is a property of some matter. It is enough to build electromagnetism, as Maxwell did.
There is a theorem, that electric field is not stable. If particles interact with each other only by static electric force, then these particles are not force equilibrium, the system is not stable. Sorry that I forget this theorem's name, if someone knows, please edit this anwser, thank you.
I believe, your confusion comes from the contradiction between microcosmic model and macroscopical model.
I will discuss later, but now my computer run out of power. Sorry.
To be continued.
As I said, I think your confusion comes from the contradiction between microcosmic model and macroscopical model. This contradiction confuses not only you, but also everyone, because the contradiction indeed exists.
Before we going on, we must clarify the language that we use. The language of previous discussions when we discussed macroscopical model, as well as some words when we discussed microcosmic model, are based on classical physics. Here, we do not need quantum mechanics in microcosmic model, semiclassical model is enough. "Electron" and "ion" are words uesd in microcosmic model, but "field" is in macroscopical model. The conception, "Equilibrium", is totally based on classical physics. By the way, in semiclassical model, the similar conception is "detailed balance".
Let us go on. I mentioned that a static electric field is not stable, this statement is based on classical physics. If you apply this theorem to microcosmic model, you will find that the conductor is not stable, or equilibrium. Acutally, you found the same conclusion before.
It is a paradox, which is from applying a not appropriate model to describe microcosmic world. It is the origin of quantum mechanics historically, which start from explaining the spectrum of Hydrogen and why it is stable.
As for your second question, you use two different discription ways for the conductor, one is macroscopical and the other one is microscopical. In macroscopical description, there is no ions and electrons, the field in the conductor must be zero. In microscopical description, the conductor contains lots of ions and electrons, and the field in the conductor fluctuate widely, hence the field in the conductor is non-zero. But both models cannot explain the reason why the conductor is stable, or equilibrium. In macroscopical description, we usually treat conductor is rigid, or very hard with a very large modulus of elasticity. In microscopical description, "quantization" guarantees it stable, which does not appear in classical physics.
When we talk about the first question, we use macroscopical description, such as electromagnetism and classical mechanics, and we say "the field inside a conductor must be zero in order for the system to be equilibrium". This statement is alway based on the macroscopical description.
Hence, your confusion just is that you confuse two different descriptions, and forget the statement has its own conditions.
I hope it can help you.
Equilibrium in the sense of this question means there are no net forces on the objects that make up the system: the charges contained in the conductor. Note that we need a model of an ideal conductor here. A neutral ideal conductor is thought of as containing equal large amounts of unbound, infinitely small (not electrons) positive and negative charges.
Consider one of the charges, one that happens to lie on the Gaussian surface. If the field there was non-zero, the charge would experience a force and accelerate: the system is not in equilibrium.
Thus, in equilibrium, there are no forces on charges within the conductor, and the electric field is zero everywhere within the conductor.