What's the rigorous definition of phase and phase transition?
Does this mean that gas and liquid are the same phases? Because in the phase diagram they are connected and they have the same symmetry(translation and rotation). If they are not the same phase, how to call the state in large pressure and large temperature? Liquid or gas?
Yes. From the modern point of view, liquid and gas are in the same phase. Because, as the asker has mentioned, they are continuously connected in the phase diagram through the "supercritical" regime. By definition, two states of matter are in the same phase if they can be smoothly deformed to each other without going through phase transitions. Historically, liquid and gas are named as different phases (by mistake) because people thought that "there must be one different phase at each side of the phase transition" (as argued in Diracology's answer). But this idea is wrong.
We can not declare different phases just by observing phase transitions. Otherwise, for example, in the following phase diagram on the left, we could have declared states A and B to be in different phases, simply because they are separated by phase transitions, as we can first go out of the blue phase and then reenter it. This way of separating phases is clearly stupid. Any reasonable person would agree that A and B should belong to the same phase in this case. Now we just deform the left phase diagram to the right by squeezing the intermediate red phase to a first-order transition line, then why we suddenly get confused about whether A and B are in the same phase or not? Definitely, they should still remain in the same phase! The liquid-gas transition is indeed a situation like this. So a logically consistent definition will have to define liquid and gas as a single phase.
Does this mean that above the critical point the transition from gas to liquid is not a phase transition, but below the critical point, the transition from gas to liquid is a phase transition?
Yes.
If answers to my first and second question are "yes", does this mean even in the same phase there still can have phase transition? This conclusion is so weird!
Given the example of the above phase diagrams, one will not feel weird about the fact that there can be (first-order) phase transitions inside a single phase. In fact, first-order transitions often appear by merging two second-order transitions together (this can be explained by Landau's theory). So going across a first-order transition is like going out of the phase and back again immediately, which definitely can happen inside a single phase. However, I am not aware of any example that continuous phase transitions can also happen within a single phase. So I conjecture that if a phase transition happens inside a phase, it must be first-order. The liquid-gas transition is one example of my conjecture.
From the Landau's paradigm, what's the symmetry breaking and order parameter in the gas-liquid phase transition? It seems that the symmetry is same in gas and liquid. Gas-liquid phase transition must be able to be explained by Landau's paradigm but Landau's paradigm says that there must be symmetry breaking in phase transition. There is an answer. I admit that from a modern point of view phase transition is not necessary due to symmetry breaking, but I don't think that gas-liquid transition has been beyond Landau's paradigm.
No symmetry breaking is associated with the liquid-gas transition. Landau's paradigm only says that there must be spontaneous symmetry breaking in second-order transitions, but not in first-order transitions. In fact, nothing can be said about first-order transitions, because first-order transitions can happen anywhere in the phase diagram without any reason. The liquid-gas transition is indeed a case like this.
Even though the liquid-gas transition is not a symmetry breaking transition, it can still be described within Landau's paradigm phenomenologically (who says that Landau's theory only applies to symmetry breaking transitions?). We can introduce the density $\rho$ of the fluid as the order parameter. Because no symmetry is acting on this order parameter, so there is no symmetry reason to forbid odd-order terms like $\rho, \rho^3, \cdots$ to appear in Landau's free energy. However the first order term can always be absorbed by redefining the order parameter with a shift $\rho\to\rho+\rho_0$, then the Landau free energy takes the general form of
$$F= F_0 + a \rho^2 + b \rho^3+ c \rho^4+\cdots.$$
First-order transition happens by driving the parameter $a$ to $a<9b^2/32c$. From this example, we can see that (within Landau's paradigm) if a phase transition happens without breaking any symmetry, it must be first-order. Again the liquid-gas transition is one such example.
So if given one phase, we firstly find the symmetry is same in this phase and then check several order parameters also same in this phase. However, how do you prove you must not be able to construct some weird order parameters such that in one part of this phase is zero and in another part of this phase is nonzero? For example, in a solid phase of water which has the same crystal structure, how to prove any order parameter that you can construct will not be zero in one part of the phase and nonzero in another part?
Indeed, you can never rule out the possibility that some weird order parameter is hiding there to further divide the phase into more phases. That is actually why the solid phase of water is divided into so many different crystal phases. Each crystal structure is associated with a different symmetry breaking pattern. Sometimes the symmetries are just so complicated that you may miss one or two of them if you are not careful enough. In that case, you will also miss the order parameters associated with the missing symmetry, until you see a specific heat anomaly in the experiment where you did not expect, then you start to realize that oh there is a missing order parameter that actually changes across this transition, and one needs to add some additional symmetry to explain it. This is actually the typical way that physicists work every day, they never figure out the full classification of phases until they see the evidence for new phases and phase transitions. I think this is also the fun part of condensed matter physics: there are always new phases of matter waiting for us to discover.
If you consider the infinite-dimensional manifold of local Hamiltonians parameterized by the various coupling constants for all possible local couplings, then a "phase transition" is defined to be a point in this parameter space at which the thermodynamic-limit free energy density is not analytic in the coupling constants. These phase-transition points typically form lower-dimensional submanifolds in the full parameter space. A "phase" is defined to an open set of points in parameter space whose boundary consists entirely of phase transitions - i.e. a maximal (connected) domain on which the free energy density is analytic. So yes, if a phase transition terminates, then you can indeed have a phase transition of which both sides are in the same phase, as in the water boiling transition. Strange but true.
Note that these definitions say nothing at all about symmetry or symmetry breaking. In practice, we often restrict ourselves to a low-dimensional submanifold of parameter space that respects some symmetry, for two reasons: (a) it make the analysis much more tractable, and (b) many real-world materials have symmetries that are respected to very high accuracy, even when you factor in impurities, disorder, etc., so this is often a very realistic approximation. When we do this, different phases within the submanifold are often characterized by different local symmetries being respected or spontaneously broken. These phase transitions are well-described by Landau theory. But these phase transitions are usually not perfectly robust, in the sense that one can often "hop around them" by temporarily breaking and then restoring the symmetry.
In fact, technically speaking, if we allow fully general local Hamiltonians with no particular symmetries, then almost every system you've ever studied is in the same phase, called the "topologically trivial phase!" All the various phase transitions we observe in the everyday world (boiling, melting, etc.) can in principle be avoided if perturb the Hamiltonian in a way that explicitly breaks all the symmetries (although again, in practice this is often very difficult to do, so e.g. it's very hard to actually go between liquid water and ice without passing through a phase transition). The exception is "topological phase transitions," which cannot be avoided by any local deformation of the Hamiltonian. These are very exotic, cannot be described by Landau theory, and are still under very active investigation today.
I will assign the first four questions. The fifth one (which I think is the most interesting) I will not address. I apologize for that and I hope someone answers it.
Does this mean that gas and liquid are the same phase? Because in the phase diagram they are connected and they have the same symmetry(translation and rotation). If they are not a same phase, how to call the state in large pressure and large temperature? Liquid or gas?
No it does not. Gas and liquid are in different phases even though they have the same symmetry. Below the critical point, they undergo a first-order phase transition which is characterized by the fact that the free energy is not an analytical function at this point. Its first derivative is discontinuous. Thus there must be one different phase at "each side" of the phase transition. We definitely can characterize these different phases by their densities (which is also discontinuous at the phase transition). The phases coexists above the critical point and that means that liquid and gas are indistinguishable. The fluid is said to be supercritical.
Does this mean that above the critical point the transition from gas to liquid is not a phase transition, but below the critial point the transition from gas to liquid is a phase transition?
Yes.
If answers to my first and second question are "yes", does this mean even in the same phase there still can have phase transition? This conclusion is so weird!
Gas and liquid as different phases of matter only makes sense bellow the critical point. Above the critical point the fluid is in the supercritical phase.
From the Landau's paradigm, what's the symmetry breaking and order parameter in the gas-liquid phase transition? It seems that the symmetry is same in gas and liquid. Gas-liquid phase transition must be able to be explained by Landau's paradigm but Landau's paradigm says that there must be symmetry breaking in phase transition. There is an answer. I admit that from modern point of view phase transition is not necessary due to symmetry breaking, but I don't think that gas-liquid transition has been beyond Landau's paradigm.
Landau's original idea was aimed to explain second-order phase transitions, such as a paramagnetic-ferromagnetic transition. Second-order phase transitions are the ones which are associated to changes of symmetries. In this case, the order parameter is associated to order and disorder of the system, this change of symmetry is continuous and therefore the free energy is analytical - it can be Taylor expanded in terms of the order parameter. Gas-liquid phase transition is a first order phase transition. We can assign the density as an "order parameter" but one has to be careful in writing the free energy in terms of this order parameter. The free energy will no longer be analytic. I do not have experience on the subject but I have seen people changing the Landau theory by considering more and more terms in the free energy in trying to describe a first-order transition.