Assembly: why some x86 opcodes are invalid in x64?

The 06 and 07 opcodes in 32-bit mode are the instructions PUSH ES and POP ES. In 64-bit mode, the segment registers CS, DS, ES, and SS are no longer used to determine memory addresses: the processor assumes a base address of 0 and no size limits. As there's now usually no reason for applications (other than the operating system itself) to access these registers, the push/pop opcodes for changing and accessing them were removed, leaving only mov to/from Sreg (which is just 2 total opcodes; the register number goes in the ModRM byte instead of part of the 1-byte opcode). That's totally sufficient for something that's almost never needed.

The FS and GS segment registers can still set the base address in 64-bit mode, so push and pop opcodes related to them have not been removed. (These 2-byte 0F xx opcodes were added in 386, and are a less valuable part of opcode space than the old 1-byte opcodes for 8086 segment registers).

Push/pop or mov of segment registers is not how OSes would typically set FS or GS segment bases, though: that would require a GDT or LDT entry, and could only set a base within the low 32 bits. 64-bit OSes would use the associated MSRs to read and write the bases directly, not the architectural registers. (Modern 32-bit OSes do that, too, unless running on old hardware that doesn't support the segment base MSRs.)

For all CPUs there's something like an "opcode space". For example, if a CPU used 8-bit opcodes then there'd be a max. of 256 instructions it could have. The larger opcodes are the more opcodes you can have, but the harder it is to fetch and decode them quickly.

80x86 is a relatively old architecture. It started out with a modest opcode space consisting of mostly 1-byte and 2-byte opcodes. Each time CPU manufacturers add a new feature it takes more opcodes from the opcode space. They ran out of opcodes. They ran out quickly.

To work around it they started doing things like adding escape codes and prefixes to artificially extended the opcode space. For an example, for recent AVX instructions you're looking at a VEX prefix followed by an old/recycled escape code (e.g. 0xF0), followed by an old/recycled address/operand size prefix (e.g. 0x66), followed by another 4 bytes. It's not pretty.

At the same time there's old instructions that are rarely used now (AAD, AAM, etc) and instructions with multiple/redundant opcodes (INC/DEC) that were consuming valuable "1-byte" opcodes. These couldn't/can't be removed entirely due to backward compatibility.

However; when 64-bit was being designed there simply wasn't any 64-bit code to be compatible with - backward compatibility didn't matter. The 1-byte opcodes being consumed by "not very important" instructions could be recycled; making those instructions invalid in 64-bit code (but freeing up some of the valuable 1-byte opcodes).

Most of those 1-byte opcodes (the entire 1-byte INC/DEC group if I remember right) got recycled immediately for the REX prefix that was needed to support 64-bit operands. Some weren't and became "free for future extensions" (with the restriction that the extension can only work in 64-bit code because those instructions are still valid in 16-bit and 32-bit code).