What's the difference between X.509 and PKCS#7 Certificate?
You've evolved to mostly right, but to add several points and expand on @CoverosGene' answer more than I felt comfortable doing in an edit:
X.509 defines a certificate (and some other things not relevant here) in ASN.1, a (very!) general data structuring method which has several defined encodings, of which DER Distinguished Encoding Representation is quite common and is used here.
PEM format -- for several types of data of which a certificate is only one -- is much as you say just binary (DER) data encoded in base64 (edit) broken into lines normally every 64 chars (but there are variations), plus header and trailer lines consisting of dashes + BEGIN or END + the type of data, in this case CERTIFICATE + dashes. Those lines look redundant to a human but they are expected and mostly required by software. PEM (Privacy Enhanced Mail) was actually a complete standard for secure email that has now been mostly forgotten (see below) except for its encoding format. (edit) As of 2015 there is RFC 7468 describing in detail most use of 'PEM' formats for modern crypto data.
PKCS#7 was defined by RSA (the company, not the algorithm) as a multi-purpose format for encrypted and/or signed data. It was turned over to IETF and evolved into CMS Cryptographic Message Syntax in RFC 2630, then RFC 3369, then RFC 3852, then RFC 5652, hence the wording of the Windows (inetopt) prompt. "PKCS#7" is often used to mean both the original RSA PKCS#7 and the IETF successor CMS, in the same way "SSL" is often used for both the original Netscape protocol and the IETF successor TLS Transport Level Security.
The .p7b or .p7c format is a special case of PKCS#7/CMS: a SignedData structure containing no "content" and zero SignerInfos, but one or more certificates (usually) and/or CRLs (rarely). Way back when this provided a standard way to handle (edit) the set of certificates needed to make up a chain (not necessarily in order).
PKCS#7/CMS is (are?) also ASN.1 and depending on circumstances can be either DER or BER, a closely-related encoding with some very minor differences that most DER decoders handle.
While PKCS#7/CMS like any DER or BER object can be PEM-formatted, I've not seen any implementation other than
openssl (edit) it is rare for certs. (Java CertificateFactory can read PKCS7/CMS-certs-only from DER or PEM, but CertPath.getEncoded writes it only to DER.) In contrast both DER and PEM formats for a single cert are common.
PKCS#7/CMS is also used as the basis for S/MIME secure email (multiple rfcs starting from 5751). Basically PEM encoded PKCS#7 into ASCII text which email systems of the 1980s could easily handle, while S/MIME represents CMS as MIME entities which are encoded in several ways modern email systems can handle.
OpenSSL confused matters by implementing, in order: a pkcs7 command which handles the certs-CRLs-only case not full PKCS#7; a crl2pkcs7 command which actually handles CRLs and certs, but again not the rest of PKCS#7; a smime command which actually handles both S/MIME and PKCS#7/CMS for most cases of encrypted and/or signed messages; and a cms command which actually handles both S/MIME and PKCS#7/CMS for a more complete set of cases.
So I would describe the options as: a cert in PEM or DER format; a (single) cert in a PKCS#7 container or for short just p7, and mention PEM only in the rare case it applies; or a cert chain in PKCS#7 or p7. The semantic difference between a single cert and a cert chain is at least as important as the format difference between a cert by itself or in a container.
And this doesn't even reach the widespread confusion between a certificate by itself (for some other entity, most often a CA root or anchor) and the combination of privatekey PLUS certificate -- or usually chain -- that you use to prove your own identity, for example as an SSL/TLS server or when signing S/MIME email. That uses the originally-Microsoft PFX Personal Information Exchange format, or its standardized form PKCS#12 or "p12".
PKCS #7 can be thought of as a format that allows multiple certificates to be bundled together, either DER- or PEM- encoded, and may include certificates and certificate revocation lists (CRLs).
Per RFC2315, PKCS#7 is
a general syntax for data that may have
cryptography applied to it, such as digital signatures and digital
envelopes. The syntax admits recursion, so that, for example, one
envelope can be nested inside another, or one party can sign some
previously enveloped digital data.