Why is it impossible to verify whether a file has been modified since creation?

I think you will want more of a philosophical answer than a technical one, given what you are rejecting.

A file is just a discrete collection of bits. Relevance and meaning are overlaid onto those bits by a human, but ultimately, it's just bits. How would it be possible to determine if the bits you have are in the same sequence in some unknown previous state? Answer: saving that state in a way that can be trusted to be used as a means of comparison.

That's why TLS/SSL uses 3rd party CAs to verify certificates, and why digitally signing files is useful. They provide a trusted means of having a state to compare. It's not perfect, but very effective if performed correctly.

Imagine you're on a desert island and I hand you a print out of the US constitution, claiming that it is an exact copy (no words changed). With nothing to compare it against, you have no way to verify that, right?

As @schroeder says, a digital file (and its metadata) is just a collection of bits. How do you determine if the bits you have in front of you are in the same sequence as when they were first written? Well, you need some "baseline" version of the document to compare against. There are many cryptographic tools that will provide this baseline: hashes, MACs, digital signatures, and finally timestamping servers may actually be the closest to what you're looking for.

Trusted timestamping is the process of securely keeping track of the creation and modification time of a document

But ultimately, unless the original copy of the file was submitted to some form of crypto at the time it was created, you're in the desert island scenario with no baseline for comparison.