Spark Shell "Failed to Initialize Compiler" Error on a mac

Spark is incompatible with Java 9, which is the version brew cask install java will install if it is up to date. If you did install Java 9, what you need to do is install Java 8 instead:

brew cask uninstall java
brew tap caskroom/versions
brew cask search java
brew cask install java8

As ZackK wrote spark is incompatible with Java9, so you can check the versions of java you have in your machine and choose a compatible version, assuming you have one.

$ sudo update-alternatives --config java

Which in my case returned:

There are 2 choices for the alternative java (providing /usr/bin/java).

• 0/usr/lib/jvm/java-8-openjdk-amd64/jre/bin/java 1081 auto mode
• *1/opt/java/jdk-9.0.4/bin/java 1 manual mode

The asterisk in front of 1 denotes the active version. Choosing 0 changed it to a compatible version.

$ java -version

which returned: openjdk version "1.8.0_151"

After the change spark-shell worked.