CMake Error: The following variables are used in this project, but they are set to NOTFOUND

It appears that CMake is unable to find the htmlcxx library.

In the whalebot documentation, htmlcxx is listed as a dependency.

You need to download htmlcxx, unzip it, then install it:

cd <path to unzipped htmlcxx>
./configure --enable-static=on --enable-shared=off
make
sudo make install

You may need to add #include <cstddef> to the top of html/tree.h to get it to build successfully. It will install to usr/local/ by default.

You also need icu installed if you don't already have it:

sudo apt-get install libicu-dev

Finally, you can now build and install whalebot. Again, making might fail if you have a reasonably up-to-date boost installation.

In line 57 of webspider/src/webspider_options.cpp, you need to replace boost::filesystem::initial_path().native_directory_string() with boost::filesystem::initial_path().string(). Then you should be good to build and install:

cd <path to unzipped whalebot>
mkdir build
cd build
cmake .. -DCMAKE_BUILD_TYPE=Release
make
sudo make install

This too will install to usr/local/ by default.