Apple - Quartz Filters not working on High Sierra—is it fixable?
This is a known issue (I reported it directly to Apple engineering) and they are working on a fix in a future update. In the meantime it does mean you need to use 3rd party software or find a workaround.
Without knowing what your ultimate aim is, one workaround I have discovered is to first export the PDF as an image file (e.g. a .jpg image) and then open the newly created .jpeg file in Preview and export it again. When you export it, select the PDF option and the Quartz Filter you want. This will then save the file as a PDF again, but this time in grayscale or B&W.
Of course, this is not ideal in the case of multiple pages. One possible option may be to use Automator, but I haven't pursued this as I already have access to 3rd party software.
However, I can update this answer as soon as it's fixed (the Apple Engineers have committed to letting me know as soon as it's fixed - and I will be following up with them!
UPDATE
After further investigation Apple's Engineers contacted me on 9 August 2018 to advise they have determined that this issue only occurs when PDFs have been originally produced by certain 3rd party products, typically older ones. As a result, a fix for this may no longer be in the pipeline because as time goes by users are less likely to come across PDFs originally produced by these older legacy products.
In my own testing I have found the problems do not occur with PDFs originally created with MS Office 2016 or later (both Mac and Windows versions). However, the problem does occur with PDFs originally created with older versions of MS Office (e.g. 2010, 2013) and other software such as PDFCreator.
The age of the original software is not necessarily indicative of whether a PDF will have issues. For example, PDFs produced with Adobe InDesign CS5 (2010 vintage) and QuarkXPress 9 (2011 vintage) work fine with Quartz Filters.
At present this has been left open by the Engineers until macOS Mojave is released. However, if it is not rectified by then it will be closed as it's unlikely to ever be.
Workaround for multi-page PDFs
In my own testing I have found a workaround that works perfectly for multipage PDFs, albeit somewhat ironically. However, it is an easy workaround:
- Open the problematic PDF in Preview
- Go to File > Export as PDF...
- Give it a name, choose a location and click on the Save button
- Open the newly created PDF in Preview
- Go to File > Export...
- Select PDF as your format
- Choose the Quartz Filter you want (e.g. Black & White or Grey Tone)
- Give it a name, choose a location and click on the Save button
- Now your newly exported multi-page PDF will have the Quartz Filter applied