Which gets the measurements right, JMeter or Apache ab?

In your setup, JMeter is saturating itself faster than it can saturate your web server.

You are running a very optimized C web server on superior hardware and bench-marking it with a relatively heavy Java app on lesser hardware. Optimized C machine code will (probably) always be faster than Java bytecode. JMeter is not able to keep up with Nginx and is therefore giving you strange results as it hits hardware limitations. Java does lots of nice things in the background that manage hardware resources, but also create unpredictable behavior at extreme resource usage. ApacheBench, on the other hand, is a light-enough C program that it can saturate the server and can produce consistent results because it has excess capacity after saturating your web server.

JMeter is great for bench-marking heavy-er dynamic applications that need some time to process requests. All the extra data it provides helps with web apps like that. When you're dealing with static file serving (just about the fastest thing a web server can do) on highly optimized web servers, you need a tool fast enough to keep up.

Jmeter tells you how long each request actually took. AB just does some very basic math to get the overall average. So, the direct answer to your question is that jmeter gets it right and ab just makes a rough guess by giving you the mean across everything.

But, sure, if you put the two tools side by side and rate them for speed then it is clearly the case that ab is going to out perform jmeter. Jmeter just does more, it records more data and is processing more logic so it takes longer to turn around a single request. The simple fact is that Jmeter is a fully featured load testing tool, AB is, well, not.

The thing is, the aim of a load testing tool is not to be the fastest kid on the block, instead it is about being able to build a realistic representation of the sort of load your app might be hit with when it goes live. In this respect jmeter wins hands down, so it really depends on what your requirements are. If you just want to generate as many requests as possible using the least amount of hardware then ab is a nice choice but if you want to build a representative test, with transactional journeys, conditional logic and all sorts of other useful stuff, then jmeter is the way to go. Think of it like this: they are both Apache projects but AB was, I think, designed to test the apache web server, JMeter, however, was designed to test Tomcat.

Now, I'm guessing that jmeter was producing inconsistent results because it was hitting a limit on the machine it was running on. I'm betting you were running in GUI mode and had at least one listener active, like this you are asking the tool to do a lot. If you need a high rate of requests then Jmeter has a lean and mean mode. Typically, for large volumes the best practice is to execute tests at the command line with very few listeners; there's lots of info about this subject on the apache jmeter site.

Another point you should consider, if you're really getting into load testing, is that in order to really get benefit from this sort of thing you need to first decide what sort of load you need your site to support and only then should you design a test that represents this. This is achieved using pacing and simulated wait times. The problem with telling a thread that is should just go away and run as fast as it possibly can is that it will iterate as fast as its local conditions allow it to, but there will always be something that puts the breaks on, even ab is limited; no matter how lightweight a tool is it still does something. But if you pace your requests then you remove this problem and as a rather useful added bonus you end up with consistency between runs and between builds of the code, so even if your server speeds up or slows down (with changes to the code base) your test will still make the same rate of requests - which is pretty useful for benchmarking.

If you want to take JMeter further then have a look at the Constant Throughput Timer and then use multiple threads to build the level of traffic you need to represent.