How to use JMH properly? Example with ArrayList

The test is badly designed; in your test, because the arraylist is created only once for multiple invocations, the array-based code just overwrites the same array a bunch of times, whereas the arraylist version adds more and more, and needs to grow.

One trivial fix is to clear it first. Another fix is to stop using state here and just make the creation of the object (be it the 100k person array, or the person arraylist, presized for 100k persons) part of the test harness. Once you take care of this, the results are the exact same taking into account the error, there is no performance different at all between arrays and arraylists for this.

MyBenchmark.capacityTestArray             avgt    5  1,325 ± 0,059  ms/op
MyBenchmark.capacityTestArrayListEnsured  avgt    5  1,287 ± 0,157  ms/op

I simplified by removing the Params state entirely, and making the creation of the list and array part of each test's outlay:

    static final int LEN = 100_000;
    
    public void capacityTestArray() {
        Person[] people = new Person[LEN];
        for (int i = 0; i < LEN; i++) {
            people[i] = new Person(i, new Address(i, i), new Pet(i, i));
        }
    }

    public void capacityTestArrayListEnsured() {
        List<Person> p = new ArrayList<Person>(LEN);
        for (int i = 0; i < LEN; i++) {
            p.add(new Person(i, new Address(i, i), new Pet(i, i)));
        }
    }

(keeping all annotations and the Person, Address, etc classes the same).

Alternatively, take your existing code and just toss a list.clear() at the top.

As soon as you understand the difference between Trial, Iteration and Invocation, your question becomes very easy to answer. And what place to better understand these then the samples themselves.

Invocation is the a single execution of the method. Let's say there are 3 threads and each execute this benchmark method 100 times. This means Invocation == 300. That is why you get very similar results using this as the set-up.

Iteration would be 3 from the example above.

Trial would be 1, when all the threads execute all their methods.

Invocation, though has a scary documentation has its usage, like a sorted data structure; but I've used in various other places too. Also the notion of operation can be "altered" with @OperationsPerInvocation - which is another sharp tool.

---

Armed with this - it gets easy to answer. When you use Iteration, your ArrayList will grow constantly - which internally means System::arrayCopy, while your array does not.

Once you figure this out, you need to read the samples and see that your second problem is that your @Benchmark methods return void. And, contrary, to the other answer - I would not suggest to bulk everything with the test method itself, but this raises the question on what do you want to test, to begin with. Do not forget that these are just numbers, in the end, you need to reason about what they mean and how to properly set-up a JMH test.