arp --delete is not deleting an entry; merely marks entry as "incomplete"

The entry will be removed, just be patient.

(If you want the shortest possible answer: incomplete == deleted)

Let's say "delete" is the wrong word for the action. What's really happening here is that the entry is manually set to the state "request sent, no answer" (thus "incomplete" ARP process) as if the machine would be really unreachable.

Now, the entry will be completely removed soon unless it gets a new valid ARP response in the meantime. In that case the entry would be re-added anyways even if it was removed instead of being marked as incomplete. So there's no actual pro or con to this behaviour.

But keep in mind that we're talking of a cache. Deleting things from caches is hard and expensive. It's way more efficient to invalidate an entry and wait if it gets replaced before it is finally removed. But for the system it's totally no difference if the entry is gone from the list or just marked incomplete.

Just to complement all other answers, I found this link very useful.

In some cases using ip is more appropriate, like the command:

# ip -s -s neigh flush all

Results may depend on your linux kernel.

Apart from the other fine answers it worths mentioning that it is possible to completely eredicate the arp cache by removing it. One of the less painful ways is:

ip link set arp off dev eth0 ; ip link set arp on dev eth0

This should remove all entries, be they in whatever state.

Alternative methods include downing and upping the interface and similar ways to make the arp cache completely removed and recreated.