Sudden MySQL freeze, with spike in dirty pages, but without obvious write queries

You may need to tune your writes for InnoDB and possibly your ext4 volume


I noticed you have innodb_write_io_threads set to 4 (default value). You need to increase that so dirty pages can get flushed to their respective .ibd files more robustly. Please set it 16.


That pause to perform write might be do to ext4. Why ?

You have innodb_flush_method set to O_DIRECT. This is supposed to make disk writes more stable. Nevertheless, I a surprise for you. I answered ib_logfile opened with O_SYNC when innodb_flush_method=O_DSYNC about a year ago. I mentioned a Percona blog that says that O_DIRECT is faked in ext4 with the latest kernel.

  1. You need to make sure you have the latest Linux kernel (3.2 or greater)
  2. Change innodb_flush_log_at_trx_commit to 0

After a long experimentation, I've found the problem to be the infamous swap insanity.

The problem has been fixed by using the innodb-numa-interleave option.