Alex L. Demidov

DevOps/SRE consultant

MariaDB Gets Unexpected OOM

MariaDB has recently surprised me by getting itself OOM-killed, even though the VM had a couple of GB of spare memory (as I thought). At first, I suspected it was some kind of memory spike, but the VM memory graph showed that it had been sitting just a few MB from the memory limit for some time before the OOM. That was unexpected, too. Started digging around and found that pmm-agent ate 1GB of RAM, but it is still not enough. Finally, checked the OOM task dump in the logs, and, indeed, MariaDB was eating about 25% more memory than my calculations estimated. The 11 GB innodb_buffer with the default other settings gave me an expected 12 GB, and even the memory_used system variable reported the same 12 GB, but the actual process RSS was 15 GB.

Digging through the web, mailing lists and documentation discovered that it is a known and documented problem with the default system malloc. The solution is to replace it with jemalloc or tcmalloc, which is also documented. In my case, memory consumption dropped by 40%.

Comments