OutOfMemoryException running EPiServer out of web context

TL;DR: If you run EPiServer out of web context, like in a test setup, and get OutOfMemoryExceptions it might be the cache not being trimmed in time. You can work around that by setting the /configuration/system.web/caching/cache[@percentagePhysicalMemoryUsedLimit] to a lower value (you can try 95, or 50 if you don't care about cache).

The problem: Cache fills up memory unless running in the web server

I ran into an issue running EPiServer (with Commerce) in a console context for integration tests where it ran out of memory running one of the tests. I had been working on optimizing cache so it made sense more memory was used but I expected the automatic cache trimming to evict cache objects to prevent the application from running out of memory. Doing the same operation in a web context did not cause the process to run out of memory. What was going on?

Investigation: Deep-dive into System.Web.Caching source code

Since I couldn't find any detailed documentation I started digging around in the reflected source code for System.Web.Caching.Cache and related classes (which is what is used as the default cache implementation in EPiServer). Here's what I concluded, though I can't guarantee all the conclusions are correct:

The cache monitors both the private bytes used by the process as well as the amount of physical memory in use on the machine. If either metric goes too high it will start trimming the cache. You can configure the limits using the privateBytesLimit and percentagePhysicalMemoryUsedLimit attributes of the /configuration/system.web/caching.cache element in your config, but it has sensible defaults for most situations.

A weakness though, is that it seems to skip monitoring private bytes for a non-IIS process. So that metric will be effectively disabled, falling back to the total physical memory usage metric. By default the "high load" threshold of that metric is set to 99 %, and the "medium load" (when I think it will start trimming the cache, though not that aggressively) is set to 97 %. Albeit high, that seems reasonable for a web server given that it will free up memory by trimming the cache when it approaches this limit.

But the catch is that the cache trimming only runs every so often, from two minute intervals down to five second intervals. Which means that if you put a lot of data into the cache in a short time when the cache is already under high load you could run out of memory.

Solution: Config lower cache limit to make it trim in time

For some reason, perhaps because it can measure the private bytes, the cache trimming was run in time to avoid the OutOfMemoryException in the web context. But I could solve my issue in the console context by lowering the total physical memory usage threshold in config, in my case in the test assembly's .dll.config, by setting percentagePhysicalMemoryUsedLimit="50". I don't really care if the cache usage is optimal in the test so I just set a very low limit to avoid these issues. Probably lowering it to 90 or even 95 % would have done the trick.