We have an app that maintains a large number of concurrent GraphQL subscriptions (order of 500+) writing frequent updates into a single InMemoryCache during the initial load and UI maintains about the same number of watch queries. Under this load we observed the app’s event loop becoming almost completely saturated — CPU pinned 100%+ with a visible processing backlog.
Profiling pointed to the executeSelectionSet / executeSubSelectedArray caches (cacheSizes['inMemoryCache.executeSelectionSet'] / ...executeSubSelectedArray) With our working set of distinct (selectionSet, entity) pairs exceeding the default cap, it seems that the LRU was continuously evicting and immediately re-requesting the same entries. it effectively blocks the event loop from doing anything else.
(This was captured right before the event loop is fully saturated. Once fully saturated, we can’t even run the profiler)
As a workaround, we increased cacheSizes['inMemoryCache.executeSelectionSet'] well above the default (4× in our case) before constructing any InMemoryCache, which noticeably reduced CPU/event-loop pressure under the same load. And we’ve also replaced watch queries with subscriptions.
Questions
Is this a known failure mode when the ESS/sub-selected-array LRU size is undersized relative to write/subscription volume?
Is there any built-in instrumentation/warning for high LRU eviction rates on these caches, so consumers can detect thrashing before it manifests as CPU saturation?
Are there recommended guidelines for sizing cacheSizes relative to subscription count / entity count, beyond “raise the default and re-measure”?
Is there a way to evict some of the entries that we know we no longer need?
Is this a known failure mode when the ESS/sub-selected-array LRU size is undersized relative to write/subscription volume?
Yes! Those caches make it possible to skip recomputing cache reads for entire subtrees in the query. When writing a query to the cache, the cache will invalidate any field that changes data so the next time the cache read happens, it know to recompute that subtree. If the ESS cache is too small, you’ll be constantly recomputing cache reads all the time since it can’t hold all the computed values (if you do a grep for cache.diff in the repo, these are what trigger most of the cache reads, and this happens a lot, even for a single request). So the fix you have is correct here! We do our best to pick defaults that we think will work for most apps, but of course there are exceptions like yours
Is there any built-in instrumentation/warning for high LRU eviction rates on these caches, so consumers can detect thrashing before it manifests as CPU saturation?
Are there recommended guidelines for sizing cacheSizes relative to subscription count / entity count, beyond “raise the default and re-measure”?
The guidance we have is in the Memory Management docs page. Take a look to see if this answers what you’re looking for. PRs welcome if you have suggestions on how to improve that page to make it more clear.
Is there a way to evict some of the entries that we know we no longer need?
Unfortunately, not really, at least not without touching implementation internals. Feel free to submit a feature request for this as well if you feel it would be useful.
