Slow updates with large cache

Client SDKs
,
1

I have a mutation that is returning a small amount of data and am seeing the app stall for seconds. It seems to me that the overhead seems related to the size of the data in the ROOT_QUERY, or perhaps in the cache as a whole.

Below is a screenshot of the profile.

I’d love to form a mental model about why this might be slow, and to learn what I can do about it. One note is that the data is not very normalized; there’s a lot of data underneath the root query. Would normalization help with this update somehow? If so, why?

One other piece of trivia is it seems like broadcastWaches happens twice for mutations, so they appear to be about twice as slow.

image
image2381×2461 405 KB

1 Like
2

I would assume that normalization might help you big time here - the cache can bail out on equal objects - and something like { __typename: "User", id: 1, bestFriend: { __ref: "User:2" } } will stay equal when User:2 is updated in the cache, but if it’s not normalized, all “parent entities” will be updated, too - and of course, your object will be stored with many copies in a lot of places.

3

Why don’t you normalize?

ex:

const client = new ApolloClient({
  link: ApolloLink.from(links),
  cache: new InMemoryCache({
    dataIdFromObject(responseObject) {
      switch (responseObject.__typename) {
        case 'Product':
          return `Product:${responseObject.sku}`;
        default:
          return defaultDataIdFromObject(responseObject);
      }
    },
  }),
});
4

@lenz , when you say we should normalize, do you mean specifically normalize the top-level object that is the result of the large query so that it is not cached directly under ROOT_QUERY (but otherwise let the result to be one big blob ), or do you mean normalize the blob into small constituent parts?

We have tried the former (i.e. give an id to the query result) and it doesn’t seem to help (in fact, it seems to have doubled the duration of the InMemoryCache2.batch call from 1s to 2s). Normalizing smaller parts of this query results would not be trivial (the results come from large JSON objects without particularly natural keys), and also I don’t think we would benefit from it conceptually – these objects are not shared by the results of different queries.

I would like to understand what’s going on and get a mental model for what’s suitable to store in the cache. I can’t quite imagine why normalizing matters much for us. To re-iterate, our problem seems to be that Apollo is processing the results of a small query, and this processing seems to be slow because the cache contains a lot of data (in particular, one large object that’s not normalized). I imagine that, upon receiving the results of the small query, Apollo needs to compare the results with cached data to see what needs updating. How does one big not normalized object in the cache affect this change detection algorithm? In our particular case, the results of the small query have nothing to do with the results of the big query – the __typename’s at the top level are different. So, normalized or not, I can’t quite imagine why the large result in the cache matters; I would expect the updating algorithm to not look at the large object at all.

For reference, the un-normalized object that makes up most of our cache consists of maybe ~200k GraphQL objects. When this object is in the cache, processing the results of the small query take ~1s (when the large object is cached directly under ROOT_QUERY-><largeQueryName(largeQueryArgs)>, and ~2s when the large object is cached under its own typename/id.

5

You might need to be a bit more specific to how that is organized - I had assumed that you do some kind of small updates to your large in-cache object, so from that perspective I’d recommend you split that large object into many smaller normalized objects.

Is the small result not updating the large one at all?

In that case, another thought:
Is the large result maybe hitting the default cache memoization sizes and causes stuff on screen to be constantly recalculated?

In that case, do a read through Memory management and do a console.log(client.getMemoryInternals()) to see if you are hitting the limits there.

6

First, thanks a lot for helping us!
I’ve stared at this a bunch running experiments, and I feel there must be something wrong here.

So, I have two queries that have nothing to do with each other – they return completely non-overlapping data. One query returns a large result; let’s call this query getStacks(). The second query returns a tiny result; let’s call this query dummyQuery(). Now, depending on whether the large result for getStacks is in the cache, processing the results of dummyQuery is either very fast or very slow – i.e. if I run getStacks with policy no-cache, then processing the results of dummyQuery takes 1ms. If getStacks results are cached, then processing dummyQuery results takes ~500ms. Additionally, processing the results of dummyQuery takes even longer if I turn this query into a mutation – it seems to usually take double (1s), but sometimes it seems to take 2s.
I believe this duration is all about apollo cache updating; it has nothing to do with other parts of my app re-rendering or such; I’m pasting the code that’s doing the measurement and the relevant Chrome profiles.

Another perhaps interesting fact is that, if the results of dummyQuery are not put in the cache when the query returns (for example because I run it with no-cache, or if dummyQuery is a mutation whose result is not normalized), then processing these results is fast.

So, it would appear that the presence of the large getStacks results in the cache affects the caching of dummyQuery results.

Is this supposed to ever happen / how can we investigate this further?

Here are the profiles showing the 500ms processing of dummyQuery results and 2s processing of dummyQuery results when it is a mutation. I’m also putting a screenshot of the mutation one below.

Here is the code doing the measurement, for clarity.

  async function onClick() {
    const start = Date.now();
    const {data} = await client.mutate({
      mutation: DUMMY_MUTATION,
      //fetchPolicy: "no-cache",
    });
    console.log(
      `done mutation. took ${Date.now() - start}ms. stats:`,
      stats,
      JSON.stringify(stats, null, 2),
    );

This is the output of client.getMemoryInternals(), printed immediately
after the dummyQuery results have been cached.

{
  "limits": {
    "parser": 1000,
    "canonicalStringify": 1000,
    "print": 2000,
    "documentTransform.cache": 2000,
    "queryManager.getDocumentInfo": 2000,
    "PersistedQueryLink.persistedQueryHashes": 2000,
    "fragmentRegistry.transform": 2000,
    "fragmentRegistry.lookup": 1000,
    "fragmentRegistry.findFragmentSpreads": 4000,
    "cache.fragmentQueryDocuments": 1000,
    "removeTypenameFromVariables.getVariableDefinitions": 2000,
    "inMemoryCache.maybeBroadcastWatch": 5000,
    "inMemoryCache.executeSelectionSet": 50000,
    "inMemoryCache.executeSubSelectedArray": 10000
  },
  "sizes": {
    "print": 25,
    "parser": 8,
    "canonicalStringify": 4,
    "links": [],
    "queryManager": {
      "getDocumentInfo": 9,
      "documentTransforms": []
    },
    "cache": {
      "fragmentQueryDocuments": 0
    },
    "addTypenameDocumentTransform": [
      {
        "cache": 9
      }
    ],
    "inMemoryCache": {
      "executeSelectionSet": 45973,
      "executeSubSelectedArray": 10000,
      "maybeBroadcastWatch": 13
    },
    "fragmentRegistry": {}
  }
}

The queries look like this:

type Query {
    getStacks(snapshotID: Int!, filters: [StacksFilter!]): SnapshotData!
}

type Mutation {
    dummyMutation: DummyType!
}

type DummyType {
    id: ID!
    dummyNumber: Int!
}

type SnapshotData {
    id: ID!
    # Goroutines grouped by process.
    GoroutinesByProcess: [GoroutinesGroup!]!
}

# GoroutinesGroup describes a set of goroutines with identical stack traces.
type GoroutinesGroup {
    # The IDs of the goroutines in this group, grouped by process.
    IDs: [GoroutinesIDs!]!
    Data: [FrameData!]!
    # Frames are ordered leaf to root.
    Frames: [FrameInfo!]!
}

The data inside GoroutinesGroup is not normalized, fwiw.

Screenshot from 2024-04-04 15-42-24
Screenshot from 2024-04-04 15-42-241190×1043 166 KB

7

Your getStacks query has so many results it just cannot be memoized with the current cache sizes, so it will re-execute all calculations on every cache access.

Try bumping the "inMemoryCache.executeSubSelectedArray" from 10000 to something like 50000.

"inMemoryCache.executeSelectionSet" is also getting dangerously close to the limit of 50000, so maybe make that 100000.

8

Thanks!

Instead of messing with these cache size limits, is there perhaps a way to tell Apollo to treat the results of the getStacks query (or, more generally, objects of type SnapshotData) as “blobs” that are opaque to the cache and count as a single object towards these limits? I would like the results of getStacks to be cached on a per-query-arguments basis, but otherwise I don’t want the cache to look through them, attempt to normalize parts of the results, etc. What makes SnapshotData special is that its results are not normalized and no other queries overlap with it.

I could modify the query schema to return a String and deal with encoding and decoding JSON out of it, but I like Apollo giving me structured types.

9

You could simulate some kind of scalar, or try it with a typePolicy that stores that value outside of the cache, but there’s no simple option for that.

Really, just bump the cache value - I’m pretty sure all your performance problems here will be gone.

1 Like
10

I have the same issue, the InMemoryCache is stuck for like 30s when fetching around 12K~ entities