Apollo Client cache not feasible for large data sets?

jhgrund · January 16, 2025, 2:20pm

Hey

We’re using Apollo Client, and are in general satisfied. However, we’ve run into performance issues for some of our clients that have very large accounts, which leads to some large requests and store caches.

The UI will stutter and drop frames to a varying degree, but it’s very noticeable. It seems to increase/occur whenever a request comes in and tries to update the cache.

Based on discussion here: Slow updates with large cache

we’ve looked into our cache sizes and limits, and have adjusted accordingly.

We seem to have a pretty large cache, but have adjusted the limits to accommodate it:

{
    "limits": {
        "parser": 1000,
        "canonicalStringify": 1000,
        "print": 2000,
        "documentTransform.cache": 2000,
        "queryManager.getDocumentInfo": 2000,
        "PersistedQueryLink.persistedQueryHashes": 2000,
        "fragmentRegistry.transform": 2000,
        "fragmentRegistry.lookup": 1000,
        "fragmentRegistry.findFragmentSpreads": 4000,
        "cache.fragmentQueryDocuments": 1000,
        "removeTypenameFromVariables.getVariableDefinitions": 2000,
        "inMemoryCache.maybeBroadcastWatch": 5000,
        "inMemoryCache.executeSelectionSet": 250000,
        "inMemoryCache.executeSubSelectedArray": 150000
    },
    "sizes": {
        "print": 26,
        "parser": 38,
        "canonicalStringify": 15,
        "links": [],
        "queryManager": {
            "getDocumentInfo": 27,
            "documentTransforms": []
        },
        "cache": {
            "fragmentQueryDocuments": 0
        },
        "addTypenameDocumentTransform": [
            {
                "cache": 27
            }
        ],
        "inMemoryCache": {
            "executeSelectionSet": 128121,
            "executeSubSelectedArray": 92948,
            "maybeBroadcastWatch": 49
        },
        "fragmentRegistry": {}
    }
}

It should be normalized, at least to a very high degree. We also have lint rules to ensure that everywhere an ID field is available, it is included in the query fragment on that type.

Any advice on how we can improve performance for the client/cache while managing large data sets would be much appreciated.

Thanks!

jerelmiller · January 16, 2025, 3:41pm

Hey @jhgrund

Have you taken any performance profiles to see where the the time is spent? Its difficult to recommend anything without knowing where the bottleneck is.

jhgrund · January 17, 2025, 10:25am

Sure, I just found it very hard to get anything useful out of except a lot of store operations and recomputes.

There’s a bunch of long running tasks of many seconds in the profile, and they are so deeply nested a screenshot doesn’t do it justice.

Here’s a short video from one of the 4 second long running tasks from a profile:

jhgrund · January 23, 2025, 9:00am

Here’s a screenshot from another perf slice:

The store seems to go into a loop or something somehow, causing the entire page to hang. Is that a symptom of something we do wrong somewhere?

It can only be reproduced for our clients with large hierarchies

The request that kicks off the most recalculations from the store is a very simple one

export const LATEST_TIMESTAMP_QUERY = gql(/* GraphQL */ `
  query LatestValueReceivedForInstallation($installationId: ID!) {
    node(id: $installationId) {
      ... on Installation {
        id
        name
        dataLastReceivedAt
      }
    }
  }
`);

It only receives id and string scalars on a single node, no nested types or anything.

lenz · January 23, 2025, 9:25am

Can you maybe send one of those profiles to lenz@apollographql.com ? From just the screenshot it’s impossible to tell anything here.

jhgrund · January 23, 2025, 9:34am

Would love to, but I can’t save them, chrome throws error saying

Failed to save timeline: Invalid string length (RangeError)

when I try to save the profile. Apparently, after some googling, it’s too large to save, even if I tried to make the profile short in length

I’ve tried screen recording the trace a bit for you here:

jhgrund · January 23, 2025, 9:35am

I’ve also found this (old) bug in the mean time, but can’t tell if it was ever solved or just auto closed?

github.com/apollographql/apollo-client

Infinite loop when using a cache-and-network fetchPolicy

opened 12:53PM - 09 Dec 20 UTC

closed 06:28PM - 06 Jul 21 UTC

alinpetrusca

✔ confirmed 🛬 fixed-in-prerelease ♾ Infinite

**Intended outcome:** I have a component that retrieves some data using useQuer…y and renders a chart. Until now I used these options: ``` fetchPolicy: 'cache-and-network', nextFetchPolicy: 'cache-first' ``` But now I realized that I have some data that may change and I need it to be up-to-date. I still want to use the cache, so I think that **cache-and-network** fetchPolicy would be the best in this case. **Actual outcome:** When I change the fetchPolicy to **cache-and-network** or **network-only** I end up having an infinite loop. This doesn't happen when I use **no-cache**. This is how the component looks like: ``` export const GET_CHART_DATA = gql` query getChartData( $year: Int! $previousYear: Int! $currentYearStartDate: Date $currentYearEndDate: Date $previousYearStartDate: Date $previousYearEndDate: Date $product: String $channel: String ) { currentYearData: reportings( year: $year service: "OTHERS" date_Gte: $currentYearStartDate date_Lte: $currentYearEndDate productType_In: $product channelType_In: $channel dateType: "VISITS" ) { edges { node { date value } } } previousYearData: reportings( year: $previousYear service: "OTHERS" date_Gte: $previousYearStartDate date_Lte: $previousYearEndDate productType_In: $product channelType_In: $channel dateType: "VISITS" ) { edges { node { date value } } } budgetData( year: $year type: "OTHERS" date_Gte: $currentYearStartDate date_Lte: $currentYearEndDate ) { edges { node { date value } } } } `; const ChartContainer = ({ dateRangeVariables, filters }) => { const { loading, data, error } = useQuery(GET_CHART_DATA, { variables: { ...dateRangeVariables, product: filters.product.map(({ value }) => value).join(','), channel: filters.channel.map(({ value }) => value).join(',') }, fetchPolicy: 'cache-and-network' }); if (loading) { return <div>Loading</div>; } if (error) { return <div>Error</div>; } // structure data for chart... return <div>Chart</div>; }; ``` **Versions** @apollo/client 3.2.0 I assume this is a cache issue, but I'm not sure if it's my fault or not and I don't know how to fix it. Any ideas? Thank you for your time! **Edit**: I also think that defaultOptions is not working correctly. I have these options set when I create the apollo client instance: ``` defaultOptions: { query: { fetchPolicy: 'cache-and-network', nextFetchPolicy: 'cache-first' } } ``` First time when I render a component that queries some data I see the network request. If I switch to another view and then come back, I see no request (it's retrieved from the cache - **cache-first?**). Anyway, if I use **fetchPolicy: 'cache-and-network'** option inside useQuery I always see the request at component mount. Is this the expected behavior ?

The simple query I linked uses fetchPolicy: 'cache-and-network' and we’re using "@apollo/client": "3.12.2"

jhgrund · January 23, 2025, 9:39am

Memory also climbs throughout the trace

lenz · January 23, 2025, 10:28am

That issue was related to memoization cache sizes, and while I agree that your problem here looks like it would hit a memoization cache limit, in your initial post you show a level of usage that doesn’t.

Maybe you can try to profile with Firefox instead or record a replay.io replay?
I really can’t read anything for half of that video and would need to look around in that data myself.

jhgrund · January 23, 2025, 2:15pm

Sent you a performance profile from Firefox

lenz · January 23, 2025, 2:23pm

Hmm. This solidifies my gut feeling that you might be hitting memoization cache limits.

Which gives me an idea: getMemoryInternals only shows the limits that would be in place if the cache would be created now - but not necessarily the limits that were in place when the cache was created. Could it be that you set these limits too late and they don’t get picked up?

Topic		Replies	Views
Dealing with large objects in the cache Client SDKs client , web	2	856	April 3, 2024
Slow updates with large cache Client SDKs client , web	9	931	May 1, 2024
useFragment is not scalable as the number of fragment grows Client SDKs client	7	415	January 23, 2024
Handling in memory cache in large application Client SDKs client , web	3	300	February 6, 2025
Apollo GraphQL client caching strategies Client SDKs client , web	1	474	January 10, 2023

Apollo Client cache not feasible for large data sets?

Related topics