Preview: Connectors N+1 batch support

Connectors
1

We’re hard at work on a solution for N+1 queries, which frequently occur in federated GraphQL APIs because of how entities work. Today, Connectors support single-item entity resolvers with the entity: true option, but we have a new pattern for entity resolvers that allow making one request to fetch a batch of entities.

This is a very early preview with only the most basic support, but we’re excited to show you the design and get your feedback.

Check out our examples repo for instructions on trying out the preview. We’ll also share more examples as we develop the feature.

How batching works

The key to understanding connectors batching is knowing how the query planner fetches entities in batches. Here’s an example schema that provides a list of products:

type Query {
  products: [Product]
    @connect(
      source: "v1"
      http: { GET: "/products" }
      selection: "id" # this endpoint only provides ids
    )
}

type Product {
  id: ID!
  name: String
  price: String
}

Because the Query.products endpoint doesn’t have all the information on products, we need a way to fetch more product information given product identifiers (keys.) The query plan would look something like this:

QueryPlan {
  Sequence {
    Fetch(service: "users.v1 http: GET /products") {
      { users { id } }
    }
    Flatten(path: "users.@") {
      Fetch(service: "users.v1 http: POST /products-batch") {
        ... on User { id }
        =>
        { name price }
      }  
    } 
  }
}

The first fetch collects Product entity keys to use in the second request. The entity keys look like this:

[
  { "__typename": "Product", "id": "p3" },
  { "__typename": "Product", "id": "p1" },
  { "__typename": "Product", "id": "p2" }
]

This is the data available in the $batch variable. Using the connectors mapping language, you can transform this data in a number of ways.

ids: $batch.id                         # { "ids": ["p3", "p1", "p2"] }
$batch { id }                          # [{ "id": "p3" }, { "id": "p1" }, { "id": "p2" }]
$({ filters: { "$in": $batch.id } })   # { "filters": { "$in": ["p3", "p1", "p2"] } }

The selection argument plays an important role with batching too. Your selection must result in a list of objects. For example:

Response:  { "data": [{"id": "p3", "name": "Product 3", "price": 111}, ...] }
Selection: "$.data { id name price }"
Result:    [{"id": "p3", "name": "Product 3", "price": 111}, ... ]

To enable this pattern, add a connector to the Product type:

type Product
  @connect(
    source: "v1"
    http: { POST: "/products-batch", body: "ids: $batch.id" }
    selection: "id name body" # this endpoint provides the rest of the data
  )
{
  id: ID!
  name: String
  price: String
}

Now the query plan results in just two requests, one for the list of product IDs, and one for a batch of products with more information!

The last thing to know is that batch order is handled for you. It’s critical that we correctly match data from your endpoint with the entity keys from the query planner. Internally, we use the shape of the $batch.anyField expressions to match items in the result of your selection mapping to guarantee data consistency, regardless of the sorting your endpoint supports.

Limitations of the preview

  • We haven’t finished all the necessary validations, so it’s possible to write a connector that fails at runtime.
  • Currently, you’re limited to expressions like $batch.id. Expressions with subselections like $batch { id } will fail.
  • Currently, the only two patterns supported are POST request bodies:
@connect(
  http: {
    POST: "http://my.api/products-batch"
    body: "ids: $batch.id"
  }
  selection: "id name price"
)

# This is equivalent to:
curl -X POST http://my.api/products-batch -d '{"ids": ["p1", "p2", "p3"]}'

or JSON-stringified URL parts:

@connect(
  http: { GET: "http://my.api/products-batch?json={$batch.id->jsonStringify}" }
  selection: "id name price"
)

# This is equivalent to:
curl http://my.api/products-batch?json=%5B%22p1%22%2C%22p2%22%2C%22p3%22%5D
# or more legibly:
curl http://my.api/products-batch?json=["p1","p2","p3"]
  • We have plans to support lists in URLS like these.
    GET: "/products?ids={$batch.id->join(',')}"
# /products?ids=p3,p1,p2
GET: "/products" queryParams: "id: $batch.id"
# /products?id=p3&id=p1&id=p2
    Please let us know what other request formats you need!

Feedback wanted!

Please let us know here in the community if our design for batching doesn’t meet your needs, or if you have any questions about how we can support your batch endpoints. Thank you!

10 Likes
2

@Jonathan_Wondrusch did you see this update? Thoughts?