GraphQL's role in the new presentation layer

Written by Erik Mogensen updated: Wednesday August 16 2017 13:25

In the new presentation layer, CUE Front, which is an option provided for Escenic Content Engine version 6.0 and onwards, we've adopted GraphQL, a "graph query language" developed by Facebook. As time has passed, our understanding of GraphQL has evolved, and this has spurred us to use GraphQL in some rather innovative ways.

Background

To start off with, we have created a rather large GraphQL schema that describes the information structure of the web services provided by a typical Escenic Content Engine. This even includes GraphQL schema types for the different Escenic content types, and their fields. This means that if you add a boolean field to a content type, then the GraphQL schema will include a corresponding boolean field definition.

The schema includes resolvers, so what you get is a fully functional GraphQL engine that allows you to follow the links provided by the underlying AtomPub web services. The GraphQL API is a facade in front of the web services. GraphQL effectively allows us to avoid a deeply nested callback pyramid or long promise chain, with all of the headaches involved in maintaining that.

Deviating from the norm

This way of using GraphQL is pretty normal, and quite fashionable, however this way of using GraphQL has a few big limitations. First of all is that it unequivocally introduces an extremely tight coupling between the client and the server. The results are generally not cacheable, since the queries are ad-hoc, and typically passed to the server using HTTP POST (which in itself is uncacheable). What we were aiming for was a simpler "JSON API" that could have very high read efficiency, meaning that we had to get caching in place.

Our solution is to store the queries on the server side rather than expecting (or allowing) clients to send ad-hoc queries. The server has a small set of queries that are pulled up using a simple heuristic based on the subject of the page being served. There is a default query type for each type of content available, and it's possible to override queries for different parts of the site. This means that each "page" that CUE Front will be rendering will have a specific query associated with it, and thus, a particular result too. This in turn makes the GraphQL responses cacheable, since each client will get the same result. Readers familiar with GraphQL might find this odd, because one of the strengths of GraphQL is the ability for clients to specify exactly what they need in their UI. This ability, however, comes at a cost of stronger coupling, something we want to avoid.

In order to facilitate how developers work, we've crafted our server with some small extras. Let's say that a path of a resource is /mysite/weather/2016-storm-surge.html. The JSON that's supposed to come back from this is the data needed to render the web page. The page itself might be associated with a specific GraphQL query (stored on the server), called story-weather.graphql—consisting of the name of the type of content and its home section. On a local development environment, the developer can suffix the URI with /edit to get right into the GraphiQL editor: /mysite/weather/2016-storm-surge.html/edit. We even added a "save" button, which stores the query under the file name in question, replacing the query. This gives developers the ability to grab "just right" amount of data quickly by simply adding /edit, modifying the query, hit "save" and go back to using the (updated) JSON result.

GraphQL for fun

Another innovation we've worked on only uses half of the GraphQL technology stack. We realized that the "first part" of the technology stack is pretty interesting on its own:

By utilising just these technologies, we make it possible to express pretty rich "queries" that are never executed, but instead are transformed into a different type of query, to be "executed" elsewhere, in a completely different context.

The problem domain is that we want to have a way of defining Solr queries in a relatively abstract form. Abstract meaning that the query itself is not concrete, but re-usable depending on different contexts. For example, given a story you might want to search for other stories that have similar tags as the one you're showing. The "similar tags" clause is an abstract clause that needs some extra processing before being sent to Solr. In our system we need to be able to allow developers to provide queries such as (tag:sometag AND section:somesection AND NOT (tag:breaking AND tag:important)). but where somesection and sometag are substituted for real tags at query time. We could of course roll our own editor, but the type system and tools that surround GraphQL make it a relatively good match for the job.

The type schema in GraphQL started out a bit like this:

type Query {
  and: Search
}

type Search {
  and: Search
  or: Search
  not: Search
  tag: String
  section: String
}

This simple schema allows you to write pretty complicated queries, something like

query {
  and {
    tag(name:"sometag")
    section(name:"somesection")
    not {
      and {
        tag(name:"breaking")
        tag(name:"important")
      }
    }
  }
}

By using aliases instead of "name" field parameters, the query looks even simpler, albeit a bit backwards.

query {
  and {
    sometag: tag
    somesection: section
    not {
      and {
        breaking: tag
        important: tag
      }
    }
  }
}

This corresponds 1-to-1 to the query described earlier:

(tag:sometag AND section:somesection AND NOT (tag:breaking AND tag:important))

Notice how the schema defines things as being of type String — this is actually false. In fact, the return type is never seen or used by anything, because the GraphQL query is never executed per se.

Taking the GraphQL query and running it through the GraphQL parser provides us with an abstract syntax tree, or AST for short. The AST gives us information on what selections are done at what level, and what their aliases are and so on. It is a trivial task to transform the AST to the actual Solr query which then can be used to query Solr for actual data. The schema can be used to power the GraphiQL UI too, with tooltips and code completion and on-line help without any extra work. And when the GraphiQL "play" button is pressed, the server then (instead of executing the query using the GraphQL engine) transforms the query to a Solr query and returns the Solr response to the user instead of a GraphQL response. Add the save button, and you start to get a feel of this becoming an IDE for Solr queries.

As you can see, the GraphQL schema is not used to describe the shape of the data desired, but the shape of a query that's used to then fetch data from Solr.

We are currently investigating ways of making it possible to supply boosting information to data sources, so that individual terms or clauses could impact the order of the search results.

GraphQL for fun and profit

Content Engine's Widget Framework consists of a lot of types of widgets, all configured and assembled in large structures called templates. The widgets in a template are grouped into areas and subgroups, indicating where on any page the widget ends up. The widgets have "data sources" that tell it where to get the data.

With CUE Front, it is possible to carry the concept of widgets forward into yet another GraphQL editor. Together with a customer, we defined a GraphQL schema that describes concepts such as widgets, layouts and so on. A GraphQL query using that schema, therefore, describes a page layout in terms of various groups and areas and their widgets. Again, the abstract syntax tree is used to understand the details of the query, and "execute" it, not by using the built in GraphQL engine or the schema's resolvers, but rather just walking through the AST and building up the response by inspecting every node and doing what it's supposed to do.

No, the resulting object no longer "resembles" the GraphQL query in that the keys aren't a 1-to-1 mapping, but that was never GraphQL's main selling point. The output of the cook is changed to define widges, layouts and their contents, rather than the more typical result of a GraphQL query. The templates no longer interact with the underlying data structures; the structure of which templates to include where comes directly from the cook.

Conclusion

This is for sure not what the creators of GraphQL intended when they set out, but the choices that the GraphQL team have made along the way make for quite a useful toolbox. The tools are nicely independent and loosely coupled, so they can be reused in ways that requires some imagination to understand.

Ultimately, we might move forward from the use of these GraphQL tools as we bump into the various limitations that are there for when GraphQL is used as intended. But for now, we'll live with these quirks to be able to move quickly.