About the new presentation layer

Written by Erik Mogensen updated: Tuesday March 21 2017 10:57

Our new presentation layer, called CUE Front is a radical shift away from the existing presentation layer provided in Content Engine. Below, we provide an overview of the different technological choices we made, and some of the technologies in use. We offer the reasoning behind them, and some of the problems they aim to solve.

To start off with, there are a few important architectural choices:

Then there's the actual list of technologies:

To see how they interact, you can take a look at the documentation; the point of this post is to give interested readers an idea of why they are used.

Let's start with these architectural choices, since they permeate the whole solution.

Single Request paradigm

One of the premises of the new presentation layer is the idea that the rendering engine that produces the actual HTML should make no more than a single HTTP call to the back-end, to get all the data needed to produce the HTML in question. This is a premise taken from the Guardian's core development principles which says that ... "you cannot make two calls to the content API or any other 3rd party".

In CUE Front it means that the Twig rendering engine (the waiter) can make a single HTTP request to its next hop (the cook) and expect a JSON response containing everything needed to render the final web page.

Statelessness

Statelessness is something that's inherited from the stateless nature of HTTP, the fundamental building block of the web. Essentially it means that each HTTP request must be understood in isolation, that it must not rely on previous interactions in order to function correctly.

This is an important premise, as it allows us to scale the different components of the stack without having to worry about things like session binding or replication. These things are important when building highly available and fault tolerant systems.

Twig

Content Engine has traditionally used JSP (JavaServer Pages) as its presentation layer technology. JSPs can be extremely efficient, as the JSP source code is transpiled to Java code, which is then compiled to Java bytecode, which eventually gets optimised using JiT to very efficient processor instructions. However, JSPs have a crucial flaw, namely the ability for developers to write arbitrary code in the files. This happens from time to time, and as such, you have to be a Java developer to write or maintain JavaServer pages.

Twig was chosen as its replacement because it is born from the ubiquitous PHP community. Twig is an actively maintained templating engine, with good support in various IDEs like IntelliJ. Crucually it also disallows writing arbitrary code in the templates, and they are compiled down to PHP bytecode, and perform pretty well. It's used by Drupal 8, CraftCMS and has very similar variants in lots of other environments like Jinja2 and Django.

A note on choosing PHP here: PHP is inherently stateless; each request is by default isolated from previous runs. It's just the way things are. Most other environments out there have an inherent capability of introducing state to the server; while for PHP you have to go out of your way to achieve this. The way we see it, the stateless nature of PHP ensures that the application suddenly doesn't accure state, and by that, violate HTTP's stateless nature.

GraphQL

GraphQL is a technology that is usually used by APIs by embedding GraphQL queries in client code, and with a single network operation (HTTP POST) send the query to the server, and get a single response containing exactly the data they want/need for a particular screen. For CUE Front it solves a slightly different but related problem. In CUE Front it is necessary to get many separate pieces of content in order to assemble the data needed to render the page. It would be possible to write lots of complicated asynchronous code using Promises and so on, but this would lead to a large, impenetrable chunk of code, which would be very hard to maintain.

GraphQL allows us to describe (in a succinct manner), which parts of which content items to retrieve, and which additional requests to make. The query itself is stored on the server, and is loosely tied to a "page template" on the rendering side. The query is relatively easy to understand and modify, without having to deal with maintaining long promise chains, promise aggregations, and so on. GraphQL includes an in-browser IDE, and is supported in major stand-alone IDEs like IntelliJ

Tachyons

Writing CSS seems like a simple thing, however, writing maintainable CSS is quite difficult. As some people have discovered, it becomes increasingly difficult to maintain CSS codebases that keep on growing as the site evolves. "Deleting unused CSS is hard and time consuming," so it generally is left alone.

Functional CSS, the idea of using single-property CSS with low specificity is also known under the names Atomic CSS and Immutable CSS. Tachyons is a set of functional CSS selectors with names such as db (for display: block) and br3 (for border-radius: 0.5rem). To seasoned CSS authors, it takes some getting used to, but the end result is that most, if not all of the CSS needed is already there. The result is less CSS code, and more presentational feel of the HTML, which ends up having (non-semantic) classes like db br3.

It is important to note that the CSS in Tachyons always has the lowest possible specificity. Even the "desktop overrides" have a single class specificity. This makes it possible to use Tachyons for most of the components and switch over to using a custom CSS class when that's called for. As the site evolves over the years, it should be possible to keep the amount of custom CSS written to a minimum, avoiding bloated CSS downloads, and perhaps even avoiding cache busting techniques, since the Tachyons CSS is more or less immutable.

To build a site using tachyons, there are two things you need to know:

  1. To read tachyons you need to understand what class="f3 fw4 fw6-ns ma3 dn-l" means. This can be done by going to tachyons' list of properties and Ctrl+F to find .fw6-ns for example. It'll tell you that fw6-ns controls the font-weight property and sets it to 600. The -ns suffix is tachyons way of saying "not small".
  2. To write tachyons, you need to be able to quickly figure out how to apply a left margin on an element. Here too, you go to the list of properties and Ctrl+F to find margin-left to see that there are loads of ml(short for margin left), alongsidemh (for margin horizontal), and nl (for negative left) classes.

Pretty soon you'll be proficient in reading and writing f3 fw4 fw6-ns ma3 dn-l understanding that it meanse "Font-size 3ish, font-weight 400%, font-weight 600% on medium and large displays, medium margin, and display: none on large devices.

Sass

Even with Tachyons in place as a way to eliminate most custom CSS, there will always be some custom component that screams for a custom CSS class with a corresponding entry in the CSS class. Sass helps keep that CSS maintainable by allows us to use e.g. scoping, variables and imports. Another use case for writing CSS is to extend the language of Tachyons when something is missing. For example there are no tachyons as of yet that can e.g. apply a clearfix to odd and even rows or every third rows; adding a new single-use, low specificity CSS rule using Sass allows you the possibility of reusing the breakpoints and global variables (like color) defined elsewhere.

Make

When starting this project, it was apparent that we would need to have some build system, if only to compile the sass to scss. There are a plethora of build tools out there, but many of them end up being hard-to-understand, some with multiple lines of nested function calls, requiring knowledge of a programming language to make the simplest of changes.

By using make, we declare the programming language to use be the modest command line, which in a way is a lowest common denominator when it comes to executing things.

Docker

CUE Front uses all of these technologies, with different base tools, like node.js for the cook and Browsersync, PHP for the waiter and the style guide, Python for chopping up images and the sass compiler, and Make to build the project. It is not so hard to install everything on ones own machine, so long as that machine is running Linux.

In order to better support Windows and macOS development, we decided to provide Dockerfiles and a Docker Compose file. With these files, a developer should be able to get a working environment without having to install each of those separate tools.

Atomic design

A problem facing template developers is the time it takes to develop a new feature. Often the feature is going to be built up of existing pieces of content. There will be headlines, links, images and buttons and some layout. The new feature should be able to re-use existing designs of these things.

Atomic design helps with this by splitting web pages up into smaller building blocks, that are assembled in larger structures to make full pages. When designers are tasked with building a feature, there is already a library of components from which to build. Small and large components can be picked from the shelf and composed to the new feature. Atomic design gives designers a way to structure the templates in a way that is reasonably well known.

PatternLab

An important feature of template development is the ability to see what you're doing quickly, without having to spin up a whole stack of processes and a CMS to host content. This feature is important for the template developer's own sake: Rapid turnaround. Template developers want a drop dead simple stack that simply works. Any rendering engine needs data in order to work, and in our case, the data is typically provided by the cook (based on the single request paradigm) through a HTTP call. In order to make changes to the JSON data coming from cook, it is sometimes desirable to get the "full stack" running locally, and this is resource intensive and complicated to set up. While Docker makes this simpler, there are lots of moving parts and a designer might not want to understand the full picture.

The solution is to re-use the templates with a static copy of the data. The waiter is designed to do a single HTTP request to get the JSON data and then render that data using the twig templates. If you have an off-line copy of that JSON data (it's a single file, remember) then you should be able to pre-render the HTML given that JSON data. Patternlab does this exact thing, along with providing a UI to navigate the pre-rendered HTML. The end result is that a designer can work completely in Patternlab and be confident that the changes made to the templates will work the same way in the final web page.

The standard set-up includes some stock JSON data, but once the project gets going, Patternlab allows you to include many more JSON files, representing different pages on the site. In this way, the style guide can include quite a lot of full-page renderings of various pages (including search results, topic pages, section fronts, video walls, long forms), alongside smaller components that make up the site (according to Atomic Design).

The output of the Patternlab doubles as a style guide, allowing and encouraging designers to maintain a "living style guide" for the web site. Designers can create new templates and include them in the style guide-rendered full page views and show them to stakeholders without having to deal with Content APIs or CMSes. This increases the speed at which designers can implement new features, check what it looks like on the final site, and get feedback from stakeholders.

Browsersync

When a designer is working on a feature, they usually have the rendered template open in one or more browser tabs, different browsers, perhaps even a few devices. The problem is that whenever a change is made to a template or a Sass file, then you have to manually reload each browser to see what changed. The more browsers you have open, the more likely you are to miss something by forgetting to reload.

To solve this problem, we've integrated Browsersync, which automatically picks up on changes to CSS and then injects the new styles to the web pages. It also reloads web pages when the template files have been modified. This increases the speed at which a designer can iterate over the designs, and see the design live in various browsers and devices.

Conclusion

CUE Front is rich in features, and includes a lot of technologies to be able to provide those features. A quick look at the code might leave you with questions like why are these technologies there in the first place? This post should explain most of those questions, and explain the roles the different technologies play in the whole CUE Front ecosystem.