JSON Home: Why, What, and How (with Clojure)
Having RESTful components programmatically traverse links in hypermedia — much like you just did when you clicked on a link to get here — is an appealing capability. A format like HAL is useful for a resource to dynamically augment itself at run time with links to other resources. But what about something to describe the top-level resources / resource types? How about some sort of well-known entry point that a HATEOAS consumer can start from, like the home page for a web site? Something that is fundamentally about discovery, is self-describing to some degree, and is generated at build time (rather than run time)?
Web maestro Mark Nottingham has been thinking about that for quite some time and proposed JSON Home, a “home document” format for non-browser HTTP clients. I’ve been following JSON Home with interest for some years, particularly back when I was working on capabilities interfaces at Atlassian. I thought I’d try it out with one of the new services we’re building at mPort to augment its HATEOAS capabilities. It is being built to be accessed by our ecosystem partners, as well as our own user interfaces, so hypermedia controls are quite important for conveying an orderly transition of states and for providing contextual references to documentation to developers using the API.
As this service is dealing with a lot of transformations of structured data, we are using Clojure as the main language for the service. With that comes the use of some excellent libraries like buddy-auth (for working with JWT),and Malcolm Sparks’ yada (for handling requests) and bidi (for resource routing).
Although we’ll soon be able to use a number of techniques to auto-generate meta-resources (as well as API docs using Swagger) at build time and meta-data at run time (like HAL), we’re keeping it simple at this stage of the project.
First, we can take a look at the API Home Document. Because we’re using Clojure, we have idiomatic access to EDN, which is a little more pliable than, and easily converted to, JSON. Here is the contents of a text file called json-home.edn:
If you read Mark’s example in the spec, you’ll see it follows his example fairly closely. Let’s break down the document a bit to help explain the items and our choices. Note: none of the URLs in the document can be followed (yet).
api links
describedBy a link to the documentation for the API. This is useful for the developers reading the home document in a response their consumer system has received.
author a link to contact the author(s) of the API. In our case, it’s an email address and subject. This could be easily used by a consumer to send the content of 5xx responses, or the developers to ask questions about 4xx response if the API documentation wasn’t clear.
resources
tag URL-based tag link relations are used for the resource names so they could be followed for documentation about semantics, and also tags like me@example.com,2016:widgets feel more cumbersome (in EDN they’d need to be strings rather than keywords).
authSchemes You can see from the document that we expect Bearer authentication in requests. It’s not apparent that we’re expecting JWT tokens in particular, so this is something where the spec may be able to have more specificity.
allow For the /scan/ resource, the hint is only the GET method is allowed, whereas for an individual scan/scan/{scan-id}, GET, POST, PUT, and DELETE are allowed. If the OPTION method is allowed on a resource, then the same methods in this hint should be nominated in the response.
formats For the resources in this service, the consumer may request response entities be represented in EDN or JSON. At some point, I intend to include text/html, as purely semantic HTML is a perfectly good format for representing data transferred between systems (if you use XML, you implicitly agree). It also happens to be very useful in a resource-oriented client architecture or self-contained system for avoiding a costly and cumbersome JSON-to-HTML translation. Furthermore, HTML already has hypermedia controls built-in, so there’s really no need for JSON Home or HAL at all! 😉 Many thanks to Stefan Tilkov for polluting my mind with this idea.
acceptRanges For collection-type resources, either items or none to hint whether idiomatic pagination is supported or not. Response headers of Accept-Ranges: items and Vary: Range would be expected from the Scans collection resource, but no ranges are currently supported for an individual Scan.
acceptPost, acceptPut, etc. The media types of supported representations for each of the supported request methods.
href The URI reference for the resource. The base URI is that of this home document.
hrefTemplate, hrefVars These indicate which parts of the URI are identifiers and where they go, and a link to the documentation to what each of the identifiers mean. Architecturally speaking, I’m not convinced about publishing URI templates to consumers, particularly with HAL in the picture, as URIs are supposed to be opaque (but let’s set that aside for now).
What, no CURIE?
You’ll note the repetition of the https://mport.com/api-docs/scan-data in URLs throughout the document. This verbosity could be addressed using CURIEs just like HAL allows (see below). The JSON Home spec doesn’t explicitly mention CURIEs, so implementations probably won’t map them.
Now that we have a document to share with consumers, we have to respond with it from a well-known location. Of course, / is the most well-known location for any HTTP host, so it makes sense for an HTTP API to have its “home page” there, too.
The following Clojure file web.clj is used to set up the “world wide webiness” of our service. It does the following:
- Reads in the
json-home.ednfile (above) as a JVM resource on the class path (it gets there because its source is in theresourcesdirectory) — L11 - Add a render function for Clojure maps to JSON Home responses — L14,15
- Add a render function for Clojure maps to JSON responses containing HAL (this should not be needed after this request is implemented) — L17,18
- Define a resource that allows
GETrequests, responding with our home document in one of a number of media types — L20–29 - Binds that resource to the
/route — L34 - Binds a
404response to any other requested routes — L36 - Includes those routes in the HTTP server handlers — L41
web.clj
And that’s all there is to it. The yada and bidi libraries (using ring, Aleph, and netty) have taken care of the heavy lifting.
That is the provider side, so what does it look like for the consumer? The following examples show what the response looks like for a JSON-loving distributed component making a request to our RESTful API, and also even for a human (developer) who might want to look at our home document in a way that is easier to read (yada does this EDN-to-HTML niceness for us).
“/” from a JSON Home consumer
Accept: application/json-home
“/” from a browser
Accept: text/html
A splash of HAL
An additional thought is that we can also (statically) include HAL meta-data in the API Home Document. This allows consumers requesting / for a response entity in the application/hal+json representation would also get some traversal capability.
That simply means adding something like the following to the json-home.edn file at the same level as the :api and :resources items:
We’re not sure yet if we want to do that, but it’s an interesting exercise nonetheless.
To conclude, I think the JSON Home Document format is interesting, and hypermedia controls are beneficial for non-HTML formats in RESTful systems.
Given that JSON Home is especially focussed on a well-known entry point for service capability discovery, it does the job well.
Personally, I think JSON Home should use application/home+json as the media type designation. This conforms to the idea of “suffix” types which indicate the underlying format and allows many consumers to use out-of-the-box JSON parsing pipelines before beginning the interpretation of the JSON Home specifics. This works great for application/hal+json in libraries that conform to the handling of suffix types.
HAL is complementary, particularly at the resource level for traversing link relations calculated at run time. HAL could probably be expanded a little to deal with method hints, media type hints, and standardized link relations to semantics and syntax documentation.
Indeed, there are formats other than HAL for representing hypermedia controls in non-HTML formats. I find them to be overly verbose and attempting to solve a problem that has already been solved (i.e. use HTML5 — you really should watch that video!). HAL is a simple and focussed middle ground those using XML and JSON content models.
Clojure is a pleasure to work with, as are the libraries and their contributors in the Clojure ecosystem.
Given most API providers aren’t in the habit of being so disciplined to provide hypermedia links for transitions, most API consumers are oblivious to the possibility. I find myself wondering: is it worth going to the trouble of generating these things if most developers won’t even use them?
