status.gallery: a LowJS reactive web app — Part 1
This is the next in a series of #LowJS articles, which started with sample code for an introduction to Hotwire. It used a reactive Spring WebFlux backend with Thymeleaf templates and Kotlin Coroutines, and one of the examples sent updates to the frontend using HTML content in Server-Sent Events (SSE).
We continue now using a full-blown, real-world application (called “status.gallery”) which expands on the use of the above technologies and includes Kotlin Flows in the mix — the latest reactive streams abstraction in Kotlin to replace RxJava and Project Reactor wrappers.
Given the amount of material to cover, I’ve split it up into four separate articles covering:
- LowJS frontend (Hotwire, fragments, SSE, Bootstrap)
- Reactive backend (Spring WebFlux, Kotlin Coroutines, Kotlin Flows)
- Types and application design (Arrow, Mockk, async logging)
- The build (Gradle Kotlin build script, browser dependencies, JVM dependencies)
which also form the basis of a talk for YOW! 2021.
Overview
This web app presents a gallery of panels summarizing the health status of various systems accessible by a URL. Each observed system (which I call an “Observee”) can have one or more instances (an “Observee Instance”), and the aggregate recent health status of each instance is used to calculate the overall health of the system.
This should be very familiar to anyone who uses information radiators or “dashboards” for production health status. This app is not intended to replace observability and monitoring tools like Humio or Datadog — it’s a simple page intended to show on a large screen as one means of alerting humans to issues, without the need for any systems outside your network, so it works alongside those more sophisticated tools or provides a simple alternative in constrained environments.
As it’s expected to run on a large screen, no interaction is expected, although I’ve built in mechanisms to support drilling into the information to answer questions like “which instance?” and “why is it showing as unhealthy?”.
The code is available to view on Bitbucket.
LowJS frontend
For the rationale of why #LowJS, see “The alternative history” section in the introductory article.
I seriously explored HTMX for this app. It is an neat little library to apply dynamic HTML capabilities to any element without writing your own JavaScript. It’s a reasonable glimpse of what a future version of HTML could be if browsers baked in more powerful declarative capabilities of markup — in a similar way to how CSS has evolved to remove the need for imperative JavaScript animations. I didn’t use HTMX due to only very basic handling of SSE that I couldn’t work around for this app, but I expect this will be improved over time and it will be an excellent choice. For now, Hotwire remains at the top of my #LowJS list.
If you’re familiar with the original sample, you’ll see similarities in the pinger Turbo Frame and loadStream SSE.
Bootstrap
In another throwback to simpler times, I’ve used the Bootstrap toolkit which does what it says on the box and provides neat, simple, mostly CSS-driven components that I’ve used for layout, typography, and color.
I’m no good at UI but I found using Bootrap’s components very straightforward and I’m very pleased with the simplicity and the results.
Let’s look at the significant HTML files used to build the frontend.
html-head.html
In the interests of DRY, we have a fragment template for the content that will be common to all pages’ <head> elements.
It specifies the character set asUTF-8 and defines the responsive viewport attributes. For a little eye candy, it also sets the “theme color” to a suitable green, which (at least) Safari will use to color the title bar (if allowed by the user). Then it loads Bootstrap’s stylesheet and script, Hotwire’s Turbo script, and the application’s custom stylesheet.
In a SaaS web app we’d normally just request the JS and CSS from the preferred CDN of the library. In this case, though, our app might be running “behind the firewall” and perhaps constrained from accessing systems other than the ones it’s monitoring. So instead we get a copy of those files during the build process and include them in our static assets (more about this in part 4).
So, yes, #LowJS does have some JavaScript, but thankfully we don’t have to write much of it at all.
index.html
You’ll see some attributes from the th namespace various elements in the source. These are processed by Thymeleaf and don’t make it to the markup rendered by the browser. Alternately you can use data-th-... instead of th:... if you don’t want to specify the xmlns:th — we do this in HTML fragment files (as there’s no <html> preamble in which to declare an XML namespace).
After Thymeleaf replaces a dummy <link> tag with the common <head> fragment and we set the page title, we encounter a little JavaScript. We expect JavaScript to be enabled in the browser, but just in case it’s not everything will still work if the user can manually interact.
If JavaScript isn’t enabled, the user can click the link manually and navigate to the status page. You can use a textual browser or (where provided) set a “Disable JavaScript” option in the browser (e.g. Chrome, Safari) to try this out. When JavaScript is available, however, then the Turbo script will have loaded and the directives in the turbo-frame custom tags will be performed.
The turbo-frame will load a fragment of HTML from status that shows the gallery of Observees and their current status. When it is “clicked” by the JavaScript in the <head> element (see above) the fragment will be loaded in-place and the user won’t see any page loading or change in the browser address bar. This is the benefit of an SPA without all the many drawbacks.
index.js
This script’s primary job is to automatically click the anchor tag used lower down to automatically load the Turbo Frame content.
We also have logic to update a clock in the page header so users can correlate the current UTC time with the time shown in the status update for each system being monitored. As well as not being strictly needed, it’s actually also solvable with another SSE stream with the UTC time coming from the server.
While I believe by now there should be HTML-only capabilities for such basic stuff as “keep this <time> element updated with the current time, displayed using this format”, for me this is an acceptable volume of JavaScript to be burdened with.
status.js
This script’s only job is to connect the SSE to Turbo Streams. It’s referenced from both index.html and status.html. It would be really nice if there was a mechanism inHotwire to instead do this declaratively. Alas, there isn’t at this point.
status.html
Along with including the same fragment for the <head> element, we have a “meta refresh” tag which instructs the browser to automatically refresh the page every 3 seconds. You won’t see these much these days, but they were useful in the “old days” before AJAX for web apps very much like this one, where information was being constantly updated and the user didn’t want to have to refresh manually.
One of the ideas of progressive enhancement is that the application will still be usable (perhaps with reduced capability) if there’s no JavaScript available. For the web apps I write, I strive to have them work even in textual browsers like lynx and w3m. I also strive to meet accessibility guidelines. With a couple of tweaks to the classes (e.g. by using a toggle setting for extra contrast) this dashboard gets a good audit report.
Although lynx still doesn’t support “meta refresh”, w3m does. If you have w3m installed, use w3m -o color=1 -o meta_refresh=1 http://localhost:8080 to navigate to the home page. You can then navigate the Status link to /status and watch the status of the systems change every 3 seconds.
w3mIn the case of JavaScript being present, we don’t need or use the “meta refresh” (and remember — that part of status.html isn’t included in the Turbo Frames in index.html). Instead, the Turbo Stream is connected and then the gallery of Observees with their current status is constructed.
This list is rendered by a Thymeleaf HTML template on the server side, and the same template is used for the SSE data coming in from the Turbo Stream (see below). Any time an event is received on the Turbo Stream, the DOM is updated with the latest HTML fragment within that event.
This is in contrast to most frontends using AJAX which transform JSON (or XML) into DOM updates using JavaScript to deserialize the payload and process it into HTML. Instead, we just send the HTML we want to replace the existing HTML. The only JavaScript involved is the the Hotwire library used to match the correct DOM element and replace it with the event data.
The turbo-frame element in this page is what’s included in the turbo-frame referenced in index.html. It establishes the containing elements for the gallery, including the unordered list <ul> element which is used to iterate over the list of Observees and their health status.
This list of Observees is “static” on the frontend. It needs to be re-requested (e.g. by refresh) to obtain any changes. Now it’s time for the Turbo Stream to do its work and update the DOM dynamically.
observee-status.turbo-stream.html
Very similar to the markup in the Turbo Frame in the above status.html, this is the template used to iterate over each event in the SSE stream.
The Turbo Stream’s SSE is supplied by a reactive stream on the backend (more about this in part 2). Thymeleaf will process the template every time a new item is published on that reactive stream. Any time the status of a particular Observee is changed, the <li> list item for it is re-interpolated with the event on the reactive stream, the contents of which are from the observee-status-li.html. It is then sent as a new message on the SSE.
The Turbo Stream library will then use the content of each message on the SSE to update the target DOM element (in our case, the <li> of the particular Observee that has had a status update).
observee-status-li.html
This fragment paints each item in the “gallery” — whether it’s the initial “static” list of Observees and their health status at the time of request, or as a message event in the SSE stream.
Thymeleaf is interpolating a number of values from the ObserveeInfo view model which indicate the status and information about each Observee.
We have four health status possibilities. The various ways these are represented are:
The HealthStatus value is from the enum on the backend (more on this in part 4). The Status value is how we would describe the health in a word (“it’s ailing!”), while the Color is how we tend to describe the health as a traffic light color (“it’s amber!”).
The Alert and Color values are based on Bootcamp theme colors and the CSS color is from our custom stylesheet.
These words and color names are combined to provide the visual distinctions for the various heath status possibilities, catering for users with issues distinguishing color or text.
Along with the label we’ve given the Observee (which will be the name the system being checked is commonly known as), it’s location (a URL that can be clicked on to visit the system), current health status and the time at which that health status was calculate.
A badge with the number of instances being monitored for that Observee is also shown. Down the track, we’ll be able to hover over that badge to see the status of each instance, and drill-down to look at more detail.
As each updated item is rendered, the existing element that needs to be replaced is specified in the target attribute in the Turbo Stream element of observee-status.turbo-stream.html— this means that any time the status of a particular Observee is changed, just its section of the gallery is updated. Crisp and flicker-free!
But the API!
Some may think the lack of an XML or JSON data transfer means there’s no “API” for this app for remote consumers. The thing is, the browser is a remote consumer, it is using a RESTful API, and the HTML fragments in the SSE event stream are the data being transferred.
Look at what a curl http://localhost:8080/status.stream sees:
If you had a remote consumer that wasn’t a browser using Turbo Streams, that content model is still very structured and parsable. You’d ignore most of the CSS class specifiers, but some of them (like list-group-item-success or green) are of semantic value.
And of course you still have the option of having a view template on the server which seriealizes a ObserveeInfo into JSON and a method on the controller which specifies JSON as the media type.
This reactive, fast, responsive, auto-updating, minimal data transfer frontend is all achieved with just a few JavaScript expressions of our own for progressive enhancement, tied into a small non-SPA library to make the magic happen.
The next article will cover the reactive backend, and it’s use of Spring WebFlux, Kotlin Coroutines and Kotlin Flows.
