I did it my way. Again.

I’ve redone it. Over three years ago I wrote about a site transition from Ruby on Rails to Sinatra. It ran great, going from something with a footprint that was too large for my tiny server to run down to something that took only 10% of the system’s memory and only used the CPU when there were web requests or calendar updates. But you know me. I’m Impossibly Stupid, so I thought that even that was not small enough to satisfy my quest for minimalism.

Examination of the real-world use of the new calendaRSS turned up something interesting. With the transition to message queues to do the brunt of the API work behind the scenes, it turns out that even the microservices layer that Sinatra was providing was more bloat than was necessary. For the kind of site I’m running, there’s absolutely no need to leave a Rack environment constantly running to process the requests that generate dynamic web pages. Taking a step back, I saw the possibility for a RAGtag solution.

Common Gateway Interface (CGI)

CGI has been the OG way to get a web server to process an HTML form for nearly 30 years. There is no additional component that is left running to process requests, which makes it the perfect RAGtag replacement for my RAM-thrifty site. There is some small overhead in having to start a new process with every endpoint request, but the actual requests that need to do that (i.e., write data rather than read it) are few and far in between for my calendaRSS API.

With that approach in mind, I started to write a simple script based on my existing Ruby code to:

1. Receive the form request
2. Do some basic validation to make sure it isn’t abuse (e.g., check that the supplied endpoint is supported)
3. Publish the important info to a message queue (the same MQTT interface that the Sinatra microservice used)
4. Redirect the browser to the response page

That’s it. It’s less that 100 physical lines of code. It only includes external libraries for CGI, MQTT, and Digest. Why Digest? Because I’m Impossibly Stupid. If you look in that list above, you’ll see that one thing the script doesn’t do is wait for a response from the message queue. No matter the request, the response time is less than a tenth of a second. It’s completely fire and forget! So in order to coordinate the request with a response, it gets a unique identifier that they can both figure out by simply fingerprinting the form parameters (via Digest, of course).

So what we have now is an external request that has been decoupled and put on an internal waiting list. Nothing of substance has changed! That means we need to refactor the old code that generated a response in Rails and Sinatra to somehow update the pages of the web site. How to do that, how to do . . .

Continuous Integration/Continuous Deployment (CI/CD)

Some software developers and their managers seem to think CI/CD is some relatively new practice, but it’s not. It was essentially the way every good project I ever worked on was organized decades ago, but now automation, faster hardware, and the Internet have simply compressed the planning timeline down to “continuous”. Kind of the same way “agile” was nothing all that new, but people with no sense of history reinvented the wheel and rebranded it and suddenly thought they were hot shit. So it goes with CI/CD.

Anyway, for those not in the know, a CI/CD pipeline is a way of coordinating the work of multiple individuals such that the pieces they’ve all made can be assembled together to build a final product that’s shipped out the door. It’s sort of like the digital version of an automobile assembly line, with everything moving in one direction towards a common goal. The way most web apps work isn’t like that, though, but rather more like an auto repair shop, with specific things that need to be done to a specific vehicle which then needs to be returned to the same specific person that brought it in.

When an HTTP request comes in to a modern web app, the dynamic content is rendered as it is needed and the response is sent back the other direction. And that makes sense if the content is constantly changing or needs to be heavily modified based on who the user is. But it is often the case that you truly want everyone in the web’s wide world to see the same thing. That’s called static content, and I decided to see what it would be like to make calendaRSS work that way.

This is actually not a new idea for web sites. Back in the early days, before Web 2.0, content was largely published as static files like that. Many were even written by humans, typed by hand into a text editor! Technology has changed a lot since then, but the simplicity of static sites has a lot of appeal, and so many people have been starting to use static site generators more again when they don’t want to fuss with all the whiz-bang features of Web 2.0.

One generator of particular note is Jekyll. Yes, it was also written in Ruby just like all my old code for calendaRSS, but it’s not interesting for that reason. It’s interesting because it forms the basis for GitHub Pages, and that served as a demonstration to me that using a CI/CD pipeline for web publishing wasn’t completely nuts.

So I updated my code to move the rendering of web pages, still largely the .erb templates I created for the Rails version of the site, into the worker services that monitor the message queue. Those new/updated files (identified by that Digest fingerprint) are then put into the public web space, where they can be fetched by any interested party. And because I am now fully using a CI/CD pipeline, I was finally easily able to make use of an entirely separate computer to do all that work. My tiny VPS is now leaner and meaner than ever before!