Common Lisp WWWism summary

⁰⁰⁰ September 13, 2019 -- (tmsr tech)

This article is part of a series on Common Lisp WWWism, more specifically a summary of what's been discussed so far and some conclusions. Maybe there isn't much to be said about these "summary" posts, but at least in this particular case I believe a synthetic view of what's been said oughta clear up the reasoning behind my decisions on the subject.

Let's begin by outright enumerating everything I've written so far on CL WWWistics:

There's also tons of lines in the logs about all this, you're more than welcome to dig them out and let them enlighten you on the matter.

Notice that there's about a couple of items discussed in all these blog posts: the HTML generator known as CL-WHO and the Hunchentoot web server. Together they form something very similar to Apache and PHP, although CL-WHO isn't a PHP, while Hunchentoot isn't exactly an Apache.

More precisely, Hunchentoot is a sort of proto-Apache along with a sort of mod_lisp, in that it provides the basic tools one needs to make a dynamic web site: the TCP stuff, "virtual host" configuration, per-URL dispatching¹; and some of what PHP itself implements, namely GET/POST handling, plugged into a language for general-purpose processing and automation, i.e. Common Lisp. Other than that, Hunchentoot says absolutely nothing about what or how content is generated: those basic tools are used to serve stuff, not specify the stuff itself; it's the user's business what (Common Lisp) primitives he or she uses to generate their web pages.

Now, this Hunchentoot, while separating this from that so neatly and generally being a pleasure to work with, comes with some very big problems. Even if we left alone the usual CL-on-Unix retardation and the HTTP-in-CL string munging crap, its worker model is frail. Fast, yes, but the thing hasn't been tested on heavy workloads, so I for one don't know what happens when ten thousand connections come in simultaneously; the amusing part being that yes, perhaps I should have tested this before pointing fingers at the thing, but as it is, I have no heavy workloads to test it on!

Other than that, there is no "Hunchentoot versus Apache" contest to speak of, no matter how you choose to compare the two. So if I set out to write a Common Lisp web server from scratch, given my experience so far, I'm not sure I'd do a much better job than Hunchentoot (let alone Apache), at least not unless I spent a year or so (though I'm afraid this could easily turn into "a decade or so") reviewing WWW servers in more depth. And so, the question comes, is this really worth doing under the circumstances?

As for CL-WHO: it's an honest tool for HTML generation, light-years ahead of PHP's templating, since it's the exactly proper abstraction level to work at². Maybe this doesn't mean much, but I'm quite proud of the thing, since it's helped me to write a basic blog scaffolding (this blog) and a code referencer without much effort, and all this stuff just works the way I'd intended it to. So given the emerging Republican paradigm of HTML generation via IRC bots, I'm supposing there's still a place for it somewhere near the problems I was initially trying to solve.

Then there's the CL-on-PC thread that I'd rather not get into now: the hardware isn't there, standard Common Lisp works poorly with Unix abstractions, and so on and so forth. Maybe there's a Lisp out there that'd solve this?

Then there's the political aspect. Consider: at some point in the '90s, some dudes wrote a web server and some other dudes pulled a HTML templating language out of their asses -- meaning, they never even specified the thing, because why the fuck, right? Either way, I guess many man-centuries were put into making these two pieces work properly, and so this is how today we have the A and P in LAMP. As for the other side of the comparison: in 2000-something, some dudes set out to write an Apache -- in Lisp. Which was great, except nobody, save for its authors maybe, used it; so if the thing implodes under its own weight, I would also have to foot the bill for these guys not being able to steal WWWism from those other guys. So it occurs to me that maybe Hunchentoot's name was predestined?

So as far as I'm concerned, I'm to preserve as much of this Lisp WWW (and other) stuff as can be preserved, and that's about it. I'll publish the current Tar Pit's innards and I'll do my best to help anyone who wants to bring Common Lisp WWWisms up. What I won't do is sink more time into perpetuating current WWW shistacks, I'd rather burn it all and start from scratch if it comes to it. Or who knows, maybe at some point Hunchentoot's strengths, or maybe Common Lisp's, will find their place somewhere, but until then, I'll have everything published and shelved, and that's that.

So then, reviewing my plan: I'll spend the rest of September getting out all the yet-unpublished V patches; then I'll work out a concrete plan for migrating this blog to MP-WP, since I suspect that won't be trivial³; then I'll get an instance of Stan's logger running; so more generally, I'll be using existing Republican infrastructure instead of trying to reinvent the wheel; and finally, I'll reprioritize the huge to-do list (plus what's landed on this table meanwhile) and start producing other things, be they this language, that other Unix or whatever.

Similarly to what the mod_alias, mod_rewrite and all those others do.↩
It seems that the CL-WHO illustration piece is still a tough one to digest; mind you, I'm the only one to blame for my poor presentation skills.

Now, consider that any use of PHP we know of actually involves the interaction between two languages: the "preprocessor"/templating/whatever-you-wanna-call-it language, i.e. PHP, and the "preprocessee" language, that is, HTML. This brings us to the practical problem that the interaction between these two languages goes both ways. Take the following minimal example:
```
<body>
Some text, etc.
<?php
    do_some_processing();
?>
Some more text, etc.
</body>
```
The separation between HTML and PHP is clearly delineated by the <? ... ?> tags, isn't it? Well yes, but most of the times you're actually going to do stuff along the lines of:
```
<body>
Some text, etc.
<?php
    do_some_processing();
    echo '<a href="' . $url . '">' . $description . '</a>';
?>
</body>
```
meaning that you will be forced to "call into" HTML from the templating language whether you want it or not, which interleaving CL-WHO does explicitly, correctly and at little expense -- learning how to use it, basically. Meanwhile, on the PHPistic side of things, just read the simple snippet above... how's that for pain?↩
Leaving aside moving the content from one place to the other and other similar labor: moving to MP-WP will change URLs, thus breaking the existing ones. I'm not even sure that I want to fix this -- I'd rather use the standard Trilema URL schema -- which I expect will upset some people.↩