A few changes to Feedparse's HTTP requester (prelude to Feedbot)

This post introduces s-xml-feedparse-http-fixes:

• the V patch; and
• my seal.

So what do these changes slash "HTTP fixes" consist of, more precisely? Two pieces of code are being repaired -- one of the repairs being required by Feedbot, while the other is more or less an opportunity for making the code a tad clea[nr]er. The latter (simpler) one goes as follows:

+(defvar *http-request-timeout* 60)

...

-  (parse-feed-string (http-request-with-timeout url 60)))
+  (parse-feed-string (http-request-with-timeout url *http-request-timeout*)))

In other words, we're calling http-request-with-timeout with an operator-configurable timeout setting -- as it should have been in the first place, but let's not go into too many details, lest I end up burning myself and applying ice on the wound.

The former (more important) change is along the lines of:

+(defun http-request-thread (url timeout)
+  (handler-case
+      (drakma:http-request url :connection-timeout timeout)
+    ;; Note: this will have to be revisited once we have a sane view of
+    ;; all the conditions returned by drakma.
+    (t (c)
+      (return-from http-request-thread (values nil c)))))

... and a bit later:

   (let ((req-thread (sb-thread:make-thread
-            #'(lambda (url timeout)
-                (drakma:http-request url :connection-timeout timeout))
+             #'http-request-thread

This means that the thread we're creating now runs the code in http-request-thread instead of an anonymous function; but more importantly, that the call to Drakma's http-request is now guarded by a handler-case that, long story short, acts as a wildcard exception catcher.

Feedbot benefits from this change by making sure that all HTTP requests -- that, as we know from above, occur on a separate thread -- are deterministic, i.e. the CL run-time doesn't end up throwing exceptions at us if somehow a request times out or who knows what other condition out of the operator's control. I know, this isn't the most fortunate of implementation choices, but well... if I do the right thing(tm) and fix Drakma at this very moment rather than later, and then that other thing, and then the other one, then I might get to publish Feedbot in a decade or so. Let's say that the debt is manageable -- for now, but not for long.

Up next: Feedbot, part the first.