Feedbot [ii]: the rss announcer

This post builds upon the previous, introducing the second component of Feedbot: the so-called feed entry announcer. Similarly to previous instances of our series, we introduce the V patch, then we discuss implementation details.

• The V patch; and
• my seal.

Below is reproduced verbatim the documentation of the entry announcer, more exactly: the message queue; its manipulation; and the announcer loop.

IV. The message queue

Once new feed entries are found by the feed checker, they are to be distributed to the recipient(s) via e.g. IRC messages. We decouple feed checking and announcements by introducing a specialized producer-consumer data structure, the message queue.

A message queue is a list of messages.

A message is a list of the form:

  (msg :to rcpt :sent sent
       :feed-id feed-id :feed-title feed-title :entry entry)

where: msg is the symbol MSG; rcpt is a string denoting a recipient; sent is a boolean that, when set to true, marks the message as sent; feed-id is a string denoting the feed id of the entry's associated feed; feed-title is the title of the feed; and entry is a new entry.

Note: feed-title is an optimization which saves the announcer an extra feed db lookup when the message is sent. See below for more details.

V. Message queue manipulation

There are three fundamental operations on message queues: pushing new messages, retrieving messages and deleting sent messages.

As in the case of the feed db: a. low-level operations; b. used in conjunction with with-msg-queue; c. are used to implement the functionality described above. The implementation of with-msg-queue is reproduced below, along with the message queue methods:

(defmacro with-msg-queue ((queue) bot &body body)
  "Execute code within the thread-safe `msg-queue' scope of `bot'."
  (with-gensyms (queue-mutex)
    `(with-slots ((,queue msg-queue) (,queue-mutex queue-mutex))
         ,bot
       (with-mutex (,queue-mutex)
         ,@body))))

(defmethod feedbot-process-msg-queue ((bot feedbot) func)
  "Process messages in the msg queue `bot' accoding to `func'.

Returns the updated message queue."
  (with-msg-queue (msg-queue) bot
    (loop for msg in msg-queue do
         (funcall func msg))
    msg-queue))

(defmethod feedbot-pushnew-to-msg-queue ((bot feedbot) &rest msgs)
  "Push new messages to the msg queue of `bot'.

Returns the updated message queue."
  (with-msg-queue (msg-queue) bot
    (loop for msg in msgs do
         (push-msg-to-queue! msg-queue msg))
    msg-queue))

(defmethod feedbot-delete-sent-msgs ((bot feedbot))
  "Delete sent messages from the msg queue of `bot'.

Returns the updated message queue."
  (with-msg-queue (msg-queue) bot
    (setf msg-queue (delete-msgs-from-queue-if! msg-queue
                                                #'get-msg-sent!))
    msg-queue))

VI. The entry announcer

The entry announcer periodically scans the message queue for new (unsent) messages from the feed checker and announces the associated entries, i.e. sends them to the recipient.

Additionally, if the number of sent messages is over a certain threshold (see *max-sent-msgs*), then they are garbage collected. To eliminate this check, set *max-sent-msgs* to NIL.

To test feedbot feed checker and announcer functionality, run e.g.:

> (defvar *feedbot*
     (make-instance 'feedbot:feedbot))
> (defvar *ttp* "http://thetarpit.org/rss.xml")
> (feedbot:feedbot-get-or-create-feed *feedbot* *ttp*)
> (feedbot:feedbot-add-rcpts *feedbot* *ttp* "spyked")
> (feedbot:feedbot-start-checker-thread *feedbot*)
> (feedbot:feedbot-start-announcer-thread *feedbot*)

The next and final part of this series will implement the IRC-facing side of Feedbot.