11.7 Gnus Reference Guide

It is my hope that other people will figure out smart stuff that Gnus can do, and that other people will write those smart things as well. To facilitate that I thought it would be a good idea to describe the inner workings of Gnus. And some of the not-so-inner workings, while I’m at it.

You can never expect the internals of a program not to change, but I will be defining (in some details) the interface between Gnus and its back ends (this is written in stone), the format of the score files (ditto), data structures (some are less likely to change than others) and general methods of operation.

11.7.1 Gnus Utility Functions

When writing small functions to be run from hooks (and stuff), it’s vital to have access to the Gnus internal functions and variables. Below is a list of the most common ones.

gnus-newsgroup-name

This variable holds the name of the current newsgroup.

gnus-find-method-for-group

A function that returns the select method for group.

gnus-group-real-name

Takes a full (prefixed) Gnus group name, and returns the unprefixed name.

gnus-group-prefixed-name

Takes an unprefixed group name and a select method, and returns the full (prefixed) Gnus group name.

gnus-get-info

Returns the group info list for group (see section Group Info).

gnus-group-unread

The number of unread articles in group, or t if that is unknown.

gnus-active

The active entry (i.e., a cons cell containing the lowest and highest article numbers) for group.

gnus-set-active

Set the active entry for group.

gnus-add-current-to-buffer-list

Adds the current buffer to the list of buffers to be killed on Gnus exit.

gnus-continuum-version

Takes a Gnus version string as a parameter and returns a floating point number. Earlier versions will always get a lower number than later versions.

gnus-group-read-only-p

Says whether group is read-only or not.

gnus-news-group-p

Says whether group came from a news back end.

gnus-ephemeral-group-p

Says whether group is ephemeral or not.

gnus-server-to-method

Returns the select method corresponding to server.

gnus-server-equal

Says whether two virtual servers are essentially equal. For instance, two virtual servers may have server parameters in different order, but this function will consider them equal.

gnus-group-native-p

Says whether group is native or not.

gnus-group-secondary-p

Says whether group is secondary or not.

gnus-group-foreign-p

Says whether group is foreign or not.

gnus-group-find-parameter

Returns the parameter list of group (see section Group Parameters). If given a second parameter, returns the value of that parameter for group.

gnus-group-set-parameter

Takes three parameters; group, parameter and value.

gnus-narrow-to-body

Narrows the current buffer to the body of the article.

gnus-check-backend-function

Takes two parameters, function and group. If the back end group comes from supports function, return non-nil.

(gnus-check-backend-function "request-scan" "nnml:misc") ⇒ t

gnus-read-method

Prompts the user for a select method.

11.7.2 Back End Interface

Gnus doesn’t know anything about NNTP, spools, mail or virtual groups. It only knows how to talk to virtual servers. A virtual server is a back end and some back end variables. As examples of the first, we have nntp, nnspool and nnmbox. As examples of the latter we have nntp-port-number and nnmbox-directory.

When Gnus asks for information from a back end—say nntp—on something, it will normally include a virtual server name in the function parameters. (If not, the back end should use the “current” virtual server.) For instance, nntp-request-list takes a virtual server as its only (optional) parameter. If this virtual server hasn’t been opened, the function should fail.

Note that a virtual server name has no relation to some physical server name. Take this example:

(nntp "odd-one"
      (nntp-address "ifi.uio.no")
      (nntp-port-number 4324))

Here the virtual server name is ‘odd-one’ while the name of the physical server is ‘ifi.uio.no’.

The back ends should be able to switch between several virtual servers. The standard back ends implement this by keeping an alist of virtual server environments that they pull down/push up when needed.

There are two groups of interface functions: required functions, which must be present, and optional functions, which Gnus will always check for presence before attempting to call ’em.

All these functions are expected to return data in the buffer nntp-server-buffer (‘ *nntpd*’), which is somewhat unfortunately named, but we’ll have to live with it. When I talk about resulting data, I always refer to the data in that buffer. When I talk about return value, I talk about the function value returned by the function call. Functions that fail should return nil as the return value.

Some back ends could be said to be server-forming back ends, and some might be said not to be. The latter are back ends that generally only operate on one group at a time, and have no concept of “server”; they have a group, and they deliver info on that group and nothing more.

Gnus identifies each message by way of group name and article number. A few remarks about these article numbers might be useful. First of all, the numbers are positive integers. Secondly, it is normally not possible for later articles to “re-use” older article numbers without confusing Gnus. That is, if a group has ever contained a message numbered 42, then no other message may get that number, or Gnus will get mightily confused.(4) Third, article numbers must be assigned in order of arrival in the group; this is not necessarily the same as the date of the message.

The previous paragraph already mentions all the “hard” restrictions that article numbers must fulfill. But it seems that it might be useful to assign consecutive article numbers, for Gnus gets quite confused if there are holes in the article numbering sequence. However, due to the “no-reuse” restriction, holes cannot be avoided altogether. It’s also useful for the article numbers to start at 1 to avoid running out of numbers as long as possible.

Note that by convention, back ends are named nnsomething, but Gnus also comes with some nnnotbackends, such as ‘nnheader.el’, ‘nnmail.el’ and ‘nnoo.el’.

In the examples and definitions I will refer to the imaginary back end nnchoke.

11.7.2.1 Required Back End Functions

(nnchoke-retrieve-headers ARTICLES &optional GROUP SERVER FETCH-OLD)

articles is either a range of article numbers or a list of Message-IDs. Current back ends do not fully support either—only sequences (lists) of article numbers, and most back ends do not support retrieval of Message-IDs. But they should try for both.

The result data should either be HEADs or NOV lines, and the result value should either be headers or nov to reflect this. This might later be expanded to various, which will be a mixture of HEADs and NOV lines, but this is currently not supported by Gnus.

If fetch-old is non-nil it says to try fetching “extra headers”, in some meaning of the word. This is generally done by fetching (at most) fetch-old extra headers less than the smallest article number in articles, and filling the gaps as well. The presence of this parameter can be ignored if the back end finds it cumbersome to follow the request. If this is non-nil and not a number, do maximum fetches.

Here’s an example HEAD:

221 1056 Article retrieved. Path: ifi.uio.no!sturles From: sturles@ifi.uio.no (Sturle Sunde) Newsgroups: ifi.discussion Subject: Re: Something very droll Date: 27 Oct 1994 14:02:57 +0100 Organization: Dept. of Informatics, University of Oslo, Norway Lines: 26 Message-ID: <38o8e1$a0o@holmenkollen.ifi.uio.no> References: <38jdmq$4qu@visbur.ifi.uio.no> NNTP-Posting-Host: holmenkollen.ifi.uio.no .

So a headers return value would imply that there’s a number of these in the data buffer.

Here’s a BNF definition of such a buffer:

headers = *head head = error / valid-head error-message = [ "4" / "5" ] 2number " " <error message> eol valid-head = valid-message *header "." eol valid-message = "221 " <number> " Article retrieved." eol header = <text> eol

(The version of BNF used here is the one used in RFC822.)

If the return value is nov, the data buffer should contain network overview database lines. These are basically fields separated by tabs.

nov-buffer = *nov-line nov-line = field 7*8[ <TAB> field ] eol field = <text except TAB>

For a closer look at what should be in those fields, see section Headers.

(nnchoke-open-server SERVER &optional DEFINITIONS)

server is here the virtual server name. definitions is a list of (VARIABLE VALUE) pairs that define this virtual server.

If the server can’t be opened, no error should be signaled. The back end may then choose to refuse further attempts at connecting to this server. In fact, it should do so.

If the server is opened already, this function should return a non-nil value. There should be no data returned.

(nnchoke-close-server &optional SERVER)

Close connection to server and free all resources connected to it. Return nil if the server couldn’t be closed for some reason.

There should be no data returned.

(nnchoke-request-close)

Close connection to all servers and free all resources that the back end have reserved. All buffers that have been created by that back end should be killed. (Not the nntp-server-buffer, though.) This function is generally only called when Gnus is shutting down.

There should be no data returned.

(nnchoke-server-opened &optional SERVER)

If server is the current virtual server, and the connection to the physical server is alive, then this function should return a non-nil value. This function should under no circumstances attempt to reconnect to a server we have lost connection to.

There should be no data returned.

(nnchoke-status-message &optional SERVER)

This function should return the last error message from server.

There should be no data returned.

(nnchoke-request-article ARTICLE &optional GROUP SERVER TO-BUFFER)

The result data from this function should be the article specified by article. This might either be a Message-ID or a number. It is optional whether to implement retrieval by Message-ID, but it would be nice if that were possible.

If to-buffer is non-nil, the result data should be returned in this buffer instead of the normal data buffer. This is to make it possible to avoid copying large amounts of data from one buffer to another, while Gnus mainly requests articles to be inserted directly into its article buffer.

If it is at all possible, this function should return a cons cell where the car is the group name the article was fetched from, and the cdr is the article number. This will enable Gnus to find out what the real group and article numbers are when fetching articles by Message-ID. If this isn’t possible, t should be returned on successful article retrieval.

(nnchoke-request-group GROUP &optional SERVER FAST INFO)

Get data on group. This function also has the side effect of making group the current group.

If fast, don’t bother to return useful data, just make group the current group.

If info, it allows the backend to update the group info structure.

Here’s an example of some result data and a definition of the same:

211 56 1000 1059 ifi.discussion

The first number is the status, which should be 211. Next is the total number of articles in the group, the lowest article number, the highest article number, and finally the group name. Note that the total number of articles may be less than one might think while just considering the highest and lowest article numbers, but some articles may have been canceled. Gnus just discards the total-number, so whether one should take the bother to generate it properly (if that is a problem) is left as an exercise to the reader. If the group contains no articles, the lowest article number should be reported as 1 and the highest as 0.

group-status = [ error / info ] eol error = [ "4" / "5" ] 2<number> " " <Error message> info = "211 " 3* [ <number> " " ] <string>

(nnchoke-close-group GROUP &optional SERVER)

Close group and free any resources connected to it. This will be a no-op on most back ends.

There should be no data returned.

(nnchoke-request-list &optional SERVER)

Return a list of all groups available on server. And that means all.

Here’s an example from a server that only carries two groups:

ifi.test 0000002200 0000002000 y ifi.discussion 3324 3300 n

On each line we have a group name, then the highest article number in that group, the lowest article number, and finally a flag. If the group contains no articles, the lowest article number should be reported as 1 and the highest as 0.

active-file = *active-line active-line = name " " <number> " " <number> " " flags eol name = <string> flags = "n" / "y" / "m" / "x" / "j" / "=" name

The flag says whether the group is read-only (‘n’), is moderated (‘m’), is dead (‘x’), is aliased to some other group (‘=other-group’) or none of the above (‘y’).

(nnchoke-request-post &optional SERVER)

This function should post the current buffer. It might return whether the posting was successful or not, but that’s not required. If, for instance, the posting is done asynchronously, it has generally not been completed by the time this function concludes. In that case, this function should set up some kind of sentinel to beep the user loud and clear if the posting could not be completed.

There should be no result data from this function.

11.7.2.2 Optional Back End Functions

(nnchoke-retrieve-groups GROUPS &optional SERVER)

groups is a list of groups, and this function should request data on all those groups. How it does it is of no concern to Gnus, but it should attempt to do this in a speedy fashion.

The return value of this function can be either active or group, which says what the format of the result data is. The former is in the same format as the data from nnchoke-request-list, while the latter is a buffer full of lines in the same format as nnchoke-request-group gives.

group-buffer = *active-line / *group-status

(nnchoke-request-update-info GROUP INFO &optional SERVER)

A Gnus group info (see section Group Info) is handed to the back end for alterations. This comes in handy if the back end really carries all the information (as is the case with virtual and imap groups). This function should destructively alter the info to suit its needs, and should return a non-nil value (exceptionally, nntp-request-update-info always returns nil not to waste the network resources).

There should be no result data from this function.

(nnchoke-request-type GROUP &optional ARTICLE)

When the user issues commands for “sending news” (F in the summary buffer, for instance), Gnus has to know whether the article the user is following up on is news or mail. This function should return news if article in group is news, mail if it is mail and unknown if the type can’t be decided. (The article parameter is necessary in nnvirtual groups which might very well combine mail groups and news groups.) Both group and article may be nil.

There should be no result data from this function.

(nnchoke-request-set-mark GROUP ACTION &optional SERVER)

Set/remove/add marks on articles. Normally Gnus handles the article marks (such as read, ticked, expired etc.) internally, and store them in ‘~/.newsrc.eld’. Some back ends (such as IMAP) however carry all information about the articles on the server, so Gnus need to propagate the mark information to the server.

action is a list of mark setting requests, having this format:

(RANGE ACTION MARK)

range is a range of articles you wish to update marks on. action is add or del, used to add marks or remove marks (preserving all marks not mentioned). mark is a list of marks; where each mark is a symbol. Currently used marks are read, tick, reply, expire, killed, dormant, save, download, unsend, and forward, but your back end should, if possible, not limit itself to these.

Given contradictory actions, the last action in the list should be the effective one. That is, if your action contains a request to add the tick mark on article 1 and, later in the list, a request to remove the mark on the same article, the mark should in fact be removed.

An example action list:

(((5 12 30) 'del '(tick)) ((10 . 90) 'add '(read expire)) ((92 94) 'del '(read)))

The function should return a range of articles it wasn’t able to set the mark on (currently not used for anything).

There should be no result data from this function.

(nnchoke-request-update-mark GROUP ARTICLE MARK)

If the user tries to set a mark that the back end doesn’t like, this function may change the mark. Gnus will use whatever this function returns as the mark for article instead of the original mark. If the back end doesn’t care, it must return the original mark, and not nil or any other type of garbage.

The only use for this I can see is what nnvirtual does with it—if a component group is auto-expirable, marking an article as read in the virtual group should result in the article being marked as expirable.

There should be no result data from this function.

(nnchoke-request-scan &optional GROUP SERVER)

This function may be called at any time (by Gnus or anything else) to request that the back end check for incoming articles, in one way or another. A mail back end will typically read the spool file or query the POP server when this function is invoked. The group doesn’t have to be heeded—if the back end decides that it is too much work just scanning for a single group, it may do a total scan of all groups. It would be nice, however, to keep things local if that’s practical.

There should be no result data from this function.

(nnchoke-request-group-description GROUP &optional SERVER)

The result data from this function should be a description of group.

description-line = name <TAB> description eol name = <string> description = <text>

(nnchoke-request-list-newsgroups &optional SERVER)

The result data from this function should be the description of all groups available on the server.

description-buffer = *description-line

(nnchoke-request-newgroups DATE &optional SERVER)

The result data from this function should be all groups that were created after ‘date’, which is in normal human-readable date format (i.e., the date format used in mail and news headers, and returned by the function message-make-date by default). The data should be in the active buffer format.

It is okay for this function to return “too many” groups; some back ends might find it cheaper to return the full list of groups, rather than just the new groups. But don’t do this for back ends with many groups. Normally, if the user creates the groups herself, there won’t be too many groups, so nnml and the like are probably safe. But for back ends like nntp, where the groups have been created by the server, it is quite likely that there can be many groups.

(nnchoke-request-create-group GROUP &optional SERVER)

This function should create an empty group with name group.

There should be no return data.

(nnchoke-request-expire-articles ARTICLES &optional GROUP SERVER FORCE)

This function should run the expiry process on all articles in the articles range (which is currently a simple list of article numbers.) It is left up to the back end to decide how old articles should be before they are removed by this function. If force is non-nil, all articles should be deleted, no matter how new they are.

This function should return a list of articles that it did not/was not able to delete.

There should be no result data returned.

(nnchoke-request-move-article ARTICLE GROUP SERVER ACCEPT-FORM &optional LAST)

This function should move article (which is a number) from group by calling accept-form.

This function should ready the article in question for moving by removing any header lines it has added to the article, and generally should “tidy up” the article. Then it should eval accept-form in the buffer where the “tidy” article is. This will do the actual copying. If this eval returns a non-nil value, the article should be removed.

If last is nil, that means that there is a high likelihood that there will be more requests issued shortly, so that allows some optimizations.

The function should return a cons where the car is the group name and the cdr is the article number that the article was entered as.

There should be no data returned.

(nnchoke-request-accept-article GROUP &optional SERVER LAST)

This function takes the current buffer and inserts it into group. If last in nil, that means that there will be more calls to this function in short order.

The function should return a cons where the car is the group name and the cdr is the article number that the article was entered as.

The group should exist before the back end is asked to accept the article for that group.

There should be no data returned.

(nnchoke-request-replace-article ARTICLE GROUP BUFFER)

This function should remove article (which is a number) from group and insert buffer there instead.

There should be no data returned.

(nnchoke-request-delete-group GROUP FORCE &optional SERVER)

This function should delete group. If force, it should really delete all the articles in the group, and then delete the group itself. (If there is such a thing as “the group itself”.)

There should be no data returned.

(nnchoke-request-rename-group GROUP NEW-NAME &optional SERVER)

This function should rename group into new-name. All articles in group should move to new-name.

There should be no data returned.

11.7.2.3 Error Messaging

The back ends should use the function nnheader-report to report error conditions—they should not raise errors when they aren’t able to perform a request. The first argument to this function is the back end symbol, and the rest are interpreted as arguments to format if there are multiple of them, or just a string if there is one of them. This function must always returns nil.

(nnheader-report 'nnchoke "You did something totally bogus")

(nnheader-report 'nnchoke "Could not request group %s" group)

Gnus, in turn, will call nnheader-get-report when it gets a nil back from a server, and this function returns the most recently reported message for the back end in question. This function takes one argument—the server symbol.

Internally, these functions access back-end-status-string, so the nnchoke back end will have its error message stored in nnchoke-status-string.

11.7.2.4 Writing New Back Ends

Many back ends are quite similar. nnml is just like nnspool, but it allows you to edit the articles on the server. nnmh is just like nnml, but it doesn’t use an active file, and it doesn’t maintain overview databases. nndir is just like nnml, but it has no concept of “groups”, and it doesn’t allow editing articles.

It would make sense if it were possible to “inherit” functions from back ends when writing new back ends. And, indeed, you can do that if you want to. (You don’t have to if you don’t want to, of course.)

All the back ends declare their public variables and functions by using a package called nnoo.

To inherit functions from other back ends (and allow other back ends to inherit functions from the current back end), you should use the following macros:

nnoo-declare

This macro declares the first parameter to be a child of the subsequent parameters. For instance:

(nnoo-declare nndir nnml nnmh)

nndir has declared here that it intends to inherit functions from both nnml and nnmh.

defvoo

This macro is equivalent to defvar, but registers the variable as a public server variable. Most state-oriented variables should be declared with defvoo instead of defvar.

In addition to the normal defvar parameters, it takes a list of variables in the parent back ends to map the variable to when executing a function in those back ends.

(defvoo nndir-directory nil "Where nndir will look for groups." nnml-current-directory nnmh-current-directory)

This means that nnml-current-directory will be set to nndir-directory when an nnml function is called on behalf of nndir. (The same with nnmh.)

nnoo-define-basics

This macro defines some common functions that almost all back ends should have.

(nnoo-define-basics nndir)

deffoo

This macro is just like defun and takes the same parameters. In addition to doing the normal defun things, it registers the function as being public so that other back ends can inherit it.

nnoo-map-functions

This macro allows mapping of functions from the current back end to functions from the parent back ends.

(nnoo-map-functions nndir (nnml-retrieve-headers 0 nndir-current-group 0 0) (nnmh-request-article 0 nndir-current-group 0 0))

This means that when nndir-retrieve-headers is called, the first, third, and fourth parameters will be passed on to nnml-retrieve-headers, while the second parameter is set to the value of nndir-current-group.

nnoo-import

This macro allows importing functions from back ends. It should be the last thing in the source file, since it will only define functions that haven’t already been defined.

(nnoo-import nndir (nnmh nnmh-request-list nnmh-request-newgroups) (nnml))

This means that calls to nndir-request-list should just be passed on to nnmh-request-list, while all public functions from nnml that haven’t been defined in nndir yet should be defined now.

Below is a slightly shortened version of the nndir back end.

;;; nndir.el --- single directory newsgroup access for Gnus
;; Copyright (C) 1995,1996 Free Software Foundation, Inc.

;;; Code:

(require 'nnheader)
(require 'nnmh)
(require 'nnml)
(require 'nnoo)
(eval-when-compile (require 'cl))

(nnoo-declare nndir
  nnml nnmh)

(defvoo nndir-directory nil
  "Where nndir will look for groups."
  nnml-current-directory nnmh-current-directory)

(defvoo nndir-nov-is-evil nil
  "*Non-nil means that nndir will never retrieve NOV headers."
  nnml-nov-is-evil)

(defvoo nndir-current-group ""
  nil
  nnml-current-group nnmh-current-group)
(defvoo nndir-top-directory nil nil nnml-directory nnmh-directory)
(defvoo nndir-get-new-mail nil nil nnml-get-new-mail nnmh-get-new-mail)

(defvoo nndir-status-string "" nil nnmh-status-string)
(defconst nndir-version "nndir 1.0")

;;; Interface functions.

(nnoo-define-basics nndir)

(deffoo nndir-open-server (server &optional defs)
  (setq nndir-directory
        (or (cadr (assq 'nndir-directory defs))
            server))
  (unless (assq 'nndir-directory defs)
    (push `(nndir-directory ,server) defs))
  (push `(nndir-current-group
          ,(file-name-nondirectory
            (directory-file-name nndir-directory)))
        defs)
  (push `(nndir-top-directory
          ,(file-name-directory (directory-file-name nndir-directory)))
        defs)
  (nnoo-change-server 'nndir server defs))

(nnoo-map-functions nndir
  (nnml-retrieve-headers 0 nndir-current-group 0 0)
  (nnmh-request-article 0 nndir-current-group 0 0)
  (nnmh-request-group nndir-current-group 0 0)
  (nnmh-close-group nndir-current-group 0))

(nnoo-import nndir
  (nnmh
   nnmh-status-message
   nnmh-request-list
   nnmh-request-newgroups))

(provide 'nndir)

11.7.2.5 Hooking New Back Ends Into Gnus

Having Gnus start using your new back end is rather easy—you just declare it with the gnus-declare-backend functions. This will enter the back end into the gnus-valid-select-methods variable.

gnus-declare-backend takes two parameters—the back end name and an arbitrary number of abilities.

Here’s an example:

(gnus-declare-backend "nnchoke" 'mail 'respool 'address)

The above line would then go in the ‘nnchoke.el’ file.

The abilities can be:

mail

This is a mailish back end—followups should (probably) go via mail.

post

This is a newsish back end—followups should (probably) go via news.

post-mail

This back end supports both mail and news.

none

This is neither a post nor mail back end—it’s something completely different.

respool

It supports respooling—or rather, it is able to modify its source articles and groups.

address

The name of the server should be in the virtual server name. This is true for almost all back ends.

prompt-address

The user should be prompted for an address when doing commands like B in the group buffer. This is true for back ends like nntp, but not nnmbox, for instance.

11.7.2.6 Mail-like Back Ends

One of the things that separate the mail back ends from the rest of the back ends is the heavy dependence by most of the mail back ends on common functions in ‘nnmail.el’. For instance, here’s the definition of nnml-request-scan:

(deffoo nnml-request-scan (&optional group server)
  (setq nnml-article-file-alist nil)
  (nnmail-get-new-mail 'nnml 'nnml-save-nov nnml-directory group))

It simply calls nnmail-get-new-mail with a few parameters, and nnmail takes care of all the moving and splitting of the mail.

This function takes four parameters.

method

This should be a symbol to designate which back end is responsible for the call.

exit-function

This function should be called after the splitting has been performed.

temp-directory

Where the temporary files should be stored.

group

This optional argument should be a group name if the splitting is to be performed for one group only.

nnmail-get-new-mail will call back-end-save-mail to save each article. back-end-active-number will be called to find the article number assigned to this article.

The function also uses the following variables: back-end-get-new-mail (to see whether to get new mail for this back end); and back-end-group-alist and back-end-active-file to generate the new active file. back-end-group-alist should be a group-active alist, like this:

(("a-group" (1 . 10))
 ("some-group" (34 . 39)))

11.7.3 Score File Syntax

Score files are meant to be easily parsable, but yet extremely malleable. It was decided that something that had the same read syntax as an Emacs Lisp list would fit that spec.

Here’s a typical score file:

(("summary"
  ("Windows 95" -10000 nil s)
  ("Gnus"))
 ("from"
  ("Lars" -1000))
 (mark -100))

BNF definition of a score file:

score-file      = "" / "(" *element ")"
element         = rule / atom
rule            = string-rule / number-rule / date-rule
string-rule     = "(" quote string-header quote space *string-match ")"
number-rule     = "(" quote number-header quote space *number-match ")"
date-rule       = "(" quote date-header quote space *date-match ")"
quote           = <ascii 34>
string-header   = "subject" / "from" / "references" / "message-id" /
                  "xref" / "body" / "head" / "all" / "followup"
number-header   = "lines" / "chars"
date-header     = "date"
string-match    = "(" quote <string> quote [ "" / [ space score [ "" /
                  space date [ "" / [ space string-match-t ] ] ] ] ] ")"
score           = "nil" / <integer>
date            = "nil" / <natural number>
string-match-t  = "nil" / "s" / "substring" / "S" / "Substring" /
                  "r" / "regex" / "R" / "Regex" /
                  "e" / "exact" / "E" / "Exact" /
                  "f" / "fuzzy" / "F" / "Fuzzy"
number-match    = "(" <integer> [ "" / [ space score [ "" /
                  space date [ "" / [ space number-match-t ] ] ] ] ] ")"
number-match-t  = "nil" / "=" / "<" / ">" / ">=" / "<="
date-match      = "(" quote <string> quote [ "" / [ space score [ "" /
                  space date [ "" / [ space date-match-t ] ] ] ] ")"
date-match-t    = "nil" / "at" / "before" / "after"
atom            = "(" [ required-atom / optional-atom ] ")"
required-atom   = mark / expunge / mark-and-expunge / files /
                  exclude-files / read-only / touched
optional-atom   = adapt / local / eval
mark            = "mark" space nil-or-number
nil-or-number   = "nil" / <integer>
expunge         = "expunge" space nil-or-number
mark-and-expunge = "mark-and-expunge" space nil-or-number
files           = "files" *[ space <string> ]
exclude-files   = "exclude-files" *[ space <string> ]
read-only       = "read-only" [ space "nil" / space "t" ]
adapt        = "adapt" [ space "ignore" / space "t" / space adapt-rule ]
adapt-rule      = "(" *[ <string> *[ "(" <string> <integer> ")" ] ")"
local           = "local" *[ space "(" <string> space <form> ")" ]
eval            = "eval" space <form>
space           = *[ " " / <TAB> / <NEWLINE> ]

Any unrecognized elements in a score file should be ignored, but not discarded.

As you can see, white space is needed, but the type and amount of white space is irrelevant. This means that formatting of the score file is left up to the programmer—if it’s simpler to just spew it all out on one looong line, then that’s ok.

The meaning of the various atoms are explained elsewhere in this manual (see section Score File Format).

11.7.4 Headers

Internally Gnus uses a format for storing article headers that corresponds to the NOV format in a mysterious fashion. One could almost suspect that the author looked at the NOV specification and just shamelessly stole the entire thing, and one would be right.

Header is a severely overloaded term. “Header” is used in RFC 1036 to talk about lines in the head of an article (e.g., From). It is used by many people as a synonym for “head”—“the header and the body”. (That should be avoided, in my opinion.) And Gnus uses a format internally that it calls “header”, which is what I’m talking about here. This is a 9-element vector, basically, with each header (ouch) having one slot.

These slots are, in order: number, subject, from, date, id, references, chars, lines, xref, and extra. There are macros for accessing and setting these slots—they all have predictable names beginning with mail-header- and mail-header-set-, respectively.

All these slots contain strings, except the extra slot, which contains an alist of header/value pairs (see section To From Newsgroups).

11.7.5 Ranges

GNUS introduced a concept that I found so useful that I’ve started using it a lot and have elaborated on it greatly.

The question is simple: If you have a large amount of objects that are identified by numbers (say, articles, to take a wild example) that you want to qualify as being “included”, a normal sequence isn’t very useful. (A 200,000 length sequence is a bit long-winded.)

The solution is as simple as the question: You just collapse the sequence.

(1 2 3 4 5 6 10 11 12)

is transformed into

((1 . 6) (10 . 12))

To avoid having those nasty ‘(13 . 13)’ elements to denote a lonesome object, a ‘13’ is a valid element:

((1 . 6) 7 (10 . 12))

This means that comparing two ranges to find out whether they are equal is slightly tricky:

((1 . 5) 7 8 (10 . 12))

and

((1 . 5) (7 . 8) (10 . 12))

are equal. In fact, any non-descending list is a range:

(1 2 3 4 5)

is a perfectly valid range, although a pretty long-winded one. This is also valid:

(1 . 5)

and is equal to the previous range.

Here’s a BNF definition of ranges. Of course, one must remember the semantic requirement that the numbers are non-descending. (Any number of repetition of the same number is allowed, but apt to disappear in range handling.)

range           = simple-range / normal-range
simple-range    = "(" number " . " number ")"
normal-range    = "(" start-contents ")"
contents        = "" / simple-range *[ " " contents ] /
                  number *[ " " contents ]

Gnus currently uses ranges to keep track of read articles and article marks. I plan on implementing a number of range operators in C if The Powers That Be are willing to let me. (I haven’t asked yet, because I need to do some more thinking on what operators I need to make life totally range-based without ever having to convert back to normal sequences.)

11.7.6 Group Info

Gnus stores all permanent info on groups in a group info list. This list is from three to six elements (or more) long and exhaustively describes the group.

Here are two example group infos; one is a very simple group while the second is a more complex one:

("no.group" 5 ((1 . 54324)))

("nnml:my.mail" 3 ((1 . 5) 9 (20 . 55))
                ((tick (15 . 19)) (replied 3 6 (19 . 3)))
                (nnml "")
                ((auto-expire . t) (to-address . "ding@gnus.org")))

The first element is the group name—as Gnus knows the group, anyway. The second element is the subscription level, which normally is a small integer. (It can also be the rank, which is a cons cell where the car is the level and the cdr is the score.) The third element is a list of ranges of read articles. The fourth element is a list of lists of article marks of various kinds. The fifth element is the select method (or virtual server, if you like). The sixth element is a list of group parameters, which is what this section is about.

Any of the last three elements may be missing if they are not required. In fact, the vast majority of groups will normally only have the first three elements, which saves quite a lot of cons cells.

Here’s a BNF definition of the group info format:

info          = "(" group space ralevel space read
                [ "" / [ space marks-list [ "" / [ space method [ "" /
                space parameters ] ] ] ] ] ")"
group         = quote <string> quote
ralevel       = rank / level
level         = <integer in the range of 1 to inf>
rank          = "(" level "." score ")"
score         = <integer in the range of 1 to inf>
read          = range
marks-lists   = nil / "(" *marks ")"
marks         = "(" <string> range ")"
method        = "(" <string> *elisp-forms ")"
parameters    = "(" *elisp-forms ")"

Actually that ‘marks’ rule is a fib. A ‘marks’ is a ‘<string>’ consed on to a ‘range’, but that’s a bitch to say in pseudo-BNF.

If you have a Gnus info and want to access the elements, Gnus offers a series of macros for getting/setting these elements.

gnus-info-group

gnus-info-set-group

Get/set the group name.

gnus-info-rank

gnus-info-set-rank

Get/set the group rank (see section Group Score).

gnus-info-level

gnus-info-set-level

Get/set the group level.

gnus-info-score

gnus-info-set-score

Get/set the group score (see section Group Score).

gnus-info-read

gnus-info-set-read

Get/set the ranges of read articles.

gnus-info-marks

gnus-info-set-marks

Get/set the lists of ranges of marked articles.

gnus-info-method

gnus-info-set-method

Get/set the group select method.

gnus-info-params

gnus-info-set-params

Get/set the group parameters.

All the getter functions take one parameter—the info list. The setter functions take two parameters—the info list and the new value.

The last three elements in the group info aren’t mandatory, so it may be necessary to extend the group info before setting the element. If this is necessary, you can just pass on a non-nil third parameter to the three final setter functions to have this happen automatically.

11.7.7 Extended Interactive

Gnus extends the standard Emacs interactive specification slightly to allow easy use of the symbolic prefix (see section Symbolic Prefixes). Here’s an example of how this is used:

(defun gnus-summary-increase-score (&optional score symp)
  (interactive (gnus-interactive "P\ny"))
  ...
  )

The best thing to do would have been to implement gnus-interactive as a macro which would have returned an interactive form, but this isn’t possible since Emacs checks whether a function is interactive or not by simply doing an assq on the lambda form. So, instead we have gnus-interactive function that takes a string and returns values that are usable to interactive.

This function accepts (almost) all normal interactive specs, but adds a few more.

‘y’

The current symbolic prefix—the gnus-current-prefix-symbol variable.

‘Y’

A list of the current symbolic prefixes—the gnus-current-prefix-symbol variable.

‘A’

The current article number—the gnus-summary-article-number function.

‘H’

The current article header—the gnus-summary-article-header function.

‘g’

The current group name—the gnus-group-group-name function.

11.7.8 Emacs/XEmacs Code

While Gnus runs under Emacs, XEmacs and Mule, I decided that one of the platforms must be the primary one. I chose Emacs. Not because I don’t like XEmacs or Mule, but because it comes first alphabetically.

This means that Gnus will byte-compile under Emacs with nary a warning, while XEmacs will pump out gigabytes of warnings while byte-compiling. As I use byte-compilation warnings to help me root out trivial errors in Gnus, that’s very useful.

I’ve also consistently used Emacs function interfaces, but have used Gnusey aliases for the functions. To take an example: Emacs defines a run-at-time function while XEmacs defines a start-itimer function. I then define a function called gnus-run-at-time that takes the same parameters as the Emacs run-at-time. When running Gnus under Emacs, the former function is just an alias for the latter. However, when running under XEmacs, the former is an alias for the following function:

(defun gnus-xmas-run-at-time (time repeat function &rest args)
  (start-itimer
   "gnus-run-at-time"
   `(lambda ()
      (,function ,@args))
   time repeat))

This sort of thing has been done for bunches of functions. Gnus does not redefine any native Emacs functions while running under XEmacs—it does this defalias thing with Gnus equivalents instead. Cleaner all over.

In the cases where the XEmacs function interface was obviously cleaner, I used it instead. For example gnus-region-active-p is an alias for region-active-p in XEmacs, whereas in Emacs it is a function.

Of course, I could have chosen XEmacs as my native platform and done mapping functions the other way around. But I didn’t. The performance hit these indirections impose on Gnus under XEmacs should be slight.

11.7.9 Various File Formats

11.7.9.1 Active File Format

The active file lists all groups available on the server in question. It also lists the highest and lowest current article numbers in each group.

Here’s an excerpt from a typical active file:

soc.motss 296030 293865 y
alt.binaries.pictures.fractals 3922 3913 n
comp.sources.unix 1605 1593 m
comp.binaries.ibm.pc 5097 5089 y
no.general 1000 900 y

Here’s a pseudo-BNF definition of this file:

active      = *group-line
group-line  = group spc high-number spc low-number spc flag <NEWLINE>
group       = <non-white-space string>
spc         = " "
high-number = <non-negative integer>
low-number  = <positive integer>
flag        = "y" / "n" / "m" / "j" / "x" / "=" group

For a full description of this file, see the manual pages for ‘innd’, in particular ‘active(5)’.

11.7.9.2 Newsgroups File Format

The newsgroups file lists groups along with their descriptions. Not all groups on the server have to be listed, and not all groups in the file have to exist on the server. The file is meant purely as information to the user.

The format is quite simple; a group name, a tab, and the description. Here’s the definition:

newsgroups    = *line
line          = group tab description <NEWLINE>
group         = <non-white-space string>
tab           = <TAB>
description   = <string>

This document was generated on January 25, 2015 using texi2html 1.82.