diff options
Diffstat (limited to 'usr.sbin/sendmail/contrib/xla')
| -rw-r--r-- | usr.sbin/sendmail/contrib/xla/README | 207 | ||||
| -rw-r--r-- | usr.sbin/sendmail/contrib/xla/xla.c | 532 |
2 files changed, 739 insertions, 0 deletions
diff --git a/usr.sbin/sendmail/contrib/xla/README b/usr.sbin/sendmail/contrib/xla/README new file mode 100644 index 0000000..a72fd03 --- /dev/null +++ b/usr.sbin/sendmail/contrib/xla/README @@ -0,0 +1,207 @@ + XLA - Extended Load Average design for Sendmail R6 + -------------------------------------------------- + + Christophe Wolfhugel - Herve Schauer Consultants + wolf@grasp.insa-lyon.fr, wolf@hsc-sec.fr + + +WARNING: this extension is supplied as a contribution to Sendmail. +Should you have trouble, questions, please contact me directly, and +*not* the Sendmail development team. + + +ABSTRACT + +Sendmail currently furnishes a limitation mecanism which is based on +the system load average, when available. Experience has prooven that +this was not sufficiant for some particular situations, for example +if you have slow and/or overloaded links. This can easily cause both +system and network congestions with Sendmail having to handle a large +number of simultaneous sessions on the same overloaded link, causing +most of the SMTP sessions to timeout after a long time. The system +load average is also generally too slow to react when your system +gets a burst of incoming or outgoing SMTP sessions which on some +stations can easily cause system unavailabilities. + +The extended load average module has been designed in order to furnish +a way of limitation the load generated by Sendmail to both your +system and your network. This design can be used either alone or as +complementary to the system load average if your system supports it. + +Limitation is based on the number of incoming/outgoing SMTP sessions, +and remote hosts are classified in classes. The system administrator +will define a maximum number of incoming SMTP sessions as well as +a maximum total (incoming + outgoing) sessions for each class of +hosts. A class can be either an individual machine or a network. + +When the limit is reached for a given class, all incoming SMTP +connections will be politely refused. When the limit is reached for +all classes, the SMTP connections will be refused by the system +(which one could consider as less politely :)). +On outgoing mail, messages will be queued for delayed processing. + +The extended load average parameters are given in the Sendmail +configuration file, and when not present, Sendmail behaves the +usual way. + + +COMPILATION + +Copy the xla.c module in the src sub-directory, edit the Makefile +in order to define XLA (-DXLA). Also add the xla.[co] module name +in the list of files so that it gets compiled. + +Regenerate sendmail by removing all objects, or at least those +containing references to XLA (this list may vary, so use grep to +get the module list). This will generate a new sendmail executable +containing the xla code. + +Debugging level 59 has been assigned to this module and when used +it provides some output (sendmail -d59.x). Please check the source +code to see which levels are supported. + + +CONFIGURATION + +The extended average uses a new set of configuration lines in the +sendmail.cf file. All newly introduced line begin with the letter L +(capital L). + +Before detailling the syntax, first an example (this can be placed +at any section of the sendmail.cf file, note that the order is +important). Fields are separated by (one or more) tabs/spaces. + +# File name used to store the counters +L/etc/sendmail.la +# Classes definition +# Lname #queue #reject +L*.insa-lyon.fr 8 3 +L*.univ-lyon1.fr 6 4 +L* 15 16 + +The first line defines the working file which will be used in order +to have the occurences of Sendmail read and update the counters. The +format of this file is described in the "Design" section. +This line is mandatory and the specified file must be absolute (ie +begin with a slash). + +Then you can specify one or more classes. The last class (*) is also +mandatory and should be in last position as the first match will stop +the search and if there is no match the behavior of Sendmail is unknown. + +Each class has three fields separated by one or more tabs/spaces. + +L{mask} {queue_#} {refuse_#} + +The {mask} is a simple mask. It can be either an explicit host name +(like grasp.insa-lyon.fr) or a mask starting with "*." or just "*". +No other variants are allowed. + +Lgrasp.insa-lyon.fr will match exactely any session to/from this host. + +L*.insa-lyon.fr will match any session to/from any machine in the + insa-lyon.fr domain as well as from the machine + named "insa-lyon.fr" if it exists. + +L* will match any session, and thus should also be + last in the list to act as a catchup line. + +The {queue_#} is the maximum number of SMTP sessions in the given class +for both incoming and outgoing messages. The {refuse_#} indicates when +to refuse incoming messages for this class. The interaction between +those counters is somewhat subtle. It seems logical that a standard +configuration has {queue_#} >= {refuse_#}, and in fact in most +configurations they can be equal (that's why what I use in my environment). +Thus, this is not mandatory. If {queue_#} < {refuse_#} outgoing messages +will be lower priority than incoming messages and once a class gets loaded +the outgoing messages are blocked first. + +I use very low values in some situations, for example I have a customer +connected to the Internet via a 9600 bps line, they also have internal +users sending burst of messages (10, 20 small messages coming in just +one or two seconds). Both situations were unsupportable. The line is +too slow to handle many simultaneous connections and the mail server +does not have the ressources to handle such a heavy load (it's a 12 Megs +Sun 3 also doing Usenet news). + +I have defined following section in the configuration file, and experience +shows the benefits for everyone. Fake domain for the example: customer.fr. + +L/etc/sendmail.la +L*.customer.fr 8 8 +L* 3 3 + +This means that there might not be more than 8 simultaneous SMTP sessions +between the mail server and any other internal host. This is to protect +the station from heavy loads like users (or applications !) sending +several tenths of messages in just a few seconds). +No more than 3 SMTP sessions are authorized with any other host, this is +to save the load of the slow 9600 line to the Internet. + +Drawback is that is you have 3 * 2 Megs sessions established from/to the +outside, all your mail will be held until one slot gets available, on +a 9600 bps line just make your counts, il blocks your line during over +one hour. + + +DESIGN + +Sendmail will analyze the "L" lines in the configuration file during +startup (or read the initialized structure from the frozen file). +When started in daemon mode (and only there), any existing working file +will be cleared and a new one is created. Each class gets a record in +the sendmail.la work file. The size of this record is a short integer +(generally two bytes) and represents the count of active sessions in +the given class. Read/Write operations in this file are done in +one operation (as anyway the size is far below one disk sector). The +file is locked with Sendmail's lockfile() function priori to any +access. + +Handling incoming SMTP sessions. + +There is interaction is two points in the Sendmail source code. First +on the listen system call: if all slots in all classes are in use, +a listen(0) is done so that the system rejects any incoming SMTP session. +This avois to fork and then reject the connexion. + +If there are some free slots, nothing better than accepting the +connection, then forking can be done. The child process then checks if +the adequate class is full or not. If full, it rejects the connection +with a "421 Too many sessions" diagnostic to the sender (which should +then appear when the remote users do a mailq). If the treshold {reject_#} +is not reached, the connection is accepted and the counter is sendmail.la +is updated. + +Handling outgoing SMTP sessions. + +As soon as Sendmail needs to connect to a distant host, the adequate class +is checked against {queue_#} and if no slots are available, the message is +queued for further processing. + +Sendmail's connection caching. + +Sendmail-R6 introduces a new design: connection caching, ie several SMTP +sessions can be opened at the same time. This could cause some problems +when sending mail, as after having a few connections opened, all slots +could be in use and generate a partial delivery of the message. In +order to deal with this, xla.c uses following design "for a given +sendmail process, only the first connection in a given class is counted". +This can be done because sendmail does not do parralel message sending +on the different channels. + +End of connection. + +As soon as a connection is closed, the counters will be automatically +updated. + + + +Please look at the code to understand of all this works. Comments, +suggestions, questions welcome. + + + + Christophe Wolfhugel + Herve Schauer Consultants + Paris, France + May 23, 1993 diff --git a/usr.sbin/sendmail/contrib/xla/xla.c b/usr.sbin/sendmail/contrib/xla/xla.c new file mode 100644 index 0000000..627d383 --- /dev/null +++ b/usr.sbin/sendmail/contrib/xla/xla.c @@ -0,0 +1,532 @@ +/* + * (C) Copyright 1993-94, Herve Schauer Consultants + * + * This module written by Christophe.Wolfhugel@hsc-sec.fr + * is to be used under the same conditions and terms (and absence + * or warranties) than the other modules of the Sendmail package. + * + * ABSOLUTELY NO WARRANTY. USE AT YOUR OWN RISKS. + * + * Version: 940417, applied a patch from Paul Graham <pjg@acsu.buffalo.edu> + * (lockfile syntax in xla.c did not reflect anymore the + * new one used by sendmail, now fine). + * + */ + + +#ifdef XLA + +#ifndef MAXLARULES +# define MAXLARULES 20 +#endif + +# include "sendmail.h" + +typedef struct { + short queue; /* # of connexions to have queueing */ + short reject; /* # of conn. to reject */ + short num; /* # of increments this process */ + char *mask; /* Mask (domain) */ + } XLARULE; + +char *XlaFname; /* Work file name */ +char XlaHostName[1024]; /* Temporary */ +int XlaNext; /* # of XlaRules */ +pid_t XlaPid; /* Pid updating the tables */ +XLARULE XlaRules[MAXLARULES]; /* The rules themselves */ +short XlaCtr[MAXLARULES]; /* Counter (for work file only) */ + +extern bool lockfile(); + +/* +** XLAMATCH -- Matches a fnmatch like expression. +** +** Parameters: +** mask -- mask to match the string too; +** name -- string. +** +** Mask can either be a plain string or a simplified fnmatch like mask: +** *.string or string.* +** No other alternatives are accepted. +** +** Returns: +** none. +** +** Side Effects: +** none. +*/ + +bool +XlaMatch(mask, name) + char *mask, *name; +{ + int l1, l2; + + l1 = strlen(mask); l2 = strlen(name); + if (l1 == 1 && mask[0] == '*') return(TRUE); + if (mask[0] == '*' && mask[1] == '.') { + if (l2 < (l1 - 2)) return(FALSE); + if (strcasecmp(&mask[2], name) == 0) return(TRUE); + if (strcasecmp(&mask[1], name + l2 - l1 + 1) == 0) return(TRUE); + return(FALSE); + } + if (l1 < 3) return(FALSE); + if (mask[l1 -1] == '*') { + if (l2 < l1 - 1) return(FALSE); + if (strncasecmp(mask, name, l1 - 1) == 0) return(TRUE); + return(FALSE); + } + if (strcasecmp(mask, name) == 0) return(TRUE); + return(FALSE); +} + +/* +** XLAZERO -- Zeroes the used variables +** +** Just initializes some variables, called once at sendmail +** startup. +** +** Parameters: +** none. +** +** Returns: +** none. +** +** Side Effects: +** none. +*/ + +xla_zero() +{ + if (tTd(59, 1)) { + printf("xla_zero\n"); + } + XlaFname = NULL; + XlaNext = 0; + XlaPid = 0; + memset((char *) &XlaRules[0], 0, sizeof(XLARULE) * MAXLARULES); +} + + +/* +** XLAINIT -- initialized extended load average stuff +** +** This routine handles the L lines appearing in the configuration +** file. +** +** L/etc/sendmail.la indicates the working file +** Lmask #1 #2 Xtended LA to apply to mask +** #1 = Queueing # of connections +** #2 = Reject connections. +** +** Parameters: +** line -- the cf file line to parse. +** +** Returns: +** none. +** +** Side Effects: +** Builds several internal tables. +*/ + +xla_init(line) + char *line; +{ + char *s; + + if (tTd(59, 1)) { + printf("xla_init line: %s\n", line); + } + if (XlaFname == NULL && *line == '/') { /* Work file name */ + XlaFname = newstr(line); + if (tTd(59, 10)) + printf("xla_init: fname = %s\n", XlaFname); + return; + } + if (XlaNext == MAXLARULES) { + syserr("too many xla rules defined (%d max)", MAXLARULES); + return; + } + s = strtok(line, " \t"); + if (s == NULL) { + syserr("xla: line unparseable"); + return; + } + XlaRules[XlaNext].mask = newstr(s); + s = strtok(NULL, " \t"); + if (s == NULL) { + syserr("xla: line unparseable"); + return; + } + XlaRules[XlaNext].queue = atoi(s); + s = strtok(NULL, " \t"); + if (s == NULL) { + syserr("xla: line unparseable"); + return; + } + XlaRules[XlaNext].reject = atoi(s); + if (tTd(59, 10)) + printf("xla_init: rule #%d = %s q=%d r=%d\n", XlaNext, + XlaRules[XlaNext].mask, + XlaRules[XlaNext].queue, XlaRules[XlaNext].reject); + XlaNext++; +} + + +/* +** XLACREATEFILE -- Create the working file +** +** Tries to create the working file, called each time sendmail is +** invoked with the -bd option. +** +** Parameters: +** none. +** +** Returns: +** none. +** +** Side Effects: +** Creates the working file (sendmail.la) and zeroes it. +*/ + +xla_create_file() +{ + int fd, i; + + if (tTd(59, 1)) + printf("xla_create_file:\n"); + if (XlaFname == NULL) return; + fd = open(XlaFname, O_RDWR|O_CREAT, 0644); + if (fd == -1) { + XlaFname = NULL; + syserr("xla_create_file: open failed"); + return; + } + if (!lockfile(fd, XlaFname, NULL, LOCK_EX)) { + close(fd); + XlaFname = NULL; + syserr("xla_create_file: can't set lock"); + return; + } + if (ftruncate(fd, 0) == -1) { + close(fd); + XlaFname = NULL; + syserr("xla_create_file: can't truncate XlaFname"); + return; + } + if (write(fd, XlaCtr, sizeof(XlaCtr)) != sizeof(XlaCtr)) { + XlaFname == NULL; + syserr("xla_create_file: can't write XlaFname"); + } + close(fd); +} + + +/* +** XLASMTPOK -- Checks if all slots are in use +** +** Check is there are still some slots available for an SMTP +** connection. +** +** Parameters: +** none. +** +** Returns: +** TRUE -- slots are available; +** FALSE -- no more slots. +** +** Side Effects: +** Reads a file, uses a lock and updates sendmail.la if a slot +** is free for use. +*/ + +bool +xla_smtp_ok() +{ + int fd, i; + + if (tTd(59, 1)) + printf("xla_smtp_ok:\n"); + if (XlaFname == NULL) return(TRUE); + fd = open(XlaFname, O_RDWR, 0644); + if (fd == -1) { + XlaFname = NULL; + syserr("xla_smtp_ok: open failed"); + return(TRUE); + } + if (!lockfile(fd, XlaFname, NULL, LOCK_EX)) { + close(fd); + XlaFname = NULL; + syserr("xla_smtp_ok: can't set lock"); + return(TRUE); + } + if (read(fd, XlaCtr, sizeof(XlaCtr)) != sizeof(XlaCtr)) { + close(fd); + XlaFname = NULL; + syserr("xla_smtp_ok: can't read XlaFname"); + return(TRUE); + } + close(fd); + for (i = 0; i < XlaNext; i++) { + if (XlaCtr[i] < XlaRules[i].reject) + return(TRUE); + } + return(FALSE); +} + + +/* +** XLAHOSTOK -- Can we accept a connection from this host +** +** Check the quota for the indicated host +** +** Parameters: +** name -- host name or IP# (string) +** +** Returns: +** TRUE -- we can accept the connection; +** FALSE -- we must refuse the connection.1 +** +** Side Effects: +** Reads and writes a file, uses a lock and still updates +** sendmail.la is a slot gets assigned. +*/ + +bool +xla_host_ok(name) + char *name; +{ + int fd, i; + + if (tTd(59, 1)) + printf("xla_host_ok:\n"); + if (XlaFname == NULL) return(TRUE); + fd = open(XlaFname, O_RDWR, 0644); + if (fd == -1) { + XlaFname = NULL; + syserr("xla_host_ok: open failed"); + return(TRUE); + } + XlaPid = getpid(); + if (!lockfile(fd, XlaFname, NULL, LOCK_EX)) { + close(fd); + XlaFname = NULL; + syserr("xla_host_ok: can't set lock"); + return(TRUE); + } + if (read(fd, XlaCtr, sizeof(XlaCtr)) != sizeof(XlaCtr)) { + close(fd); + XlaFname = NULL; + syserr("xla_smtp_ok: can't read XlaFname"); + return(TRUE); + } + strncpy(XlaHostName, name, sizeof(XlaHostName) -1); + XlaHostName[sizeof(XlaHostName) -1] = 0; + i = strlen(name) - 1; + if (i >= 0 && XlaHostName[i] == '.') XlaHostName[i] = 0; + for (i = 0; i < XlaNext; i++) { + if (XlaMatch(XlaRules[i].mask, XlaHostName)) { + if (XlaCtr[i] < XlaRules[i].reject) { + if (XlaRules[i].num++ == 0) { + XlaCtr[i]++; + lseek(fd, i*sizeof(XlaCtr[i]), SEEK_SET); + if (write(fd, &XlaCtr[i], sizeof(XlaCtr[i])) != sizeof(XlaCtr[i])) + XlaFname = NULL; + } + close(fd); + return(TRUE); + } + close(fd); + return(FALSE); + } + } + close(fd); + return(TRUE); +} + +/* +** XLANOQUEUEOK -- Can we sent this message to the remote host +** +** Check if we can send to the remote host +** +** Parameters: +** name -- host name or IP# (string) +** +** Returns: +** TRUE -- we can send the message to the remote site; +** FALSE -- we can't connect the remote host, queue. +** +** Side Effects: +** Reads and writes a file, uses a lock. +** And still updates the sendmail.la file. +*/ + +bool +xla_noqueue_ok(name) + char *name; +{ + int fd, i; + + if (tTd(59, 1)) + printf("xla_noqueue_ok:\n"); + if (XlaFname == NULL) return(TRUE); + fd = open(XlaFname, O_RDWR, 0644); + if (fd == -1) { + XlaFname = NULL; + syserr("xla_noqueue_ok: open failed"); + return(TRUE); + } + if (!lockfile(fd, XlaFname, NULL, LOCK_EX)) { + close(fd); + XlaFname = NULL; + syserr("xla_noqueue_ok: can't set lock"); + return(TRUE); + } + XlaPid = getpid(); + if (read(fd, XlaCtr, sizeof(XlaCtr)) != sizeof(XlaCtr)) { + close(fd); + XlaFname = NULL; + syserr("xla_noqueue_ok: can't read XlaFname"); + return(TRUE); + } + strncpy(XlaHostName, name, sizeof(XlaHostName) -1); + XlaHostName[sizeof(XlaHostName) -1] = 0; + i = strlen(name) - 1; + if (i >= 0 && XlaHostName[i] == '.') XlaHostName[i] = 0; + for (i = 0; i < XlaNext; i++) { + if (XlaMatch(XlaRules[i].mask, XlaHostName)) { + if (XlaCtr[i] < XlaRules[i].queue) { + if (XlaRules[i].num++ == 0) { + XlaCtr[i]++; + lseek(fd, i*sizeof(XlaCtr[i]), SEEK_SET); + if (write(fd, &XlaCtr[i], sizeof(XlaCtr[i])) != sizeof(XlaCtr[i])) + XlaFname = NULL; + } + close(fd); + return(TRUE); + } + close(fd); + return(FALSE); + } + } + close(fd); + return(TRUE); +} + + +/* +** XLAHOSTEND -- Notice that a connection is terminated. +** +** Updates the counters to reflect the end of an SMTP session +** (in or outgoing). +** +** Parameters: +** name -- host name or IP# (string) +** +** Returns: +** none. +** +** Side Effects: +** Reads and writes a file, uses a lock. +** And still updates sendmail.la. +*/ + +xla_host_end(name) + char *name; +{ + int fd, i; + + if (tTd(59, 1)) + printf("xla_host_end:\n"); + if (XlaFname == NULL || XlaPid != getpid()) return; + fd = open(XlaFname, O_RDWR, 0644); + if (fd == -1) { + XlaFname = NULL; + syserr("xla_host_end: open failed"); + return; + } + if (!lockfile(fd, XlaFname, NULL, LOCK_EX)) { + close(fd); + XlaFname = NULL; + syserr("xla_host_end: can't set lock"); + return; + } + if (read(fd, XlaCtr, sizeof(XlaCtr)) != sizeof(XlaCtr)) { + close(fd); + XlaFname = NULL; + syserr("xla_host_end: can't read XlaFname"); + return(TRUE); + } + strncpy(XlaHostName, name, sizeof(XlaHostName) -1); + XlaHostName[sizeof(XlaHostName) -1] = 0; + i = strlen(name) - 1; + if (i >= 0 && XlaHostName[i] == '.') XlaHostName[i] = 0; + for (i = 0; i < XlaNext; i++) { + if (XlaMatch(XlaRules[i].mask, XlaHostName)) { + if (XlaRules[i].num > 0 && XlaRules[i].num-- == 1) { + if (XlaCtr[i]) XlaCtr[i]--; + lseek(fd, i*sizeof(XlaCtr[i]), SEEK_SET); + if (write(fd, &XlaCtr[i], sizeof(XlaCtr[i])) + != sizeof(XlaCtr[i])) + XlaFname = NULL; + } + close(fd); + return; + } + } + close(fd); +} + +/* +** XLAALLEND -- Mark all connections as closed. +** +** Generally due to an emergency exit. +** +** Parameters: +** name -- host name or IP# (string) +** +** Returns: +** none. +** +** Side Effects: +** Reads and writes a file, uses a lock. +** And guess what: updates sendmail.la. +*/ + +xla_all_end() +{ + int fd, i; + + if (tTd(59, 1)) + printf("xla_all_end:\n"); + if (XlaFname == NULL || XlaPid != getpid()) return; + fd = open(XlaFname, O_RDWR, 0644); + if (fd == -1) { + XlaFname = NULL; + syserr("xla_all_end: open failed"); + return; + } + if (!lockfile(fd, XlaFname, NULL, LOCK_EX)) { + close(fd); + XlaFname = NULL; + syserr("xla_all_end: can't set lock"); + return; + } + if (read(fd, XlaCtr, sizeof(XlaCtr)) != sizeof(XlaCtr)) { + close(fd); + XlaFname = NULL; + syserr("xla_all_end: can't read XlaFname"); + return(TRUE); + } + for (i = 0; i < XlaNext; i++) { + if (XlaCtr[i] > 0 && XlaRules[i].num > 0) { + XlaCtr[i]--; XlaRules[i].num = 0; + } + } + lseek(fd, 0, SEEK_SET); + if (write(fd, XlaCtr, sizeof(XlaCtr)) != sizeof(XlaCtr)) { + XlaFname = NULL; + } + close(fd); +} +#endif /* XLA */ |
