Installing INN part 2: [Previous] [Up to Table of Contents] [Next]

2.  Feeding a large number of sites

     Innd tries to keep as many batchfiles open for as long as possible.
It  will  normally  open  as  many  as  it  can, using all the available
descriptors minus a fixed number for internal  use  (log  files,  etc.).
You  can  explicitly set the number of files to open by using the "-o"

     If you have more outgoing feeds than  available  descriptors,  innd
will recycle the files on a a "least recently used" basis.  If most of
your feeds get most  articles  (or  you  have  vastly  more  feeds  then
available  descriptors),  this  will lead to "file thrashing," closing
and opening all the excess feeds on each article.  To reduce  this,  you
can  have  innd  use  an  internal  buffer for a site by using the "I"
parameter in the newsfeeds file.  If a site does not have its  batchfile
open,  the server will not try to open it until there is more data to be
written then will fit in the buffer.   For  example,  suppose  innd  was
started with "-o10" and there are 12 sites, all with "I512" in their
newsfeeds entry.  If each article generates 50 bytes (a pathname  and  a
Message-ID), then innd will close and re-assign two descriptors every 10
or so articles.

     A better alternative is to use funnels and an  exploder.   Funnels,
specified  in  the  newsfeeds file, let multiple sites send their output
down a single stream.  The advantage of funnels is that this stream  can
be a channel; the primary disadvantage is that the funnel specifies what
data is to be written, not the individual sites.  (Since most feeds will
want either "Wn" or "Wnm" entries, this is usually not a problem.)

     In order for the funnel output to be useful,  it  usually  must  be
split  into  individual, per-site, files.  Programs that do this type of
splitting  are  called  "exploders."   INN  provides  two   exploders,
filechan and buffchan.

     Filechan is the simplest, and most inefficient, exploder.  It  does
not  keep  any  files open and is very system-call intensive.  It can be
used to provide behavior (and performance!) that  is  similar  to  B2.11
inews.   It  can, however, be used as the funnel for an unlimited number
of sites.

     Buffchan keeps all its output files open all the time.   It  should
not  be  used  for  more  sites  then  a  single  process can have open.
Buffchan also has flags to automatically flush its  files,  as  well  as
close  and  re-open  them, every specified number of articles.  (The re-
open capability is useful for things like nntplink in  its  "watch  the
batchfile" mode.)  Using buffchan with the  "-l1\  -c50"  flags  will
give behavior that is similar to the C News relaynews.

     Buffchan can be run as a full exploder ("Tx")  in  the  newsfeeds
file.   This  means that you can use ctlinnd to send a command line down
buffchan's input stream.   (Innd  will  also  send  a  command  whenever
newsgroups  are  modified.)   The only useful message is "flush" which
will close, and re-open, the specified site files.  You should also note
that the flow is one-way; full exploders cannot send any acknowledgement

[Source:"Installing InterNetNews 1.5.1"][][Revision: 1.19 1996/11/10]
[Copyright: 1991 Rich Salz, 1996 Internet Software Consortium]
Installing INN Part 2 : [Previous] [Up to Table of Contents] [Next]
You can find a summary and links related to this topic
as part of the Mib Software Usenet RKT.