rkt logo

INN FAQ Part 7

NOTE: The maintainers of the INN FAQ stopped publishing in December 1997.
An important update to this topic is provided by Mib Software in the Usenet RKT for Subscribers.

Subject: (7.45) INN is only slowly responding

(From: Erland Sommarskog <sommar@sophocles.algonet.se>)
Q: We started a new news server just a month ago. SS-20, lots of memory,
23 GB of news spool. The first week it ran fast as a jaguar, but now
it crawls like a snail. It takes forever to connect, we can't keep
up with our feeds which have to flush our queues, it takes to forever
to connect. When we monitor the box it spends an awful lot of time
in kernel mode, and seems to be doing a tremendous amount of disk
access. Where did we go wrong? If it matters, we're not running
expire, but dexpire and histtrim.

A: The problem is with the history database. This database consists of
three files: history, history.pag and history.dir. The dir files
contains a hash table. For optimal performance, the hash file must
not be too small, or you will get many collisions. The initial size
of the history database is based on Usenet traffic as it was a
couple of years ago, and no one was considering a 23 GB spool. Now,
for people who is running expire this is not a problem, because when
you run expire, the history file is rebuilt, and the size for the
new database is taken from the old one. But if you are running
dexpire and histtrim, this never happens, and innd will spend most
of its time reading overflow buckets.
What to do then? Rebuild the history database. And the simlpest
way to do it, is to run expire. A simple approach is to run expire
every now and then, with an expire.ctl that safe won't expire very
many articles. (But add a short expiration time on some junk group,
so that expire get something to work with.)
A better approach is to discontinue use of histtrim, and run expire
daily basis. Dexpires produces an output which you easily can trans-
form into an expire.ctl. I discourage use of histtrim, since it
may delete articles from history and yet are on the system.

(From: James Brister <brister@vix.com>)
A good indicator of your performance characteristics would be how much
smaller is the number generated by this,

   head -1 /var/news/etc/history.dir | perl -ane 'print 2 ** $F[7], "\n";'

than the size of your history text file. If it's bigger you're OK. if it's
smaller, then lookups for the message ids at the tail of the history text
file (past the byte indexed by the number just generated) will be much
slower than for those at the front.

See also: INN FAQ #7.6, INN FAQ #7.7 and INN FAQ #7.41


[Source: INN FAQ Part 7 Archive-name: usenet/software/inn-faq/part7]
[Last Changed: $Date: 1997/09/16 01:25:57 $ $Revision: 2.25 $]
[Copyright: 1997 Heiko Rupp, portions by Tom Limoncelli, Rich Salz, et al.]