| Subject: (7.53) Makehistory is slow on inn 1.x , x<5.1 |
|---|
From: "Otto J. Makela" <otto@cc.jyu.fi> There is a rather serious bug in the standard distribution inn 1.4 (and all unoff versions, but has been fixed in inn 1.5.1) makehistory, which affects innd performance very much. Makehistory can be used to create the history dbz database files from the raw text file and for example dexpire does this with every histtrim operation. The problem is that makehistory is given just one parameter specifying the size of the expected dbz database in lines, but it uses this parameter also for the dbzfresh() tagmask parameter where the maximum key value is expected to be given. This means the dbz database is built with a very small hash table causing most of the history database to be accessed without hashing, a very serious performance hit. As noted in the inn FAQ section 7.45, you can check the size of your dbz hash table with the following commands: head -1 history.dir | perl -ane 'print 2 ** $F[7], "\n";'if the number returned is smaller than your history text file, you are being affected by this problem. The problem becomes worse as the history file grows, so an indication of this problem is that your news server worked fine at first (small history file) but started really beating the living daylights out of the hard disk where history is after it grew a bit larger. This problem can be patched with the following extremely simple change to makehistory, which just multiplies the number of lines in the dbzfresh call with 70 which is (supposedly) an average number of characters per history text file line. This same fix has been implemented in inn 1.5.1 makehistory. -- diff -u expire/makehistory.c{.orig,}
--- expire/makehistory.c.orig Mon Jul 31 22:18:46 1995
+++ expire/makehistory.c Wed Apr 23 11:26:50 1997
@@ -125,7 +125,8 @@
/* Open the new database, using the old file if desired and possible. */
(void)dbzincore(1);
if (IgnoreOld) {
- if (dbzfresh(p, dbzsize(size), HIS_FIELDSEP, 'C', dbztagmask(size)) < 0) {
+ /* Assume average history line length of 70 characters */
+ if (dbzfresh(p, dbzsize(size), HIS_FIELDSEP, 'C', dbztagmask(size*70)) < 0) {
(void)fprintf(stderr, "Can't do dbzfresh, %s\n",
strerror(errno));
if (temp[0])
------------------------------[Last Changed: $Date: 1997/09/16 01:25:57 $ $Revision: 2.25 $] [Copyright: 1997 Heiko Rupp, portions by Tom Limoncelli, Rich Salz, et al.] |