INN FAQ Cookbook example of setting up NOV ("overchan").

INN FAQ Part 4

NOTE: The maintainers of the INN FAQ stopped publishing in December 1997.
An important update to this topic is provided by Mib Software in the Usenet RKT for Subscribers.

Subject: (4.19) Cookbook example of setting up NOV ("overchan").
Now that you have your other feeds working, you might want to set up a NOV feed so that your NOV database is built. Newsreaders use the NOV database to speed up their queries. Christophe.Wolfhugel@grasp.insa-lyon.fr (Christophe Wolfhugel) (with many modifications from Tom Limoncelli and further input from davek@melita.com (Dave Kennedy) ) writes: Step 1: Upgrade to INN 1.4 or higher: Most of the bugs in 1.3 were related with overchan. In fact, the only reason why many people used 1.3 without any problems was due to the fact that they were not using overchan (and they didn't hit on some of the bugs that appeared for SVR4 users, all of which were fixed in 1.4) Step 1.5: Make sure _PATH_OVERVIEWDIR in config.data is NOT set to "/var/spool/news". There is a big performance boost to be realized by putting the NOV files outside the /var/spool/news hierarchy. To find out why, read "Subject: overchan can't keep up." in part 7 of this FAQ. You might want to read this anyway since it gives advice about other things to do to get better NOV performance. "/var/spool/news/over.view" is becoming the standard place to put your ".overview" files. If you do not use this location, make /var/spool/news/over.view a symbolic link to the correct place. For performance reasons, it is a good idea to set _PATH_OVERVIEWDIR to the actual location of the files. NB: if you change config.data, you must do a "make all" and "make install". It is not sufficient to just give the -D option to overchan and expireover, as nnrpd also needs to know where overview data is. If it doesn't, it won't complain nor use your overview data, but assume, there is none and generate it on the fly which is noticeable slower than using the database. Step 2: Make sure INN is working. Get everything else working before you try to get overchan to work. You'll only confuse yourself. Step 3: Ponder if you have enough disk space. NOV uses up an additional 10%-20% of your news spool. This is a good 100 Mb if you have a full feed. The real space savings come when you delete your separate databases for trn, nn, and tin and use one unified database. All serious newsreaders have NOV support. Step 4: Edit "overview.fmt" (it's in the $INN/site directory, or you can edit it where it was installed, in /usr/lib/news ) to include "Xref:full" as the last line. (i.e. uncomment out the last line). Step 5: Add this entry to your "newsfeeds" file. overchan gets it's data from a special feed. # This feeds header data to NOV: OVERVIEW!::Tc,WO:/usr/local/news/bin/overchan Read the "newsfeeds" man to make sure you understand what you've just done. Do a "ctlinnd checkfile" to make sure the newsfeeds file has the proper syntax, then do a "ctlinnd reload newsfeeds nov" to make it official. Step 6: If you changed your $inn/site files, then: % cd $inn/site % make install Step 7: Let innd know that files have been updated: % ctlinnd reload overview.fmt "Enabled XRef:" % ctlinnd reload newsfeeds "Added OVERVIEW - overchan entry" Step 8: You must run "expireover -s" at least once a month. Once a week is even better. This is necessary to remove overview data of for some reason or other left over entries. Here is a good crontab entry for "news" to run: 0 5 * 1 /usr/lib/news/bin/expireover -s Step 9: (optional) To create the original database: (run this as "news") % /usr/local/news/bin/expireover -a This step will take a long time depending on the number of articles already in your system. But, if you skip this step, client access will be slow for articles that came in before you started "overchan". This is not a serious problem; you will get a lot of warnings in your "news.daily" output until you have received at least one new article in each newsgroup. Note: "a lot of warnings" means one for every newsgroup. This can make your news.daily report >6000 lines. The lines will all look like: overchan cant open clari/local/washington/.overview, No such file or directory overchan cant open clari/local/sfbay/.overview, No such file or directory overchan cant open uc/news/.overview, No such file or directory Step 9: Change the invocation of news.daily: In the crontab file for "news", edit the "news.daily" line to be something like: news.daily delayrm expireover (the expireover is required if you use overchan) Step 10: Inform your users that you now support "NOV, the News OverView database" and suggest that people switch to newsreaders that use newsreaders that are compliant with the Overview format. Step 11: You are done. Step 12: In a few weeks, drop support for mthreads, nnmaster, etc. (assuming you've upgraded to replacements that use NOV). Delete all those old databases that might have been maintained and enjoy the newly gained functionality and regained disk space! Step 13: If you are running tin (mostly the 1.2 versions) then you will get "bad overview" messages. These don't come from inn, but from tin. Solution edit the source (art.c) to increase the buffer size for overview information from 1024 bytes to at least 4096 bytes. ------------------------------ [Source: INN FAQ Part 4 Archive-name: usenet/software/inn-faq/part4] [Last Changed: $Date: 1997/08/26 01:26:21 $ $Revision: 2.19 $] [Copyright: 1997 Heiko Rupp, portions by Tom Limoncelli, Rich Salz, et al.]

Subject: (4.19) Cookbook example of setting up NOV ("overchan").

Now that you have your other feeds working, you might want to set up a
NOV feed so that your NOV database is built. Newsreaders use the NOV
database to speed up their queries.

Christophe.Wolfhugel@grasp.insa-lyon.fr (Christophe Wolfhugel) (with
many modifications from Tom Limoncelli and further input from
davek@melita.com (Dave Kennedy) ) writes:

Step 1: Upgrade to INN 1.4 or higher: Most of the bugs in 1.3 were
related with overchan. In fact, the only reason why many people used
1.3 without any problems was due to the fact that they were not using
overchan (and they didn't hit on some of the bugs that appeared for
SVR4 users, all of which were fixed in 1.4)

Step 1.5: Make sure _PATH_OVERVIEWDIR in config.data is NOT set to
"/var/spool/news". There is a big performance boost to be realized by
putting the NOV files outside the /var/spool/news hierarchy.

To find out why, read "Subject: overchan can't keep up." in part 7 of
this FAQ. You might want to read this anyway since it gives advice
about other things to do to get better NOV performance.

"/var/spool/news/over.view" is becoming the standard place to put
your ".overview" files. If you do not use this location, make
/var/spool/news/over.view a symbolic link to the correct place. For
performance reasons, it is a good idea to set _PATH_OVERVIEWDIR to the
actual location of the files. NB: if you change config.data, you must
do a "make all" and "make install". It is not sufficient to just give
the -D option to overchan and expireover, as nnrpd also needs to know
where overview data is. If it doesn't, it won't complain nor use your
overview data, but assume, there is none and generate it on the fly
which is noticeable slower than using the database.

Step 2: Make sure INN is working. Get everything else working before
you try to get overchan to work. You'll only confuse yourself.

Step 3: Ponder if you have enough disk space. NOV uses up an
additional 10%-20% of your news spool. This is a good 100 Mb if you
have a full feed. The real space savings come when you delete your
separate databases for trn, nn, and tin and use one unified database.
All serious newsreaders have NOV support.

Step 4: Edit "overview.fmt" (it's in the $INN/site directory, or you can
edit it where it was installed, in /usr/lib/news ) to include
"Xref:full" as the last line. (i.e. uncomment out the last line).

Step 5: Add this entry to your "newsfeeds" file. overchan gets it's data
from a special feed.

# This feeds header data to NOV:

OVERVIEW!:*:Tc,WO:/usr/local/news/bin/overchan

Read the "newsfeeds" man to make sure you understand what you've
just done. Do a "ctlinnd checkfile" to make sure the newsfeeds
file has the proper syntax, then do a "ctlinnd reload newsfeeds nov"
to make it official.

Step 6: If you changed your $inn/site files, then:

	% cd $inn/site
	% make install

Step 7: Let innd know that files have been updated:

	% ctlinnd reload overview.fmt "Enabled XRef:"
	% ctlinnd reload newsfeeds "Added OVERVIEW - overchan entry"

Step 8: You must run "expireover -s" at least once a month. Once a
week is even better. This is necessary to remove overview data of
for some reason or other left over entries. Here is a good crontab entry
for "news" to run:

0 5 * * 1 /usr/lib/news/bin/expireover -s

Step 9: (optional) To create the original database:

(run this as "news")

	% /usr/local/news/bin/expireover -a

This step will take a long time depending on the number of articles
already in your system. But, if you skip this step, client access
will be slow for articles that came in before you started "overchan".
This is not a serious problem; you will get a lot of warnings in your
"news.daily" output until you have received at least one new article
in each newsgroup.

Note: "a lot of warnings" means one for every newsgroup. This can
make your news.daily report >6000 lines. The lines will all look like:

overchan cant open clari/local/washington/.overview, No such file or directory
overchan cant open clari/local/sfbay/.overview, No such file or directory
overchan cant open uc/news/.overview, No such file or directory

Step 9: Change the invocation of news.daily:

In the crontab file for "news", edit the "news.daily" line to be
something like:

   news.daily delayrm expireover

(the expireover is required if you use overchan)

Step 10: Inform your users that you now support "NOV, the News OverView
database" and suggest that people switch to newsreaders that use
newsreaders that are compliant with the Overview format.

Step 11: You are done.

Step 12: In a few weeks, drop support for mthreads, nnmaster, etc.
(assuming you've upgraded to replacements that use NOV). Delete all
those old databases that might have been maintained and enjoy the newly
gained functionality and regained disk space!

Step 13: If you are running tin (mostly the 1.2 versions) then you
will get "bad overview" messages. These don't come from inn, but from
tin. Solution edit the source (art.c) to increase the buffer size for
overview information from 1024 bytes to at least 4096 bytes.

------------------------------

[Source: INN FAQ Part 4 Archive-name: usenet/software/inn-faq/part4]
[Last Changed: $Date: 1997/08/26 01:26:21 $ $Revision: 2.19 $]
[Copyright: 1997 Heiko Rupp, portions by Tom Limoncelli, Rich Salz, et al.]