December 21, 2001

HTTPsync is client-side-only software that performs fast
and efficient incremental updates of collections of files
from standard HTTP servers.  

This file describes HTTPsync version 3.00.  An article
describing HTTPsync appears in the July 1999 issue of
Dr. Dobb's Journal.

Contents
   Theory of Operation
   Obtaining and Compiling HTTPsync
   Usage
   Packing List syntax
   Reusable Subroutines
   Additional Information

Theory of Operation
-------------------
A packing list is stored on the server which describes the
files, sizes, time stamps, and read/write/execute permissions
for all files in the collection.  HTTPsync first obtains the
packing list specified at run time by a URL, then compares
to the local files.  HTTP GET requests are made for only those
files which are needed.  Files created will always be in or
below the current working directory, owned by the current
process.  Read/Write/Execute permissions are transferred, but
symlinks and SUID bits are stripped.  Files are made to match
exactly, local changes are overwritten.


Obtaining and compiling HTTPsync
--------------------------------
HTTPsync is provided under a license which meets the Open
Source Definition, and is GPL compatible.
Refer to the source code itself for the license.

1. Get the source for HTTPsync:
    http://www.mibsoftware.com/httpsync/httpsync.c

2. Compile it. This is usually a

	make httpsync

or

	gcc -o httpsync httpsync.c

In the case of unresolved symbols errors, then add appropriate
libraries. On Solaris:

	gcc -o httpsync httpsync.c -lnsl -lsocket

On WinSock systems, be sure to define WIN32 and link
against WSOCK32.LIB.  For example, using MSVC:
	
	cl httpsync.c /DWIN32 wsock32.lib

3. If you want to use SSL, you must compile with USE_SSLEAY
or SSL_OPENSSL defined.  You must supply the cryptographic
libraries. (On UNIX they are often named -lssl -lcrypto.  HTTPsync
has been tested to work with OpenSSL.)

4. HTTPsync does not need any special permissions to run.  There
are no configuration files.  It just needs to be in the path.

Usage
-----
1. To place a collection on a server, create a packing list
   (usually by running httpsync -m, providing a list of files
   on the standard input, each beginning with "./")  The
   output of the unix 'find' utility works well, for example:
       find . -type f | httpsync -m
   Store this packing list at the root of the collection, or
   use "R" to specify a different root.

2. To obtain a collection from a server, provide the URL
   of a packing list, and specify an HTTP proxy server, if any.
	httpsync @http://domain.com/path/to/packing.lst

   If a proxy server should be used:
	httpsync -x proxy.domain.com @http://domain.com/path/to/packing.lst

3. As of HTTPsync version 3.0, SSL for https: URLs is supported when
   you compile with either the USE_SSLEAY or USE_OPENSSL macros defined.
   No validation of certificates is done.

4. To use HTTP Basic Authentication, specify a username on the command
   line.    If you must provide a password on the command line, be aware that
   it may be visible to other users who can view the parameters of your your
   running processes. If you are running interactively, you should
   not specify a password on the command line. (You will be prompted.)
       httpsync -u username [-p password] @http://hostaddress/URI

5. If you wish to automate HTTPsync, or wrap with a GUI, see the source
   code for operation when given the '-a' flag.  (The -t flag may also
   be useful.)


Packing list syntax
-------------------
There are four types of lines in an HTTPsync packing list.

- Lines beginning "./"  specify a file or directory.  Subdirectories
  for files are created automatically, if needed.  See the
  output of HTTPsync -m for the fields in order.  File
  permissions are given in base 8, sizes in base 10.  Dates
  must always be GMT.  File names must not contain the character
  sequence '..', which is restricted on the client side so that
  files outside the current directory hierarchy are not modified.
  
- Lines beginning with "O./" specify obsolete files or
  directories.

- Lines beginning with "R" modify the URL used for HTTP
  requests.  The local file directory remains the current
  directory.  This allows CGI URLs for access control,
  authentication, etc.

- Lines beginning with '#' are ignored as comments, except
  a line beginning with the sequence "#-#httpsync ", which
  gives the version of the file format.  This is checked to
  prevent future packing list formats from confusing older
  httpsync clients.

Running the output of "find . -type f" through HTTPsync will
generate a valid packing list.  

Files larger than the largest long value are not handled by
HTTPsync.

Reusable Subroutines
--------------------
HTTP/1.1 persistent connections save the setup and tear down
overhead when multiple files are to be transferred from a
server.  See the routines beginning "HTTPaccess_" for an
easily reused method.

Additional Information
----------------------
Additional usage notes, and original copies of HTTPsync are
available at http://www.mibsoftware.com/httpsync/