Clobberd 4.16 Install              Copyright (C) 1996-2000 Jason Nunn
---------------------              Clobberd comes under the GNU
                                   General Public License.
                                   Darwin, Northern Territory, Australia


4.16 is a Stable release.


Requirements
------------

  - Linux 2.2 or better
  - Glibc 2.0.x or better
  - Ncurses 2.x or better.
  - linux proc fs installed (linux 2.2.x and up)
  - sendmail
  - ppp 2.3.x configured kernel (for net traffic monitoring)
  - TCP/IP networking configured kernel

Everything has been written in C, except for clobberedit, which has been
written in C++. you will need both a C and C++ compiler installed on your
machine.


Notices
-------

- many additions in this revision. this revision replaces the last stable
revision which was 4.10.

- Again, if you are using redhat 6.1, then you must replace the ncurses
libraries, as the particular version that redhat installs has bugs in it.
grab ncurses version 5.0 off ftp.gnu.org.

- thanks to Tzvetomir Tzvetkov <ceco@net-surf.net> for providing a small
patch for clobberd (see updates).

- The complexity of clobberd now is quite high. i am doing a bit of work
on the manual as well.

- A companion distribution is on the drawing board that will contain
companion programs that use clobberd. One of these is a billing and
mailing program called ant that was written for accessnt.com.au. Also the
manual will be included in this sister distribution. see clobberd page for
more info later on.

- nif monitoring for outward traffic will have their uploads and download
volumes the wrong way. i'm trying to figure away around this, but for now,
you'll just have to put up with it.

- there is a bug (well it's not really a bug, rather a condition that
clobberd can't properly deal with) when clobberd monitors NIF's as a user
(nb/ don't confuse this will normal PPP user monitoring, as this is not
effected). in linux there is no way to distinguish between one NIF
session, and another (there is no such thing as a NIF ID, or session
counter within linux). therefore, if a NIF (ie eth0), goes up and down
fast enough so that clobberd can't detect the "down" state, then clobberd
won't detect that the NIF is a new session. the result is that volumes
will go backward, and minus values will happen. Sorry, i can't do anything
about this. I'll be having a look at the new 2.3.x code to see if there
are any developments in this area. if not, i'll email the linux
maintainers... The hack to get around this is to make sure that when a NIF
goes down, it remains down for at least the clobberd sleep time. for ppp
user monitoring, this is ok, clobberd can detect when a ppp interface goes
down and then up, because clobberd in this case monitors the user and the
NIF by the state of the utmp file. also, another hack for this is to have
very short sleep times, so that the sampling resolution is increased.


Updates since 4.10 (last public release)
----------------------------------------

not only has alot of new features been added, the code itself has been
emensely improved and modulised. i've spent alot of time making the code
elegant so it is easier to read and hence will reduce bugs. i've also
spend alot of time putting commments into the code.


- amended s_net_readln() (line termination bug)

- vulcanised routines to handle memory, net and read errors in a more
unixy style. General code improvements here and there. integrated client
net functions from clobbertool and clobberedit:  many "behind the seens"
stuff.

- vulcanised locking mechanisms (when client programs read and modify data
records via tcp). this is so that third party clients can be used with
clobberd without causing any security holes etc.

- bug (condition) spotted in tedit_us::add_user(). previously allowed you
to add white space etc. fixed.

- created a linked list and binary tree library, and a set of interface
routines for them to reduce code redundancy and bugs.  un-recursed deep
recursive routines to reduce stack load (incase of buggy kernels with
memory leaks).

- In the UM user lists, a "*" beside the user denotes that the user is
currently active. a '+' denotes that they are on more than once. A '%'
denotes an inactive NIF. A '$' denotes an active NIF (based on a request
by phoenix@accessnt.com.au).

- Removed "Email Host" field from general settings. Changed "Email
Address" in user settings, to a multifunction email entry, which can
either contain is fully qualified email address (ie
"jsno@downunder.net.au"), an address of a SMTP server (ie
"downunder.net.au"), or nothing which will mean "localhost". In the last
two cases, clobberd will email the user at the SMTP server. (change based
on a request by curl@tcc-comp.com.au).

- The General Settings, changed "Root Email Name" back to "Root Email
Address". (change based on a request by curl@tcc-comp.com.au)

- clobberedit will now display "*"'s, when you enter password (requested
by curl@tcc-comp.com.au).

- fixed small bug in us.cc tedit_us::add_user(). If you had us default set
to off, and you created a new entry, you would get un-inited fields. fixed.

- modified User settings to facilite grouping of user settings . i did
this by creating several different types of user setting entries:

  - Normal
  - Group
  - Link
  - Union
  - Effective

'Normal' is a normal User setting entry. 'Group' is a normal US entry,
accept that it is flagged group so that it can be identifed as a group
setting (for report purposes, and syntax purposes in clobberedit). a
'Link' entry is a US entry that points to a group entry. a 'Union' entry
is the same as a link accept that the union is composed of both the link
and the linked entry. An 'Effective' entry is a composed entry that is
"effective" to a given user.

When you are viewing a US user list, a '@' beside the user denotes that
the entry is a group, a '^' denotes that it's a link, a '~' denotes a
union link, and nothing denotes a normal entry.

- an extentive new logging system has been implemented.
 
    - Messages are more meaningful.
    - Time is in a more compressed format
    - Per user logging has been added
         - Logs session activity
         - log entries are generated when daily and total purges are done.

- clobberedit now has a "running statistics" window in the root screen and
  runs continiously. it provides various statistics- totals, averages, and
  a 24 hour connection tally.

This information is also logged when daily and total purges are done.

- expanded connection stats in clobberedit

- expanded boot.conf. some compile-time defs moved into boot.conf

- clobberd (and clients) will read tcp port number from /etc/services.

- created various sorted statistical lists:

    - "Total Time Used list"
    - "Total Network Volume list"
    - "Connection Efficiency list"  (users Total Time / total connections)

- expanded email facility. You now have: Normal, CC root and none. CC root
  tells clobberd to send a copy of a users email to root email (as 
  specified in general settings). (thanks to Tzvetomir Tzvetkov's
  <ceco@net-surf.net> for suggesting the CC to root idea).

- flashing isn't done by clobberd anymore. to replace it, a set of
  flashing methods have been created in boot.conf. clobberd will fork a
  child, and run one of these programs to do the job of instantly
  messaging a user. even though a universal messenger (aggreed upon by
  everyone that is multi-platform) doesn't exist yet, when it does become
  available, it will now be possible to use a command line utility to page
  people.

  at the moment, the only method available is to pass a program a
  username (intended for write). later, other methods will be available,
  like passing programs IP numbers, or ICQ numbers etc.

- you can both email a user AND flash a user now

- added extra network handshaking. if client is wrong version, then
  server will end the connection.

- added archive facility

- removed 'report' facility. it's redundant. you can write a perl script
  that reads output from userstats to perform reporting.

- turned clobbering code into a seperate program called 'killuser'.
  clobberd will fork and execute killuser to terminate users.

- turned the notifying code into a seperate program called 'clob_notify'.

- removed clobbertool, and added the following utilities:

    - clob_archive
    - clob_dpurge
    - clob_tpurge
    - clob_userstat   (no password required)
    - clob_userstats  (lists all users, or groups of users)
    - clob_list       (sorted statistical lists of users)
    - clob_gstat      (dumps running statistics)

   nb/ passwords are manditory now.

- adjusted the way a session and new users start time is inited. making it
consistant to how volume is done.

- daily and total purge functions won't purge anymore if any record lock's
  exist. if you are running them in a cron job, then they will fail if you
  are editting user data. if this happens, crontab will email you the
  failure. you will then have to run the purgers manually.. infact any of
  the command line utilities will behave exactly the same as clobberedit
  when it comes to locked data. You will just have to be careful not to be
  editting data when the cron jobs are running.

- clobberd now treats network interfaces (NIF's) as users, and monitors
  them in the same way, however....

     - NIF's are not regulated
     - no connection tallies are recorded for them
     - they will not effect the general statistics in any way.
     - no session logs are recorded for nif's
     - the only thing that is generated, is a UM record stats, a session.log,
       daily.log and total.log files. the session.log will only tell you
       when the nif went up or down.

- changed scanning method (behind the scenes change) to remove any
  possible latency problems. under new model, you can now have very short
  sleep times and hence reduce sampling lose.

- 15 second sleep time is now set as default. the lowest sleep time you
  have have now is 1 second. faster sleep times are now possible.



suggestions rejected (ideas that are out of clobberd's scope)
-------------------------------------------------------------

- a costing facility/billing facility (like freeside and ispd).

  --> reason: billing has never been in the scope of clobberd, and never
      will be as long as i'm maintaining it. if you want to do billing, 
      write a simple script that reads output from clob_userstats. it
      shouldn't be too hard to do this.

      if you want a billing facility, ant in the sister distrubution
      (clobberd-goodies.tar.gz) may interest you. ant uses statistics
      produced by clobberd to bill and mail users.


- Radius support

  --> reason: i don't have the hardware or money to do this.



Synopsis
--------

Clobberd is a daemon that runs on a single host, and monitors user
activity and network interface activity.

 - For user monitoring, clobberd monitors the utmp file (typically
   /var/run/utmp), monitors their time online, what network interfaces
   they are using (ie PPP), and how much network traffic they are
   creating. It monitors all this information, and regulates them
   according to a set of restrictions.

 - For network interface monitoring, it will simply monitor the amount of
   traffic, and the time it is active. no regulation is done.

Users are able to view their stats via a web page, and a remote
maintenance facility allows you to maintain user data from anywhere on
your network.

For a detailed description of clobberd, please refer to the manual or the
web site: www.downunder.net.au/~jsno/clobberd.

4.x has been written for my personal use only. However, i have released it
under the GNU General Public License for anybody to use, and is now
distributed on an "AS IS" basis.


Installation
------------

- If you are currently running 4.10, then backup your files. from 4.9
  upwards, the clobberd.db data file format is backward compatable. you
  can retain this data file, and use it in this version. for any versions
  before 4.9, you will have to convert them.

- extract distribution.

- run 'Uninstall.old_version' as root. This will remove any old 4.x
  clobberd files that are on your machine. If you are running a very old
  version like 3.2 or 2.8, then you will probably have look around your fs
  and check the /etc, /usr/sbin paths as well.

- Edit directives in the make.config file-

    - comment out the "development" directives.
    - uncomment "release" directives
    - check that those paths exist, if they don't either adjust the paths 
      for your given Un*x flavour, or create the paths.

- make the distribution- type in "make".

- If there's no compilation problems, install it- "make install". Nb/ make
  sure you are root user and this point.

- Go into the /var/local/clobberd file, and edit the boot.conf file where
  appropriate (see manual for directive meanings). Once done, make sure
  that this file has a privillege 0600 at all times, so that your users
  can not view your password.

- You may like to customise the m_*.txt files in /var/local/clobberd/mesg.

- Add the following network service entry to /etc/services-

      clobberd        303/tcp         clobberd

if you have a service that is already using 303, then change it to 304.

- Put "/usr/local/sbin/clobberd in your start up script. This can vary
  emensely. On the redhat bundled machine i am using, I put clobberd at
  the end of my /etc/rc.d/rc.sysinit file-

  ---$--------------------
  # clobberd
  #
  echo -n "Starting up clobberd..."
  /usr/sbin/local/clobberd &
  ---$--------------------


  The next step is to place the following entry in the shutdown script. In
  my case, it's the /etc/rc.d/init.d/halt file.

  ---$--------------------
  # clobberd
  #
  echo "Shutting down clobberd.."
  killall -15 clobberd
  sleep 3
  ---$--------------------

  this is very important, as it will do two things-

   - shuts down clobberd nicely
   - doing this produces a nice log entry in all log files.


- Edit the root crontab and put the following entries in-

  ---$--------------------
  # daily purge 00:00am
  #
  0 0 * * * root /usr/local/sbin/clob_dpurge -p <passwd>

  # total purge on the 1st of each month at 00:00am
  #
  0 0 1 * * root /usr/local/sbin/clob_tpurge -p <passwd>

  # auto-archive on the 1st of each month at 01:00am
  #
  0 1 1 * * root /usr/local/sbin/clob_archive -p <passwd>
  ---$--------------------

  the first entry is mandatory. the daily purge entry for example purges
  the daily monitors for each user and produces a set of logs. The second
  entry (clob_tpurge) you can either choose to do manuallly, or have
  cron run it. The third entry (clob_archive) is completely optional.

  Important: when running within cron, never never never run all the above
  at the same time. have their scheduling seperated by at least 1-2
  minutes.

  Nb/ Purging functions are very flexable. Daily time can be modified to
  mean "weekly time" etc. All you need to do is tailor the appropriate
  message files (/var/lib/clobberd/m_*.txt), and set purge programs to
  run in the cron jobs at your choosen times.

- notify your users to goto http://<clobberd server>/cgi-bin/clob_userstat.cgi
  they will be able to access their stats.



Contact
-------

   Snail: Jason Nunn
          32 Rothdale Rd
          Moil NT 0810
          Australia

   Email: jsno@downunder.net.au or jsno@accessnt.com.au

     WWW: www.accessnt.com.au/~jsno/clobberd

listserv: majordomo@tcc-comp.com.au
          (send "subscribe clobberd" in the message body.
           IMPORTANT: do not send email to this address
           without subscribing to it first)


                                   ooo0ooo
