Chapter 20

System Accounting (Reference)

This chapter provides reference information about system accounting.

This is a list of reference information in this chapter.

The `runacct` Script

The main daily accounting script, runacct, is normally invoked by the cron command outside of prime business hours. The runacct script processes connect, fee, disk, and process accounting files. This script also prepares daily and cumulative summary files for use by the prdaily and monacct scripts for billing purposes.

The runacct script takes care not to damage files if errors occur. A series of protection mechanisms are used to recognize an error, provide intelligent diagnostics, and complete processing in such a way that the runacct script can be restarted with minimal intervention. It records its progress by writing descriptive messages into the active file. Files used by the runacct script are assumed to be in the /var/adm/acct/nite directory, unless otherwise noted. All diagnostic output during the execution of the runacct script is written into the fd2log file.

When the runacct script is invoked, it creates the lock and lock1 files. These files are used to prevent simultaneous execution of runacct. The runacct program prints an error message if these files exist when it is invoked. The lastdate file contains the month and day the runacct script was last invoked, and is used to prevent more than one execution per day. If the runacct script detects an error, a message is written to the console, mail is sent to root and adm, locks might be removed, diagnostic files are saved, and execution is ended. For instructions on how to start the runacct script again, see "How to Restart the runacct Script".

To allow the runacct script to be restarted, processing is broken down into separate re-entrant states. The statefile file is used to keep track of the last state completed. When each state is completed, the statefile file is updated to reflect the next state. After processing for the state is complete, the statefile file is read and the next state is processed. When the runacct script reaches the CLEANUP state, it removes the locks and ends. States are executed as shown in the following table.

Table 20-1 runacct States

State	Description
`SETUP`	The `turnacct` `switch` command is executed to create a new `pacct` file. The `/var/adm/pacct`n process accounting files (except for the `pacct` file) are moved to the `/var/adm/Spacct`n.MMDD files. The `/var/adm/wtmpx` file is moved to the `/var/adm/acct/nite/wtmp.`MMDD file (with the current time record added on the end) and a new `/var/adm/wtmp` file is created. The `closewtmp` and `utmp2wtmp` programs add records to the `wtmp`.MMDD file and the new `wtmpx` file to account for users currently logged in.
`WTMPFIX`	The `wtmpfix` program checks the `wtmp.`MMDD file in the `nite` directory for accuracy. Because some date changes cause the `acctcon` program to fail, the `wtmpfix` program attempts to adjust the time stamps in the `wtmpx` file if a record of a date change appears. This program also deletes any corrupted entries from the `wtmpx` file. The fixed version of the `wtmp.`MMDD file is written to the `tmpwtmp` file.
`CONNECT`	The `acctcon` program is used to record connect accounting records in the file `ctacct.`MMDD. These records are in `tacct.h` format. In addition, `acctcon` creates the `lineuse` and `reboots` files. The `reboots` file records all the boot records found in the `wtmpx` file.
`PROCESS`	The `acctprc` program is used to convert the `/var/adm/Spacct`n.MMDD process accounting files into total accounting records in the `ptacct`n.MMDD files. The `Spacct` and `ptacct` files are correlated by number so that if the `runacct` script fails, the `Spacct` files are not processed.
`MERGE`	The `acctmerg` program merges the process accounting records with the connect accounting records to form the `daytacct` file.
`FEES`	The `acctmerg` program merges ASCII `tacct` records from the `fee` file into the `daytacct` file.
`DISK`	If the `dodisk` script procedure has been run, which produces the `disktacct` file, the `DISK` program merges the file into the `daytacct` file and moves the `disktacct` file to the `/tmp/disktacct.`MMDD file.
`MERGETACCT`	The `acctmerg` program merges the `daytacct` file with the `sum/tacct` file, the cumulative total accounting file. Each day, the `daytacct` file is saved in the `sum/tacct.`MMDD, file so that the `sum/tacct` file can be re-created if it is corrupted or lost.
`CMS`	The `acctcms` program is run several times. This program is first run to generate the command summary by using the `Spacct`n files and write the data to the `sum/daycms` file. The `acctcms` program is then run to merge the `sum/daycms` file with the `sum/cms` cumulative command summary file. Finally, the `acctcms` program is run to produce `nite/daycms` and `nite/cms`, the ASCII command summary files from the `sum/daycms` and `sum/cms` files, respectively. The `lastlogin` program is used to create the `/var/adm/acct/sum/loginlog` log file, the report of when each user last logged in. If the `runacct` script is run after midnight, the dates showing the time last logged in by some users will be incorrect by one day.
`USEREXIT`	Any installation-dependent (local) accounting program can be included at this point. The `runacct` script expects it to be called the `/usr/lib/acct/runacct.local` program.
`CLEANUP`	Cleans up temporary files, runs the `prdaily` script and saves its output in the `sum/rpt.`MMDD file, removes the locks, and then exits.

Caution - When restarting the runacct script in the CLEANUP state, remove the last ptacct file because it will not be complete.

Daily Accounting Reports

The runacct shell script generates five basic reports upon each invocation. The following table describes the five basic reports generated.

Table 20-2 Daily Accounting Reports

Report Type	Description
"Daily Report"	Shows terminal line utilization by `tty` number.
"Daily Usage Report "	Indicates usage of system resources by users (listed in order of user ID).
"Daily Command Summary "	Indicates usage of system resources by commands, listed in descending order of memory use. In other words, the command that used the most memory is listed first. This same information is reported for the month with the monthly command summary.
"Monthly Command Summary "	A cumulative summary that reflects the data accumulated since the last invocation of the `monacct` program.
"Last Login Report "	Shows the last time each user logged in (arranged in chronological order).

Daily Report

This report gives information about each terminal line used. The following is a sample Daily Report.

Oct 16 02:30 2001  DAILY REPORT FOR venus Page 1


from Mon Oct 15 02:30:02 2001
to   Tue Oct 16 02:30:01 2001
1       runacct
1       acctcon

TOTAL DURATION IS 1440 MINUTES
LINE         MINUTES  PERCENT  # SESS  # ON  # OFF
console      868      60       1       1     2
TOTALS       868      --       1       1     2

The from and to lines specify the time period reflected in the report. This time period covers the time the last Daily Report was generated to the time the current Daily Report was generated. Then, comes a log of system reboots, shutdowns, power failure recoveries, and any other record dumped into the /var/adm/wtmpx file by the acctwtmp program. For more information, see acct(1M).

The second part of the report is a breakdown of terminal line utilization. The TOTAL DURATION tells how long the system was in multiuser mode (accessible through the terminal lines). The following table describes the data provided in the Daily Report.

Table 20-3 Daily Report Data

Column	Description
`LINE`	The terminal line or access port.
`MINUTES`	The total number of minutes that the line was in use during the accounting period.
`PERCENT`	The `TOTAL` `DURATION` divided by the total number of `MINUTES`.
`# SESS`	The number of times this line or port was accessed for a login session.
`# ON`	Same as `SESS`. (This column no longer has meaning. Previously, it listed the number of times that a line or port was used to log in a user.)
`# OFF`	This column reflects the number of times a user logs out and any interrupts that occur on that line. Generally, interrupts occur on a port when `ttymon` is first invoked after the system is brought to multiuser mode. If the `# OFF` exceeds the `# SESS` by a large factor, the multiplexer, modem, or cable is probably going bad, or there is a bad connection somewhere. The most common cause is an unconnected cable dangling from the multiplexer.

During real time, you should monitor the /var/adm/wtmpx file because it is the file from which the connect accounting is derived. If the wtmpx file grows rapidly, execute the following command to see which tty line is the noisiest.

# /usr/lib/acct/acctcon -l file < /var/adm/wtmpx

If interruption is occurring frequently, general system performance will be affected. Additionally, the wtmp file might become corrupted. To correct this problem, see "How to Fix a Corrupted wtmpx File".

Daily Usage Report

The Daily Usage Report gives a breakdown of system resource utilization by user. A sample of this report follows.

Oct 16 02:30 2001  DAILY USAGE REPORT FOR skisun Page 1


     LOGIN  CPU  (MINS)  KCORE-   MINS    CONNECT  (MINS) DISK   # OF   # OF  # DISK  FEE
UID  NAME   PRIME NPRIME PRIME    NPRIME  PRIME    NPRIME BLOCKS PROCS  SESS  SAMPLES
0    TOTAL  72    148    11006173 51168   26230634 57792  539    330    0     2150    1
0    root   32    76     11006164 33664   26230616 22784  0      0      0     127     0
4    adm    0     0      22       51      0        0      0      420    0     0       0
101  rimmer 39    72     894385   1766020 539      330    0      1603   1     0       0

The following table describes the data provided in the Daily Usage Report.

Table 20-4 Daily Usage Report Data

Column	Description
`UID`	User ID number.
`LOGIN NAME`	Login (or user) name of the user. Identifies a user who has multiple login names.
`CPU (MINS)`	Amount of time, in minutes, that the user's process used the central processing unit. Divided into `PRIME` and `NPRIME` (non-prime) utilization. The accounting system's version of this data is located in the `/etc/acct/holidays` file.
`KCORE-MINS`	A cumulative measure of the amount of memory in Kbyte segments per minute that a process uses while running. Divided into `PRIME` and `NPRIME` utilization.
`CONNECT (MINS)`	Amount of time in minutes, that the a user was logged into the system, or "real time." Divided into `PRIME` and `NPRIME` use. If these numbers are high while the `# OF PROCS` is low, you can conclude that the user logs in first thing in the morning and hardly touches the terminal the rest of the day.
`DISK BLOCKS`	Output from the `acctdusg` program, which runs the disk accounting programs and merges the accounting records (`daytacct`). For accounting purposes, a block is 512 bytes.
`# OF PROCS`	Number of processes invoked by the user. If large numbers appear, a user might have a shell procedure that has run out of control.
`# OF SESS`	Number of times a user logged on to the system.
`# DISK SAMPLES`	Number of times disk accounting was run to obtain the average number of `DISK BLOCKS`.
`FEE`	Often unused field that represents the total accumulation of units charged against the user by the `chargefee` script.

Daily Command Summary

The Daily Command Summary report shows the system resource use by command. With this report, you can identify the most heavily used commands and, based on how those commands use system resources, gain insight on how best to tune the system.

These reports are sorted by TOTAL KCOREMIN, which is an arbitrary gauge but often a good one for calculating drain on a system.

A sample daily command summary follows.