autolog
make FITS file logs

Version 1.2 (2000 February 20)

autolog prints a formatted synopsis of the contents of a FITS header based on a format file containing a list of header keywords to extract, the format to use, and a heading. The output is a single line of text for each file on the command line piped to stdout.

The output format is designed to facilitate compiling a table of the contents of a large number of FITS files. autolog can be used as-is or with a wrapper script. For example, autolog is used by the autologger wrapper script run by the caliban transfer-disk daemon to generate a running observing log of all files written by the OSU data-taking system during a night.

Index:

Usage:

  autolog [-H] [-Ffmtfile] [-s] fits1 fits2 ...
where:
-H
do not print a header. By default, a header is printed as the first line of output. This option is usually used if autolog is appending to an existing file and you do not not wish to have redundant headers in that file.

-Ffmtfile
Use fmtfile as the format file. By default, it will use the autolog.fmt file in the current working directory. [unless you change fmtfile in autolog.h to point to a global format file]. See Format Files below.

-s
Work silently, suppressing all FITS error messages. This is useful if you are running autolog on a wildcard (*) file list, and want to ignore errors generated by stray non-FITS files that might get scanned. The default behavior is to print all error messages. [This feature is new to version 1.2]

fits1 fits2 ...
the names of the FITS files to be logged. autolog will accept wildcard entries for the filenames, thus 
           autolog *.fits
will print a formatted log of all *.fits files in the current working directory to stdout.

[Menu]
Usage Examples:

Create a log file named "night01.log" using all *.fits file in the directory /home4/ritchey/data/, using format file nt01.fmt for the output format. 

        % autolog -Fnt01.fmt *.fits >night01.log
Create a log file "foo.log" for all files foo.001 through foo.029 in the current working directory. Use the format file foo.fmt. 
        % autolog -Ffoo.fmt foo.0[0-2][0-9] >foo.log
Later, append entries for files foo.030 through foo.061 to foo.log, but suppress printing the header over again: 
        % autolog -H -Ffoo.fmt foo.0[3-5][0-9] >>foo.log
        % autolog -H -Ffoo.fmt foo.06[0-1] >>foo.log
The formatting of format files is described below.

[Menu]

Format Files:

The format of the output logfile table is specified using the format file. A template, autolog.fmt, is provided to give you a start.

#: Add a comment to a format file

   Syntax: # this is a comment...

   The hash character (#) is used as a comment character ONLY if used as the
   first character in a line of the file (autolog uses a relatively crude
   parser).
[Menu]

DELIMITER: Define the table field-delimiter character

   Syntax: DELIMITER   

   DELIMITER specifies the field delimiter character to print between fields
   in a record.  It is either a single printable ASCII character, or one of
   the following special definitions for non-printing and/or potentially
   confusing delimiter characters:

        DELIMITER SPACE or DELIMITER SPC
            Print a space character between fields

        DELIMITER TAB
            Print a TAB character between fields

        DELIMITER CSV
            Print a comma (,) between fields.  This makes the output
            a Comma-Separated-Variable (csv) file, suitable for reading
            with common spreadsheet programs, like Excel or Lotus.

   A common example is "DELIMITER |", which puts a bar (|) between fields.
[Menu]

LOGTIME: Timestamp log entries.

   Syntax: LOGTIME [LOC|UT]

   LOGTIME will write a timestamp (hh:mm:ss) in the last record of each log
   entry, recording when the log entry was made.  

        LOGTIME LOC or LOGTIME
            The timestamp is to be the local time (e.g., EST)
            LOC is the default time system.

	LOGTIME UT
            The timestamp is to be in UTC time,

   LOGTIME is optional, and if omitted from the format file, no timestamp
   will be written.
[Menu]

Header Field Keywords

   Syntax: KEYWORD keyfmt Heading 

   Where:
        KEYWORD = FITS header keyword.  Must be exact.
        keyfmt  = C-style format spec for the output field 
                  (e.g., %6.1f, %-10.10s)
        Heading = text string (no spaces) for the field header to be
                  printed at the top of the output.
Fields will appear in the the output records in the order they are given in the format file, even though the keywords in the FITS headers proper may appear in a different order.

Blank or absent header keywords will result in spaces (if string values) or 0 [zero] in the appropriate numerical format (if a numerical value) appearing in the respective fields as place-holders.

Fields may be turned off by putting a # (comment character) in front of them to make them into a comment. This way when you want to turn it back on, you don't have to mess around trying to remember what the format was before...

The allowed format codes are as follows, only the main C format codes are supported: 

   code    type         example    description
  ---------------------------------------------------------------------
    s      string       %-25.25s   25 character string, left justified
                        %10.10s    10 character string, right justified
    i      integer      %2i        2-digit integer
    f      float        %7.1f      7-digit floating number, 1 digit
                                   precision (e.g., 2435.1)
    e      exponential  %9.2e      6-digit exponential, 2 digits
                                   precision (e.g., -1.23e-15
  ---------------------------------------------------------------------
In all cases, the first number (left of the .) gives the width of the field. This will be taken to be the width of the field for the header string as well. Header strings longer than this size are simply truncated.

[Menu]

Example Format File:

        OBJECT   %-25.25s  Object
        RA       %-10.10s  RA
        DEC      %-9.9s    Dec
        EQUINOX  %7.1f     Equinox
        EXPTIME  %7.1f     ExpTime
        ST       %-10.10s  LST
        DATE-OBS %-10.10s  Date
        UT       %-10.10s  UTC
        HJD      %9.1f     HJD
        OBSERVAT %-12.12s  Observatory
        FILENAME %-15.15s  Filename
Note that any number of spaces may appear between the three arguments, but *no spaces* may appear within a given argument. For example: 
        OBJECT  %-25.25S   Object Name
will result in the string "Object" appearing as the header over the column with the contents of the OBJECT header card, with "Name" being ignored. The parser is very simple-minded.

[Menu]

Requirements:

At least CFITISO version 1.40 or later is needed to compile autolog (see COMPILING below).

Notice

As of 1998 Nov 19, we strongly recommend that you use version 2.0 or later of CFITSIO, as this will provide support for Year2000-compliant FITS date formats. These formats will be the default for all OSU-built instruments (MOSAIC/TIFKAM, ANDICAM, DANDICAM, OSIRIS, and CCDS) starting 1999 January 1.

At runtime, you need either an autolog.fmt file in the current working directory, or a format file specified with the -F flag. See the template .fmt file (autolog.fmt) that came with the source for a heavily annotated example.

[Menu]

Errors:

If the specified source file is either not a FITS file, or if it is a corrupted FITS file, autolog will print an error message to stderr, but not print anything else for that file. You can suppress these messages using the "-s" flag.

All autolog error messages print the name of the errant file and diagnostics as to what caused the error. See the CFITSIO manual for details of error codes (available online at HEASARC).

[Menu]

Bugs:

Old versions of the CFITSIO routines (v2.027 and earlier) are very intolerant of minor FITS header formatting errors in optional keywords. For example, if a character argument in a card omits the closing quote ('), it produces the error message: 

     Error: status=205: string missing closing quote
      This keyword string value has no closing quote:
      INSTRUME= 'CTIO OSIRIS          /
      Failed to find the END keyword in header (ffgphd).
      Failed to read the required primary array header keywords.
      ffopen could not interpret primary array header of file:
      foo.fits
This problem was fixed in CFITSIO v 2.028 (1999 Jan 27), so the "fix" is to install the latest CFITSIO routines and recompile autolog.

The old bug whereby any FITS error would cause the cfitsio routines to report false errors for all subsequent bona-fide FITS files in a wildcard list has been fixed in recent versions of cfitsio.

There are probably lots and lots more bugs lurking in the works. This tool is pretty rude & crude. Send all bug reports to Rick Pogge (pogge@astronomy.ohio-state.edu).

[Menu]

Compiling:

Must be compiled with the HEASARC cfitsio libraries, v1.40 or later. We recommend, however, that you load the latest version of the cfitsio libraries (currently 2.036, 2000 Feb 4): 

    gcc -o autolog autolog.c -I/home/cfitsio -L/home/cfitsio -lcfitsio
where 
    /home/cfitsio/ is where the fitsio.h and libcfitsio.a files live
The gccfits script is provided with the source code as an example.

On Solaris systems, you also have to include the socket and nsl libraries explicitly, thus 

    gcc -o autolog autolog.c -I/home/cfitsio -L/home/cfitsio -lcfitsio 
        -lsocket -lnsl
FITSIO is freely available and well supported and documented, has has emerged as the benchmark FITS generation/reading package. It is senseless to use anything else to create or read FITS format data files. See the FITSIO website
http://heasarc.gsfc.nasa.gov/docs/software/fitsio/.
for the latest version of cfitsio and the associated documentation.

The most recent version of CFITSIO that we have used with this code is v2.036 (2000 Feb 4).

[Menu]

Downloading autolog:

The source code for autolog is available as a gzip-compressed tar archive. The latest version is 1.2 (2000 Feb 20):

Download (19Kb)
See the Installation instructions or the 00README file provided with the ftp distribution for details.

[Menu]

Author:

Richard Pogge
The Ohio State University Department of Astronomy
pogge@astronomy.ohio-state.edu
Last Update: 2000 Feb 20

[Menu]

Updated: 2000 Feb 20 [rwp]