YALO Telescope
Observation Template (Obs) Files

Updated: 2000 September 11

Table of Contents:


Overview

The ANDICAM is a 2-channel instrument with separately configurable CCD and IR cameras. To enable efficient queue scheduling of observations, all data with ANDICAM are acquired by means of Observation Template Files (or "obs files" for short). Obs files define the instrument, detector, and target parameters and configurations needed to acquire a single observation. In this sense, an obs file represents the smallest schedulable "unit observation". For example, to take a single long CCD image while simultaneously taking a set of short IR integrations dithered between each IR image using the internal dithering mirror, a single obs file is created that describes the observing "sequence" to be followed by each channel of the instrument. The CCD and IR data-taking sequences described by the obs file are then executed in parallel by the Prospero data-taking program.

Each obs file consists of three basic blocks:

Target Block:
Contains the target name, coordinates, and project ID information.

CCD Block:
Contains the CCD observing parameters (filter, exposure time, number of sequential images, binning factor, etc.).

IR Block:
Contains the IR observing parameters (filter, exposure time, number of co-adds and sequential images, dithering parameters, etc.).

While obs files have been primarily designed for efficient simultaneous CCD and IR imaging, they also provide a consistent and logical way to schedule single-channel observations (i.e., CCD-only or IR-only imaging) with ANDICAM. This allows us to simplify the suite of commands needed to take data in each of the ANDICAM's 3 modes, which in turn simplifies the work of the on-site observers and the queue manager. It also ensures that the unused detector channel is kept idle while the other is active.

A "unit observation" means that the choice of filters and the base integration times for each channel are fixed, but multiple images with those filters and integration times may be acquired. Similarly, IR channel observations can be dithered between images using internal tip/tilt mirror. If observations with different filters or exposure times are required, the astronomer must create separate unit observations to build up the data set, each unit having its own obs file associated with it. Some examples are described below.

Obs files can be generated using the web form described in these pages, or by using interactive commands in the newest version of Prospero (v4.5 or later) at the telescope. In principle, you could create obs files by hand with an editor, but the format is very strict and any typing errors could cause your observation to fail. This help file describes how to use the web forms for creation, management, and submission of obs files for your program.

Once you have created all of the obs files for your program, you need to submit them along with detailed observing instructions to the queue manager for evaluation, scheduling, and implementation. This is done with a separate project submission form. See the Phase II Observing Program Electronic Submission Instructions for details.

[Contents]


How to Create Obs Files

Obs files are created in three steps using a multi-part web form. This form is accessed by logging into the Observing Preparation Tools pages, using the Project ID assigned to your project when it was approved for implementation and scheduling.

Step 1: Enter Target Information

The first step in creating a set of obs files for a target is to specify the information for the target proper:

  1. The name of the target.
  2. The RA and Dec of the target, and the equinox of the coordinates.
  3. Indicate whether the target is a solar system object (i.e., a moving target requiring ephemerides or other time-dependent sources of coordinates).
  4. The imaging mode you wish to use:
    1. CCD-only (no IR imaging).
    2. IR-only (no CCD imaging).
    3. Dual CCD+IR Imaging (simultaneous imaging in both channels)
Once you have entered this information, hitting the "Next>" button on the form will load the exposure parameter entry form.


Step 2: Enter Exposure Parameters

After entering the target info, you will be given a blank form for entering the exposure parameters for the observations. As described above, obs files are used to perform unit observations with a given instrument configuration. These units can be combined into multi-part imaging "sequences" using Prospero scripts that execute each of the separate obs files in turn.

For example, to take BVI images of an object, you would create three separate obs files, one for each filter (B, V, and I). A filter change is an "instrument configuration change". You would then have the on-site observers execute the obs files one at a time by hand, or provide them with a Prospero script to execute them in a sequence (thus reducing three execution steps into one: executing the script). For programs that require a number of obs files executed in a particular sequence, chaining them together into scripts will enhance the efficiency of the observing (and eliminate the possibility of operator mistakes).

Once you have entered the exposure parameters for a given configuration, hitting the "Next>" button will then validate and analyze the entries.


Step 3: Validate Entries & Save the Obs File

You will next be shown a summary of the observation parameters you entered on the previous form along with an execution time analysis. At this point you need to review the parameters for errors. The project PI's are responsible for making sure that everything is correct before submission. You can use the browser's "Back" button to return to the exposure parameter entry form, make changes, and iterate until everything is ready before submitting the final obs file.

Execution Time Analysis

The exposure parameters you entered on the previous form are analyzed to make an estimate of the total amount of time that will be required to execute the requested exposures sequence. This estimate includes the unit integration times in each channel, number of images to be acquired, detector reset and readout times, IR mirror dithering motion overheads, etc., as appropriate. In general, the estimates are good to about 5% or so. The estimates do not, however, include instrument configuration overheads (e.g., changing filters or entering user parameters like the object name and coordinates), telescope configuration, etc. These latter will add to the total time it takes to execute your observation program. In general, instrument-related overheads add only ~10 seconds for each target/instrument change.

The estimates provided by the execution time analysis are meant to serve two purposes:

  1. To aid the proposer in optimizing 2-channel observation to minimize dead-time.
  2. To provide guidance to the queue manager in scheduling the observations.
From the proposer's perspective, the first provides a way to make sure that time that could be spent collecting more photons is not wasted because the CCD and IR operations are mismatched. For example, you might find that your first choices of integration times leave 64s of "dead time" on the IR channel while the CCD is still integrating and reading out. The web tools provided allow you to assess the impact is of adding an extra 60-second IR image to the obs files to close the gap. The web tools have been designed to make this kind of optimization exercise straightforward.

If you need to adjust any of the exposure parameters, use the "Back" button on your browser to return to the exposure parameter entry form in Step 2, make the changes, and then repeat the validation step. You can iterate as often as you like until you get things the way you want them. This way you avoid making lots of intermediate files that pile up on the disk (you can always clean up later with the Obs File Manager, but why make extra work?

It bears repeating that the estimates of execution times do not include such factors as the time required to setup the observation (e.g., setting the filters, exposure times, object names, etc.), nor do they include the computer-to-disk data-transfer times. They only provide estimates of the time it takes to perform those operations that occur from the moment the first exposure in the sequence begins until the last image is readout. All of the other configuration and data-transfer overheads cannot be predicted with any precision, the former because it depends critically on the previous configurtion of the instrument, and the latter because the inter-machine data-transfer system has a roughly factor of 4 range of transfer speeds (roughly 1Mb/sec to 4Mb/sec) governed by such ineffibles as the disk caching, how busy the up/downstream machines are, etc. As such, the queue manager has to empirically estimate any additional overheads based on past experience, and the adjust the queue schedule accordingly on subsequent nights based on how long it takes the observers to execute the program in practice (e.g., using the times reported by the nightly data logs).

Telescope pointing, target acquition, and guide-star acquisition/lock times are not known a priori, and largely treated as overhead absorbed by the observatory operations (everybody pays more or less equally averaged over time). However, if your program requires a lot of additional operations, e.g., many offsets around the target position between observations, it will cost you in terms of total execution time. If your program proves excessively costly in additional overhead, the queue manager will contact you regarding how to modify your program to make it more efficient. Difficult programs are difficult to schedule and execute. Keeping it simple is a virtue.

Name it & Save it

If everything is satisfactory, you are then asked to give the obs file a name. Each project is assigned a working directory for its obs files. The filenames of obs files should be simple and reflect as much as possible the contents of the file. For example, if the obs file is to acquire simultaneous V and H band images of a star cluster named NGC007, you might name the obs file ngc007vh. Choosing simple, unique, descriptive names will help the on-site observers execute your program. If you make complicated names that are hard to type, it will slow down the process; please keep it simple (the filenames are ultimately for them, not you).

After selecting a name, save the final obs file to your project's working directory by hitting the "Save Obs File" button at the bottom of the form. The obs file is stored on disk where you can view it later with the Obs File Manager (which also provides tools for renaming and deleting files). You will also be shown a copy of the final, saved obs file on your browser screen. We recommend that you print out your browser screen with the final version for your records.


Notes:

Safe Saving

To protect existing obs files for your project from accidental deletion, you are prevented from overwriting existing obs files with the form. If you try, the form will warn you and ask you to either (a) provide another, unique name and resubmit the form for processing, or (b) to go to the Obs File Manager and either delete or rename the existing file, and then resubmit the form again. Careful use of the "back" button on the browser lets you do this simply.

Making more than one obs file for a target

The multi-step process is designed to allow multiple iterations between any sets of forms, provided you take care to use the browser Back button and the various form buttons carefully. The target information (name and coordinates) are If you reload the form by following a link, bookmark, or other means, you will be given a blank form. By now simple web forms are familiar to most users, so this needs little detailed explanation.

Trouble? How to Get Help

No system is perfect. If you have problems using these forms, especially during the check-out period, contact Rick Pogge at Ohio State (pogge@astronomy.ohio-state.edu) and describe your problem, the time it occurred, and any other info you think might help him debug things. The more feedback the better at this stage.

[Contents]


Obs File Creation Form Entries

The following are descriptions of the entries in the obs file creation forms. These entries are accessible from within the forms themselves by clicking on the highlighted item. If you have come to this page from the web form, remember use the "Back" button on your browser to continue filling out the form in progress.

Project Information (login)

On logging into the YALO Observing Preparation Tools pages, you need to provide your name and Project ID.

Your Name

To keep track of who created the obs files for a project, we ask you to enter your name in the box provided.

Project ID

Project ID codes are assigned by the YALO scheduling committee when time is awarded on YALO. Select your project ID from among the entries in the pull-down menu. If you don't know your ID number, or don't find it in the menu, contact the current queue manager or NOAO program coordinator (as appropriate) immediately.

[Contents]


Step 1: Enter Target Information:

The first form asks for the target name and coordinates, and the imaging mode to be used. Project PIs (or their CoIs) are responsible for entering the target name and coordinates correctly. Neither the queue manager nor the on-site observers will enter, validate or correct target names or coordinates. Although if there are problems with your coordinates, the queue manager will be in touch.

Target Name

Enter the target name of the object you wish to observe. This is how the target name will appear in all observing logs and FITS image headers (the OBJECT keyword). The target name you enter in step 1 is carried forward through all subsequent obs file creation steps.

You can enter up target names up to 60 characters long (including spaces), but only the first 16 characters will appear in the observing logs (all 60 will appear in the image FITS headers).

Solar System Objects

If your target is a solar system object (i.e., a moving target), be sure to check the "Solar System Object" box. You will also need to provide both approximate coordinates for the object, as well as arranging to send the queue manager ephemerides for minor bodies (asteroids and moons) separately from this form.

Target RA, Dec, & Equinox

Enter the Right Ascension, Declination, and Equinox of the target. RA should be in hh:mm:ss.s format, and Declination in dd:mm:ss format. The sign of the declination goes in the first place, thus:
    -00:00:32
    +12:12:34
etc. are valid declinations. Please do not enter decimal coordinates.

The default equinox is J2000.0 coordinates, and we would prefer that you precess coordinates to J2000.0 before entering them here. If you cannot precess the coordinates, enter the Equinox in the box provided (e.g., 1950.0 for B1950.0 coordinates; omit the "J" or "B").

If your target is a solar system object, enter approximate coordinates for the observing season to assist in queue scheduling, and remember to check the "Solar System Object" box next to the Target Name. All of the planets are already in the YALO telescope controller system, but for minor planets or planetary moons you will have to provide the queue manager with an accurate ephemeris. Contact the queue manager for instructions on how to submit ephemerides for solar system targets.

Imaging Mode

Select the imaging mode for the observations. This is one of:

You may change the imaging mode used for a given target by returning to Step 1 with the Back button on your browser.

[Contents]


Enter Exposure Parameters

The second form asks you to enter the CCD and/or IR imaging exposure parameters for your target for the imaging mode selected. This includes filter selection, and setting up the dithering parameters for IR imaging.

CCD Imaging Parameters:

CCD Filter

Select a filter from the list provided. Only one CCD filter can be specified in a given obs file. To make a series of observations through more than one CCD filter, you need to create a separate obs file for each filter.

CCD Base Integration Time

Enter the integration time, in seconds, of a single CCD image. Integration times up to 1800s are allowed. You can integrate for longer than 1800s, but cosmic ray contamination becomes severe. There is no default integration time, and this field may not be left blank.

An integration time of 0 seconds will result in a "bias" or "zero" frame being acquired. If you do not wish to take CCD images with this obs file, back up to the target info form and select "IR-only Mode" as the Imaging Mode.

Number of CCD Images

Select the total number of CCD images to be acquired. Each CCD image will have the base integration time entered above. The total integration time will be the base CCD integration time multiplied by the number of CCD images. Default is 1 image. The total CCD integration time will be the number of CCD images times the base integration time.

NOTE: you cannot select 0 CCD images. If you do not wish to take CCD images with this obs file, back up to the target info form and select "IR-only Mode" as the Imaging Mode.

CCD Binning

Select the on-chip CCD binning factor to use for these images. You may bin the CCD by 1x1, 2x2, or 4x4 pixels.

IR Imaging Parameters:

IR Filter

Select a filter from the list provided. Only one IR filter can be specified in a given obs file. To make a set of observations through more than one IR filter, you need to create a separate obs file for each filter.

IR Base Integration Time

Enter the integration time, in seconds, of a single IR array image. Times up to 1800s are allowed, but the background will typically saturate in a few minutes, especially at K. There is no default integration time, and this field may not be left blank.

The minimum IR detector integration time is 4 seconds. If you enter a base integration time between 0 and 4 seconds, you will get at 4 seconds of integration anyway. Above 4 seconds, the requested integration time will be achieved. See the ANDICAM manual for an explanation.

Number of IR Images

Select the total number of IR images to be acquired. Each IR image will have the base integration time entered above. Default is 1 image. The total effective IR integration time will be the number of IR images times the base integration time.

NOTE: You cannot select 0 IR images. If you do not wish to take IR images with this obs file, back up to the target info form and select "CCD-only Mode" as the Imaging Mode.

Number of CoAdds per IR image

Each IR image can be composed of a number of separate images averaged together ("co-added") in the detector control computer before being stored as a single image file on disk. Each IR CoAdd will have the base integration time entered above. For example, if you specify 5 IR images of 3 co-adds each, the instrument will acquire 15 images of the base integration time, averaging them in groups of 3, and storing them as 5 separate FITS files. Default is 1 Co-add/image.

In general, co-adding is most useful when a target is sufficiently bright that you are restricted to very short integration times to avoid saturation, and so would need a very large number of single images to build up sufficient signal. Co-adding helps build up signal while producing a reasonable number of final image files, and with the additional benefit of a slightly reduced overhead penalty compared to taking a large number of single images. The cumulative readout noise penalty, however, is the same regardless.

Dithering:

An internal tip-tilt mirror in the IR channel beam can be used to make small "dithering" offsets between IR images. Dithering is the standard way to reduce the effects of bad pixels and flat-field artifacts on the IR array by shifting and combining multiple images taken at slightly offset positions on the array. No dithering is done between IR Co-Adds.

To dither between images, check the "Dither between images" box, and select scale the dithering pattern ("dither scale") from the pull-down menu. The default dither scale is "10", which corresponds to a dither throw of approximately 5 arcseconds on the YALO 1-meter. Dither scales range from 10 to 100 in steps of 10 (5-50 arcsec in steps of 5 arcsec).

The IR tip/tilt mirror dithers the image around the IR detector in a fixed hexagonal offset pattern. There are 7 dither positions: the first (1) centered on the array, and the following 6 (2-7) arranged in a lop-sided hexagon, stepped in (roughly) 120-degrees segments. The hexagon is lopsided due to the 45-degree difference between the tip/tilt actuator plane and the surface of the diagonal pickoff mirror. This pattern is repeated (modulo 7) until the total number of IR Images requested have been acquired. At the end of a dithering sequence, the mirror is returned to the home (centered) position. Tests at the telescope show that the dithering is repeatable to about 1 pixel (0.2 arcsec).

If you do not wish to dither between images, check the "No Dithering between images" box. This will keep the IR tip/tilt mirror centered during the entire obs file's IR imaging sequence.

[Contents]


Step 3: Validate & Save the Obs File

Once the exposure parameters have been entered in step 2, the obs file creation form generates the obs file and estimates the execution times for the CCD and/or IR images requested. You are asked to first validate your entries, using the estimated execution times to help adjust your exposure parameters to optimize your observations, and then select a filename for the obs file and save it to your project's working disk.

Estimated Execution Times

The estimated execution times give the approximate amount of real time required to execute the requested exposures, from the first detector array reset until the end of the last detector readout (not including any instrument or telescope configuration overheads). The various data-acquisition overheads were measured on 2000 September 6 with the current ANDICAM setup, and yield estimates based on our CCD and IR Array readout models that are accurate to within 5%. Be warned, your mileage *will* vary, so don't expend too much effort time optimizing out the last second of dead time in dual-channel observations - reality *will* have the last word. For those interested in factoring in instrument configuration overheads, at most this amounts to 10-seconds per observing template (assming change of name, filter, exposure time, and CCD binning factors). Telescope pointing, target acquition, and guide-star acquisition/lock times are totally up for grabs. If your program requires a lot of hand offsets around the field, it will cost total observation execution time. If your program proves excessively costly in overhead, the queue manager will contact you regarding modification of your program.

Estimated CCD Execution Times

This estimate takes into account:
  1. The base integration time per CCD image.
  2. The number of CCD images requested.
  3. The CCD binning factor selected.
  4. The time required to readout the CCD (vertical and horizontal clocking times and the readout amp integration time per pixel). This varies with the binning selected.
  5. The number of readout amplifiers being used (2 at present, not user-selectable).
  6. The pre-exposure "setup" overhead, including array erase cycles (~6-sec independent of binning).

For example, a single, full-frame, unbinned 2048x2048 CCD image requires approximately 80 seconds over and above the base integration time to erase and readout the array. Binning 2x2 will reduce the setup and readout overhead to approximately 28 seconds.

Taking many short exposures incurs a greater readout "penalty" compared to taking fewer longer exposures to achieve the same total integration time. For example, a single 300 second during full-frame, unbinned image will require approximately 379 seconds to execute, while dividing the 300 seconds of integration time into two (2) exposures of 150 seconds each requires 459 seconds (32% more time). Doing three exposures of 100 seconds each would require ~539 seconds to execute (42% more time).

Estimated IR Execution Times

This estimate takes into account:
  1. The base integration time per IR image (the minimum integration time is 4 seconds).
  2. The number of IR images requested.
  3. The number frames co-added per IR image requested.
  4. The array readout time per quadrant (approx 4 seconds).
  5. The pre-exposure reset time (approx 4 seconds).
  6. The tip/tilt mirror overhead if dithering between images (approx. 2 seconds/position).

The algorithm for computing the execution time of IR images is complicated by execution latencies encountered during readout and storage due to the finite clock interval allowed in the hardware. In general, the times computed are good to 5%, although if a large number of images (>10) has been requested with a comparable number of co-adds per image, the calculator will systematically underestimate the total execution time by as much as 10% due to cumulative small latencies in the system.

[Contents]


Enter a Filename for the Obs File

Each YALO project is assigned a private directory on the queue server for storing their observation template files.

This field asks you to provide a filename for your new obs file. The filenames of obs files should be kept simple, and reflect as much as possible the contents of the file. For example, if the obs file is to acquire simultaneous V and H band images of a star cluster named NGC007, you might name the obs file ngc007vh. Including the object name in the filename like this helps to ensure relatively unique names.

Filenames are case sensitive, and may contain only letters and number, no special characters (e.g., +, -, _, etc.) are allowed.

If the filename you provide will overwrite an existing obs file in your directory, you will be given a warning and asked to confirm that you really want to overwrite the existing file. Detailed instructions are provided if this occurs.

Note that the obs files will be given the .obs files extension automatically by the web forms, so you should not include a file extension with the filename you type in the box provided.

[Contents]


Editing Existing Obs Files

Once you have a set of Obs files created, you can later go back and edit them with the Obs File Editor. This form will read in the contents of a existing Obs file and let you modify the entries, either to replace an old Obs file or to create a new Obs file using an old one as a template. In cases where readout or overhead times change substantially (e.g., in September 2000), the editor can be used to re-evaluate old Obs files and modify them to reduce deadtime in DUAL mode.

The basic entries are the same as on the Obs file creation form described above. The one potential point of confusion is if you are editing an Obs file that was originally either CCD-only or IR-only imaging mode. In these cases, the editor form provides boxes for the unused channel (after entries for the active channel). These entries will be ignored unless you decide to change the imaging mode as part of your editing session. It's easier to try than read about, trust me, you'll figure it out.

The Obs File Editor forms are a new tool released on 2000 Sept 10.

[Contents]


Viewing/Renaming/Deleting Obs Files with the Obs File Manager

After a while, you can get quite a collection of Obs files. To provide a way to manage these semi-sensibly, we have provided the Obs File Manager Form. The file manager is a table-based web form that shows you all of the obs files that have been created for your project, and lets you view, delete, or rename individual files. This lets you clean up after making your obs files, fix ugly filenames, etc., without having to go back through the creation or editor forms.

Obs files "deleted" from the active list are moved to the wastebasket for your project, where they are held until you decide to either delete them once and for all, or to "undelete" them and return them to the active list. If there are any files in your project's wastebasket when you open up the file manager, they will appear in a separate "wastebasket table" beneath the table of active files. This is a safety feature to prevent you from accidentally deleting obs files (you can delete them, we just make you think about it first).

Editing existing obs files is not an option during the early parts of the testing phases (the file parser is not working yet), but we hope to have something working eventually.

The instructions for using the File Manager are described in a separate document.

[Contents]


Example Obs Files

Example 1: I-band CCD Imaging

This is an obs file to take one 300-second CCD image of MB99018 with the I-band filter. The IR channel is kept idle during the observation (CCD-only mode).

# Prospero Observation Template File
# Created: 2000 Sep 9 [13:35:50] by saveobs.pl Version 1.3
# For: R. Pogge
#
PROJECT=O39901
IMGTYPE=OBJECT
OBJECT=MB99018
RA=18:01:07.7
DEC=-28:31:41
EQUINOX=2000.0
MODE=CCD
#
# CCD Imaging Parameters
#
# Estimated Execution Time: 379 sec
#
CCD:
   FILTER=0 # I
   EXPTIME=300.0
   NIMGS=1
   CCDBIN=1
END
Example 2: IR K-band Imaging

This obs file takes a sequence of five (5) 60-second images at K of a Seyfert 1 galaxy, dithering between images with a dither scale of 20 units (about 10 arcsec). The CCD channel is kept idle during this observation (IR-only mode).

# Prospero Observation Template File
# Created: 2000 Sep 7 [13:27:45] by saveobs.pl Version 1.3
# For: Rick Pogge (OSU)
#
PROJECT=O39901
IMGTYPE=OBJECT
OBJECT=Mrk 1347 K
RA=12:39:47.3
DEC=-23:27:16.0
EQUINOX=2000.0
MODE=IR
#
#  IR Imaging Parameters
#
# Estimated Execution Time: 350 sec
#
IR:
   FILTER=3 # K
   EXPTIME=60.0
   NCOADDS=1
   NIMGS=5
   DITHER=T
   DSCALE=20
END
Example 3: Dual IR/CCD Imaging

This obs file takes simultaneous V and K-band images of 47 Tucanae, one 180-sec integration at V and three 75-sec integrations at K, dithering by a factor of 40 between IR images.

# Prospero Observation Template File
# Created: 2000 Sep 7 [13:41:59] by saveobs.pl Version 1.3
# For: bailyn
#
PROJECT=Y20006
IMGTYPE=OBJECT
OBJECT=47 tuc
RA=00:26:00
DEC=07:02:22
EQUINOX=2000.0
MODE=DUAL
#
# CCD Imaging Parameters
#
# Estimated Execution Time: 259 sec
#
CCD:
   FILTER=6 # V
   EXPTIME=180.0
   NIMGS=1
   CCDBIN=1
#
#  IR Imaging Parameters
#
# Estimated Execution Time: 255 sec
#
IR:
   FILTER=3 # K
   EXPTIME=75.0
   NCOADDS=1
   NIMGS=3
   DITHER=T
   DSCALE=40
END
Note that instead of three 60 second integrations at K, the astronomer has increased the base integration time to 75 seconds to roughly equalize the amount of real-time required to execute the CCD and IR observations (259-sec compared to 255-sec for CCD and IR, respectively). Attempting to optimize the difference by more than 10-20 seconds, however, is usually a waste as this time can easily be consumed by various system latencies that are unpredictable (not all dual-channel operations can be made parallel).

[Contents]


If you came here from one of the entry forms, use the "BACK" button on your browser to return to the obs file generation form in progress.


Updated: 2000 Sept 11 [rwp/osu]