version 014: 21-Feb-2000 - config.names added entry for data.rows sdded entry for stat.dat - create_di.csh modified call to types2input - data2in.csh added call to htmlinfo.perl modified call to det_type.perl modified call to types2input - det_type.perl modified to reflect change in header.info now reads filename configuration information now writes to output file directly - input2table.perl now using config.names entry for data.rows added check if any object spectra positions have been found remove carriage return from data rows get information of location of spectra from stats file and no longer from call to find_location. This is now possible because the statistics are generated in prepare.csh and useful because getting this information is very CPU time intensive. - gen_reduce.perl now using config.names entry for data.rows remove carriage return from data rows cosmetic output changes improved handling of titles written for stars without existing name alias - mkrepair.perl modified to reflect change in header.info now reads filename configuration information now writes to output file directly - prepare.csh added call to htmlinfo.perl added call to det_stat.perl modified call to det_type.perl modified call to mkrepair.perl - types2input.csh now obsolete - moved to obsolete directory - types2input.perl added check for type of arc modified to reflect change in header.info changed calling sequence - subroutines/extract_header.csh added "END" as identifier for the end of the information belonging to one data file to header information file - created htmlinfo.perl - created det_stat.perl - created Routines.pm version 013: 17-Oct-1999 - added utility: u2s.perl creates entries usable for stars.coo (or addstars.coo) from the information of unknown entries in frames.type - created "obsolete" directory this will hold scripts that have been retired from the main development - retired input2table.csh this script is out of date and input2table.perl has been used for quite a while. - created Scripts.pm a perl module that holds functions used in several scripts. used in input2table.perl,types2input.perl - default.options added the parameter "directory". It's value can be "yes" for sorting output files into directories to make things look nicer and easier to clean up or "no" for using the old, flat structure. added the parameters "arc-dt", "bias-dt" and "flat-dt". They define the time delay in seconds between two consecutive frames of these types to split them into different .in files. added the parameters "cosmiscrays.threshold" and "cosmicrays.fluxratio". They change the default values when a numeric value is given. - split prepare.csh into prepare.csh (first pass) and data2in.csh (second pass). - created config.names this file is read by the scripts and holds the (global) file names and directory names used in prepare.csh,types2input.perl,types2input.csh,data2in.csh, create_di.csh,input2table.perl,gen_reduce.csh - changed name of tmpfile1 to header.info - prepare.csh added part to create directory structure (if directory option is "yes"). modified to use "subroutine" code - added subroutine directory to hold csh code that is used in more than one script. - mkrepair.perl frame.info and header.info are now deleted as the last action of the repair script. Because of this, they do not get re-generated in data2in.csh if they already exist. - created find_badpix.perl eventually this is supposed to automatically find the location of bad pixels ... not even close to being finished. - gen_reduce.perl changed bad pixel mask value to conform with what the man page for text2mask describes. - types2input.perl now splits arc|bias|flat also according to time. If the time difference is larger than specified in the options files a split occurs. - added utility: quicklook quicklook will plot one or two dimensional FITS images using the PGPLOT perl module. - README: some more updates done. version 012: 02-Oct-1999 - prepare.csh: changed arguments to mkrepair.perl call to reflect changes in script - added addframe.type This file in the data directory is used by det_type.perl for additional information - mkrepair.perl: now also reads the local addstars.coo when it exists for non object frames (arc,bias and flat) ra and dec are set to 0 in the repair script if no other value can be inferred. changed calling arguments to accomodate addstars.coo addition the directory is now passed as well as the file names for the header and type information. The local add* files used are hardcoded. - det_type.perl: started coding on the extended detection of types from the object header keyword. This is especially helpful for early data. - rename_dao.csh: The year is no longer fixed in the script but the user is now asked for it. Also general code improvements and added some checks. - substantially updated README (but still more work needs to be done) - added gen_reference.csh version 011: 08-Apr-1999 - bias, flat and arc table entries are now sorted according to frame number again. (input2table.perl) Switching to perl brought another improvement in the runtime of input2table (also see timings under version 008) modified (input2table.perl) : 0:09.66 - added utility: rename_dao.csh - input2table.perl: enabled object-arc1 and object-arc2 options. spectra that exceed the flat row index range are now reported and the rows written to object.table adjusted to a typical range generated from all spectra that are within the range. arc table has the arcs appended with the same rows as the objects removed system call to tail arc table now generated after the object table and arc entries in the object table point to the entry with the same number of rows in the arc table. - the file /addstars.coo can hold coordinates of objects that have only been observed in the specific night and helps keep stars.coo small - det_type.perl: added code for reading /addstars.coo arc source (thar, cdne or fear) is now listed in frame.type The source is found from the FITS header string by a simple matching algorithm. The algorithm currently can handle chemical element abbreviations and 2 elements only. When calculating the angle between 2 stars the argument to acos can be (just) larger than one due to round off errors. This resulted in some stars not being detected if their position was the one in the coordinates file. This problem is now resolved by forcing the argument to acos to be +-1 in these cases. If the type has been determined as object but no star was found within the search range then the coordinates are now printed like in the unknown type case. That makes it easier to use the output of "grep unknown frame.type" to be used as input for (add)stars.coo - prepare.csh: adapted det_type.perl call - gen_reduce.perl: added SITe5 CCD data modified to handle /addstars.coo modified to handle arc#####a entries in the arc table. For these entries the arc#####.in and .out files are being used as they are identical versions with different columns used. removed a bug in setting up the CCD size information which prevented everything but the first entry from working Now writing correct header strings for arcs according to type arc final file name now determined by arc type version 010: 26-Mar-1999 - removed bug with path handling in call to find_location in input2table.csh This problem did not show up if the scripts directory was parallel to the data directory. - added options file (see README entry for default.options and reduce.options) - prepare.csh copies default.options to /reduce.options if the file does not exist - rewrote input2table in perl. This new version is able to read the options files which steer how the script determines bias, flat and arc frame numbers. - which input2table script is being used is now determined by a symlink called input2table. Remember to invoke input2table instead of input2table.csh as before. The perl version of the script is further developed than the csh version. version 009: 24-Mar-1999 - integrated repair.in into mkrepair.perl - changed prepare.csh to reflect mkrepair changes - repair script now finds telescope and year from filenames in rawdat - default values for header strings can be set in the file add.info in the data directory (see README) - gen_reduce now uses row numbers from arc.table for arc header string - arcs should now be split if telescope is 1.8m and difference in position is bigger than 5 deg. - added support for + in bias column (any table) - find_location is a symlink that points to the location finding routine that should be used (currently either find_location.csh or find_location.perl) version 008: - modified gen_reduce.perl to take arc data-rows column into account when generating .sum.lis files (don't forget to remove them if you want them regenerated) - added data-rows column to arc.table values are standard row values found from flat fields - changed input2table.csh to call find_location.perl and moved find_locate outside of awk script for objects. original (find_location.csh) : 2:22.42 original (find_location.perl) : 1:49.02 modified (find_location.csh) : 2:01.40 modified (find_location.perl) : 0:14.09 - rewrote find_location.csh in perl using PDL This makes future enhancements easier and improves performance: csh perl 1 input file : 0:03.22 0:02.17 14 input files: 0:33.21 0:03.55 This shows that the startup cost for PDL is considerable but still better than running the old version while having a clear advantage when multiple input files are being used The perl version currently uses median values (which should make it more robust against cosmic rays) while the csh version uses averages. - removed short->ushort conversion bug in find_location.csh version 007: - now checking for existing *.in files (Stephenson) If they exist a warning is printed and they are NOT overwritten ... Remove all *.in files if you want them regenerated and rerun prepare.csh - repair script is now set to permission 755 - arc .sum.lis files now have correct x-range for extraction (Stephenson) - star imdivide now has correct range for SITE-2 (Stephenson) - badpixelfile now has (more) correct values for SITE-2 (Stephenson) - correct fluxratio and threshold values now used for SITE-2 (Stephenson) - updated stars.coo (Stephenson) - delete raw flat field files again right after combining them (Stephenson) - restructured combined flat field handling code - now running a search for the row range on every object file (Stephenson) - types2input now has execute permission (Stephenson) version 006: SORRY, no change log was kept for version 006 and earlier ...