There are versions of the I/O API callable from both
Fortran and from
C . This document describes the Fortran
interface; the C interface is very
similar. The major difference between the two are that Fortran LOGICAL
functions returning .TRUE. or .FALSE. correspond to C functions returning
1 or 0, Fortran ".EXT"
include-files correspond one-to-one to
C ".h" include files, and the C calls
look much like the Fortran calls, except that file descriptions are
passed via pointers to data structures typedeffed in fdesc3.h
instead of a COMMON found in FDESC3.EXT
There are 9 routines you'll need to know:
INIT3() and SHUT3()
to start up and shut down
the I/O API;
OPEN3() to open files;
DESC3() to get file descriptions;
READ3(), INTERP3(),
XTRACT3(), and DDTVAR3()
to access data values; and WRITE3()
to store data to files. Additionally,
M3EXIT() is a useful utility routine which works
with the I/O API to generate exit (or error) messages to the
program log,
call SHUT3(), and then terminate the program with a user-supplied
exit status (which should be 0 for success and nonzero for failure).
There are three INCLUDE files you'll have to worry about.
Each has extensive in-line documentation describing how it is used.
PARMS3.EXT
contains the dimensioning parameters and the "magic number"
parameters used to control the operation of various routines in the I/O API.
FDESC3.EXT has
commons that hold file descriptions (more about that later); it needs
PARMS3.EXT for its own dimensioning. Finally,
IODECL3.EXT
has declarations and usage comments for the various functions in the
I/O API (it's really a short manual on the I/O API in its own right).
csh setenv command. Additionally, there are four
standard logical names:
LOGFILE, SCENFILE, and EXECUTION_ID, which may be used for the
program log file, scenario-description file, and execution identifier,
and GRIDDESC , for ASCII grid and
coordinate system databases used by utility routine
DSCGRID().
For programming purposes, the significant facts are that names should
not contain blanks (except at the end: "foo " is OK;
"f oo" is not), and are at most 16 characters long.
When you run a program that uses the I/O API, you begin with a
sequence of setenv commands that set the values for the
program's logical names, much as you begin a (normal) Cray Fortran
program with a sequence of ASSIGN commands for its files.
For example, if "myprogram" has logical names
"foo" and "bar" that I
want to connect up to files "somedata.mymodel"
and "otherdata.whatever" from directory
"/tmp/mydir", the script for the program would
look something like:
...
setenv foo /tmp/mydir/somedata.mymodel
setenv bar /tmp/mydir/otherdata.whatever
setenv qux "/tmp/mydir/volatilestuff.mymodel -v"
setenv LOGFILE /tmp/mydir/mymodel.log
setenv SCENFILE /tmp/mydir/test17a.description
setenv EXECUTION_ID TEST17A
/user/mydir/myprogram
...
VOLATILE files are indicated by a
trailing -v in the setenv command, as
above, in order to tell the I/O API to perform disk-synch
operations before every input and after every output operation on that
file. Such files can be accessed by other programs while the
generating program is still running, and are readable even if it fails
to do a SHUT3() or
M3EXIT() (or if it crashes unexpectedly).
BUFFERED virtual files can be used to
provide safe, structured exchange of data -- of "gridded",
"boundary", or "custom" types only -- between
different modules in the same program. If you setenv
the value of a logical name to the value BUFFERED, as
given below:
...
setenv qux BUFFERED
...
/user/mydir/myprogram
...
then the I/O API will establish in-memory buffers and time indexing
for "qux" instead of creating a physical file on disk. One module
can then use WRITE3() (see below) to export data
for sharing, which other modules would then use
READ3() or INTERP3() to import. Note
that since these routines associate the data with its simulation
date-and-time, the system will notice the error (and warn the user)
if you attempt to get and use data before it has been produced.
Note also that by changing the setenv in the script between
"BUFFERED" and a physical file-name, you can change between
efficient data sharing between modules and high-resolution
instrumentation of the data being shared, without changing the
underlying program at all.
COUPLING-MODE virtual files can be used to provide PVM-based data exchange between cooperating programs using exactly the unchanged I/O&API programming interface, with the kind of name-based direct-access semantics that provides, with the extra scheduling condition that requests for data that has not yet been written put the requester to sleep until it becomes available (at which time the requester is awaked and given the requested data). The decision of which files are disk-based and which are COUPLING-MODE virtual files is also made by setenv commands at program-launch, the value being of the form "virtual <communications-channel-name>":
...
setenv zok "virtual CHEM_CONC_3D_G3"
/user/mydir/myprogram
...
Except for INIT3(), all of the I/O API routines are LOGICAL functions returning TRUE for success and FALSE for failure.
There are a number of dimensioning parameters and "magic number" token values for the I/O API. Throughout the I/O API, names (logical file names, variable names, and units) are character strings with maximum length NAMLEN3 = 16; descriptions are either one or MXDESC3 = 60 lines of length at most MXDLIN3 = 80. The I/O API currently supports up to MXFILE3 = 50 open files, each with up to MXVARS3 = 120 variables.
NUMIDS in the
ID-referenced data structure, below) . Where such system-defined
variables are present, the operations READ3() and WRITE3() act on entire
time steps (all variables) at once; otherwise, they can be used to store
or retrieve time steps of individual variables one at a time. There are
moderate performance advantages to writing the variables for a time step
in the same order that they appear in the file description, and for writing
the time steps in consecutive order; however, this is not required by the
I/O API (which permits any access order to the data, for both read and
write operations). The structural types are as follows, together with
declarations for sample time step records of these data types:
(In the examples, declarations are given for M3REAL variables in terms
of REAL*4, etc, instead of merely REAL, to protect you in the cases that
your compiler has a "-r8" flag, etc., which silently changes all
REALs from 4-byte to 8-byte -- and causes you accidentally to be linked
with an incompatible version of the library; if you never use this flag,
don't worry.) The structure-types are:
...(SIZE is a fixed, user-defined dimension:)
REAL*4 ARRAY( SIZE, NLAYS, NVARS )
FTYPE3D, TSTEP3D, NCOLS3D, NROWS3D, NLAYS3D, NVARS3D, NTHIK3D,
GDTYP3D, P_ALP3D, P_BET3D, P_GAM3D, XORIG3D, YORIG3D, XCELL3D,
YCELL3D, GDNAM3D, XCENT3D, YCENT3D, VNAME3D, UNITS3D, VDESC3D
REAL*4 ARRAY( NCOLS, NROWS, NLAYS, NVARS )
NTHIK cells wide (where you may use a
negative NTHIK to indicate an internal perimeter such as
is used by ROM and RADM). The boundary array is dimensioned
as follows in terms of the dimensions NCOLS and NROWS for the array
it surrounds:
...(SIZE = ABS( NTHIK )*(2*NCOLS + 2*NROWS +4*NTHIK)
REAL*4 ARRAY( SIZE, NLAYS, NVARS )
There are accompanying diagrams illustrating the data layout for
various cases of NTHIK:
...
INTEGER*4 MAXID ! max permitted # of sites
PARAMETER ( MAXID = 100 )
...
INTEGER*4 NUMIDS ! number of actual sites
INTEGER*4 IDLIST( MAXID ) ! list of site ID's
REAL*4 XLON ( MAXID ) ! first variable in file
REAL*4 YLAT ( MAXID ) ! second variable
REAL*4 TK ( MAXID ) ! third variable
REAL*4 PRES ( MAXID ) ! fourth variable
REAL*4 RH ( MAXID ) ! fifth (last) variable
COMMON /FOO/ NUMIDS, IDLIST, XLON, YLAT, TK, PRES, RH
NVARS3D is 5.
The dimension MAXID maps into the NROWS3D
dimension in the file description data structure FDESC3.EXT,
for use by OPEN3() or DESC3(). To read or write this data, put the
first element, NUMIDS, of this common in the
array spot of READ3(),
WRITE3(), etc.:
IF ( .NOT. WRITE3( 'myfile', 'ALL', JDATE, JTIME, NUMIDS ) ) THEN
...(some kind of error happened--deal with it here)
END IF
ELEV,
TA, and QV given at up to 50 stations, each of
which may have up to 100 observation levels, is given by the following.
Note that ELEV = "height of the level above ground"
is user-specified as one of the variables rather than supplied by the
system as for ROM Type 1 files.
...
INTEGER*4 MXIDP ! max # of sites
INTEGER*4 MXLVL ! max # of levels
PARAMETER ( MAXID = 50, MXLVL = 100 )
...
INTEGER*4 NPROF ! # of actual sites
INTEGER*4 PLIST( MXIDP ) ! list of site ID's
INTEGER*4 NLVLS( MXIDP ) ! # of actual levels at site
REAL*8 X ( MXIDP ) ! array of site X-locations
REAL*8 Y ( MXIDP ) ! array of site Y-locations
REAL*8 Z ( MXIDP ) ! array of site Z-locations
REAL*4 ELEV ( MXLVL, MXIDP ) ! height of lvl a.g.l.
REAL*4 TA ( MXLVL, MXIDP ) ! variable "TA"
REAL*4 QV ( MXLVL, MXIDP ) ! variable "QV"
COMMON /BAR/ NPROF, PLIST, NLVLS, X, Y, Z, ELEV, TA, QV
...
The site dimension MXIDP maps into the NROWS3D
dimension , and the levels dimension MXLVL maps into the
NCOLS3D dimension in the file description (FDESC3) data
structures. To read or write this data, put the first element,
NPROF, of the common BAR in the
array spot of READ3(), WRITE3(), etc.:
IF ( .NOT. WRITE3( 'myfile', 'ALL', JDATE, JTIME, NPROF ) ) THEN
...(some kind of error happened--deal with it here)
END IF
...
INTEGER*4 MXNEST ! max # of nests
INTEGER*4 MXGRID ! max # of cells (total, all grids)
INTEGER*4 MXLAYS ! max # of levels
PARAMETER ( MXNEST = 10, MXGRID = 10000, MXLAYS = 25 )
...
INTEGER*4 NNEST ! # of actual nests
INTEGER*4 NLIST( MXNEST ) ! list of nest ID's
INTEGER*4 NCOLS( MXNEST ) ! # of actual cols of nest
INTEGER*4 NROWS( MXNEST ) ! # of actual rows of nest
INTEGER*4 NLAYS( MXNEST ) ! # of actual lays of nest
REAL*8 XN ( MXNEST ) ! array of nest X-locations
REAL*8 YN ( MXNEST ) ! array of nest Y-locations
REAL*8 DX ( MXNEST ) ! array of nest cell-size DX's
REAL*8 DY ( MXNEST ) ! array of nest cell-size DY's
REAL*4 NO2 ( MXGRID, MXLAYS, MXNEST ) ! variable "NO2"
REAL*4 O3 ( MXGRID, MXLAYS, MXNEST ) ! variable "O3"
COMMON /QUX/ NNEST, NLIST, NCOLS, NROWS, NLAYS,
& XN, YN, DX, DY, NO2, O3
...
The nest dimension MXNEST maps into the NROWS3D
dimension, the cells dimension MXGRID maps onto
NCOLS3D, and the layers dimension MXLAYS
maps onto NLAYS3D in the file description (FDESC3) data
structures. To read or write this data, put the first element,
NNEST, of this common QUX in the
ARRAY spot of READ3(), WRITE3(), etc.:
IF ( .NOT. WRITE3( 'nfile', 'ALL', JDATE, JTIME, NNEST ) ) THEN
...(some kind of error happened--deal with it here)
END IF
...
INTEGER NASRC ! max # of active cols per row
INTEGER NGRID ! number of rows in the matrix
PARAMETER ( NASRC = 3978, NGRID = 5400 )
...
INTEGER NS( NGRID ) ! # of actual cols per row
INTEGER IS( NACEL , NGRID ) ! column pointers
REAL CS( NACEL , NGRID ) ! col-coefficients
COMMON / GRIDMAT / NS, IS, CS
In this case, NVARS3D = 1. In the case of
NVARS3D > 1, multiple
coefficient matrices would share the same NS and IS arrays.
The active-columns dimension NASRC maps into the NCOLS3D dimension
and the matrix-rows dimension NGRID maps into the NROWS3D dimension
in the file description (FDESC3) data structures. To read or
write this data, put the first element, NS of the common
GRIDMAT in the array spot of READ3(),
WRITE3(),etc.:
IF ( .NOT. WRITE3( 'mfile', 'ALL', JDATE, JTIME, NS ) ) THEN
...(some kind of error happened--deal with it here)
END IF
..
IF ( .NOT. DESC3( 'myfile' ) ) THEN
...(error: probably the file hasn't been opened yet)
END IF
Some of the items in a file description, such as the dates and times
for file creation and update, and the name of the program which created
the file, are maintained automatically by the system. Others describe
the variables in the file: the file type (as described above), the
number of variables, their names, unit designations, and descriptions,
as well as the description of the file as a whole. Still others
dimension the data: the number of layers and the grid dimensions
(where for ID and profile files, the number of sites is mapped onto
the rows dimension; for profile files, the number of vertical levels
is mapped onto the columns dimension). Still other parts of the file
description specify the geometry of the grid: the map projection used,
its projection parameters, and the grid's location and cell-size
relative to that map projection; the vertical-grid-coordinate type
and the boundary values separating the model layers.
INIT3() is an integer
function. It returns the unit number to be used
for the program's log (if you setenv LOGFILE, the I/O API's log and
error messages will be written to this unit; otherwise, they go to
standard output, unit 6). INIT3() can be called as many times as you
want, to get the unit number for the program log.
SHUT3() is a logical function that returns TRUE if the system successfully flushed all I/O API files to disk and shut itself down, and FALSE if it failed. If it failed, there probably was a hardware problem -- not much you can do about it, but at least you ought to be able to know. It is legal to call SHUT3() and close down all files currently open, and then to call INIT3() again and open new ones. NOTE that utility routine M3EXIT() calls SHUT3() as the final step of its operation (in addition to generating a log-message with the current simulation date-and-time and the indicated message-text). CLOSE3() is a logical function that returns TRUE if the system successfully flushed the indicated file to disk and closed it, and FALSE if it failed.
setenv SCENFILE to that
file before you run the program, then OPEN3 will copy the
SCENFILE information into the headers of any output files
for that program. Also, if you setenv EXECUTION_ID to your
own identifier for the program execution, it will automate the storage
and the logging of that identifier. Finally, if you setenv
IOAPI_CHECK_HEADERS YES, then the I/O API will perform a sanity
check on internal file descriptions -- checking that grid description
parameters are in range, for example, or that vertical levels are
either systematically increasing or systematically decreasing.
The arguments to OPEN3 are the name
of the file, an INTEGER "magic number" indicating
the type of open operation, and the caller's name for logging and
audit-trail purposes. You can call OPEN3 many times for the same file
without hurting anything, if you want -- as long as you don't first open
it read-only and then try to change your mind, or try to open it as a
NEW file after it is already open. Names and values for the
mode-of-opening magic number argument are defined in
PARMS3.EXT as
the following:
INCLUDE file
FDESC3.EXT
to define the structure for the file, and then call OPEN3(). If the
file doesn't exist in either of these cases, OPEN3() will use the
information to create a new file according to your specifications,
and open it for read/write access. In the "unknown"case,
if the file already exists, OPEN3() will perform a consistency check
between your supplied file description and the description found in
the file's own header, and will return TRUE (and leave the file open)
only if the two are consistent. Sample calls to OPEN3() for an
input file 'myfile' and an output file 'my_newfile' might look like
the following:
...
IF ( .NOT OPEN3( 'myfile', FSREAD3, 'my program') ) THEN
...(some kind of error happened--deal with it here)
END IF
...
... (First, fill in the file's description for 'my_newfile'.
... Then open it:)
IF ( .NOT. OPEN3( 'my_newfile', FSNEW3, 'my program' ) ) THEN
...(some kind of error happened--deal with it here)
END IF
There are also three sample programs that
demonstrate how to use the I/O API to create various kinds of files --
gridded, boundary, and ID-referenced, with one or multiple layers, and
either time-stepped or time-independent.
NOTE: Joan Novak (EPA) and Ed Bilicki (MCNC) have declared as a software standard that modeling programs may not use FSCREA3 as the mode for opening files. FSCREA3 is reserved for use by analysis/data extraction programs only.
To get a file's description, you use the
DESC3() function.
When you call DESC3(), it puts the file's complete description in the
standard file description data structures in
FDESC3.EXT .
Note that the file must have been opened prior to calling DESC3().
A typical call might look like:
...
IF ( .NOT. DESC3( ' myfile' ) ) THEN
...(some kind of error happened--deal with it here)
ELSE
...(the FDESC3 commons now contain the file description:
... data type, dimensions, starting date&time, timestep,
... list of variables and their descriptions, etc.)
END IF
...
size argument -- you tell it how much data you expect, and
it checks that against how much data the file thinks you ought to get,
for error checking purposes. A typical INTERP3 call to read/interpolate
the variable HNO3 to 12:30 PM on February 4, 1987 might look like:
...
CHARACTER*16 FNAME, VNAME
REAL*4 ARRAY( NCOLS, NROWS, NLAYS )
...
IF ( .NOT. INTERP3( 'myfile', 'HNO3', 1987035, 123000,
& NCOLS*NROWS*NLAYS, ARRAY ) ) THEN
...(some kind of error happened--deal with it here)
END IF
With READ3() and XTRACT3(), you can use the
"magic values"
ALLVAR3' (= 'ALL', defined in
PARMS3.EXT ) or
ALLAYS3 (= -1, also defined in PARMS3.EXT) as
variable name and/or layer number
to read all variables or all layers from the file, respectively.
For time independent files, the date and time arguments are ignored.
ALLVAR3 (= 'ALL', defined in
PARMS3.EXT ) as the
variable-name. For ID-referenced,
profile, and grid-nest files, you must write an entire time step at a time
(i.e., the variable-name must be ALLVAR3).
WRITE3() is affected by standard
environment variable IOAPI_LOG_WRITE (which has default value
"YES"; normally WRITE3() generates a log message for each
write-operation successfully completed. However, if you
setenv IOAPI_LOG_WRITE NO then these messages will be
suppressed. Typical WRITE3() calls to write data for date and time
JDATE:JTIME might look like the following:
...
REAL*4 ARRAY( NCOLS, NROWS, NLAYS, NVARS )
...
IF ( .NOT. WRITE3( 'myfile', 'HNO3', JDATE, JTIME, ARRAY ) ) THEN
...(some kind of error happened--deal with it here)
END IF
IF ( .NOT. WRITE3( 'afile', 'ALL', JDATE, JTIME, ARRAYB ) ) THEN
...(some kind of error happened--deal with it here)
END IF
HHMMSS = 10000 * hour + 100 * minutes + seconds
YYYYDDD = 1000 * year + day
where the year is 4-digits (1994, say, rather than just 94), and the
day is the Julian day-number (1,...,365 or 366). By convention,
dates and times are stored in Greenwich Mean Time.
There are two utility programs,
juldate, for converting calendar dates to Julian
dates, and gregdate,
for converting Julian dates to calendar dates and reporting the
day-of-the-week. Both of these programs also report whether daylight
savings time is in effect for the specified date. There are also a
number of utility routines available for manipulating dates and times.
Note that for these routines, time steps may perfectly well be
negative -- just make sure you keep the parts all positive or
all negative; a time step of -33000 means to step three and a half
hours into the past, for example. This way of representing dates and
times is easy to understand and manipulate when you are watching code
in the debugger (you don't have to turn "seconds since Jan. 1, 1970"
into something meaningful for your model run, nor do you have to remember
whether April has 30 days or 31 when your model run crosses over from
April to May). The utility routines for manipulating dates and times
are the following:
Diagrams showing the relationship of the grid and its layers to the header attributes XORIG3D, YORIG3F, VGLVS3D, etc., are available in Postscript, X bitmap, JPEG, and GIF image formats. Note that Layer 1 is the bottom layer in the modeling grid. Some development work needs yet to be done: we need to do some more work about the definition of grids and map projections , and then to define and write utility routines having to do with map projections, transformation of locations from one map projection to another, and interpolation of data from one grid to another (on possibly different map projections).
Next: Changes from the Previous I/O API Version
To: Models-3/EDSS I/O API: The Help Pages