DATA DENSITY
Data density reflects the effective geographical distance between source data elements in the gridded field.
Data elements on a finer mesh yield a finer resolution locally of the slope (derivative) of the pressure analysis field
and its derivatives. The spatial resolution of the gridded six-hourly pressure fields provided by FNMOC has changed from
the previous Northern Hemisphere ca. 3° latitude (63x63 polar stereographic projection) grid to a global 1°
(360x181 spherical projection) grid. A comparison of the November 1996 pressure fields in the north Pacific
(18N-63N, 101W-173W) for these two grids, as well as a global 2.5° (73x144 spherical projection) grid, is provided.
In some rare cases, we have found significant local
differences in the upwelling index values calculated from the two grid sizes. For example, the index at 57N 137W derived
from the 3° grid is substantially larger than the index calculated from the 1° pressure grid. This appears to be
related to weaker local pressure gradients in the finer resolved grid, which result in lower local geostrophic winds and
upwelling values than given with the more smoothed 3° grid, which spreads a steeper gradient over the site of this
particular index. However, the pressure fields appear to be virtually identical, and the vast majority
of derived values agree to within +/-1 in the least significant digit.
Plots of the pressure fields from the 3 different surface pressure grids help to illustate any minute differences.
An attempt is made to make product calculations independent of grid size or type. In general, calculations of the
upwelling indices and other products are made from either a 3° or an "alternating 5°" computational meshes.
The computational mesh is computed by non-linear Bessel interpolation from the source gridded field, regardless of its
data density. Selected subsets of the six-hourly gridded fields are also prepared as regional text products.
PROCESSOR WORD SIZE
The effective processor word size for the FNMOC's Control Data Corporation CYBER mainframe computer is 60 bits,
as mapped from the actual 64-bit words under the NOS/BE operating system. The processor word size is 32 bits for the
ERD's Silicon Graphics Incorporated (SGI) IRIS under the IRIX (UNIX) operating system. Processor word size has a
bearing on the truncation associated with digital floating point computation. Any calculation carries a different
number of bits of significance to the next calculation, which could result in slight "round-off" differences. Double
precision calculations (64-bit floating point representation) are available in the SGI IRIS. Although double precision
may be used to achieve a rough comparability between systems, it was determined for the level of calculations required
for product preparation that double precision produced no significant difference in the products.
DATA FORMAT
Data format need not, but may, affect computational schemes. FNMOC has changed from a NEDN (historical Navy)
to GRIB (standard WMO) format
for the surface pressure fields they produce.
When processing the coarse (e.g., 3°) gridded pressure fields, there is an exact representation of the field values
from either format. However, when processing the 1° GRIB fields, round-off errors in the floating point representation
of the field are expected for the SGI regardless of word size or double precision usage. The FNMOC chose one-tenths of a
millibar for their use of the World Meteorological Organization's GRIB format binary scaled positive integer gridded field
difference values. The effective precision is one-sixteenths of a millibar for the GRIB floating point reference value.
DATA PRECISION
Data precision is a measure of the smallest unit of data retained from the available precision in the source
analysis. This is not a measure of accuracy of the data represented, but reflects the accuracy of the format's
representation of the source analysis. Data precision may effect the truncation error associated with data representation,
e.g., for the one degree GRIB format, tenths of a unit does not always have an exact representation in binary floating point
forms, so the scaling calculation affects representational truncation for some values as field values are unpacked to the
floating point form. (The values x.0 and x.5 are exact, regardless of processor, word size, or double precision usage,
while other values may not be exact ). Minor truncation errors are propagated through the field at each computational
step.
PROCESSING DETAILS FOR SPECIFIC DATA PRODUCTS
Multi-parameter Time Series Product. These time series text and graphical products are each calculated on an
interpolated 3° computational mesh centered at and presented for a particular geographic point. The product preparation
program allows for the use of six-hourly or monthly mean source fields. The program also allows for the averaging period to
be set by the user.
Daily Upwelling Time Series Product. These six-hourly and monthly time series text and graphical products are each calculated on an interpolated 3-degree computational mesh and presented for the particular geographic point. The program is typically run for a set of 15 "standard" points along the US west coast.
NA3 and NP3 Products. Monthly mean gridded North Atlantic and Pacific regional and tabular multi
parameter text products are each calculated on a standard 3-degree mesh as interpolated from the monthly mean pressure field.
The 3-degree mesh is characterized by data points at intersection of lines of latitude and longitude evenly divisible by 3.
Results of one computation are used in the next computation, allowing for potential propagation of truncation errors.
NA5 and NP5 Products. Monthly mean gridded North Atlantic and Pacific regional multi-parameter text products
are each calculated on an alternating 5-degree mesh as interpolated from the monthly mean field. The alternating 5-degree
mesh is characterized by data points at intersection of lines of latitude and longitude divisible by 10, with residual 0
on odd rows/columns and residual 5 on even rows/columns. The rows are 5 degrees apart. Results of one computation are
used in the next computation, allowing for the potential propagation of truncation errors.
Processor-Word Size | Data Format/Precision | Data Density (Projection) |
---|---|---|
CYBER-60 bits | NEDN-mb/16 | ~3 lat degree(NHPS 63x63) |
CYBER-60 bits | NEDN-mb/16 | ~2.5 degree(Spherical 73x144) |
IRIS-32 bits | GRIB-mb/16 | ~3 lat degree(NHPS 63x63) |
IRIS-32 bits | GRIB-mb/16 | ~2.5 degree(Spherical 73x144) |
IRIS-32 bits | GRIB-mb/10 | 1 degree(Spherical 360x181) |