$Id: libtcd.html 1803 2007-01-22 15:29:56Z flaterco $

Tide Constituent Database

Comments on this documentation should be returned to David Flater.

Contents

  1. Introduction
  2. Compiling libtcd
  3. Tide record (version 2.2)
  4. Public database header (version 2.2)
  5. libtcd 2.2 API
  6. V1 to V2 migration guide
  7. Changelog
  8. References

1. Introduction

libtcd provides a software API for reading and writing Tide Constituent Database (TCD) files.

The TCD file format and schema are used by XTide to retrieve constituent definitions (speeds, equilibrium arguments, and node factors), harmonic constants, subordinate station offsets and associated metadata for use in generating tide predictions.

The TCD file format and schema were originally designed by Jan Depner to improve the performance of XTide and to meet additional requirements of the U.S. Naval Oceanographic Office (NAVO).  They are now maintained primarily by David Flater.

The design goals for TCD included:

The TCD file format and schema and libtcd are in the public domain.  They are distributed in the hope that they will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

For additional background, see Jan Depner, "Format for the Oceanographic and Atmospheric Master Library (OAML) Tide Constituent Database," rev. 2003-06-06.

2. Compiling and installing libtcd

The normal route:

bash-3.1$ ./configure
bash-3.1$ make
bash-3.1$ su
bash-3.1# make install

libtcd is packaged with the popular and portable GNU automake, so all usual GNU tricks should work.  Help on configuration options can be found in the INSTALL file or obtained by entering ./configure --help.

3. Tide record (version 2.2)

This section documents the tide record schema of the Tide Constituent Database as it stands in libtcd version 2.2.

Fields designated as "header" fields are retrieved at indexing time and stored in memory for quick access.  The rest are retrieved only when individual tide stations are loaded.

Type 1 records are for reference stations; type 2 records are for subordinate stations.  Some fields are only encoded for one record type while others are encoded for both.

intX means signed X-bit integer.  uintX means unsigned X-bit integer.  TCD files are encoded with a bit-packing function that allows odd-sized fields to be stored efficiently.  int varying and uint varying indicate fields whose lengths in bits are dynamically configured for each TCD file.

Floating-point numbers are encoded in the TCD file using scaled integers.  An integer with a scale of X represents the real number obtained by dividing the integer by X.

Please note:  All character strings stored in a TCD file use the character set ISO-8859-1.

Record type TIDE_RECORD field TIDE_RECORD data type File encoding Encodable range Semantics
allheader.record_numberint32noneN/ARecords in TCD file are implicitly numbered [0, N-1].
allheader.record_sizeuint32uint16[0, 65535]Length of encoded record in bytes.
allheader.record_typeuint8uint4[0, 15]See enum TIDE_RECORD_TYPE in Section 5.  Type 1 is reference station.  Type 2 is subordinate station.  Others are as yet undefined.
allheader.latitudefloat64int25, scale 100000[-167.77216, 167.77215]Latitude in degrees north; use [-90.00000, 90.00000].  Lat,lng of 0,0 is interpreted as NULL.
allheader.longitudefloat64int26, scale 100000[-335.54432, 335.54431]Longitude in degrees east; use [-180.00000, 180.00000].  Lat,lng of 0,0 is interpreted as NULL.
allheader.reference_stationint32int18[-131072, 131071]Index, references record_number of reference station for type 2 records, or -1 for none.
allheader.tzfileint16uint10[0, 1023]Index, references table of time zone values like ":America/New_York" that control which time zone output should be rendered into.  These time zone values are defined in the zoneinfo time zone database that is included with most flavors of Unix.  Not to be confused with zone_offset, which actually affects the calibration of the results relative to real time.  See comments for zone_offset below.
allheader.namechar[90]0-terminated stringLength constrained by TIDE_RECORD.Station name.
allcountryint16uint9[0, 511]Index, references table of country names.
allsourcechar[90]0-terminated stringLength constrained by TIDE_RECORD.Where you got the data, or "" for NULL.
allrestrictionuint8uint4[0, 15]Index, references table of restrictions like "Public domain," "Non-commercial use only," "DoD/DoD contractors only."
allcommentschar[10000]0-terminated stringLength constrained by TIDE_RECORD.Human-readable text that is not provided to the user of a data set at the time of use but may be retrieved on demand.  Use "" for NULL.
allnoteschar[10000]0-terminated stringLength constrained by TIDE_RECORD.Human-readable text that MAY be delivered to the user of a data set at the time of use.  XTide will print the notes in modes where this is convenient but ignore them when it is not.
alllegaleseuint8uint4[0, 15] Index, references table of human-readable legal notices that MUST be delivered to the user of a data set at the time of use.  Use 0 for NULL.
allstation_id_contextchar[90]0-terminated stringLength constrained by TIDE_RECORD.Name of the authority that has defined the Station ID, or "" for NULL.
allstation_idchar[90]0-terminated stringLength constrained by TIDE_RECORD.An identifier for a tide station, defined by the authority specified in station_id_context, or "" for NULL.
alldate_importeduint32uint27[0, 134217727]The date on which the data set was imported into the database, encoded Year * 10000 + Month [1, 12] * 100 + Day [1, 31], or 0 for NULL.  N.B., this is not the date of creation of the TCD file, but the date on record in a tide data management package such as Harmbase.
allxfieldschar[10000]0-terminated stringLength constrained by TIDE_RECORD.Space for backward-compatible addition of expansion fields as text.  Encoding:
xfields: xfield*
xfield: field-name ":" field-value "\n"
field-name: [^:\n ]+
field-value: [^\n]* continuation*
continuation: "\n " field-value
When decoding field-values, the string "\n " is replaced by "\n" to unmangle multi-line values.  (Yes, multi-line values remain multi-line, unlike RFC 822 headers.)
alldirection_unitsuint8uint varying[0, 255]Index, references table of units like "degrees true."  This field is used to indicate the units of the direction fields.
allmin_directionint32uint9[0, 511]Direction of ebb current.  Use [0, 359], or 361 for NULL.
allmax_directionint32uint9[0, 511]Direction of flood current.  Use [0, 359], or 361 for NULL.
alllevel_unitsuint8uint varying[0, 255]Index, references table of units like "feet," "meters," "knots," "knots^2."  This field indicates the units of the datum and the amplitudes in type 1 records and the units of the level_add fields in type 2 records.*
1datum_offsetfloat32int28, scale 10000[-13421.7728, 13421.7727]The datum (Z0).
1datumint16uint7[0, 127]Index, references table of datum kinds like "Mean Lower Low Water."  For currents, set to 0 (value should be ignored).
1zone_offsetint32int13[-4096, 4095]The standard time to which epochs are adjusted, a.k.a. the meridian, in hours and minutes east of UTC, encoded Hours * 100 + Minutes.  zone_offset affects the calibration of the predictions relative to real time.  Not to be confused with tzfile, which only affects which time zone the results are rendered into.  For example, adjusting zone_offset will change high tide from 5:00 EST to 4:00 EST, while adjusting tzfile will change it from 5:00 EST to 4:00 CST (actually the same "real time" in two different time zones).  Technically, zone_offset could be done away with by specifying that all data should be calibrated with UTC; however, this would make it more difficult to identify data sets with source data by visual inspection (all of the epochs would be different).
1expiration_dateuint32uint27[0, 134217727]Use-until date for data set, encoded Year * 10000 + Month [1, 12] * 100 + Day [1, 31], or 0 for NULL.
1months_on_stationuint16uint10[0, 1023]Number of months in time series used to derive harmonic constants, or 0 for NULL.
1last_date_on_stationuint32uint27[0, 134217727]Last date in time series used to derive harmonic constants, encoded Year * 10000 + Month [1, 12] * 100 + Day [1, 31], or 0 for NULL.
1confidenceuint8uint4[0, 15]A meaningless indicator of data quality, generally initialized to 10.  (Jan wants to keep this.)
1amplitudefloat32[255]uint19, scale 10000 (each)[0.0000, 52.4287]Amplitudes of constituents.  Constituents with amplitude less than AMPLITUDE_EPSILON are not encoded.
1epochfloat32[255]uint16, scale 100 (each)[0.00, 655.35]Epochs (phases) of constituents, in degrees.  Use [0.00, 359.99].  Constituents with amplitude less than AMPLITUDE_EPSILON are not encoded.
2min_time_addint32int13[-4096, 4095]The time adjustment for Low Tide or Max Ebb events, encoded Hours * 100 + Minutes.  0 and NULL are equivalent (no adjustment).** ***
2min_level_addfloat32int17, scale 1000[-65.536, 65.535]Additive adjustment for Low Tide or Max Ebb events.***  0 and NULL are equivalent (no adjustment).
2min_level_multiplyfloat32uint16, scale 1000[0.000, 65.535]Ratio for Low Tide or Max Ebb events, or 0 for NULL (equivalent to 1, no adjustment).***
2max_time_addint32int13[-4096, 4095]The time adjustment for High Tide or Max Flood events, encoded Hours * 100 + Minutes.  0 and NULL are equivalent (no adjustment).** ***
2max_level_addfloat32int17, scale 1000[-65.536, 65.535]Additive adjustment for High Tide or Max Flood events.***  0 and NULL are equivalent (no adjustment).
2max_level_multiplyfloat32uint16, scale 1000[0.000, 65.535]Ratio for High Tide or Max Flood events, or 0 for NULL (equivalent to 1, no adjustment).***
2flood_beginsint32int13[-4096, 4095]The time adjustment for Slack Water or Min Flood before Max Flood, encoded Hours * 100 + Minutes, or 2560 for NULL.**
2ebb_beginsint32int13[-4096, 4095]The time adjustment for Slack Water or Min Ebb before Max Ebb, encoded Hours * 100 + Minutes, or 2560 for NULL.**

* In case of units = knots^2 (hydraulic currents), it should be understood that only the amplitudes of the constants are such as to give results in knots squared.  The datum and any level_add corrections are in plain knots; i.e., the square root of the amplitude is taken before the datum and any corrections are added in.

** Time corrections DO NOT incorporate adjustments to Local Standard Time when the reference and sub station are in different time zones.  Time corrections are specified as if all calculations and predictions were done in UTC.  Time zone differences are handled by adjusting the tzfile field.  This differs from current NOS practice, in which LST adjustments are incorporated.  When this "time warp" is undone for NOS sub stations that use reference stations across the International Date Line, time corrections in excess of 24 hours can result.

*** See http://www.flaterco.com/xtide/mincurrents.html regarding usage for Min Flood and Min Ebb events.

This section describes the "public portion" of the database header structure, which is declared in tcd.h and made available to applications via the get_tide_db_header operation.

char version[90];
libtcd version string.
uint32 major_rev;
libtcd major revision number.
uint32 minor_rev;
libtcd minor revision number.
char last_modified[90];
Last modification of TCD file.
uint32 number_of_records;
Number of records in TCD file.
int32 start_year;
Year corresponding to 0 index in speed, equilibrium argument, and node factor arrays.
uint32 number_of_years;
Number of years in speed, equilibrium argument, and node factor arrays.
uint32 constituents;
Number of constituents.
uint32 level_unit_types;
Number of entries in table used by get_level_units.
uint32 dir_unit_types;
Number of entries in table used by get_dir_units.
uint32 restriction_types;
Number of entries in table used by get_restriction.
uint32 datum_types;
Number of entries in table used by get_datum.
uint32 countries;
Number of entries in table used by get_country.
uint32 tzfiles;
Number of entries in table used by get_tzfile.
uint32 legaleses;
Number of entries in table used by get_legalese.
uint32 pedigree_types;
For backward compatibility.  Ignore.

The encodings of constituent speeds, equilibrium arguments, and node factors are as follows.

NameProgram data typeFile encodingEncodable rangeSemantics
Speedfloat64uint varying (max 31), scale 10000000, with offset[0.0000000, 214.7483647]*Speed of constituent in degrees per hour.
Equilibrium argumentfloat32uint varying (max 31), scale 100, with offset[0.00, bignum]Equilibrium argument in degrees.  Use [0.00, 359.99].
Node factorfloat32uint varying (max 31), scale 10000, with offset[0.0000, bignum]Node factor.

The offsets are adjustments so that the minimum value in the data set is represented with a zero and the maximum by max−offset.  For node factors, this can save bits.  However, for speeds and equilibrium arguments they are a needless complication.  Speeds in excess of the limit quoted above will overflow the integers before the offset is taken into account, and the minimum equilibrium argument is always zero anyway.

* The highest speed in the IHO list of constituents as of 2003-07-31 is 203.904625 degrees/hour for 6MS14.

5. libtcd 2.2 API

The API is provided in the C language and is made available by including the header file tcd.h.

#define NV_BYTE    int8_t
#define NV_INT16   int16_t
#define NV_INT32   int32_t
#define NV_INT64   int64_t
#define NV_U_BYTE  uint8_t
#define NV_U_INT16 uint16_t
#define NV_U_INT32 uint32_t
#define NV_U_INT64 uint64_t

#define NV_BOOL    unsigned char
#define NV_CHAR    char
#define NV_U_CHAR  unsigned char
#define NV_FLOAT32 float
#define NV_FLOAT64 double
This section may appear slightly different from one platform to the next.  The build process for libtcd generates #includes and data type definitions as needed to provide integer types of specific sizes.
#define NVFalse         0
#define NVTrue          1
#define NV_U_INT32_MAX  4294967295
#define NV_INT32_MAX    2147483647
#define NV_U_INT16_MAX  65535
#define NV_INT16_MAX    32767
Defined values.
#define LIBTCD_VERSION   "PFM Software - libtcd v2.2 - 2006-11-22"
#define LIBTCD_MAJOR_REV 2
#define LIBTCD_MINOR_REV 2
These defines describe the version of libtcd to which the header file belongs.
/* One-line character strings */
#define ONELINER_LENGTH      90
/* Verbose character strings */
#define MONOLOGUE_LENGTH  10000
#define MAX_CONSTITUENTS    255
These defines describe the sizes of the fixed-size arrays in the tide record.
typedef struct ... DB_HEADER_PUBLIC;
typedef struct ... TIDE_STATION_HEADER;
typedef struct ... TIDE_RECORD;
The structure for the database header, and the header and main part of the tide record.
enum TIDE_RECORD_TYPE {REFERENCE_STATION=1, SUBORDINATE_STATION=2};
Syntactic sugar for values of header.record_type.
#define NULLSLACKOFFSET 0xA00
Magic constant used to indicate NULL in the flood_begins and ebb_begins fields of the tide record.
#define AMPLITUDE_EPSILON 0.00005
This is the level below which an amplitude rounds to zero.
void dump_tide_record (TIDE_RECORD *rec);
Prints a low-level dump of the tide record to stderr.
NV_CHAR *get_country (NV_INT32 num);
NV_CHAR *get_tzfile (NV_INT32 num);
NV_CHAR *get_level_units (NV_INT32 num);
NV_CHAR *get_dir_units (NV_INT32 num);
NV_CHAR *get_restriction (NV_INT32 num);
NV_CHAR *get_datum (NV_INT32 num);
NV_CHAR *get_legalese (NV_INT32 num);
For fields in the tide record that are indices into tables of character string values, these functions are used to retrieve the character string value corresponding to a particular index.  The value "Unknown" is returned when no translation exists.  The return value is a pointer into static memory.
NV_CHAR *get_constituent (NV_INT32 num);
Get the name of the constituent corresponding to index num [0,constituents-1].  The return value is a pointer into static memory.
NV_CHAR *get_station (NV_INT32 num);
Get the name of the station whose record_number is num [0,number_of_records-1].  The return value is a pointer into static memory.
NV_FLOAT64 get_speed (NV_INT32 num);
Returns the speed of the constituent indicated by num [0,constituents-1].
NV_FLOAT32 get_equilibrium (NV_INT32 num, NV_INT32 year);
NV_FLOAT32 get_node_factor (NV_INT32 num, NV_INT32 year);
Get the equilibrium argument and node factor for the constituent indicated by num [0,constituents-1], for the year start_year+year.
NV_FLOAT32 *get_equilibriums (NV_INT32 num);
NV_FLOAT32 *get_node_factors (NV_INT32 num);
Get all available equilibrium arguments and node factors for the constituent indicated by num [0,constituents-1].  The return value is a pointer into static memory which is an array of number_of_years floats, corresponding to the years start_year through start_year+number_of_years-1.
NV_INT32 get_time (NV_CHAR *string);
NV_CHAR *ret_time (NV_INT32 time);
NV_CHAR *ret_time_neat (NV_INT32 time);
Convert between character strings of the form "[+-]HH:MM" and the encoding Hours * 100 + Minutes.  ret_time pads the hours with a leading zero when less than 10; ret_time_neat omits the leading zero and omits the sign when the value is 0:00.  Returned pointers point into static memory.
NV_CHAR *ret_date (NV_U_INT32 date);
Convert the encoding Year * 10000 + Month [1, 12] * 100 + Day [1, 31] to a character string of the form "YYYY-MM-DD", or "NULL" if the value is zero.  The returned pointer points into static memory.  (The compact form, without hyphens, is obtainable just by printing the integer.)
NV_INT32 search_station (NV_CHAR *string);
When invoked multiple times with the same string, returns record numbers of all stations that have that string anywhere in the station name.  This search is case insensitive.  When no more records are found it returns -1.
NV_INT32 find_station (NV_CHAR *name);
NV_INT32 find_tzfile (NV_CHAR *name);
NV_INT32 find_country (NV_CHAR *name);
NV_INT32 find_level_units (NV_CHAR *name);
NV_INT32 find_dir_units (NV_CHAR *name);
NV_INT32 find_restriction (NV_CHAR *name);
NV_INT32 find_datum (NV_CHAR *name);
NV_INT32 find_constituent (NV_CHAR *name);
NV_INT32 find_legalese (NV_CHAR *name);
Inverses of the corresponding get_ operations.  Return -1 for not found.
NV_INT32 add_restriction (NV_CHAR *name, DB_HEADER_PUBLIC *db);
NV_INT32 add_tzfile (NV_CHAR *name, DB_HEADER_PUBLIC *db);
NV_INT32 add_country (NV_CHAR *name, DB_HEADER_PUBLIC *db);
NV_INT32 add_datum (NV_CHAR *name, DB_HEADER_PUBLIC *db);
NV_INT32 add_legalese (NV_CHAR *name, DB_HEADER_PUBLIC *db);
Add the value of name to the corresponding lookup table and return the index of the new value.  If db is not NULL, the database header struct pointed to will be updated to reflect the changes.  The maximum length of name is restricted by the corresponding size definition in tide_db_default.h (restriction 30, tzfile 30, country 50, datum 70, legalese 70, including the terminating null).
NV_INT32 find_or_add_restriction (NV_CHAR *name, DB_HEADER_PUBLIC *db);
NV_INT32 find_or_add_tzfile (NV_CHAR *name, DB_HEADER_PUBLIC *db);
NV_INT32 find_or_add_country (NV_CHAR *name, DB_HEADER_PUBLIC *db);
NV_INT32 find_or_add_datum (NV_CHAR *name, DB_HEADER_PUBLIC *db);
NV_INT32 find_or_add_legalese (NV_CHAR *name, DB_HEADER_PUBLIC *db);
Add the value of name to the corresponding lookup table if and only if it is not already present.  Return the index of the value.  If db is not NULL, the database header struct pointed to will be updated to reflect the changes.  The maximum length of name is restricted by the corresponding size definition in tide_db_default.h (restriction 30, tzfile 30, country 50, datum 70, legalese 70, including the terminating null).
void set_speed (NV_INT32 num, NV_FLOAT64 value);
Set the speed for the constituent corresponding to index num [0,constituents-1].
void set_equilibrium (NV_INT32 num, NV_INT32 year, NV_FLOAT32 value);
void set_node_factor (NV_INT32 num, NV_INT32 year, NV_FLOAT32 value);
Set the equilibrium argument and node factor for the constituent corresponding to index num [0,constituents-1], for the year start_year+year.
NV_BOOL open_tide_db (NV_CHAR *file);
Opens the specified TCD file.  If a different database is already open, it will be closed.  libtcd maintains considerable internal state and can only handle one open database at a time.  Returns false if the open failed.
void close_tide_db ();
Closes the open database.
NV_BOOL create_tide_db (NV_CHAR *file, NV_U_INT32 constituents,
    NV_CHAR *constituent[], NV_FLOAT64 *speed, NV_INT32 start_year,
    NV_U_INT32 num_years, NV_FLOAT32 *equilibrium[],
    NV_FLOAT32 *node_factor[]);
Creates a TCD file with the supplied constituents and no tide stations.  Returns false if creation failed.  The database is left in an open state.
DB_HEADER_PUBLIC get_tide_db_header ();
Returns a copy of the database header for the open database.
NV_BOOL get_partial_tide_record (NV_INT32 num, TIDE_STATION_HEADER *rec);
Gets "header" portion of tide record for the station whose record_number is num [0,number_of_records-1] and writes it into rec.  Returns false if num is out of range.  num is preserved in the static variable current_index.
NV_INT32 get_next_partial_tide_record (TIDE_STATION_HEADER *rec);
Invokes get_partial_tide_record for current_index+1.  Returns the record number or -1 for failure.
NV_INT32 get_nearest_partial_tide_record (NV_FLOAT64 lat, NV_FLOAT64 lon,
                                          TIDE_STATION_HEADER *rec);
Invokes get_partial_tide_record for a station that appears closest to the specified lat and lon in the Cylindrical Equidistant projection.  Returns the record number or -1 for failure.
NV_INT32 read_tide_record (NV_INT32 num, TIDE_RECORD *rec);
Gets tide record for the station whose record_number is num [0,number_of_records-1] and writes it into recnum is preserved in the static variable current_record.  Returns num, or -1 if num is out of range.
NV_INT32 read_next_tide_record (TIDE_RECORD *rec);
Invokes read_tide_record for current_record+1.  Returns the record number or -1 for failure.
NV_BOOL add_tide_record (TIDE_RECORD *rec, DB_HEADER_PUBLIC *db);
#ifdef COMPAT114
/* Omission of db parameter was a bug. */
NV_BOOL update_tide_record (NV_INT32 num, TIDE_RECORD *rec);
#else
NV_BOOL update_tide_record (NV_INT32 num, TIDE_RECORD *rec, DB_HEADER_PUBLIC *db);
#endif
NV_BOOL delete_tide_record (NV_INT32 num, DB_HEADER_PUBLIC *db);
Add a new record, update an existing record, or delete an existing record.  If the deleted record is a reference station, all dependent subordinate stations will also be deleted.  Add and update return false if the new record is invalid; delete and update return false if the specified num is invalid or there is insufficient memory available.  If db is not NULL, the database header struct pointed to will be updated to reflect the changes.
NV_BOOL infer_constituents (TIDE_RECORD *rec);
Computes inferred constituents when M2, S2, K1, and O1 are given and fills in the remaining unfilled constituents.  The inferred constituents are developed or decided based on Article 230 of "Manual of Harmonic Analysis and Prediction of Tides," Paul Schureman, C&GS Special Publication No. 98, October 1971.  Returns false if M2, S2, K1, or O1 is missing.

For more information, see Jan Depner, "Format for the Oceanographic and Atmospheric Master Library (OAML) Tide Constituent Database," rev. 2003-06-06.

6. V1 to V2 migration guide

libtcd vs. TCD file version compatibilities

libtcd 1 can only read and modify files written by v1.  Versions of libtcd prior to 1.99 will fail inelegantly when attempting to read v2 files and will corrupt the database if a modification is attempted.  Versions 1.99 and 1.100 will detect and report the version mismatch.

libtcd 2 can read files written by v1 or v2, but it can only create or modify v2 files.  If it is necessary to create a v1 file, you must compile your application using libtcd 1.  Version 1.100 is recommended for this purpose.  It is available at ftp://ftp.flaterco.com/xtide/libtcd-1.100.tar.gz.

To convert a v1 TCD file to a v2 TCD file, use the application rewrite_tide_db in the package tcd-utils, available from http://www.flaterco.com/xtide/files.html.

Compiling old applications without changes

This option is for emergencies only, or in case of such extreme laziness that maintaining the version-integrity of the API is less important than avoiding minor changes to your applications.

Applications written against the TCD version 1 API should compile and run without changes using libtcd 2 if libtcd 2 is built with the COMPAT114 option, as follows:

bash-3.1$ ./configure --enable-COMPAT114

This will allow old applications to read and write version 2 TCD files.  However, the resulting library is NOT API-compatible with libraries built in the normal way, which will break applications that are properly written for the v2 API.  Therefore, any library built with this option is for local use only.  Only a static library will be built.

Even with COMPAT114 enabled, some fields do not behave as they did in libtcd 1.  Specifically, COMPAT114 does the following.

The following subsections detail the application changes that are required and recommended to migrate to the version 2 API without COMPAT114.

Checklist of required changes when migrating applications from v1 API to v2 API

  1. Replace all uses of the units field with level_units.
  2. Replace all uses of NAME_LENGTH and SOURCE_LENGTH with ONELINER_LENGTH.
  3. Replace all uses of COMMENTS_LENGTH with MONOLOGUE_LENGTH.
  4. Provide a third parameter, DB_HEADER_PUBLIC *db, on each invocation of update_tide_record.  It can be NULL if the calling function has no need of the db header.
  5. Stop using the tide record fields pedigree, units, avg_level_units, min_avg_level, and max_avg_level.
  6. Stop using the functions get_pedigree, find_pedigree, add_pedigree, and check_simple.

Checklist of optional changes when migrating applications from v1 API to v2 API

  1. Ensure that the max_direction and min_direction fields are initialized to 361 (the value indicating null) and/or set to their correct values when writing reference stations.  Ensure that direction_units is set properly.
  2. Change variable data types and casts to match the new type of months_on_station.  This was changed from unsigned char (uint8) to unsigned short (uint16) in libtcd 1.100.
  3. Change variable data types and casts to match the new types of the constituents and num_years parameters to create_tide_db and any fields in DB_HEADER_PUBLIC that were changed from signed to unsigned in libtcd 1.99.
  4. Make use of the new fields notes, legalese, xfields, station_id_context, station_id, and date_imported as appropriate.
  5. Make use of the new functions find_or_add..., ...legalese, get_equilibriums, get_node_factors, ret_time_neat, and ret_date as appropriate.
  6. When invoking functions that accept a DB_HEADER_PUBLIC *db parameter, if the db header has no use in the calling function except to fill in this parameter, a NULL can be passed instead.

7. Changelog

libtcd 2.2.2 (libtcd.so.0.0.2)

David Flater
2007-01-22

(Bugs, tide_db.c) Corrected more off-by-one errors in the use of calculate_bits by the function create_tide_db.  The function write_tide_db_header was rounding values when it encoded them but create_tide_db was truncating them when it was calculating the number of bits.  So if the maximum speed, equilibrium argument, or node factor value as encoded in integer form happened to round up to an exact power of 2, an invalid TCD file could have been produced.

(Nit, bit_pack.c) Eliminated the special case in calculate_bits to return 1 on the input 0.  A field whose only possible value is 0 consumes no space in the TCD file but is correctly decoded as 0.

libtcd 2.2.1 (libtcd.so.0.0.1)

David Flater
2007-01-21

(Bug, bit_pack.c) Rewrote calculate_bits to eliminate ambiguous interpretation of "range" argument (now called "value"), eliminate the possibility of roundoff error, return the correct result in the case where the input value is 1, and, as a special case, return 1 if the input value is 0.

(Bugs, tide_db.c) Corrected off-by-one errors in the use of calculate_bits by the function create_tide_db.

The preceding issues could have resulted in the production of an invalid TCD file if the number of constituents, the number of distinct speed values, the number of distinct equilibrium argument values, or the number of distinct node factor values that were required by a particular database happened to be exactly a power of 2 (including 1).  These conditions are unlikely to have occurred in normal use.  I discovered the problems while attempting to generate a test case for XTide using exactly one constituent.

libtcd 2.2 (libtcd.so.0.0.0) release 2

David Flater
2006-11-26

Added warranty disclaimer to COPYING and libtcd.html.  No source code changes.

libtcd 2.2 (libtcd.so.0.0.0)

David Flater
2006-11-22

Split out libtcd into a separate tarball (was previously bundled with XTide).

Started using automake and libtool; started building shared lib.

Resolved preprocessor define clash between automake and libtcd on VERSION.

Merged nvtypes.h and tide_db_version.h into tcd.h to reduce pollution of public include directories.

Removed extraneous #includes from tcd.h.

Changed COMPAT114 into a configure option (--enable-COMPAT114).

Caused tcd.h to be generated at configure time with the following substitutions.

libtcd 2.1.3

David Flater
2006-10-05

Changed asserts with side-effects to use require macro instead.

Minor clean-ups.

libtcd 2.1.2

David Flater
2006-08-03

Improved configure script.

libtcd 2.1.1

David Flater
2006-08-02

Added configure script.

Cleaned up nvtypes.h.

Corrected date on version 2.1 (said 2005, was 2006).

libtcd 2.1

David Flater
2006-05-28

Fixed bug in delete_tide_record:  linkage from subordinate stations to reference stations other than the one being deleted was getting broken.  Bug report credit:  August Hahn.

Fixed a memory management error in update_tide_record.  Bug report credit:  August Hahn.

Added enum for TIDE_RECORD_TYPE.

In update or delete_tide_record, if cannot allocate memory, return failure to the caller instead of exiting.

Added more error traps for invalid input.

Miscellaneous cleanups and nit fixes.

libtcd 2.0

David Flater
2004-10-15

Lots of changes to implement the revised database schema.

COMPAT114 is no longer defined by default in tcd.h (i.e. the v2 API is now effective).

Removed the unused define DEFAULT_NUMBER_OF_OBSOLETE_RECORDS from tide_db_default.h.

Fixed repeated reporting of amplitude/epoch errors.

Fixed wasted byte bug on speeds, equilibrium arguments, and node factors.

libtcd 1.100 "enhanced edition"

David Flater
2004-10-13

With luck, this will be the final release in the version 1 series and the last release that will write files that can be read by libtcd 1.02 through 1.14.  It includes an additional round of cleanups that will enhance upward compatibilty but should not break backward compatibility.

check_simple was placed inside #ifdef COMPAT114 (which is still defined in tcd.h).  AFAIK, this function is now of interest only in restore_tide_db, which uses it to determine which XML format to output.

Copied the libtcd.html descriptions of operations into the header file.

Made the db parameters to add_ and find_or_add_ operations optional (can be null).

Gave delete_tide_record a return value.  Gave update_tide_record a db parameter, conditional on COMPAT114 not being defined.

In delete_tide_record, assert that the record is found.

Deleted unused systemtime var in write_tide_db_header.

Deleted unnecessary file parameter to read_tide_db_header.

Added static function check_tide_record to do sanity checks on records before committing them to the database and called it from add_tide_record and update_tide_record.

Initialized an uninitialized var in search_station.

Added clip_string calls in find_tzfile and find_constituent.

Replaced some weird code in find_constituent with more normal code.

Got rid of unnecessary copy operations in add_ functions.

Renamed tname parms to name.

Added checks to ensure that database is open when expected to be open and closed when expected to be closed.

Added AMPLITUDE_EPSILON and used it where applicable.

Moved duplicate record-size-figuring code into a new function, figure_size.  Fixed wasted byte when record size is even multiple of 8 bits.  (The same bug exists for calculating the size of the speeds, equilibrium arguments, and node factors, but fixing that would break compatibility, and the wastage is at most 3 bytes.)

Changed months_on_station to uint16.

Fixed bounds checking on input to add_ functions -- they all have their own static limits.  Added similar bounds checking in write_tide_db_header.

Lots of documentation nits fixed.

libtcd 1.99 "collector's edition"

David Flater
2004-10-05

With luck, this will be the final release in the version 1 series and the last release that will write files that can be read by libtcd 1.02 through 1.14.  It includes a gigantic round of cleanups to libtcd code that were made in anticipation of the upcoming major revision.  Except for the bug fixes, the changes should be invisible to all wholesome client applications.  Unwholesome client code or the use of unwholesome databases built by old, buggy versions of libtcd may now trigger errors or warnings that did not appear with libtcd 1.14.

In the process of testing the cleanups, I discovered some unrelated, really horrible bugs.

(Bug) Constituents with zero phases were not being encoded by write_tide_record.  The affected reference stations in harmonics-dwf-2004-09-14 are Delfzijl, Netherlands; The Battery, New York Harbor, NY; and South Pass, LA.  write_tide_record now encodes any constituent with a positive amplitude.  Constituents with zero amplitude but nonzero phase are dropped, and negative amplitudes yield assertion failures.  Made analogous changes to update_tide_record (the counts were supposed to match).

(Bug) update_tide_record was counting constituents for subordinate stations.

(Bug) Backed out the change to the control flow in delete_tide_record made in 1.14, which broke the deletion of reference stations with sub stations.

(Bug) Fixed logic error in delete_tide_record that corrupted the database when reference stations with sub stations were deleted.

(Bug) Made read_next_tide_record return -1 if at end of file, instead of going off the deep end.

Following are details of the gigantic cleanup.

The defines NAME_LENGTH, SOURCE_LENGTH and COMMENTS_LENGTH were generalized to ONELINER_LENGTH to set the length of one-line character string buffers and MONOLOGUE_LENGTH to set the length of verbose character string buffers.

COMPAT114 is defined in tcd.h.  The effects are:

Added commentary on the commentary about OS-specific defines in nvtypes.h and tide_db_header.h, noting that they are not actually used.

Cleaned up all warnings and errors compiling under g++ -Wall:

Added asserts everywhere that I came across potentially dangerous assumptions.

Changed -O2 to -O in Makefile (Forte doesn't like O2).

Moved changelogs from source files to libtcd.html.

Made write_tide_record static; no applications should be calling this directly.

In clip_string, expanded static buffer to MONOLOGUE_LENGTH, fixed a bug that was previously masked, simplified code and added bounds checking.

Eliminated repeated calls to clip_string in find functions.

Eliminated clip_string duplication in write_tide_record.

Changed 256-char static buffers in add_pedigree, add_tzfile, add_country, add_datum, and add_restriction to ONELINER_LENGTH and added bounds checking.

Made filename MONOLOGUE_LENGTH.

Made version and last_modified ONELINER_LENGTH.

Handled overlong strings in TCD file by complaining and truncating instead of returning garbarge.

Eliminated get_string.

Eliminated repeated parsing of input in read_tide_db_header.  Changed input buffer size to ONELINER_LENGTH.

Made support for pre-version-1.02 header checksums conditional on COMPAT114.

Changed many int32 fields in the DB header that had no reason ever to be negative to uint32.  Exceptions include start_year (in case some crazy goes B.C.) and the offsets for speeds, equilibrium arguments, and node factors (required to read pre-version-1.11 TCD files).  Also changed record_size in the tide record header, but left alone the rest of the tide record because the risk of breaking client code is too great:  build_tide_db temporarily stores negative results from find functions in these fields, and tideEditor puts -1 in the record_number field to flag a new record.  -1 in reference_station means no reference, though normally it's only -1 for reference stations.

In signature of create_tide_db, constituents and num_years became unsigned.

Changed many looper, index, size, etc. variables from signed to unsigned.

Corrected unsigned scan formats in read_tide_db_header.

Removed unused fields, unused data types and overcomplicated infrastructure in tide_db_header.h and read_tide_db_header.

Corrected signature of bit_unpack in tide_db.c.

Made get_speed, get_equilibrium and get_node_factor give assertion failures instead of returning -1.0 when invoker attempts to retrieve nonsense values.  Added a missing bounds check.

Made set_speed, set_equilibrium and set_node_factor give assertion failures instead of doing nothing when invoker attempts to set nonsense values.  Added a missing bounds check.

Made record_size in TIDE_INDEX struct 32 bits.

Added a warning about TCD files with negative offsets for speeds, eq args or node factors (caused by the bug fixed in 1.11).

Changed format of date string in [LAST MODIFIED] to XTide style.  Changed identifer in [VERSION] to libtcd.

Added these functions:

Added header fields for major and minor revision number.  Added check for attempt to read higher major revision than supported.

libtcd 1.14

David Flater
2004-09-03

(tide_db.c) Certain database update operations that ended with closing and re-opening the database were broken by nulling out the filename in close_tide_db.  Incorporated patch from Jan Depner that rectifies this and tweaks the control flow in delete_tide_record.

libtcd 1.13

David Flater
2004-08-15

(create_tide_db) Write a valid end_of_file record even if someone creates a database with no tide records.

(tide_db_default) Removed a superfluous space character in table of default datum types (after Lowest Astronomical Tide).

libtcd 1.12

David Flater
2003-12-04

(tcd.h) Deleted #include <malloc.h>.

libtcd 1.11

David Flater
2003-11-16

(create_tide_db) Fixed horrible bug:  offsets for speeds, equilibrium args, and node factors were sign-reversed with respect to their usage in read_tide_db_header and write_tide_db_header, resulting in possible overflows.

(read_tide_db_header) Added handling for zero tide records, which happens on new database create.

(open_tide_db) Added check of modified flag to 2003-10-14 code.

(close_tide_db) Deleted repeat free of tindex introduced 2003-10-14.  Cleared modified flag on close.

libtcd 1.10

David Flater
2003-10-14

Incorporated patch from Phil Thornton that closes a memory leak and improves performance on repeat calls to open_tide_db.  See http://www.flaterco.com/xtide/tcd_notes.html

libtcd 1.09

Jan C. Depner
2003-09-04

Bug fix - modifying last record in file and changing size of record caused a big problem.  Duh, I should have seen that one coming.

libtcd 1.08

Jan C. Depner
2003-07-23

Bug fix - deleting last record in file caused a big problem.

libtcd 1.07

David Flater
2003-03-27

(This version was also released briefly without an updated version string.)

Check_simple altered per resolution to abolish "simplified" type 2 records.  See http://www.flaterco.com/xtide/tcd_notes.html

libtcd 1.06

David Flater
2003-03-18

(This version failed to report an updated version string.)

Added NULLSLACKOFFSET to tcd.h per 2003-03-18 change to slack offsets.  See http://www.flaterco.com/xtide/tcd_notes.html

libtcd 1.05

David Flater
2003-03-11

Incorporated nit fixes from August Hahn:  added clip_string(tname) to take care of cases when the country name had leading/trailing spaces, and removed a couple of unused variables.

libtcd 1.04

David Flater
2002-12-13

While integrating 1.03, deleted an unused variable (whoop de do).

libtcd 1.03

Wade Ladner/Jan C. Depner
2002-12-09

Small fix for Micro$oft Windoze.  We don't need to add CR's to strings (apparently).

libtcd 1.02

August Hahn
2002-11-15

Replaced the simplistic checksum with a real CRC.  If the first checksum fails it will check for an old-style checksum.

libtcd 1.01

David Flater
2002-10-01

Many changes since 2002-08-01.  Bumped version upon integration with XTide distribution.  Renamed resulting library to libtcd.

tidelib 1.00

Jan C. Depner
2002-08-01

First tested release.

tidelib 0.99

Jan C. Depner
2002-07-15

Prototype for testing.

8. References

Jan Depner, "Format for the Oceanographic and Atmospheric Master Library (OAML) Tide Constituent Database," rev. 2003-06-06.

libtcd version 1.100.

XTide.

ISO-8859-1.

Manual of Harmonic Analysis and Prediction of Tides.  Special Publication No. 98, Revised (1940) Edition (reprinted 1958 with corrections; reprinted again 1994).  United States Government Printing Office, 1994.