Skip to main content

CSV / TSV Format Reference

The csv and tsv formats provide a standard delimited text file format for mnemonic data.

Telemetry File Format

Files should be ASCII or UTF-8 encoded. New lines will be interpretted from either \n or \r\n. The conf object may define other customization of the format:

Key Value Default Description
delimit string for csv: , (comma), for tsv: \t (tab) value delimiter
quote character " (double quote character) value quote character
t auto, iso8601, s, ms, us auto time format (see below)

The first line must contain an appropriately generated 128-bit UUID in the standard 36 character format.

This may optionally be followed by one or more metadata values. These are treated as (key, value) pairs stored in XINA alongside the file. Each key must be unique in the file, must not be empty, and must not start with the $ (dollar sign) character. The values may be any JSON value. If the value starts with a [ (brace) or { (bracket) character it will be interpretted as a JSON array or object, respectively. Literal values true and false will be stored as booleans. If the value is numeric it will be stored as a number. If the value is empty it will be stored as null. Otherwise, it will be stored as a string.

The file must then have a row starting with one of two values: $mn_row or $mn_col. These indicate the end of any metadata and the start of the file data.

For $mn_row, the file must contain three columns, in this order:

  • time (Unix time or ISO8601 zoned timestamp)
  • mnemonic (name or ID)
  • value (numeric, empty string, or null)

For example (whitespace added for clarity, not required):

123e4567-e89b-12d3-a456-426614174000
bldg, 37
room, 123
$mn_row
0, v_mon, 1
0, i_mon, 5
1, t_mon, 100
2, v_mon, 1.1
2, i_mon, 4
3, t_mon,
4, v_mon, 1.2
4, i_mon, 3
5, t_mon, 101

For $mn_col, the file must first contain a time column, followed by a column for each mnemonic. The row starting with $mn_col must specify the column headers with the mnemonic name or ID for each column. Unlike $mn_row, null values must be spelled out explicitly, as empty string values will not create a point in the database.

For example, the following is equivalent to the above example (whitespace added for clarity, not required):

123e4567-e89b-12d3-a456-426614174000
bldg, 37
room, 123
$mn_col , v_mon , i_mon , t_mon
0       , 1     , 5     ,
1       ,       ,       , 100
2       , 1.1   , 4     ,
3       ,       ,       , null
4       , 1.2   , 3     ,
5       ,       ,       , 101

Time Parsing

The mode of time processing is determined by the value for t in conf. The auto mode attempts to interpret the most likely formatting for the timestamp. If the value is an integer or floating point format, it will be interpretted as a Unix timestamp, with precision based on these rules:

  • t > 1e16: error, value above typical range
  • t > 1e14: microseconds
  • t > 1e11: milliseconds
  • t > 1e8: seconds
  • t <= 1e8: error, value below typical range

Otherwise

Import Processing and Validation

The MODEL_TM_IMPORT action (reference here) provides some file processing and validation.

The UUID and any metadata are parsed from the file and included as fields in the XINA record. These are considered the single source of truth for these fields, if they are also specified explicity in the import action an error will be thrown.

CSV Mining Utility

The XINA Model CSV Mining Utility mines mnemonic data from CSV / TSV files, which must adhere to the format defined here.

Options

Back to top