Skip to main content

CSV / TSV Format Reference

The csv and tsv formats provide a standard delimited text file format for mnemonic data.

TelemetrySource File Format

Files shouldmust be either ASCII or UTF-8 encoded. New lines will be interpretted from either \n or \r\n. The conf object may define other customization of the format:

Conf Definition

Key Value Default Description
delimit string for csv: , (comma), for tsv: \t (tab) value delimiter
quote character " (double quote character) value quote character
moderow or colrowmnemonic mode (see below)
t auto, iso8601, s, ms, us auto time format (see below)

The first line must contain an appropriately generated 128-bit UUID in the standard 36 character format.

This may optionally be followed by one or more metadata values. These are treated as (key, value) pairs stored in XINA alongside the file. Each key must be unique in the file, must not be empty, and must not start with theIf $mode (dollar sign) character. The values may be any JSON value. If the value starts with a [ (brace) or { (bracket) character it will be interpretted as a JSON array or object, respectively. Literal values true and false will be stored as booleans. If the value is numeric it will be stored as a number. If the value is empty it will be stored as null. Otherwise, it will be stored as a string.

The file must then have a row starting with one of two values: $mn_row or $mn_col. These indicate the end of any metadata and the start of the file data.

For $mn_row, the file must contain three columns, in this order:

  • time t (Unix time or ISO8601 zoned timestamp)
  • mnemonic mn (name or ID)
  • value v (numeric, empty string,empty, or null)

For example (whitespace added for clarity, not required):

123e4567-e89b-12d3-a456-426614174000
bldg,t 37, room,mn    123, $mn_rowv
0,0 v_mon,, v_mon , 1
0,0 i_mon,, i_mon , 5
1,1 t_mon,, t_mon , 100
2,2 v_mon,, v_mon , 1.1
2,2 i_mon,, i_mon , 4
3,3 t_mon,, 4,t_mon v_mon,,
4 , v_mon , 1.2
4,4 i_mon,, i_mon , 3
5,5 t_mon,, t_mon , 101

ForIf $mn_colmode is col, the file must first contain a time column, followed by a column for each mnemonic. The rowcolumn starting with $mn_colheaders must specify the column headers with the mnemonic name or ID for each column. Unlike $mn_rowrow, null values must be spelled out explicitly, as empty string values will not create a point in the database.

For example, the following is equivalent to the above example (whitespace added for clarity, not required):

123e4567-e89b-12d3-a456-426614174000
bldg, 37
room, 123
$mn_colt       , v_mon , i_mon , t_mon
0       , 1     , 5     ,
1       ,       ,       , 100
2       , 1.1   , 4     ,
3       ,       ,       , null
4       , 1.2   , 3     ,
5       ,       ,       , 101

Time Parsing

The mode of time processing is determined by the value for t in conf. The auto mode attempts to interpret the most likely formatting for the timestamp. If the value is an integer or floating point format, it will be interpretted as a Unix timestamp, with precision based on these rules:

  • t > 1e16: error, value above typical range
  • t > 1e14: microseconds
  • t > 1e11: milliseconds
  • t > 1e8: seconds
  • t <= 1e8: error, value below typical range

Otherwise it will be interpretted as a zoned ISO8601 timestamp. If t is set explicitly in the configuration the time will always be interpretted in that context.

Import Processing and Validation

The MODEL_TM_IMPORT action (reference here) provides some file processing and validation.

The UUID and any metadata are parsed from the file and included as fields in the XINA record. These are considered the single source of truth for these fields, if they are also specified explicity in the import action an error will be thrown.

CSV Mining Utility

The XINA Model CSV Mining Utility mines mnemonic data from CSV / TSV files, which must adhere to the format defined here.

Options

Back to top