Skip to main content

CSV / TSV Format Reference

The csv and tsv formats provide a standard delimited text file format for mnemonic data.

Source File Format

Files must be either ASCII or UTF-8 encoded. New lines will be interpretted from either \n or \r\n. The conf object may define other customization of the format:

Conf Definition

Key Value Default Description
delimiter string auto detect (',', '\t', ';') value delimiter
quote_char character " (double quote character) value quote character
ignore_lines number 0 lines to ignore after UUID and before header
mode row or col row mnemonic mode (see below)
t auto, iso8601, s, ms, us auto time format (see below)
zone string time zone to use if not provided
offset string time offset to use if not provided

The first line must contain an appropriately generated 128-bit UUID in the standard 36 character format.

If mode is row, the file must contain three columns, in this order:

  • t (Unix time or ISO8601 zoned timestamp)
  • mn (name or ID)
  • v (numeric, empty, or null)

For example (whitespace added for clarity, not required):

123e4567-e89b-12d3-a456-426614174000
t , mn    , v
0 , v_mon , 1
0 , i_mon , 5
1 , t_mon , 100
2 , v_mon , 1.1
2 , i_mon , 4
3 , t_mon ,
4 , v_mon , 1.2
4 , i_mon , 3
5 , t_mon , 101

If mode is col, the file must first contain a time column, followed by a column for each mnemonic. The column headers must specify the mnemonic name or ID for each column. Unlike row, null values must be spelled out explicitly, as empty values will not create a point in the database.

For example, the following is equivalent to the above example (whitespace added for clarity, not required):

123e4567-e89b-12d3-a456-426614174000
t       , v_mon , i_mon , t_mon
0       , 1     , 5     ,
1       ,       ,       , 100
2       , 1.1   , 4     ,
3       ,       ,       , null
4       , 1.2   , 3     ,
5       ,       ,       , 101

Time Parsing

The mode of time processing is determined by the value for t in conf. The auto mode attempts to interpret the most likely formatting for the timestamp. If the value is an integer or floating point format, it will be interpretted as a Unix timestamp, with precision based on these rules:

  • t > 1e16: error, value above typical range
  • t > 1e14: microseconds
  • t > 1e11: milliseconds
  • t > 1e8: seconds
  • t <= 1e8: error, value below typical range

Otherwise it will be interpretted as a zoned ISO8601 timestamp. If t is set explicitly in the configuration the time will always be interpretted in that context.

Import Processing and Validation

The MODEL_TM_IMPORT action (reference here) provides some file processing and validation.

The UUID and any metadata are parsed from the file and included as fields in the XINA record. These are considered the single source of truth for these fields, if they are also specified explicity in the import action an error will be thrown.

CSV Mining Utility

The XINA Model CSV Mining Utility mines mnemonic data from CSV / TSV files, which must adhere to the format defined here.

Options

Back to top