Skip to main content

Structs DSV Format

The XINA Structs DSV format provides a standard delimited text data file format. This is recommended for data files attached to events, and forms the basis for the structs buffer file format.

Files have certain standard requirements:

  • Must be UTF-8 encoded
  • New lines will be interpretted from either \n or \r\n
  • Blank lines will be ignored
  • Lines starting with the # character are treated as comments and ignored

The conf object may define other customization of the format:

Key Value Default Description
delimiter string auto detect (',', '\t', ';') value delimiter
quote_char character " (double quote character) value quote character
ignore_lines number 0 lines to ignore at the start of the file
mode "row" or "col" auto-detect file row/col format (see below)
t "auto", "iso8601", "s", "ms", or "us" "auto" time format (see below)
zone string time zone to use if not provided
invalid "ignore", null, or number "ignore" preferred interpretation of invalid literal
nan "ignore", null, or number "ignore" preferred interpretation of 'Nan' literal
p_infinity "ignore", null, or number "ignore" preferred interpretation of positive 'Infinity' literal
n_infinity "ignore", null, or number "ignore" preferred interpretation of negative 'Infinity' literal

It is strongly recommended to include a unique appropriately generated 128-bit UUID in the standard 36 character format as a comment in the first processed line of each file. (If ignore_lines > 0, this would be the first line after that number of lines.)

The first processed uncommented line will be intepretted as the column header. If the mode property is "row", the file must contain three columns:

Name Description Alternate Names
t Unix time or ISO8601 zoned timestamp time, timestamp
k key key, mn, mnemonic, n, name
v value (numeric, empty, or null) val, value

The header is used to determine the order of the columns.

For example (whitespace added for clarity, not required):

# 123e4567-e89b-12d3-a456-426614174000
t , k     , v
0 , v_mon , 1
0 , i_mon , 5
1 , t_mon , 100
2 , v_mon , 1.1
2 , i_mon , 4
3 , t_mon ,
4 , v_mon , 1.2
4 , i_mon , 3
5 , t_mon , 101

If mode is "col", the file must first contain a time column, followed by a column for each mnemonic. The column headers must specify the mnemonic name or ID for each column. Unlike row, null values must be spelled out explicitly, as empty values will not create a point in the database.

For example, the following is equivalent to the above example (whitespace added for clarity, not required):

# 123e4567-e89b-12d3-a456-426614174000
t       , v_mon , i_mon , t_mon
0       , 1     , 5     ,
1       ,       ,       , 100
2       , 1.1   , 4     ,
3       ,       ,       , null
4       , 1.2   , 3     ,
5       ,       ,       , 101

If the mode property is not specified, the mode will be determined by the number of columns in the file. If there are exactly 3 columns with names matching the required columns for the "row" mode, that mode is used; otherwise the file is assumed to use the column mode.

Time Parsing

The mode of time processing is determined by the value for t in conf. The auto mode attempts to interpret the most likely formatting for the timestamp. If the value is an integer or floating point format, it will be interpretted as a Unix timestamp, with precision based on these rules:

  • t > 1e16: error, value above typical range
  • t > 1e14: microseconds
  • t > 1e11: milliseconds
  • t > 1e8: seconds
  • t <= 1e8: error, value below typical range

Otherwise it will be interpretted as a zoned ISO8601 timestamp. If t is set explicitly in the configuration the time will always be interpretted in that context. The ISO timestamp may use the standard format: 2023-05-31T17:55:07.000 or condensed 20230531T175507.000. If the zone property provided in the configuration, the timestamps do not require a zone. Otherwise they must include an explicit zone.

No Comments
Back to top