Introduction

Although XINA is very flexible and can be configured to meet almost any data organization requirements, we have defined standard organization principles for common use cases with pre-built front end tooling. These are not hard limitations, just recommendations based on past experience, performance benchmarks, and cost/benefit analysis.

Data Model

Abstractly, a data model (or simply model) is defined as having a set of synchronously relevant data. For example, a project might have a flight model, ETU model, etc. Models store data in independent databases, and multiple models may be importing data in parallel.

Broadly we use time as the primary method to organize and synchronize data within a model. In XINA this is represented as an 8-byte unsigned integer Unix time with microsecond precision. We use Unix time because it is:

• Widely and consistently supported
• Time zone independent
• Efficiently converted

Other time formats may be available for data export depending on project requirements.

Project Organization

A project should be defined by a single XINA group at the top level. Each model is then defined by a single XINA group, which contain all groups and databases associated exclusively with the model. These should either be defined in the project group, or may be subdivided into category groups.

A project may use a mix of both approaches or additional levels of subcategories if required, but it is recommended to either use a flat structure or single level of category groups to avoid confusion. Models may be referred to by the path relative to their project group (in the above example, model_a would be referenced as model_a or category_a.model_a, respectively).

Project and category groups may also include additional groups and databases of data or resources which are not model specific, such as journals or definitions databases. In most cases with standard structures, models will default to databases or groups within the model, but search for them up the tree if not found. A complete project group might look like:

Project Configuration

A group is defined as a project by the xs_model_project key. The value is a JSON object with the following definition:

Key	Value	Default
def_mn	relative path to mnemonic definitions database	def.mn
def_prof	path to profile definitions database	def.prof
def_plot	path to plot definitions database	def.plot

A group is defined as a category by the xs_model_category key. The value is a JSON object extending the definition of the xs_model_project key, automatically inheriting any unset values from the project configuration.

All models are required to provide an mn_def, prof_def, and plot_def database. It is strongly recommended that these be shared by the entire project, and that all models use the same temporal precision, to maximize intercompatibility between models.

Model Organization

Data within a model falls into four primary classifications:

• Telemetry
  • source data file(s) from data collection point
  • typically stored in a raw (sometimes binary) format
  • storage cost is cheap
  • accessing data means downloading files or most likely requires custom XINA tools
  • may be divided into multiple data sources (see below)
• Viewable Data
  • extracted from telemetry into XINA database(s)
  • telemetry is the single source of truth for this data, not intended to be user editable
    • (except under controlled circumstances with model-aware API calls)
  • add data is either mnemonic, instant, or interval (see below)
  • can be accessed and analyzed with built-in XINA tools
  • storage is expensive
  • optimizations may be needing depending on project requirements, data volumes
• User Metadata
  • additional data added by users, often directly through the XINA interface
  • XINA likely the primary repoository for this data
  • for example, a journal
• Definitions / References
  • may be user entered or defined outside XINA
  • may exist at model level or above (category/project level)
  • more formal and restricted than user metadata

Data Source

Abstractly, a data source (or simply source) is a single point of data import to a model. Formally, each feed of telemetry must align to a single data source, and each telemetry file must provide all data for that source for the time range of data in that file.

In many cases, a model will only have a single data source; for example, if all data is provided directly from a single instrument, or multiple components are merged into a single data stream through FEDS before import into XINA. In these cases delineation by data source is not required in model organzation, and should use this pattern:

However, in environments with multiple import points running in parallel, databases must be designed with multiple sources.

Note in this example each source has distinct databases for telemetry, instant, interval, and mnemonic data. This would be required if each data source provided all three data types. As requirements for instants and intervals are less stringent than mnemonics, in some circumstances instants and intervals could be considered a single source and populated independently:

Model Configuration

A group is recognized as a model if the xs_model key is set in the group objects. The value is a JSON object extending the definition of the xs_model_project key, automatically inheriting any unset values from the parent project or category configuration.