XINA Import

Overview

Cron jobs have been created to automate the import of the PACE/OCI telemetry archives into XINA. There are currently two separate jobs, but both work in a similar way:

• ops_oci - downloads OCI*.oci archives and imports them into the XINA model "ops_oci"
• ops_pace - downloads HSK*.HSK archives and imports them into the XINA model "ops_pace"

The jobs have been scheduled to run every 10 minutes, performing the following:

1. Download up to 5 archives (configurable). Successfully downloaded archives are added to a local file "archiveIndex.txt". Archives in this file will not be downloaded again. If necessary, this file can be edited or even deleted to force the re-downloading of any previously downloaded archives.
2. Process the archive into 10-minute segments and remove science data packets (i.e., APIDs 700 and 720).
3. Upload the processed archives to the appropriate database/model in XINA.

These cron jobs are currently running on the "xina-run" server (run-oci-xina-io) in the $HOME directory of the user "mrburkhart".

Bash Script Details

downloadArchives.sh

This script downloads archives using the "oci_download" application.

Arguments:

• FILTER : A file filter (e.g., oci*.oci)
• MAX_FILES : The maximum number of downloads to process
• DEST : The destination folder for downloaded archives
• START_DATE: The start date to filter archives

Example: downloadArchives.sh "OCI*.oci" 5 "$HOME/ops_oci/raw" "12/04/2023"

segmentArchives.sh

This script processes archives into 10-minute segments using the "tmsegment" application.

Arguments:

• MODEL : The XINA database/model name (example: 'debug')
• FILTER : A file filter (e.g., oci*.oci)
• SOURCE : The source folder with raw archives
• DEST : The destination folder for segmented archives

Example: segmentArchives.sh ops_oci "OCI*.oci" "$HOME/ops_oci/raw" "$HOME/ops_oci/segmented"

uploadToXina.sh

This script uploads the processed archives to XINA using the "xina_importarchive.sh" bash script.

Arguments:

• MODEL : The XINA database/model name (example: 'debug')
• SOURCE : The source folder containing the segmented archives

Example: uploadToXina.sh ops_oci "$HOME/ops_oci/segmented"

check.sh

This script combines the above three scripts into a single script.