XINA Import

Overview

Cron jobs have been created to automate the import of the PACE/OCI telemetry archives into XINA. There are currently two separate jobs, but both work in a similar way:

The jobs have been scheduled to run every 10 minutes, performing the following:

  1. Download up to 5 archives (configurable). Successfully downloaded archives are added to a local file "archiveIndex.txt". Archives in this file will not be downloaded again. If necessary, this file can be edited or even deleted to force the re-downloading of any previously downloaded archives.
  2. Process the archive into 10-minute segments and remove science data packets (i.e., APIDs 700 and 720).
  3. Upload the processed archives to the appropriate database/model in XINA.

These cron jobs are currently running on the "xina-run" server (run-oci-xina-io) in the $HOME directory of the user "mrburkhart".

Bash Script Details

downloadArchives.sh

This script downloads archives using the "oci_download" application.

Arguments:

Example: downloadArchives.sh "OCI*.oci" 5 "$HOME/ops_oci/raw" "12/04/2023"

segmentArchives.sh

This script processes archives into 10-minute segments using the "tmsegment" application.

Arguments:

Example: segmentArchives.sh ops_oci "OCI*.oci" "$HOME/ops_oci/raw" "$HOME/ops_oci/segmented"

uploadToXina.sh

This script uploads the processed archives to XINA using the "xina_importarchive.sh" bash script.

Arguments:

Example: uploadToXina.sh ops_oci "$HOME/ops_oci/segmented"

check.sh

This script combines the above three scripts into a single script.

Revision #2
Created 25 January 2024 19:06:06 by Michael Burkhart
Updated 25 January 2024 19:47:48 by Michael Burkhart