MOMA

MOMA Alerts

A MOMA Alert is defined as a notification of certain parameters of interest in a single MOMA Test. This page details the logic we use to extract the data for each alert. Note that this is the master reference, which means the engineers and scientists should ensure that the logic detailed here is correct. All alert data is in an Alert Panel within a typical TID record __TOC__

General Error Check

Severity : Notice

MAIF APV Comm Check

Severity : Warning

Spacewire Comm Error

Severity : Warning

Severity : Notice

MOMA Safe Event

Severity : Notice

WRP Too Fast

Severity : Notice

WRP Failed to Start

Severity : Warning

WRP Pressure Spike

Severity : Warning

Dark Counts

Severity : Warning

Laser Burst Energy Variation

Severity : Info

Laser Pulses Mismatch

Severity : Notice

Unknown SEB Sequences

Severity : Notice

Filament VMON Pegged

Severity : Warning

Pulse Count | Packet Size Mismatch

Severity : Warning

Summed Scan | TIC Mismatch

Severity : Warning

Gap in Scans

Severity : Warning

cFactor | TIC Mismatch

Severity : Warning

Scan Status | TIC Mismatch

Severity : Warning

Dropped Packets

Severity : Warning

Both EMs On

Severity : Warning

Limit Violations

Severity : Warning

MOMA C++ Mining Scripts

Introduction

The MOMA C++ mining scripts are command-line tools built on top of MOMA Data View code.

The primary purpose of these scripts is to extract data into a XINA Online-friendly format. They currently live in mine699.gsfc.nasa.gov in the /home/mpallone/cpp-playground directory (although I intend to move them into /usr/local/bin as soon as I can get permission...):

If running these executables on mine699, be sure to be logged in as a user whose momagse directory is configured and up to date, such as the 'moma' user.

Housekeeping mining script

Absolute filename: /home/mpallone/cpp-playground/extract_hk_data_main

From the file's header comment:

   /**
    * 
    * @file   extract_hk_data_mainclass.cpp 
    * @author Mark Pallone 
    * @date   December, 2015
    * 
    * A script to extract HK data from a TMFile in order to populate XINA online 
    * databases. 
    * 
    * Data will be written to the specified output directory with the preferred 
    * XINA format. That is, one datapoint per line as follows: 
    * 
    *     utc time,fsw time,relative time,hkid,science value
    * 
    * Note that times will be in microseconds, not seconds, unless the --seconds 
    * flag is used.
    * 
    * The name of the output file will be: 
    *     
    *     <tid_number>{=html}_hk_data.csv    
    * 
    * Example usage: 
    * 
    *     ./extract_hk_data_main  --infile <full path to tm.mom file>   
    *                             --outdir <full path to output directory>{=html}  
    *                             --hkids 1,2,5-10,15-20  --dhkids 30-50,55
    * 
    * HKIDs listed immediately after the --hkids flag will have *all* of their 
    * values stored. 
    * 
    * Delta HKIDs listed next to the --dhkids flag will have their initial value
    * stored, and subsequent values will only be stored if they differ from the 
    * previous value for that particular HKID. 
    * 
    * One or both of --hkids, --dhkids must be specified. 
    */

MOMA Consumables

A MOMA consumable is defined as any piece of hardware in the MOMA instrument that has a limited lifetime. What exactly "lifetime" means is dependent on the specific piece of hardware we are interested in and a single consumable may have more than one way to define its lifetime. This page details the logic we use to extract the data for each consumable. Note that this is the master reference, which means the engineers and scientists should ensure that the logic detailed here is correct. All consumable data is available on XINA Online at https://ssed.gsfc.nasa.gov/xina/xo/tool/moma-timeline.

WRP Consumable

On Time

WRP on time is determined by calculating the total time that PWM_SPEED (HKID 832) is above 500 RPM.

Cycles

The number of WRP cycles is determined by counting the number of times the PWM speed (HKID 832) goes about 30,000 RPM.

NOTE: This current logic counts some of the failed WRP Startups as a cycle. See trend for failed startups.

Filament Consumable

On Time

Filament 1 and Filament 2 on time are determined by calculating the amount of time that SEB IS_EMON_A and IS_EMON_B (HKIDs 17 and 18) are greater than 1 uA.

Cycles

Filament 1 and Filament 2 cycles are determined by counting the number of times the filaments transition from off to on. A few notes about the algorithm:

EM Consumable

On Time

EM 1 and EM 2 on times are determined by summing the on times for all scans, and then looking for any time when the EMs were constantly on outside of scanning.

SEB sequences are examined to determine the on time for individual scans. In order to avoid double-counting scans, only sum packet (packet type 27) scans are examined when in EI mode, and only science packet (packet type 26) scans are examined when in LDI mode.

Additionally, outside of science scans, the total time that the EMs are on are summed. This is determined by summing the amount of time that EM A and EM B are below -1900 V. HKIDs 22 and 23 are examined in this phase of the algorithm.

Bradley Tse pointed out a flaw in this algorithm:

There is a corner case where that may introduce its own problems. If the user stops the script after the Sequence was dumped but before scanning started (maybe because they noticed one of the parameters was incorrect), then that means the SebSequences will be duplicated.

And so EM on time for all scans after that (until the next Sequence Dump) will have 2x the amount (or however many times the script was stopped before scans occurred)

Full Cycles and Soft Cycles

EM Cycles are determined through two passes of the data.

The first pass of EM Cycles are counted from the scan status packets. The cycles are calculated differently depending on the type of mode the test is in.

If in LDI mode [1], if HKID 222 (SCN_EM1) or HKID 223 (SCN_EM2) is below the -1900V threshold, then a full cycle is added to the corresponding detector's count. If in Dark Counts mode [2], no full cycles are added here, as typically a Dark Counts run contains only 1 full cycle total. If in EI mode [3], soft counts are added to EM1 if HKID17 (IS_EMON_A) is greater than 9, and the same procedure for EM2 and HKID18 (EMON_B).

The timestamps of the scan packets are used to determine gaps in time when scans are not occurring during the test. A second pass through but looking specifically in the gaps and the binned HK data is necessary to avoid missing any full or soft cycles.

Here, HKID47 (SEB: EM_1_DAC0_SET) and HKID48 (SEB: EM_2_DAC1_SET) are looked through individually. If the HK value is > -50V, the detector is OFF. If the HK value is < -1900 the detector is ON. Full cycles are added for each occurrence where a detector changes from the OFF state to the ON state within a gap.

[1] - LDI Mode is when Filaments are OFF ([HKIDs 17/18] < 9), and the RF (HKID 235) is ON (End Amplitude >= 500A)

[2] - Dark Counts Mode is when Filaments are OFF ([HKIDs 17/18] < 9), and the RF (HKID 235) is OFF (End Amplitude < 500A)

[3] - EI Mode is when a Filament is ON ([HKIDs 17 or 18] > 9)

EM Counts

EM 1 and EM 2 counts are calculated according to the logic below. If the most recent DAC value (HKIDs 222 and 223) is zero, then ions are not counted against the multiplier.

   # dummy variable to help when counting ions in science packet (packet 26) scans
   temp_counts = 0 
   #
   for each packet in the tm.mom file: 
       if the packet indicates that filament A or filament B is on (HKIDs 105 and 102):
           set the current scan type to EI
       else:
           set the current scan type to LDI 
       #
       else if the packet type is a scan status packet (type 25): 
          if we're in LDI mode: 
              count temp_counts against whichever EM is on (or both, if they're both on) 
              set temp_counts back to 0
       #
       else if the current mode is LDI mode and the current packet type is a science packet (type 26): 
           temp_counts += packet.total_ion_count() 
       # 
       else if the current mode is EI mode and the current packet type is a sum packet (type 27): 
           count packet.total_ion_count() against whichever EM is on (or both, if they're both on) 
       #
       # special-case code for handling dark count scans: 
       else if the current mode is LDI, and the current packet is a sum packet (type 27), and the packet's RF DAC value (HKID 405) < 50:
           count packet.total_ion_count() against whichver EM is on (or both, if they're both on)

Laser Consumable

Laser Shots

Laser shots are determined by summing the number of main pulses (HKID 1050) and the number of pre pulses (HKID 1001).

Valve Consumable

MS Valve Cycles

Valve cycles are determined by examining the message log and counting the number of times a valve transitions from closed to opened. When counting valve cycles for a TID, the state of the valve cycles in the previous TID is accounted for. The exact regexes used (as of Feb 22) are:

VALVE_OPENED_REGEX = r'.*valve \d opened$'
VALVE_CLOSED_REGEX = r'.*valve \d closed$'

PCV Cycles

Counts the number of times the Pressure Control Valve (PCV) on the GC has been opened. This is found by adding all the times the PCV On bit in the GC Control register (mask 0x4000 of register 0x3A)(HKID 712) transitions from 0 to 1.

PCV On Time

This is the total powered time of the PCV. This is determined by calculating the duration of time when the PCV On bit (HKID 712) is set to 1 in a TID.

GC Flow Time is calculated the same way, but was made as a separate consumable so it could be reset when the Helium tank is refilled.

Bus Energy Consumable

Bus 1 energy is calculated by summing HKID 939, and then dividing that sum by 3600 to get watt-hours.

Bus 2 energy and Bus 3 energy are calculated in the same way, except HKIDs 940 and 941 are used.

Bus 4 energy is always assumed to be zero.

Miscellaneous

Duration

This is the sum of the durations of all TIDs. Currently, each TID's duration is calculated by simply subtracting earliest_packet.timestamp from last_packet.timestamp. However, we plan on making this calculation more accurate by ignoring rover simulation packets.

Aperture Valve

This is the number of times the "apv open" message appears in the message log. NOTE: This does not account for APV opens during commissioning since the "apv open" message is out output.

MOMA Data View

MOMA Data View is an application built with the Qt framework that is used to interpret, display, and analyze MOMA data. It can be used for both real time and post processing purposes. At this time we only support Macs. __TOC__

Installation

Prerequisites

Tunneling into momaioc

This step creates a "tunnel" to the momaioc server, which basically gives you access to all of the files on the server. The momaioc server contains our Subversion repository which is where we store all of the data, apps, and files.

1. Open up a terminal and paste this command into the terminal, replacing AUID with your actual AUID:

ssh AUID@momaioc.gsfc.nasa.gov -L 6994:localhost:3690

If that command complains about an unknown hostname, then use this command instead:

ssh AUID@128.183.20.207 -L 6994:localhost:3690

2. You should now be prompted for your password. Your password will have the following format (omitting the carrots when you actually enter it): <RSA_PIN> <RSA Generated Numbers> Be careful about entering your password because after 3 consecutive unsuccessful attempts you will be locked out.

3. You have now successfully created a tunnel to the momaioc server, which means we can now access the data and files! You do not need to do anything else with this terminal tab. It should be left open for the duration of your work session. All other terminal work should be done in a different tab.

Downloading momagse

The momagse directory contains various configuration files and all of our apps, including MOMA Data View.

1. Open a new terminal tab

2. Change to your home directory:

cd ~

3. Download/checkout the momagse directory:

svn co svn://localhost:6994/momagse)

4. You should now have the momagse directory!

Installing MOMA Data View

1. Mount the MOMA Data View dmg

open ~/momagse/Apps/momadataview.dmg

2. A Finder window should have opened with the MOMA Data View application in it. Drag and drop the application to your Applications directory.

3. Unmount the MOMA Data View dmg

4. You now have MOMA Data View installed!

Updating to a newer version

Note: The following instructions assume you have met all of the prerequisites.

1. Create your momaioc tunnel

2. Open a new terminal tab or window.

3. Update your momagse directory:

   svn up ~/momagse

4. Follow the same steps in Installing MOMA Data View except this time replace the pre-existing app.

Getting the data

1. Create your momaioc tunnel

2. Open a new tab or terminal window. If it does not already exist, create a momadata directory in your home folder and change into it.

mkdir ~/momadata
cd ~/momadata

3. If you do not have any data, then use the following commands to check out the data:

svn co svn://localhost:6994/momadata/etu etu 
svn co svn://localhost:6994/momadata/fm fm 
svn co svn://localhost:6994/momadata/qsm qsm

If you already have data, then you can update your working copy as follows:

svn up ~/momadata/etu ~/momadata/fm ~/momadata/qsm

Build Environment

The following configuration is used to build MOMA Data View:

Version History

The changelog is ordered from most recent version to oldest. Because a changelog has not been consistently maintained for the lifetime of the application, versioning only starts at 3.0.0.

5.2.0

Revision: 9728

Release Date: 2021-04-02


5.1.0

Revision: 9588

Release Date: 2020-12-18


5.0.2

Revision: 9531

Release Date: 2020-11-23


5.0.1

Revision: 9300

Release Date: 2020-10-08


5.0.0

Revision: 8994

Release Date: 2020-07-29


4.25.0

Revision: 5528

Release Date: 2017-06-28


4.24.4

Revision: 5425

Release Date: 2017-06-02


4.24.3

Revision: 5351

Release Date: 2017-05-22


4.24.2

Revision: 5328

Release Date: 2017-05-12


4.24.1

Revision: 5249

Release Date: 2017-05-1


4.24.0

Revision: 5113

Release Date: 2017-04-19


4.23.6

Revision: 5013

Release Date: 2017-04-10


4.23.5

Revision: 4985

Release Date: 2017-03-28


4.23.4

Revision: 4976

Release Date: 2017-03-24


4.23.3

Revision: 4939

Release Date: 2017-03-15


4.23.2

Revision: 4870

Release Date: 2017-03-02


4.23.1

Revision: 4776

Release Date: 2017-02-14


4.23.0

Revision: 4762

Release Date: 2017-02-10


4.22.7

Revision: 4676

Release Date: 2017-01-26


4.22.6

Revision: 4625

Release Date: 2017-01-14


4.22.5

Revision: 4623

Release Date: 2017-01-13


4.22.4

Revision: 4596

Release Date: 2017-01-06


4.22.3

Revision: 4563

Release Date: 2016-12-02


4.22.2

Revision: 4482

Release Date: 2016-11-04


4.22.1

Revision: 4475

Release Date: 2016-11-03


4.22.0

Revision: 4426

Release Date: 2016-10-27


4.21.1/2

Revision: 4391

Release Date: 2016-10-18


4.21.0

Revision: 4355

Release Date: 2016-10-12


4.20.0

Revision: 4023

Release Date: 2016-08-16


4.19.1

Revision: 3859

Release Date: 2016-07-15


4.19.0

Revision: 3845

Release Date: 2016-07-14


4.18.6

Revision: 3822

Release Date: 2016-07-08


4.18.5

Revision: 3818

Release Date: 2016-07-07


4.18.4

Revision: 3812

Release Date: 2016-07-05


4.18.3

Revision: 3773

Release Date: 2016-06-22


4.18.2

Revision: 3765

Release Date: 2016-06-20


4.18.1

Revision: 3726

Release Date: 2016-06-06


4.17.3

Revision: 3540

Release Date: 2016-04-27


4.17.2

Revision: 3442

Release Date: 2016-04-7


4.17.1

Revision: 3434

Release Date: 2016-04-7


4.17.0

Revision: 3403

Release Date: 2016-04-4


4.16.0

Revision: 3295

Release Date: 2016-03-22


4.15.3

Revision: 3265

Release Date: 2016-03-19


4.15.2

Revision: 3254

Release Date: 2016-03-15


4.15.1

Revision: 3250

Release Date: 2016-03-15


4.15.0

Revision: 3235

Release Date: 2016-03-10


4.14.0

Revision: 3187

Release Date: 2016-02-29


4.13.5

Revision: 3158

Release Date: 2016-02-25


4.13.4

Revision: 3147

Release Date: 2016-02-24


4.13.3

Revision: 3122

Release Date: 2016-02-22


4.13.2

Revision: 3097

Release Date: 2016-02-18


4.13.1

Revision: 3090

Release Date: 2016-02-18


4.13.0

Revision: 3088

Release Date: 2016-02-18


4.12.0

Revision: 3083

Release Date: 2016-02-17


4.11.2

Revision: 2980

Release Date: 2016-01-19


4.11.1

Revision: 2937

Release Date: 2016-01-05


4.11.0

Revision: 2888

Release Date: 2015-12-17

Updated to support version 2 of the tm.meta


4.10.0

Revision: 2823

Release Date: 2015-12-3


4.9.3

Revision: 2809

Release Date: 2015-11-23


4.9.2

Revision: 2786

Release Date: 2015-11-17

Bug Fixes


4.9.1

Revision: 2768

Release Date: 2015-11-11

Bug Fixes


4.9.0

Revision: 2749

Release Date: 2015-11-5

Bug Fixes

New Features

Removed


4.8.1

Revision: 2744

Release Date: 2015-11-5

Bug Fixes

New Features


4.8.0

Revision: 2732

Release Date: 2015-11-3

Bug Fixes

New Features


4.7.0

Revision: 2584

Release Date: 2015-09-24

Bug Fixes

New Features


4.6.3

Revision: 2555

Release Date: 2015-09-18

Bug Fixes

New Features


4.6.2

Revision: 2491

Release Date: 2015-09-03

Bug Fixes

New Features


4.6.1

Revision: 2477

Release Date: 2015-09-02

Bug Fixes


4.6.0

Revision: 2473

Release Date: 2015-09-01

Significant changes

New Features

Bug fixes

Bug fixes


4.5.1

Revision: 2321

Release Date: 2015-07-30

Bug fixes


4.5.0

Revision: 2303

Release Date: 2015-07-21

Significant change

Bug fixes


4.4.1

Revision: 2288

Release Date: 2015-07-16

Bug fixes


4.4.0

Revision: 2275

Release Date: 2015-07-13

Significant change

New Features

Bug fixes


4.3.0

Revision: 2238

Release Date: 2015-06-23

Significant change

New Features

Bug fixes


4.2.3

Revision: 2218

Release Date: 2015-06-15

Bug fixes


4.2.2

Revision: 2189

Release Date: 2015-06-04

Bug fixes


4.2.1

Revision: 2181

Release Date: 2015-06-03

Bug fixes


4.2.0

Revision: 2178

Release Date: 2015-06-02

New Features

Bug fixes


4.1.1

Revision: 2021

Release Date: 2015-04-02


4.1.0

Revision: 2021

Release Date: 2015-03-31


4.0.2

Revision: 1931

Release Date: 2015-03-12


4.0.1

Revision: 1931

Release Date: 2015-03-11


4.0.0

Revision: 1929

Release Date: 2015-03-10


3.3.0

Revision: 1792

Release Date: 2015-02-06


3.2.2

Revision: 1776

Release Date: 2015-02-03


3.2.1

Revision: 1770

Release Date: 2015-02-02


3.2.0

Revision: 1736

Release Date: 2015-01-26


3.1.0

Revision: 1684

Release Date: 2015-01-07


3.0.0

Revision: 1629

Release Date: 2014-12-09

MOMA MAIF Voltages

L 802 -m 50 -s -l 1.4800 -h 1.5000 \# MAIF:CDH_1P5V https://goo.gl/r6Fd9Q


L 803 -m 50 -s -l 2.0900 -h 2.1000 \# MAIF:2P7V https://goo.gl/1XKlk2


L 812 -m 50 -s -l 1.4700 -h 1.5100 \# MAIF:MAIF_1P5VMON https://goo.gl/GDR2QN


L 813 -m 50 -s -l 3.2500 -h 3.3300 \# MAIF:MAIF_3P3VMON
L 814 -m 50 -s -l 5.000 -h 5.100 \# MAIF:MAIF_5VMON
L 815 -m 50 -s -l 14.500 -h 14.700 \# MAIF:MAIF_P13VMON
L 816 -m 50 -s -l -14.700 -h -14.500 \# MAIF:MAIF_M13VMON
L 817 -m 50 -s -l 16.000 -h 16.500 \# MAIF:MAIF_MOT_VMON 
L 818 -m 50 -s -l 3.5000 -h 5.0000 \# MAIF:WRP1_SUM 
L 824 -m 50 -s -l 0.2000 -h 0.2500 \# MAIF:HEATER_IMON
L 827 -m 50 -s -l 1.0000 -h 1.2000 \# MAIF:MAIN_IMON 
L 828 -m 50 -s -l 0.5000 -h 0.7000 \# MAIF:SEB_50VAC_IMON 
L 830 -m 50 -s -l 0.4500 -h 0.7500 \# MAIF:MOTOR_SUP_IMON

MOMA Mailing List

The MOMA Mailing list is used for any general announcements pertaining to MOMA data and the data analysis software. If you interested in either of those, it is highly recommended you sign up for the mailing list so you can keep up to date. Sign up at https://lists.nasa.gov/mailman/listinfo/moma-data-and-software

MOMA Mass Calibration

This page details how mass calibration is accomplished for MOMA's mass spectrometer. The mass calibration config file is currently being used by MOMA Data View, our python tools, and XINA Online.

Mass Cal Config File Format

All Versions

Conventions

Fields

Version 1

Fields

 0 = BIN
 1 = RF_MON
 2 = RF_DAC

 [1.0, 2.0, 3.0] --> m/z = 3x^2 + 2x + 1

Example

{  
  "control_mode":2,
  "entries":[  
     {  
        "constants":[  
           0.0,
           1.0,
           0.0
        ],
        "data_points":[  
        ],
        "order":0,
        "phase_cycle":-1,
        "rms":0.0
     },
     {  
        "constants":[  
           3.04691,
           0.568583,
           0.0
        ],
        "data_points":[  
           {  
              "amu":132.905,
              "control":228.389
           },
           {  
              "amu":392.715,
              "control":685.332
           }
        ],
        "order":1,
        "phase_cycle":3,
        "rms":1.0
     },
     {  
        "constants":[  
           6.34889,
           0.992971,
           0.0
        ],
        "data_points":[  
           {  
              "amu":132.905,
              "control":127.214
           },
           {  
              "amu":392.715,
              "control":389.253
           },
           {  
              "amu":652.525,
              "control":651.162
           },
           {  
              "amu":912.335,
              "control":912.074
           }
        ],
        "order":1,
        "phase_cycle":6,
        "rms":0.999999
     }
  ],
  "version":2
}

MOMA Meta Markers

This page defines the specification that software should adhere to in order to support MOMA Meta Markers. Meta Markers were created to improve our post-processing capabilities by allowing us to add markers after the telemetry has already been generated. This functionality may be used for any number of purposes:

A Meta Marker consists of

Unlike normal telemetry markers, which are packets inside of a tm.mom file, Meta Markers are generated by post-processing software based on predefined rules. Despite their simple format, human beings do not define Meta Markers in terms of these four fields. Instead, humans define rules which software uses to generate Meta Markers. The format and other details of these rules are described below.

Note that Meta Markers are different from Annotations. Meta Markers are generated by software to augment XINA Online's trending capability, whereas Annotations are generated by human beings using MOMA Data View in order to describe events within a single TID.

Storing Markers

Markers should be stored in a .json file in the momagse/MetaMarkerRules directory.

Example: 50050.json

Meta Marker 50050 is intended to adjust marker 50 by 10 seconds. These fields are defined in a following section.

{
  "meta_markers": [
    { 
      "tids": "0-39999",
      "meta_marker_id": 50050, 
      "meta_marker_text": "Adjusted Background Collection",
      "start_conditions": [
        {
          "type": "marker",
          "marker": 50,
          "offset_in_seconds": 10
        }
      ], 
      "end_conditions": [
        {
          "type": "next_marker" 
         }
      ]
    }
  ]
}

In this example, any time marker 50 is encountered, a corresponding marker 50050 will also be created.

Meta Marker Rule Format

Note that all .json files in the momagse/MetaMarkerRules directory will be used. So, Meta Marker Rule filenames can be named anything, as long as they have a .json extension.

Root Key

Meta Marker Object Fields

Each Meta Marker object has the following fields:

The start_conditions array is a set of trigger objects.

The end_conditions array is a set of trigger objects.

Supported Trigger Types

The trigger type dictates how the marker's timestamps should be calculated. The different trigger types and their fields are described below.

Marker

The meta marker is defined relative to another marker ID.

Message

The meta marker starts when a line in the message log matches a given regular expression.

Duration

This trigger only applies to end_conditions.

Next Marker

This trigger only applies to end_conditions. If this trigger is set, then the Meta Marker end timestamp will be equal to the timestamp of the next telemetry marker.

Future Trigger Types

These trigger types are not currently supported, but might be in the future if the need arises.

Time

The marker is defined with a timestamp.

Housekeeping Value

The marker is triggered by a HK value.

Conventions

If a Meta Marker is relative to a normal telemetry marker (i.e., a marker packet in a tm.mom file), define the Meta Marker ID as original_mkid + 50000. This is convention only. If more than one Meta Marker is relative to the same marker, a different offset will be necessary.

Meta Markers cannot be relative to other Meta Markers.

A separate file should be used for each Meta Marker Rule.

Dependencies

This section details the dependencies that the Meta Markers have on the telemetry data. Ideally, any of the telemetry data that Meta Markers depend on should not change, but if it must, then whoever created the Meta Marker should be notified so that the Meta Marker can be updated.

Messages

MOMA Python Packages

This page lists useful Python (and some shell) scripts that facilitate data extraction and analysis, and links to install packages containing these scripts. A more detailed description of library capabilities can be found here.

To determine which version of 699util you have installed, you can type pip list in your terminal.

Installation Instructions

For installation instructions, go to the Python Tool Installation page, or contact MOMA software developer.

If you already have a package installed and just want to upgrade to a new package, you can:

List of Scripts

Script Name Description
c_apv_ops.py count the number of APV operations
c_emcount.py count ions against the multipliers
c_emruntime.py determine how long EMs were on
c_expectedvaluechk.py report actual vs. expected housekeeping values
c_extract_hkid_928.py stream HKID 928 (the 1 KHz RSIM current) data to a file (this HKID is too dense for c_tmfields.py)
c_filament_cycles.py count filament cycles
c_ioncount.py performs various ion counting calculations for MOMA scans
c_lasercount.py count laser pulses
c_look_for_missing_sci_data.py print errors for missing science data
c_maifruntime.py returns the amount of time the WRP was on for the telemetry file
c_momaconsume.py run various consumable scripts
c_momafilTime.py determine how long filaments were on
c_momascan.py plot voltage vs. ion counts
c_pktsummary.py print a table of packet types and number of occurrences
c_plotdatarate.py plot the instrument's data rate from the Rover's perspective
c_print_meta_markers.py print metamarkers that apply to the tm.mom file
c_pwm_cycles.py count PWM cycles
c_sync_moma_model_data.py sync new lab data to SVN
c_t0.py print t0 for various time systems
c_tid_to_mzml.py output science scans in MZML format
c_tidsummary.py output a JSON file summarizing TID data
c_tmfields.py output housekeeping values
c_tmmarker.py print markers in the tmfile
c_tmmsg.py print the message log
c_tmplot.py plot housekeeping data
c_tmsequence.py check the packet sequence numbers and report anomalies
c_tmsummary.py print a summary of each packet in the telemetry file
c_valve_2_open_time.py determine how long valve 2 was open
c_valve_cycles.py count the number of valve cycles across a range of TIDs
add_gse_pkts.py add or modify the GSE packets at the beginning of telemetry files
boxcar.py determine boxcar avaerage of two columns of data
checksum.py calculate the fletcher checksum of files or input from stdin
evref copy an expected value reference file to current directory and run expectedvaluechk.py
extract_science_data.py write science data to files
filterExpectedValues.py filter out ExpectedValue.txt lines for which there are no markers in the tm file
fixPacket1.py creates tm.mom.fix such that packet 1 is modified to match the cwd
iniget.py parse an INI file and print a specific value
make_metafile.py make a tm.meta file for a TID
output_patched_database.py apply patch files, and print out the resulting database
playtm.py replay a telemetry file packet by packet
plotref copy a plot reference file to the CWD and execute it
print_sequence_numbers_and_timestamps.py what a descriptive name
pump_down_trend.py trend APV pressure pulse information in LDI mode
rf_chk.py output RF monitor values for a given bin
runall.py execute a command across all (or a range of) TIDs
threshold_timestamp.py print the time that an HKID went above (or below) a threshold
tids.sh defines convenient functions to jump to TID folders
time_between_msgs.py print the amount of time between messages in a message log (or between two occurrences of the same message)
tmdump.py generate a hex dump of a telemetry file, separated by packets
tmexcerpt.py filter data from a telemetry file to a smaller telemetry file
tmkeys.py parse and print the telemetry database

What's with all this "c_" stuff?

Scripts that begin with "c_" are actually using MOMA Data Viewer code to examine MOMA telemetry files. In many cases, you can omit the "c_" (for example, by running tmfields.py instead of c_tmfields.py) to instead use the pure-Python version of the script. However, we no longer support the pure-Python telemetry extraction library, so its output will become less reliable as time goes on.

I want to write my own script!

MOMA Python packages will create the directory "${HOME}/custom_699_utils" and add it to your PATH. So, users can write a Python script that uses the MOMA Python telemetry extraction library, stick it in this directory, make it executable, and then use it like any other script. Future package installs will not modify the contents of this directory, so your scripts will persist.

Bug Reports

Please do not report bugs on this wiki. Instead, use the Bugzilla server we have set up for bug tracking and reporting. Once you have set up an account, you may file a bug for the python software at the page for Command-line Utils.

Releases

3.03.12

DMG (Mac) Installation Package: https://s3.amazonaws.com/699_bin/699util-3.03.12.dmg

C++ Revision: 4942

Python Revision: 2278

Release Date: 2017-03-16

3.03.11

DMG (Mac) Installation Package: https://s3.amazonaws.com/699_bin/699util-3.03.11.dmg

C++ Revision: 4934

Python Revision: 2274

Release Date: 2017-03-15

3.03.09

DMG (Mac) Installation Package: https://s3.amazonaws.com/699_bin/699util-3.03.09.dmg

C++ Revision: 4693

Python Revision: 2252

Release Date: 2017-01-30

3.03.08

DMG (Mac) Installation Package: https://s3.amazonaws.com/699_bin/699util-3.03.08.dmg

C++ Revision: 4666

Python Revision: 2247

Release Date: 2017-01-24

3.03.06

DMG (Mac) Installation Package: https://s3.amazonaws.com/699_bin/699util-3.03.06.dmg

C++ Revision: 4608

Python Revision: 2239

Release Date: 2017-01-12

3.03.05

DMG (Mac) Installation Package: https://s3.amazonaws.com/699_bin/699util-3.03.05.dmg

C++ Revision: 4592

Python Revision: 2234

Release Date: 2017-01-10

3.03.04

DMG (Mac) Installation Package: https://s3.amazonaws.com/699_bin/699util-3.03.04.dmg

C++ Revision: 4592

Python Revision: 2233

Release Date: 2016-12-16

3.03.03

DMG (Mac) Installation Package: https://s3.amazonaws.com/699_bin/699util-3.03.03.dmg

C++ Revision: 4592

Python Revision: 2231

Release Date: 2016-12-15

3.03.02

DMG (Mac) Installation Package: https://s3.amazonaws.com/699_bin/699util-3.03.02.dmg

C++ Revision: 4592

Python Revision: 2229

Release Date: 2016-12-15

3.03.01

DMG (Mac) Installation Package: https://s3.amazonaws.com/699_bin/699util-3.03.01.dmg

C++ Revision: 4591

Python Revision: 2227

Release Date: 2016-12-14

3.03.00

DMG (Mac) Installation Package: https://s3.amazonaws.com/699_bin/699util-3.03.00.dmg

C++ Revision: 4589

Python Revision: 2222

Release Date: 2016-12-14

3.02.03

DMG (Mac) Installation Package: https://s3.amazonaws.com/699_bin/699util-3.02.03.dmg

C++ Revision: 4509

Python Revision: 2215

Release Date: 2016-11-15

3.02.02

DMG (Mac) Installation Package: https://s3.amazonaws.com/699_bin/699util-3.02.02.dmg

C++ Revision: 4483

Python Revision: 2200

Release Date: 2016-11-04

3.01.01

DMG (Mac) Installation Package: https://s3.amazonaws.com/699_bin/699util-3.01.01.dmg

Revision: 2175

Release Date: 2016-10-14

3.01.00

DMG (Mac) Installation Package: https://s3.amazonaws.com/699_bin/699util-3.01.00.dmg

Revision: 2167

Release Date: 2016-10-03

3.00.02

DMG (Mac) Installation Package: https://s3.amazonaws.com/699_bin/699util/699util-3.00.02.dmg

Revision: 2156

Release Date: 2016-08-30

3.00.01

DMG (Mac) Installation Package: https://s3.amazonaws.com/699_bin/699util-3.00.01.dmg

Revision: 2155

Release Date: 2016-08-30

3.00.00

DMG (Mac) Installation Package: https://s3.amazonaws.com/699_bin/699util/699util-3.0.0.dmg

Revision: 2151

Release Date: 2016-08-22

MOMA Python Time Synchronization

This applies to r1920.

The code responsible for packet timestamp resolution is the BufferedMomaPacketList class inside of momapacketlist.py.

MOMA Script Validation Process

MOMA Scripts are thoroughly reviewed and tested before being approved for flight following the MOMA Script Validation Process. The validation process involves one or more (sometimes many more) meetings with representatives from the science, systems, electrical and software teams review each line of the script as well as one or more runs on the software testbed and engineering test unit. The intent of the process is to be thorough while still reacting quickly within the surface operations strategic and tactical flow.

MOMAScriptProcess.png

Scripts that are in the review process or that have already been approved are identified as "Activities" and listed in XINA. See link below.

The activity on XINA maintains the all the notes, versions, durations, energy, etc. that go along with the script (or in some cases scripts). During surface operations as well as integration and test activities with the ExoMars rover, the XINA activities will be accessible to the ExoMars team for access to all the support data (primarily duration, energy and data volume) and restrictions (such a environmental temperature and pressure) associated with a scipt.

For surface operations, at a minimum, the following three documents will be attached to the activity: 1. Meeting Notes. Includes attendees names, action items, etc. 2. Safety Critical Check List. Spreadsheet contain specific checks for safety critical items the will be completed by the reviewers. 3. Restrictions Document. List any environmental, electrical or other states that restrict the state MOMA must be in to run the script.

In addition, fields in the XINA contain the duration, energy, data volume, filename and version, file checksum, review status (Approved for Flight, Approved for ETU, Under Development) and other keys items.

The scripts themselves are text files and they are maintained in the NASA GSFC Subversion Version Control system (SVN). SVN maintains each checked in version and automatically replaces fields within the file such that the script can identify its SVN version number when it runs. After a script is approved for flight, it is "blessed". In this process, the filename is changed to contain the revision ID number of the script and placed in another SVN folder reserved for flight approved scripts.

Note that as of this writing in August 2016, the method of delivering an approved script to surface operations team or the integration and test team at TAS-I has not been defined.

MOMA SEB Voltages


L 001 -m 50 -s -l 4.950 -h 5.0550 # SEB:CTL_+5D_VMON https://goo.gl/Qk2mZF


L 002 -m 50 -s -l 13.000 -h 13.900 # SEB:CTL_+13A_VMON https://goo.gl/QoFHZH


L 003 -m 50 -s -l -13.600 -h -13.250 # SEB:CTL_-13A_VMON https://goo.gl/FzEuwX


L 004 -m 50 -s -l 1.5000 -h 1.5080 # SEB:CTL_+1.5D_VMON https://goo.gl/nfXBiK


L 005 -m 50 -s -l 3.3000 -h 3.3500 # SEB:CTL_+3.3_VMON https://goo.gl/k1EHS8


L 006 -m 50 -e -l -10.00 -h 10.00 # SEB:N_-5KV_MON https://goo.gl/32S5nR


https://goo.gl/nSofs2 Note that on TID 30165, the delay after turning on SEB was short, so during background collection, it was still stabilizing to 0. That is why the min/max are much bigger on that TID than others.


L 008 -m 50 -s -l -5.000 -h 5.0000 # SEB:RF_AGC_MON https://goo.gl/gU2cvi


L 009 -m 50 -s -l -1.800 -h 0.5000 # SEB:GC_EC(HV1-1)_VMON https://goo.gl/Q3J6nd


L 017 -m 50 -e -l -0.1000 -h 0.1000 # SEB:IS_EMON_A https://goo.gl/TG0CSg


L 018 -m 50 -e -l -0.1000 -h 0.1000 # SEB:IS_EMON_B https://goo.gl/Kk28je


L 019 -m 50 -s -l 0.900 -h 1.0000 # SEB:FIL_VMON https://goo.gl/i9Xu0a


L 020 -m 50 -s -l 0.0100 -h 0.0150 # SEB:FIL_IMON https://goo.gl/XA1tlv


L 022 -m 50 -e -l -0.100 -h 1.0000 # SEB:EM-1(HV2-3)_VMON https://goo.gl/fYZG7d This plot shows the same sudden change in value on the same TID as did HK 009. Believe this indicates a change in the database not reflected in previously mined data.


L 023 -m 50 -e -l -0.100 -h 1.0000 # SEB:EM-2(HV2-4)_VMON https://goo.gl/No44D3 Also shows a shift between TIDs 30339 and 30342.


L 024 -m 50 -e -l -1.000 -h 1.0000 # SEB:SRC_A_FOC_A(-100_HV1)_VMON https://goo.gl/8Gcd14 Also shows a shift between TIDs 30339 and 30342.


L 025 -m 50 -e -l -1.000 -h 1.0000 # SEB:SRC_B_FOC_A(-100_HV2)_VMON https://goo.gl/8QjIwa Also shows a shift between TIDs 30339 and 30342.


L 026 -m 50 -e -l -0.100 -h 1.0000 # SEB:LDI_EC(HV1-2)_VMON https://goo.gl/b77OsC Also shows a shift between TIDs 30339 and 30342.


L 027 -m 50 -e -l -0.100 -h 0.5000 # SEB:ROD_BIAS(HV1-3)_VMON https://goo.gl/1C3GcX


L 028 -m 50 -e -l -0.100 -h 0.5000 # SEB:OUTPUT_A(EI_ACCEL)_VMON https://goo.gl/T1t1E6


L 029 -m 50 -s -l 4.9500 -h 5.0500 # SEB:CTL_+5VREF_VMON https://goo.gl/8aNgcH


L 030 -m 50 -s -l -5.0500 -h -4.950 # SEB:CTL_-5VREF_VMON https://goo.gl/pcl63Y


L 031 -m 50 -s -l 2.5000 -h 2.8500 # SEB:RF_PRES_MON https://goo.gl/v3pqdb

Noted that the RF Pressure changes with ambient barometric pressure at GSFC and MI


L 062 -m 50 -e -l -0.100 -h 0.1000 # SEB:RF_AMP_MON https://goo.gl/Diunps


MOMA System Locale

The system locale is a FSW variable that can be set, and the scripts can use to determine what model we are using.

MOMA Trending

SEB Voltages

MAIF Voltages

Temperatures

MOMA/XINA Data Mining

java -jar /mine699/app/x3/xina_commit_watch.jar
-archive /mine699/mission/moma/archive
-config /mine699/mission/moma/config.watch.json
-data /mine699/mission/moma/data
-import /mine699/mission/moma/import
-java /mine699/env/jre1.8.0_45/bin/java
-label MOMA
-mailto "nick.dobson@gmail.com;eric.i.lyness@nasa.gov"
-mailhost mailhost.gsfc.nasa.gov
-python2 /mine699/env/python2.7/bin/python2.7
-python3 /mine699/env/python3.4/bin/python3.4
-rev /mine699/mission/moma/rev
-svn /usr/bin/svn
-svnlook /usr/bin/svnlook
-timeout 43200 >> /mine699/mission/moma/log/watch.log 2>&1 &