Processing data with runots.py

Data are processed using the umbrella run script runots.py. With this script the user specifies the instrument and step to be applied to the data.

For most instruments, this is a two-step process:

1. instrument data to raw CDF

2. raw CDF to clean .nc

For some instruments (usually those measuring waves) there is a third step:

3. clean .nc to wave-statistics .nc

The user will call runots.py from the command line, with arguments described below. The autogenerated documentation below can be a little challenging to interpret, so here’s an example:

runots.py aqd hdr2cdf gatts_file.txt config_file.yaml

The above command specifies aqd (Aquadopp collecting in currents mode) and hdr2cdf (step 1 above, instrument data to raw CDF), using the specified configuration files.

To get to the final .nc file, the user would then run runots.py aqd cdf2nc raw_cdf_file_name.cdf

To get final wave-statistics .nc files for supported instruments, the user has 2 potential options nc2waves and nc2diwasp with a clean .nc file of continuous or burst data as a positional argument

To find out what options are available for a specific instrument the user can run runots.py <insert instrument code> -h

runots.py

Run USGS CMHRP ocean time-series data processing system.

usage: runots.py [-h]
                 {abss,aqd,aqdhr,aqdwvs,eco,eofe,exo,glx,hobo,iq,lisst,mc,rbr,rdi,rsk,sgtid,sgwvs,sig,tb,tcm,troll,vec,wvs,wxt} ...

Instruments

Specify one of the instruments in the list below

instrument

Possible choices: abss, aqd, aqdhr, aqdwvs, eco, eofe, exo, glx, hobo, iq, lisst, mc, rbr, rdi, rsk, sgtid, sgwvs, sig, tb, tcm, troll, vec, wvs, wxt

Sub-commands

abss

AQUAscat1000R

runots.py abss [-h] {mat2cdf,cdf2nc} ...

Steps

Specify one of the steps in the list below

step

Possible choices: mat2cdf, cdf2nc

Sub-commands

mat2cdf

Convert instrument data to raw .cdf

runots.py abss mat2cdf [-h] gatts config

Positional Arguments

gatts

path to global attributes file (gatts formatted)

config

path to instrument configuration file (YAML formatted)

cdf2nc

Convert raw .cdf to clean .nc

runots.py abss cdf2nc [-h] [--atmpres ATMPRES] cdfname

Positional Arguments

cdfname

raw .cdf filename

Named Arguments

--atmpres

path to cdf file containing atmopsheric pressure data

aqd

Aquadopp (currents)

runots.py aqd [-h] {hdr2cdf,cdf2nc} ...

Steps

Specify one of the steps in the list below

step

Possible choices: hdr2cdf, cdf2nc

Sub-commands

hdr2cdf

Convert instrument data to raw .cdf

runots.py aqd hdr2cdf [-h] gatts config

Positional Arguments

gatts

path to global attributes file (gatts formatted)

config

path to instrument configuration file (YAML formatted)

cdf2nc

Convert raw .cdf to clean .nc

runots.py aqd cdf2nc [-h] [--atmpres ATMPRES] cdfname

Positional Arguments

cdfname

raw .cdf filename

Named Arguments

--atmpres

path to cdf file containing atmopsheric pressure data

aqdhr

Aquadopp HR

runots.py aqdhr [-h] {hdr2cdf,cdf2nc} ...

Steps

Specify one of the steps in the list below

step

Possible choices: hdr2cdf, cdf2nc

Sub-commands

hdr2cdf

Convert instrument data to raw .cdf

runots.py aqdhr hdr2cdf [-h] gatts config

Positional Arguments

gatts

path to global attributes file (gatts formatted)

config

path to instrument configuration file (YAML formatted)

cdf2nc

Convert raw .cdf to clean .nc

runots.py aqdhr cdf2nc [-h] [--atmpres ATMPRES] cdfname

Positional Arguments

cdfname

raw .cdf filename

Named Arguments

--atmpres

path to cdf file containing atmopsheric pressure data

aqdwvs

Aquadopp Waves

runots.py aqdwvs [-h] {wad2cdf,cdf2nc,nc2waves} ...

Steps

Specify one of the steps in the list below

step

Possible choices: wad2cdf, cdf2nc, nc2waves

Sub-commands

wad2cdf

Convert instrument data to raw .cdf

runots.py aqdwvs wad2cdf [-h] gatts config

Positional Arguments

gatts

path to global attributes file (gatts formatted)

config

path to instrument configuration file (YAML formatted)

cdf2nc

Convert raw .cdf to clean .nc

runots.py aqdwvs cdf2nc [-h] [--atmpres ATMPRES] cdfname

Positional Arguments

cdfname

raw .cdf filename

Named Arguments

--atmpres

path to cdf file containing atmopsheric pressure data

nc2waves

Undocumented

runots.py aqdwvs nc2waves [-h] ncname

Positional Arguments

ncname

processed .nc filename

eco

WET Labs ECO

runots.py eco [-h] {csv2cdf,cdf2nc} ...

Steps

Specify one of the steps in the list below

step

Possible choices: csv2cdf, cdf2nc

Sub-commands

csv2cdf

Convert instrument data to raw .cdf

runots.py eco csv2cdf [-h] gatts config

Positional Arguments

gatts

path to global attributes file (gatts formatted)

config

path to instrument configuration file (YAML formatted)

cdf2nc

Convert raw .cdf to clean .nc

runots.py eco cdf2nc [-h] [--atmpres ATMPRES] cdfname

Positional Arguments

cdfname

raw .cdf filename

Named Arguments

--atmpres

path to cdf file containing atmopsheric pressure data

eofe

EofE ECHOLOGGER

runots.py eofe [-h] {log2cdf,cdf2nc} ...

Steps

Specify one of the steps in the list below

step

Possible choices: log2cdf, cdf2nc

Sub-commands

log2cdf

Convert instrument data to raw .cdf

runots.py eofe log2cdf [-h] gatts config

Positional Arguments

gatts

path to global attributes file (gatts formatted)

config

path to instrument configuration file (YAML formatted)

cdf2nc

Convert raw .cdf to clean .nc

runots.py eofe cdf2nc [-h] [--atmpres ATMPRES] cdfname

Positional Arguments

cdfname

raw .cdf filename

Named Arguments

--atmpres

path to cdf file containing atmopsheric pressure data

exo

YSI EXO

runots.py exo [-h] {csv2cdf,cdf2nc} ...

Steps

Specify one of the steps in the list below

step

Possible choices: csv2cdf, cdf2nc

Sub-commands

csv2cdf

Convert instrument data to raw .cdf

runots.py exo csv2cdf [-h] gatts config

Positional Arguments

gatts

path to global attributes file (gatts formatted)

config

path to instrument configuration file (YAML formatted)

cdf2nc

Convert raw .cdf to clean .nc

runots.py exo cdf2nc [-h] [--atmpres ATMPRES] cdfname

Positional Arguments

cdfname

raw .cdf filename

Named Arguments

--atmpres

path to cdf file containing atmopsheric pressure data

glx

Geolux Wave Radar

runots.py glx [-h] {dat2cdf,cdf2nc,nc2waves} ...

Steps

Specify one of the steps in the list below

step

Possible choices: dat2cdf, cdf2nc, nc2waves

Sub-commands

dat2cdf

Convert instrument data to raw .cdf

runots.py glx dat2cdf [-h] gatts config

Positional Arguments

gatts

path to global attributes file (gatts formatted)

config

path to instrument configuration file (YAML formatted)

cdf2nc

Convert raw .cdf to clean .nc

runots.py glx cdf2nc [-h] [--atmpres ATMPRES] cdfname

Positional Arguments

cdfname

raw .cdf filename

Named Arguments

--atmpres

path to cdf file containing atmopsheric pressure data

nc2waves

Undocumented

runots.py glx nc2waves [-h] ncname

Positional Arguments

ncname

processed .nc filename

hobo

Onset HOBO

runots.py hobo [-h] {csv2cdf,cdf2nc} ...

Steps

Specify one of the steps in the list below

step

Possible choices: csv2cdf, cdf2nc

Sub-commands

csv2cdf

Convert instrument data to raw .cdf

runots.py hobo csv2cdf [-h] gatts config

Positional Arguments

gatts

path to global attributes file (gatts formatted)

config

path to instrument configuration file (YAML formatted)

cdf2nc

Convert raw .cdf to clean .nc

runots.py hobo cdf2nc [-h] [--atmpres ATMPRES] cdfname

Positional Arguments

cdfname

raw .cdf filename

Named Arguments

--atmpres

path to cdf file containing atmopsheric pressure data

iq

SonTek IQ

runots.py iq [-h] {mat2cdf,cdf2nc} ...

Steps

Specify one of the steps in the list below

step

Possible choices: mat2cdf, cdf2nc

Sub-commands

mat2cdf

Convert instrument data to raw .cdf

runots.py iq mat2cdf [-h] gatts config

Positional Arguments

gatts

path to global attributes file (gatts formatted)

config

path to instrument configuration file (YAML formatted)

cdf2nc

Convert raw .cdf to clean .nc

runots.py iq cdf2nc [-h] [--atmpres ATMPRES] cdfname

Positional Arguments

cdfname

raw .cdf filename

Named Arguments

--atmpres

path to cdf file containing atmopsheric pressure data

lisst

Sequoia Scientific LISST

runots.py lisst [-h] {csv2cdf,cdf2nc} ...

Steps

Specify one of the steps in the list below

step

Possible choices: csv2cdf, cdf2nc

Sub-commands

csv2cdf

Convert instrument data to raw .cdf

runots.py lisst csv2cdf [-h] gatts config

Positional Arguments

gatts

path to global attributes file (gatts formatted)

config

path to instrument configuration file (YAML formatted)

cdf2nc

Convert raw .cdf to clean .nc

runots.py lisst cdf2nc [-h] [--atmpres ATMPRES] cdfname

Positional Arguments

cdfname

raw .cdf filename

Named Arguments

--atmpres

path to cdf file containing atmopsheric pressure data

mc

Seabird MicroCAT

runots.py mc [-h] {asc2cdf,cdf2nc} ...

Steps

Specify one of the steps in the list below

step

Possible choices: asc2cdf, cdf2nc

Sub-commands

asc2cdf

Convert instrument data to raw .cdf

runots.py mc asc2cdf [-h] gatts config

Positional Arguments

gatts

path to global attributes file (gatts formatted)

config

path to instrument configuration file (YAML formatted)

cdf2nc

Convert raw .cdf to clean .nc

runots.py mc cdf2nc [-h] [--atmpres ATMPRES] cdfname

Positional Arguments

cdfname

raw .cdf filename

Named Arguments

--atmpres

path to cdf file containing atmopsheric pressure data

rbr

RBR

runots.py rbr [-h] {csv2cdf,cdf2nc,nc2waves,nc2diwasp} ...

Steps

Specify one of the steps in the list below

step

Possible choices: csv2cdf, cdf2nc, nc2waves, nc2diwasp

Sub-commands

csv2cdf

Convert instrument data to raw .cdf

runots.py rbr csv2cdf [-h] gatts config

Positional Arguments

gatts

path to global attributes file (gatts formatted)

config

path to instrument configuration file (YAML formatted)

cdf2nc

Convert raw .cdf to clean .nc

runots.py rbr cdf2nc [-h] [--atmpres ATMPRES] cdfname

Positional Arguments

cdfname

raw .cdf filename

Named Arguments

--atmpres

path to cdf file containing atmopsheric pressure data

nc2waves

Undocumented

runots.py rbr nc2waves [-h] ncname

Positional Arguments

ncname

processed .nc filename

nc2diwasp

Undocumented

runots.py rbr nc2diwasp [-h] ncname

Positional Arguments

ncname

processed .nc filename

rdi

Teledyne RDI ADCP

runots.py rdi [-h] {mat2cdf,cdf2nc} ...

Steps

Specify one of the steps in the list below

step

Possible choices: mat2cdf, cdf2nc

Sub-commands

mat2cdf

Convert instrument data to raw .cdf

runots.py rdi mat2cdf [-h] gatts config

Positional Arguments

gatts

path to global attributes file (gatts formatted)

config

path to instrument configuration file (YAML formatted)

cdf2nc

Convert raw .cdf to clean .nc

runots.py rdi cdf2nc [-h] [--atmpres ATMPRES] cdfname

Positional Arguments

cdfname

raw .cdf filename

Named Arguments

--atmpres

path to cdf file containing atmopsheric pressure data

rsk

RBR

runots.py rsk [-h] {csv2cdf,cdf2nc,nc2waves,nc2diwasp} ...

Steps

Specify one of the steps in the list below

step

Possible choices: csv2cdf, cdf2nc, nc2waves, nc2diwasp

Sub-commands

csv2cdf

Convert instrument data to raw .cdf

runots.py rsk csv2cdf [-h] gatts config

Positional Arguments

gatts

path to global attributes file (gatts formatted)

config

path to instrument configuration file (YAML formatted)

cdf2nc

Convert raw .cdf to clean .nc

runots.py rsk cdf2nc [-h] [--atmpres ATMPRES] cdfname

Positional Arguments

cdfname

raw .cdf filename

Named Arguments

--atmpres

path to cdf file containing atmopsheric pressure data

nc2waves

Undocumented

runots.py rsk nc2waves [-h] ncname

Positional Arguments

ncname

processed .nc filename

nc2diwasp

Undocumented

runots.py rsk nc2diwasp [-h] ncname

Positional Arguments

ncname

processed .nc filename

sgtid

Seabird Seagauge Tides

runots.py sgtid [-h] {tid2cdf,cdf2nc} ...

Steps

Specify one of the steps in the list below

step

Possible choices: tid2cdf, cdf2nc

Sub-commands

tid2cdf

Convert instrument data to raw .cdf

runots.py sgtid tid2cdf [-h] gatts config

Positional Arguments

gatts

path to global attributes file (gatts formatted)

config

path to instrument configuration file (YAML formatted)

cdf2nc

Convert raw .cdf to clean .nc

runots.py sgtid cdf2nc [-h] [--atmpres ATMPRES] cdfname

Positional Arguments

cdfname

raw .cdf filename

Named Arguments

--atmpres

path to cdf file containing atmopsheric pressure data

sgwvs

Seabird Seagauge Waves

runots.py sgwvs [-h] {wb2cdf,cdf2nc,nc2waves} ...

Steps

Specify one of the steps in the list below

step

Possible choices: wb2cdf, cdf2nc, nc2waves

Sub-commands

wb2cdf

Convert instrument data to raw .cdf

runots.py sgwvs wb2cdf [-h] gatts config

Positional Arguments

gatts

path to global attributes file (gatts formatted)

config

path to instrument configuration file (YAML formatted)

cdf2nc

Convert raw .cdf to clean .nc

runots.py sgwvs cdf2nc [-h] [--atmpres ATMPRES] cdfname

Positional Arguments

cdfname

raw .cdf filename

Named Arguments

--atmpres

path to cdf file containing atmopsheric pressure data

nc2waves

Undocumented

runots.py sgwvs nc2waves [-h] ncname

Positional Arguments

ncname

processed .nc filename

sig

Nortek Signature

runots.py sig [-h] {mat2cdf,cdf2nc,nc2waves,nc2diwasp} ...

Steps

Specify one of the steps in the list below

step

Possible choices: mat2cdf, cdf2nc, nc2waves, nc2diwasp

Sub-commands

mat2cdf

Convert instrument data to raw .cdf

runots.py sig mat2cdf [-h] gatts config

Positional Arguments

gatts

path to global attributes file (gatts formatted)

config

path to instrument configuration file (YAML formatted)

cdf2nc

Convert raw .cdf to clean .nc

runots.py sig cdf2nc [-h] [--atmpres ATMPRES] cdfname

Positional Arguments

cdfname

raw .cdf filename

Named Arguments

--atmpres

path to cdf file containing atmopsheric pressure data

nc2waves

Undocumented

runots.py sig nc2waves [-h] ncname

Positional Arguments

ncname

processed .nc filename

nc2diwasp

Undocumented

runots.py sig nc2diwasp [-h] ncname

Positional Arguments

ncname

processed .nc filename

tb

TruBlue

runots.py tb [-h] {csv2cdf,cdf2nc,nc2waves} ...

Steps

Specify one of the steps in the list below

step

Possible choices: csv2cdf, cdf2nc, nc2waves

Sub-commands

csv2cdf

Convert instrument data to raw .cdf

runots.py tb csv2cdf [-h] gatts config

Positional Arguments

gatts

path to global attributes file (gatts formatted)

config

path to instrument configuration file (YAML formatted)

cdf2nc

Convert raw .cdf to clean .nc

runots.py tb cdf2nc [-h] [--atmpres ATMPRES] cdfname

Positional Arguments

cdfname

raw .cdf filename

Named Arguments

--atmpres

path to cdf file containing atmopsheric pressure data

nc2waves

Undocumented

runots.py tb nc2waves [-h] ncname

Positional Arguments

ncname

processed .nc filename

tcm

Lowell Tilt Current Meter

runots.py tcm [-h] {csv2cdf,cdf2nc} ...

Steps

Specify one of the steps in the list below

step

Possible choices: csv2cdf, cdf2nc

Sub-commands

csv2cdf

Convert instrument data to raw .cdf

runots.py tcm csv2cdf [-h] gatts config

Positional Arguments

gatts

path to global attributes file (gatts formatted)

config

path to instrument configuration file (YAML formatted)

cdf2nc

Convert raw .cdf to clean .nc

runots.py tcm cdf2nc [-h] [--atmpres ATMPRES] cdfname

Positional Arguments

cdfname

raw .cdf filename

Named Arguments

--atmpres

path to cdf file containing atmopsheric pressure data

troll

AquaTROLL

runots.py troll [-h] {csv2cdf,cdf2nc} ...

Steps

Specify one of the steps in the list below

step

Possible choices: csv2cdf, cdf2nc

Sub-commands

csv2cdf

Convert instrument data to raw .cdf

runots.py troll csv2cdf [-h] gatts config

Positional Arguments

gatts

path to global attributes file (gatts formatted)

config

path to instrument configuration file (YAML formatted)

cdf2nc

Convert raw .cdf to clean .nc

runots.py troll cdf2nc [-h] [--atmpres ATMPRES] cdfname

Positional Arguments

cdfname

raw .cdf filename

Named Arguments

--atmpres

path to cdf file containing atmopsheric pressure data

vec

Nortek Vector

runots.py vec [-h] {dat2cdf,cdf2nc,nc2waves} ...

Steps

Specify one of the steps in the list below

step

Possible choices: dat2cdf, cdf2nc, nc2waves

Sub-commands

dat2cdf

Convert instrument data to raw .cdf

runots.py vec dat2cdf [-h] gatts config

Positional Arguments

gatts

path to global attributes file (gatts formatted)

config

path to instrument configuration file (YAML formatted)

cdf2nc

Convert raw .cdf to clean .nc

runots.py vec cdf2nc [-h] [--atmpres ATMPRES] cdfname

Positional Arguments

cdfname

raw .cdf filename

Named Arguments

--atmpres

path to cdf file containing atmopsheric pressure data

nc2waves

Undocumented

runots.py vec nc2waves [-h] ncname

Positional Arguments

ncname

processed .nc filename

wvs

Aquadopp Waves

runots.py wvs [-h] {wad2cdf,cdf2nc,nc2waves} ...

Steps

Specify one of the steps in the list below

step

Possible choices: wad2cdf, cdf2nc, nc2waves

Sub-commands

wad2cdf

Convert instrument data to raw .cdf

runots.py wvs wad2cdf [-h] gatts config

Positional Arguments

gatts

path to global attributes file (gatts formatted)

config

path to instrument configuration file (YAML formatted)

cdf2nc

Convert raw .cdf to clean .nc

runots.py wvs cdf2nc [-h] [--atmpres ATMPRES] cdfname

Positional Arguments

cdfname

raw .cdf filename

Named Arguments

--atmpres

path to cdf file containing atmopsheric pressure data

nc2waves

Undocumented

runots.py wvs nc2waves [-h] ncname

Positional Arguments

ncname

processed .nc filename

wxt

Vaisala WXT

runots.py wxt [-h] {csv2cdf,cdf2nc} ...

Steps

Specify one of the steps in the list below

step

Possible choices: csv2cdf, cdf2nc

Sub-commands

csv2cdf

Convert instrument data to raw .cdf

runots.py wxt csv2cdf [-h] gatts config

Positional Arguments

gatts

path to global attributes file (gatts formatted)

config

path to instrument configuration file (YAML formatted)

cdf2nc

Convert raw .cdf to clean .nc

runots.py wxt cdf2nc [-h] [--atmpres ATMPRES] cdfname

Positional Arguments

cdfname

raw .cdf filename

Named Arguments

--atmpres

path to cdf file containing atmopsheric pressure data