TEQC: (Translate, Edit, Quality Check) History Functional Overview Future?

119
TEQC: (Translate, Edit, Quality Check) History Functional Overview Future?

Transcript of TEQC: (Translate, Edit, Quality Check) History Functional Overview Future?

Page 1: TEQC: (Translate, Edit, Quality Check) History Functional Overview Future?

TEQC:(Translate, Edit, Quality Check)

History

Functional Overview

Future?

Page 2: TEQC: (Translate, Edit, Quality Check) History Functional Overview Future?

Well, “Translate, Edit, and Quality Check” of what?

Receiver collectable data of Global Navigation Satellite Systems (GNSS),where that data are:

• Physical observables of the signals broadcast by those satellites

• Digital data encoded in those signals, e.g. SV navigation data

• Ancillary ground measurements, e.g. meteorological data

Page 3: TEQC: (Translate, Edit, Quality Check) History Functional Overview Future?

GNSS Overview (circa Jan 2015)

The players in GNSS (Global Navigation Satellite Systems): GPS: USA, 21 + 3 MEO SVs -- 2 orbits every 1 sidereal day, CDMA

4 IIA + 12 IIR + 7 IIR-M + 8 IIF

GLONASS: Russia, 21 + 3 MEO SVs -- 17 orbits every 8 sidereal days, FDMA 24 fully operational + 2 spares + 1 under check + 2 (GLONASS-K) in flight tests

Galileo: ESA, MEO 27 + 3 SVs -- 5 orbits every 3 sidereal days, CDMA GIOVE-A and GIOVE-B offline, 4 IOV, 2 FOC in wrong orbits

QZSS: Japan (JAXA), at least 8 IGSO SVs -- 1 inclined orbit every 1 sidereal day, CDMA 1 SV until 2015(?), signals complement GPS and Galileo

Beidou: China, 5 GEO + 5 IGSO SVs + 32 MEO SVs, CDMA MEO -- 13 orbits every 7 sidereal days

5 GEO + 5 IGSO + 5 (?) MEO

SBAS: GEO -- 1 orbit every 1 sidereal day, CDMA

WAAS - USA (3) EGNOS - ESA (4) MSAS - Japan (2) GAGAN - India (2) SDMC- Russia (3) SNAS – China (0)

IRNSS: India, 4 IGSO + 3 GEO – CDMA (L-band + S-band, 3 SVs launched to date)

Page 4: TEQC: (Translate, Edit, Quality Check) History Functional Overview Future?
Page 5: TEQC: (Translate, Edit, Quality Check) History Functional Overview Future?

QZSS as viewed from Perth, Australia

Page 6: TEQC: (Translate, Edit, Quality Check) History Functional Overview Future?

Beidou as viewed from Perth, Australia

Page 7: TEQC: (Translate, Edit, Quality Check) History Functional Overview Future?

Historical background …

Late 1995/early 1996, UNAVCO wants a C-based replacement to its Fortran-based QC program(s) – of which there are now about 3-6 different ones – for GPS campaign data.

UNAVCO at that time was the University NAVSTAR Consortium which had been at UCAR, Boulder CO, since leaving CIRES, University of Colorado at Boulder.

UCAR – University Corporation for Atmospheric Research CIRES – Cooperative Institute for Research in Environmental Sciences

Page 8: TEQC: (Translate, Edit, Quality Check) History Functional Overview Future?

Historical background …

Late 1995/early 1996, UNAVCO wants a C-based replacement to its Fortran-based QC program(s) – of which there are now about 3-6 different ones – for GPS campaign data.

About the same time:• RINEX (Receiver Independent Exchange) format being more common• some RINEX version 1 still around . . .• various new features in RINEX 2 being proposed – non-standard usage• some interest in GLONASS, assuming Russia could keep the system going• new receivers coming on-line, new data formats• UNAVCO was getting into archiving of GPS data of NSF-funded campaigns• early GPS permanent stations, some with streamed formats• metadata issues (errors) that needed to be found and fixed

Page 9: TEQC: (Translate, Edit, Quality Check) History Functional Overview Future?

Historical background …

I was hired by UNAVCO to develop software to deal with some of those issues, as well as:• Wanted new program to be 100% non-interactive so it could be scripted• Minimal (none, if possible) reading of configuration files• Usable on UNIX/Linux and Microsoft DOS systems• Consult with Dr. Werner Gurtner to firm up new RINEX 2 version• Consult with IGS (International GPS Services) for their needs• Work via NDAs with receiver manufacturers to be able to read their

formats• Collaborate with NGS to help with one of their data issues

Page 10: TEQC: (Translate, Edit, Quality Check) History Functional Overview Future?

Historical background …

I was hired by UNAVCO to develop software to deal with some of those issues, as well as:• Wanted new program to be 100% non-interactive so it could be scripted• Minimal (none, if possible) reading of configuration files• Usable on UNIX/Linux and Microsoft DOS systems• Consult with Dr. Werner Gurtner to firm up new RINEX 2 version• Consult with IGS (International GPS Services) for their needs• Work via NDAs with receiver manufacturers to be able to read their

formats• Collaborate with NGS to help with one of their data issues

First version of that software as teqc on 17 March 1997

Page 11: TEQC: (Translate, Edit, Quality Check) History Functional Overview Future?

International Electromagnetic Spectrum Allocation

L1L2L5

GPS Carrier Frequencies

MHz

Page 12: TEQC: (Translate, Edit, Quality Check) History Functional Overview Future?

upcoming GPS Block III

antenna array of GPS IIR-M

L1: 19.03 cmL2: 24.42 cm

L1C/A signal is 25.6 W L1 antenna gain is 13 dBi L1C/A broadcast at 500 W

Page 13: TEQC: (Translate, Edit, Quality Check) History Functional Overview Future?

GPS Signal Waveform – L1 Channel

Using a right-handed circularly polarized 1575.42 MHz carrier …

1 0 1 1 0

01

Page 14: TEQC: (Translate, Edit, Quality Check) History Functional Overview Future?

How does one put binary information into a single-frequency carrier?

BPSK - Binary PSKQPSK - Quadrature PSK+ others

One way: Phase-shift Keying (PSK)

(in-quadrature)

(in-phase)

I + Q = QPSK

Page 15: TEQC: (Translate, Edit, Quality Check) History Functional Overview Future?

What’s in the GPS signals?

pseudo-random Gold code

pseudo-random M-sequence

orbit + other data

Page 16: TEQC: (Translate, Edit, Quality Check) History Functional Overview Future?

GPS Signal Spectra Evolution

Page 17: TEQC: (Translate, Edit, Quality Check) History Functional Overview Future?

Power Details on Modernized GPS L1

Page 18: TEQC: (Translate, Edit, Quality Check) History Functional Overview Future?

Power Spectra of GPS, Galileo, GLONASS and Beidou in E2-L1-E1/B1

Page 19: TEQC: (Translate, Edit, Quality Check) History Functional Overview Future?

Power Spectra of GPS, Galileo, GLONASS and Beidou in E5-L5/B2

Page 20: TEQC: (Translate, Edit, Quality Check) History Functional Overview Future?

Overview: Where we started (1980-2003) …

Page 21: TEQC: (Translate, Edit, Quality Check) History Functional Overview Future?

… and where things are circa 2015:

Page 22: TEQC: (Translate, Edit, Quality Check) History Functional Overview Future?

What’s the “data”? GPS L5:

• carrier frequency GPS L5 at 1176.45 MHz• signals just GPS L5 signal on L5 carrier

• components L5 has in-phase (I) and quadrature phase (Q) could track I, or Q, or I+Q

• observables phase – cycles of the carrier frequency pseudorange – receiver time shift of Gold code signal-to-noise (SNR) on carrier frequency doppler shift of carrier frequency

Page 23: TEQC: (Translate, Edit, Quality Check) History Functional Overview Future?

What’s the “data”? GPS L1 and L2:

• carrier frequencies GPS L1 at 1575.42 MHz GPS L2 at 1227.60 MHz• signals L1C/A + L1P(Y) (+ L1 M-code + L1C) L2C and L2P(Y) (+ L2 M-code)

• components L2C has CM and CL (civilian moderate and civilian long) L1P(Y), L2P(Y) = in-phase (I) L1C/A, L2C = quadrature phase (Q)

• observables phase, pseudorange, signal-to-noise (SNR), doppler

Page 24: TEQC: (Translate, Edit, Quality Check) History Functional Overview Future?

Determining the pseudorange to each SV…• find time offset of each 1023-bit code• time offset x speed of light = pseudorange to SV of specific PRN Gold code

… and PRN serves as the ID for each SV in GPS, SBAS, Galileo, Beidou, QZSS

Page 25: TEQC: (Translate, Edit, Quality Check) History Functional Overview Future?

more about the observables …

Phase – • measured in cycles of the carrier frequency• arbitrary offset• phase lock loop for tracking (plus new methods)Pseudorange – • time offset converted to distance, e.g. meters• range to SV plus receiver clock offset from GPS time• found from time offset of Gold codeSignal-to-noise (SNR) – • now measured in dB-Hz• old units are relative and somewhat obscure• exact method used is not knownDoppler – • phase change per unit time• today really only needed for certain RTK applications

Page 26: TEQC: (Translate, Edit, Quality Check) History Functional Overview Future?

GLONASS is differentRussian Federal Space Agency

• http://glonass-iac.ru/en/• http://glonass-iac.ru/en/GLONASS/

Orbit Slot Number: 1-24 – this is the unique SV identifier, tied to a physical location of one of the 3 orbital planesFrequency Channel Number: k, not fixed over time on an SV k = 0, 1, …, 13 from 1998 to 2005 k = -7, -6, …, +6 from 2005 to present

G1 = 1602.00 MHz + k*9/16 MHzG2 = 1246.00 MHz + k*7/16 MHzG3 = 1202.025 MHz (CDMA, GLONASS-K only)

You need k of the SV to convert GLONASS phase back and forth from units of cycles to units of distance for that SV.

Page 27: TEQC: (Translate, Edit, Quality Check) History Functional Overview Future?

SV Navigation (and other broadcast) Information

GPS (Galileo, QZSS, Beidou are similar) – • Keplerian orbital + perturbation parameters• SV clock offset and drift• SV health• 4-parameter ionospheric models for both L1 and L2• UTC offset model from GPS time• nominally updated every two hours• 50 bits per second! (300 bits/subframe)

GLONASS (SBAS is similar) –• orbital position, velocity, (and acceleration)• SV health• UTC and GPS offset from GLONASS time• updated every 30 minutes

Page 28: TEQC: (Translate, Edit, Quality Check) History Functional Overview Future?

SV Navigation (and other broadcast) Information

GPS (Galileo, QZSS, Beidou are similar) – • Keplerian orbital + perturbation parameters• SV clock offset and drift• SV health• 4-parameter ionospheric models for both L1 and L2• UTC offset model from GPS time• nominally updated every two hours• 50 bits per second! (300 bits/subframe)

GLONASS (SBAS is similar) –• orbital position, velocity, (and acceleration)• SV health• UTC and GPS offset from GLONASS time• updated every 30 minutes• has that pesky frequency channel number k for each GLONASS SV

Page 29: TEQC: (Translate, Edit, Quality Check) History Functional Overview Future?

Meteorological DataMet pack with RS-232 interface to receiver – • typically temperature, relative humidity, and pressure• Vaisala: also rain & hail increments, wind speed and direction• NMEA or other ASCII strings are returned upon data request• receiver manufacturers just store whatever ASCII is returned

Page 30: TEQC: (Translate, Edit, Quality Check) History Functional Overview Future?

Quick review …

• GNSS with GPS now being one of many• many GHz carrier frequencies using RH circularly-polarized waves• navigation and other information is sent digitally in the carrier using PSK (phase-shift keying) or BOC (binary offset carrier)• allows SV orbit determination at any specified time• SV id is done with a 1023-bit PRN or similar code• receiver time offset of the PRN code determines the pseudorange to SV• carrier cycle count, SNR, and doppler determined by receiver tracking• other ancillary information, e.g. met, can be collected from external devices, e.g. met packs, by the receiver

Page 31: TEQC: (Translate, Edit, Quality Check) History Functional Overview Future?

Ok, where the heck is all this stored?

Manufacturer binary formats – • Trimble .r00/.t00/.t01/.t02 proprietary formats; convert to .dat/.tgd with Trimble’s runpkr00• Leica MDB and LB2• Septentrio Binary Format (SBF)• Javad JPS, Topcon TPS• many others (Ashtech, AOA, Magellan, NovAtel, …)BINEX, JPL Soc format, RTCM – • Same idea, except not receiver specific RINEX – • ASCII• highly formatted (from Fortran code legacy)• different RINEX files for observation data, navigation data, and met data• no provision for tilt or other ancillary site data

Page 32: TEQC: (Translate, Edit, Quality Check) History Functional Overview Future?

Popping the hood on RINEX (2.xx)

signal carrier (MHz)

pseudo- range

phase SNR doppl

GPS L1C/A, L1P(Y) 1575.42 C1, P1 L1 S1 D1

L2C, L2P(Y) 1227.60 C2, P2 L2 S2 D2

L5 1176.45 C5 L5 S5 D5

GLONASS G1 1602+k*9/16 C1, P1 L1 S1 D1

G2 1246+k*7/16 C2, P2 L2 S2 D2

SBAS L1C/A 1575.42 C1 L1 S1 D1

L5 1176.45 C5 L5 S5 D5

Page 33: TEQC: (Translate, Edit, Quality Check) History Functional Overview Future?

Popping the hood on RINEX 2.11

signal carrier (MHz)

pseudo- range

phase SNR doppl

Galileo E2-L1-E1 1575.42 C1 L1 S1 D1

E6 1278.75 C6 L6 S6 D6

E5b 1207.14 C7 L7 S7 D7

E5a+b 1191.795 C8 L8 S8 D8

E5a 1176.45 C5 L5 S5 D5

Page 34: TEQC: (Translate, Edit, Quality Check) History Functional Overview Future?

Extending RINEX 2.11 (unofficial)

signal carrier (MHz)

pseudo- range

phase SNR doppl

GPS L1C 1575.42 C7 L7 S7 D7

GLONASS G3 1202.025 C7 L7 S7 D7

Beidou B1/E2 I/Q 1561.098 C1 L1 S1 D1

B1-2/E1 I/Q 1589.742 C2 L2 S2 D2

B3/E6 I/Q 1268.52 C6 L6 S6 D6

B2/E5b I/Q 1207.14 C7 L7 S7 D7

Page 35: TEQC: (Translate, Edit, Quality Check) History Functional Overview Future?

Extending RINEX 2.11 (unofficial)

signal carrier (MHz)

pseudo- range

phase SNR doppl

QZSS L1C/A 1575.42 C1 L1 S1 D1

L1C 1575.42 C7 L7 S7 D7

L1-SAIF 1575.42 C8 L8 S8 D8

L2C 1227.60 C2 L2 L2 L2

L5 1176.45 C5 L5 S5 D5

LEX S/L 1278.75 C6 L6 S6 D6

Page 36: TEQC: (Translate, Edit, Quality Check) History Functional Overview Future?

RINEX Constellation ID

SV code SV ID

GPS blank or G PRN (1-32)

GLONASS R Slot # (1-24)

Galileo E PRN (1-??)

Beidou C PRN (1-37)

QZSS J PRN – 192 (1-8?)

SBAS S PRN – 100 (20-42)

SV ID in RINEX is PRN-based – except for GLONASS (which is orbital slot #)

Page 37: TEQC: (Translate, Edit, Quality Check) History Functional Overview Future?

How can teqc help with any of this?

• Quick metadata extraction for verification

• What GNSS SVs and observables are being tracked

• Translation to RINEX

• Metadata editing during or after translation to RINEX

• Detailed time windowing

• GNSS constellation or specific SV filtering

• Creating or reorganizing RINEX into time-windowed files

• Quality checking of GNSS data

Page 38: TEQC: (Translate, Edit, Quality Check) History Functional Overview Future?

Semantics:

Teqc is a command line program and does not have a GUI:

teqc {option list} {filename1} {filename2} …

Page 39: TEQC: (Translate, Edit, Quality Check) History Functional Overview Future?

Semantics:

Teqc is a command line program and does not have a GUI:

teqc {option list} {filename1} {filename2} …

Teqc is designed to work with UNIX/Linux shell environments:

• Output to “stdout” and “stderr”, so can use redirection

• Input can be via “stdin” via piping

Page 40: TEQC: (Translate, Edit, Quality Check) History Functional Overview Future?

Semantics:

Teqc is a command line program and does not have a GUI:

teqc {option list} {filename1} {filename2} …

Teqc is designed to work with UNIX/Linux shell environments:

• Output to “stdout” and “stderr”, so can use redirection

• Input can be via “stdin” via piping

teqc {options} file > newfile

cat file | teqc {options} > newfile

Page 41: TEQC: (Translate, Edit, Quality Check) History Functional Overview Future?

Semantics:

In UNIX/Linux shell syntax:

• “stdout” is represented by “1” or nothing

• “stderr” is represented by “2”

teqc {options} file 2>&1 | more

teqc {options} file 2> file.err > file.out

Page 42: TEQC: (Translate, Edit, Quality Check) History Functional Overview Future?

Semantics:

In UNIX/Linux shell syntax:

• “stdout” is represented by “1” or nothing

• “stderr” is represented by “2”

teqc {options} file 2>&1 | more

teqc {options} file 2> file.err > file.out

teqc {options} file 2>> file.err >> file.out

Appending to an existing file:

Page 43: TEQC: (Translate, Edit, Quality Check) History Functional Overview Future?

Semantics:

DOS shell does not provide all these capabilities! Options:

• Install something like Cygwin on Windows

• Samba mount Windows directory on UNIX/Linux system

• teqc itself provides some internal replacements

Page 44: TEQC: (Translate, Edit, Quality Check) History Functional Overview Future?

Semantics:

DOS shell does not provide all these capabilities! Options:

• Install something like Cygwin on Windows

• Samba mount Windows directory on UNIX/Linux system

• teqc itself provides some internal replacements

teqc {options} file 2> file.err > file.out

teqc {options} +err file.err +out file.out file

teqc option equivalent:

Page 45: TEQC: (Translate, Edit, Quality Check) History Functional Overview Future?

Semantics:

DOS shell does not provide all these capabilities! Options:

• Install something like Cygwin on Windows

• Samba mount Windows directory on UNIX/Linux system

• teqc itself provides some internal replacements

teqc {options} file 2> file.err > file.out

teqc {options} +err file.err +out file.out file

teqc option equivalent:

teqc {options} file 2>> file.err >> file.out

teqc option equivalent:

teqc {options} ++err file.err ++out file.out file

Page 46: TEQC: (Translate, Edit, Quality Check) History Functional Overview Future?

General Rules:

teqc {option list} {filename1} {filename2} …

teqc assumes that what it is reading will be RINEX … unless it is told otherwise or it determines otherwise

teqc assumes the output will be RINEX … unless it is told otherwise, i.e. +binex <BINEX_type(s)>, or unless “special” outputs are requested, e.g.:

++igs to output IGS rx, antenna, radome designations

+config to output current teqc config options

+qc to obtain a data “quality check”

++sym to obtain a qc “symbol” table

+help to obtain all teqc options (for this version)

+version to obtain teqc version and build information

+id to obtain +version plus other information

… and there are others

Page 47: TEQC: (Translate, Edit, Quality Check) History Functional Overview Future?

First Obvious Assumption:

teqc has to be able to read the input file(s).

For RINEX, any version 1 up to an including 2.11

For Trimble .r00/.t00/.t01/.t02 you need to convert to .dat or .tgd using Trimble’s runpkr00: runpkr00 –g –d file.t02

teqc will try to auto-identify the input format. You can test thisby executing: teqc +mdf filename

If teqc selects the wrong format, you can always override, e.g.Javad JPS format: teqc –javad jps filename

If teqc selects the wrong GPS week for the start of the data,you can override: teqc –week 1839 filename

teqc cannot read everything, e.g. Magellan or NovAtel formats

Page 48: TEQC: (Translate, Edit, Quality Check) History Functional Overview Future?

• RINEX observation header metadata: -O.xxxx

Metadata Editing

-O.rn string change receiver number

-O.at string change antenna type

-O.an string change antenna number

-O.pe h E N change antenna height (and/or E N)

-O.px X Y Z change antenna X, Y, and Z

-O.pg lat lon el change antenna (converted) X, Y, and Z

-O.rt string change receiver type

Page 49: TEQC: (Translate, Edit, Quality Check) History Functional Overview Future?

• RINEX observation header metadata: -O.xxxx

Metadata Editing

Shell requirement: if string has whitespace in it, remember to encapsulate with double-quotes, e.g. -O.rt “AOA SNR-8000 ACT”

-O.rn string change receiver number

-O.at string change antenna type

-O.an string change antenna number

-O.pe h E N change antenna height (and/or E N)

-O.px X Y Z change antenna X, Y, and Z

-O.pg lat lon el change antenna (converted) X, Y, and Z

-O.rt string change receiver type

Page 50: TEQC: (Translate, Edit, Quality Check) History Functional Overview Future?

• RINEX observation data: -O.xxxx

Data Editing

You can ask for any observables …but you are only going to get data for the ones that have data.

-O.obs l1+l2+c1+p1+p2+s1+s2+d1+d2

Can use all these during translation, editing RINEX, qc, etc.!

-O.int interval (original interval, sometimes needed when decimating)

-O.obs string change observables to those in string

-O.dec interval decimate epochs to interval seconds

-O.-obs string remove all observables in string

Page 51: TEQC: (Translate, Edit, Quality Check) History Functional Overview Future?

• filtering out entire constellations –

–G to remove all GPS SVs–R to remove all GLONASS SVs–E to remove all Galileo SVs–S to remove all SBAS SVs–C to remove all Beidou SVs–J to remove all QZSS SVs

• filtering out specific SVs from a constellation –

–G2-3 to remove GPS PRN 2 and 3+G1,4-32 to keep just GPS PRNs 1 and 4 – 32… and so on

Constellation/SV Filtering

Page 52: TEQC: (Translate, Edit, Quality Check) History Functional Overview Future?

• explicit start/end times

-st to specify window start-e to specify window end+d{unit} to specify window length

Example: specify start time and length of 2 hours:

teqc –st 2015-01-20T00:00:00 +dh 2 filename

Filtering in Time – time windowing

teqc –st 2015-01-20T00:00:00 +dm 120 filename

… or, equivalently:

Page 53: TEQC: (Translate, Edit, Quality Check) History Functional Overview Future?

• explicit start/end times

-st to specify window start-e to specify window end+d{unit} to specify window length

Example: specify start time and length of 2 hours:

teqc –st 2015-01-20T00:00:00 +dh 2 filename

Filtering in Time – time windowing

teqc –st 2015-01-20T00:00:00 +dm 120 filename

… or, equivalently:

Number of input files can be one to many …Only one output (which can be redirected to a file).

Page 54: TEQC: (Translate, Edit, Quality Check) History Functional Overview Future?

• “time binning” with –tbin option

-tbin N{unit} file_prefix {unit} can be d, h, m, s-ast timestring similar to –st option

Example: specify length of 2 hours; input is one or more RINEX obs

teqc +obs + –tbin 2h temp filename1 {filename2 …}

Filtering in Time – time windowing

teqc +obs + +nav + +met + -tbin 1h filename.mdb

Example with translation of Leica MDB file creating hourly files

Page 55: TEQC: (Translate, Edit, Quality Check) History Functional Overview Future?

• “time binning” with –tbin option

-tbin N{unit} file_prefix {unit} can be d, h, m, s-ast timestring similar to –st option

Example: specify length of 2 hours; input is one or more RINEX obs

teqc +obs + –tbin 2h temp filename1 {filename2 …}

Filtering in Time – time windowing

teqc +obs + +nav + +met + -tbin 1h filename.mdb

Example with translation of Leica MDB file creating hourly files

Number of input files can be one to many …But also one to many output files possible.

Page 56: TEQC: (Translate, Edit, Quality Check) History Functional Overview Future?

• With –st, -e, +d{unit} – by design, uses:

inclusive epoch match for both the starting and ending epochs (if found)

Filtering in Time – time windowing

• With –tbin – by design, uses:

inclusive epoch match for the starting epoch (if found), exclusive epoch match for the ending epoch (if found)

Page 57: TEQC: (Translate, Edit, Quality Check) History Functional Overview Future?

• With –st, -e, +d{unit} – by design, uses:

inclusive epoch match for both the starting and ending epochs (if found)

Filtering in Time – time windowing

• With –tbin – by design, uses:

inclusive epoch match for the starting epoch (if found), exclusive epoch match for the ending epoch (if found)

This is the classic fence post problem:

The explicit time windowing results in N+1 posts and N gaps, the time binning results in N posts and N gaps.

Page 58: TEQC: (Translate, Edit, Quality Check) History Functional Overview Future?

What else?

teqc +meta file(s) information about gaps and metadata

teqc [+all] –O.sum . file(s) which SVs and how may observables

teqc +qcq file(s) qc “quick” of data

teqc +v file(s) “verify” readability of input

teqc +diag file(s) parsing diagnostics

teqc +mds file(s) metadata “short” – just time span

Page 59: TEQC: (Translate, Edit, Quality Check) History Functional Overview Future?

What else?

teqc +meta file(s) information about gaps and metadata

teqc [+all] –O.sum . file(s) which SVs and how may observables

teqc +qcq file(s) qc “quick” of data

teqc +v file(s) “verify” readability of input

teqc +diag file(s) parsing diagnostics

teqc +mds file(s) metadata “short” – just time span

Data quality check is a big topic. Remember that was the original task.

Page 60: TEQC: (Translate, Edit, Quality Check) History Functional Overview Future?

What else can we do with all this data?

Think about not having any navigation (i.e. orbital parameter) information …

… and what to do if you have navigation information.

So, now, let’s look at data quality checking!

Page 61: TEQC: (Translate, Edit, Quality Check) History Functional Overview Future?

Without any navigation information:• look for sensible parsing of binary formats • look for gaps in the data (assuming a constant sample interval)• which SVs have which observables and how many observables• try to find “clock” slips (i.e. receiver clock jumps not handled correctly)• compute mean signal-to-noise• try to find phase cycle slips (from loss of lock on phase lock loop tracking)• look at other linear combinations of observables, e.g. “multipath” combination

Page 62: TEQC: (Translate, Edit, Quality Check) History Functional Overview Future?

With navigation information:• try to compute orbital positions of SVs at any arbitrary time epoch• try to compute position of antenna (using pseudoranges at various

time epochs)• if successful, could do most of the previous tests as function of SV elevation and/or azimuth

Page 63: TEQC: (Translate, Edit, Quality Check) History Functional Overview Future?

RHCP RHCP

RHCP

mostly LHCP

mostly LHCP

Idealized GNSS Multipath

Page 64: TEQC: (Translate, Edit, Quality Check) History Functional Overview Future?

Model for phase and pseudorange measurementsbetween the satellite s and the receiver r :

Time for some math!

where ionosphere is dispersive and:

Page 65: TEQC: (Translate, Edit, Quality Check) History Functional Overview Future?

Further assume:

where c2 is proportional to the electron density in the ionosphere and:

Start with L1 – L2 and assume the same path lengths through theionosphere and neutral atmosphere:

or:

Page 66: TEQC: (Translate, Edit, Quality Check) History Functional Overview Future?

Can define the time-rate of change of the ionospheric delay:

Reminder about GLONASS:

You need that pesky frequency channel number, k, to convert each GLONASS FDMA G1 or G2 phase measurement to distance!

Page 67: TEQC: (Translate, Edit, Quality Check) History Functional Overview Future?

To obtain the multipath equations, start with Pi – Li and substitute I1 or I2from equations (5) or (6), again assuming the same path through theatmosphere for the two frequencies:

where B1 and B2 are bias terms:

that arise from phase ambiguities.

Page 68: TEQC: (Translate, Edit, Quality Check) History Functional Overview Future?

http://gpsworld.com/threes-challenge/

Multipath of GPS L5 Signal

Page 69: TEQC: (Translate, Edit, Quality Check) History Functional Overview Future?

teqc (version 2013Dec11 or later) can estimate:

M(Pn, Ln, Lq) for:

…but not yet a generalized M(Pn, Lm, Lq), n != m != q,

… or dealing with multipath of individual signal components.

GPS: L1, L2, L5SBAS: L1, L5GLONASS: G1, G2, G3Galileo: E2-L1-E1, E5a, E5b, E5a+b, E6Beidou: B1/E2, B2/E5b, B3/E6, B1-2/E1QZSS: L1C/A, L1C, L1-SAIF, L2C, L5, LEX

where n or q is the primary signal “1” as “Mnq” or “MPnq”or .mnq qc compact3 filename suffix …

Page 70: TEQC: (Translate, Edit, Quality Check) History Functional Overview Future?

M12 at P230 for GPS PRN 12, 2014-122

Page 71: TEQC: (Translate, Edit, Quality Check) History Functional Overview Future?

Some special considerations for translation and qc:

• maximum number of SVs in a constellation -- back to GLONASS: Javad receivers can be set to track non-operational GLONASS SVs and they use slot #s > 24

use something like –n_GLONASS 26 to allow up to slot #26

use something like –R25-26 to filter out the non-standard slot #s

• maximum number of SVs tracked by the receiver – (especially now with the GNSS explosion)

use something like –max_rx_SVs 48 to allow up to 48 SVs to be simultaneously tracked in any one epoch

Page 72: TEQC: (Translate, Edit, Quality Check) History Functional Overview Future?

• translation to various RINEX navigation files –

+nav GPS[,GLONASS[,SBAS[,Galileo[,Beidou[,QZSS]]]]] ... and use ‘-’ to skip one

+nav G.nav,-,-,E.nav,C.nav

e.g.

To create:

GPS RINEX nav file G.navGalileo RINEX nav file E.navBeidou RINEX nav file C.nav

Page 73: TEQC: (Translate, Edit, Quality Check) History Functional Overview Future?

• filtering out entire constellations –

–G to remove all GPS SVs–R to remove all GLONASS SVs–E to remove all Galileo SVs–S to remove all SBAS SVs–C to remove all Beidou SVs–J to remove all QZSS SVs

• filtering out specific SVs from a constellation –

–G2-3 to remove GPS PRN 2 and 3+G1,4-32 to keep just GPS PRNs 1 and 4 – 32… and so on

Reminder: Constellation/SV Filtering

Page 74: TEQC: (Translate, Edit, Quality Check) History Functional Overview Future?

• filtering out entire constellations –

–G to remove all GPS SVs–R to remove all GLONASS SVs–E to remove all Galileo SVs–S to remove all SBAS SVs–C to remove all Beidou SVs–J to remove all QZSS SVs

• filtering out specific SVs from a constellation –

–G2-3 to remove GPS PRN 2 and 3+G1,4-32 to keep just GPS PRNs 1 and 4 – 32… and so on

Reminder: Constellation/SV Filtering

You can do this during qc operations as well!

Page 75: TEQC: (Translate, Edit, Quality Check) History Functional Overview Future?

• do qc ‘lite’ on selected constellations –

–no_orbit list e.g.

–no_orbit R,C to not do orbit calculations for GLONASS (R) and Beidou (C), but do everything else possible in the qc

• do orbit calculations only and output columned coordinates –

–orbit i:w[:[s][g]]

i = the interval in secondsw = window span of interest in seconds s = sidereal-fixed frame; otherwise Earth-fixed g = geographic coordinates; otherwise cartesian (xyz)

Page 76: TEQC: (Translate, Edit, Quality Check) History Functional Overview Future?

Data Quality Check Semantics

Basic command: teqc +qc {other options} RINEX_obs_filename …

teqc will automatically look for any RINEX navigation files that match(using the filename suffix extension):

• .YYo .YYn, .YYg, .YYh, .YYe, .YYc, .YYj• .YYO .YYN, .YYG, .YYH, .YYE, .YYC, .YYJ• .obs .nav, .glo, .sbs, .gal, .cmp, .qzs• .OBS .NAV, .GLO, .SBS, .GAL, .CMP, .QZS

If you need one or more special RINEX navigation files, use –nav with a comma-delimited list with the extra filenames.

If nav info is found and ephemeris for SV, then “qc full” is attempted;otherwise or if an antenna position cannot be found: “qc lite”.

Page 77: TEQC: (Translate, Edit, Quality Check) History Functional Overview Future?

• +nav to output various RINEX navigation files --

+nav GPS[,GLONASS[,SBAS[,Galileo[,Beidou[,QZSS]]]]] ... and use ‘-’ to skip one

-nav navfile1[,navfile2[,…]]

Each navfileN: any type, any order

Remember: Use +nav for translation, -nav for qc:

• -nav to input various RINEX navigation files for qc (if needed) --

Each navfileN: fixed type and order

Page 78: TEQC: (Translate, Edit, Quality Check) History Functional Overview Future?

• filtering in time (time windowing) – only the explicit time windowing:

-st to specify window start-e to specify window end+d{unit} to specify window length

Example: specify start time and length of 2 hours:

teqc +qc –st 2015-01-20T00:00:00 +dh 2 filename

Most Filtering options available with +qc

teqc +qc –st 2015-01-20T00:00:00 +dm 120 filename

… or, equivalently:

Page 79: TEQC: (Translate, Edit, Quality Check) History Functional Overview Future?

• filtering in time (time windowing) – only the explicit time windowing:

-st to specify window start-e to specify window end+d{unit} to specify window length

Example: specify start time and length of 2 hours:

teqc +qc –st 2015-01-20T00:00:00 +dh 2 filename

Most Filtering options available with +qc

teqc +qc –st 2015-01-20T00:00:00 +dm 120 filename

… or, equivalently:

But no -tbin option with +qc!

Page 80: TEQC: (Translate, Edit, Quality Check) History Functional Overview Future?

Some special considerations for qc:

• -set_mask to set elevation mask (default is 10 degrees)• -set_hor to set horizon angle (default is 0 degrees)• -set_comp to set elevation “comparison” angle (default is 25

degrees)

Page 81: TEQC: (Translate, Edit, Quality Check) History Functional Overview Future?

Some special considerations for qc:

• -set_mask to set elevation mask (default is 10 degrees)• -set_hor to set horizon angle (default is 0 degrees)• -set_comp to set elevation “comparison” angle (default is 25

degrees)

• Use –w # to change # of columns in ASCII timeplot default is 72 columns; each column is 20 minutes for 24-hour span can change this to be any value 1 – 255

Page 82: TEQC: (Translate, Edit, Quality Check) History Functional Overview Future?

Some special considerations for qc:

• -set_mask to set elevation mask (default is 10 degrees)• -set_hor to set horizon angle (default is 0 degrees)• -set_comp to set elevation “comparison” angle (default is 25

degrees)

• Use –w # to change # of columns in ASCII timeplot default is 72 columns; each column is 20 minutes for 24-hour span can change this to be any value 1 – 255

• Use +qcq (“qc quick”) to skip report file and all compact plot files.

Page 83: TEQC: (Translate, Edit, Quality Check) History Functional Overview Future?

Some special considerations for qc:

• -set_mask to set elevation mask (default is 10 degrees)• -set_hor to set horizon angle (default is 0 degrees)• -set_comp to set elevation “comparison” angle (default is 25

degrees)

• Use –w # to change # of columns in ASCII timeplot default is 72 columns; each column is 20 minutes for 24-hour span can change this to be any value 1 – 255

• Use +qcq (“qc quick”) to skip report file and all compact plot files.

• Important, but so obvious it is now often overlooked:

You should look at the qc output with non-proportional font like Courier.

Page 84: TEQC: (Translate, Edit, Quality Check) History Functional Overview Future?

Some special considerations for qc:

• -set_mask to set elevation mask (default is 10 degrees)• -set_hor to set horizon angle (default is 0 degrees)• -set_comp to set elevation “comparison” angle (default is 25

degrees)

• Use –w # to change # of columns in ASCII timeplot default is 72 columns; each column is 20 minutes for 24-hour span can change this to be any value 1 – 255

• Use +qcq (“qc quick”) to skip report file and all compact plot files.

• Important, but so obvious it is now often overlooked:

You should look at the qc output with non-proportional font like Courier.

• Can qc raw data, but will have to at least include the window length:

teqc +qc +dh 24 trimble.tgd teqc +qcq +dh 24 trimble.tgd

Page 85: TEQC: (Translate, Edit, Quality Check) History Functional Overview Future?

The qc overview: the ASCII Timeplot

Page 86: TEQC: (Translate, Edit, Quality Check) History Functional Overview Future?

The qc Summary Lines

SUM – original summary lineSWN – time “windowed” summary lineSMP – multipath summary (if more than MP12 and MP21)

Example: +qcq of 24-hours of GNSS data, +dh 48 to window as 48 hours of data, -R to not extend GLONASS orbits:

Page 87: TEQC: (Translate, Edit, Quality Check) History Functional Overview Future?

ASCII Timeplot Symbol Table

Execute: teqc ++sym

Page 88: TEQC: (Translate, Edit, Quality Check) History Functional Overview Future?

ASCII Timeplot Symbol Table

(output of teqc ++sym continued …)

Page 89: TEQC: (Translate, Edit, Quality Check) History Functional Overview Future?

Lou’s Top Ten Things to Look At

1) In the ASCII timeplot, are the data mostly ‘o’ or ‘*’? If so, probably good.

Page 90: TEQC: (Translate, Edit, Quality Check) History Functional Overview Future?

Lou’s Top Ten Things to Look At

2) In the ASCII timeplot, if lots of ‘+’ at beginning and/or end of SV track, then the receiver’s elevation cutoff is lower than teqc’s (default = 10 degrees):

There’s more usable data; may want to use –set_mask with value < 10 (or whatever mask value is being used).

Page 91: TEQC: (Translate, Edit, Quality Check) History Functional Overview Future?

Lou’s Top Ten Things to Look At

3) In the ASCII timeplot, if lots of ‘L’ (receiver’s status on loss-of-lock): a) if at beginning and end of SV track, normal loss of lock probably at low

SV elevations: nothing to worry about b) if elsewhere, this is probably a concern: problematic tracking?

Page 92: TEQC: (Translate, Edit, Quality Check) History Functional Overview Future?

Lou’s Top Ten Things to Look At

4) In the ASCII timeplot, if anything different, then get familiar with the symbols! Keep teqc ++sym handy!

Page 93: TEQC: (Translate, Edit, Quality Check) History Functional Overview Future?

Lou’s Top Ten Things to Look At

4) In the ASCII timeplot, if anything different, then get familiar with the symbols! Keep teqc ++sym handy!

Page 94: TEQC: (Translate, Edit, Quality Check) History Functional Overview Future?

Lou’s Top Ten Things to Look At

5) In the ASCII timeplot, columns of ‘C’s (N-millisecond clock slips) are an indication of:

a) a bad RINEX translation (remember Flavor A vs. Flavor B) b) if the receiver and RINEX from it were good before, then there’s probably

something wrong with the receiver’s clock now

Page 95: TEQC: (Translate, Edit, Quality Check) History Functional Overview Future?

Lou’s Top Ten Things to Look At

6) Slips in ASCII time plot: If lots of ‘I’s – slips of the ionospheric combination – either: a) a bad RINEX translation? b) if GLONASS SV: wrong frequency channel number for the SV’s slot number? c) radio frequency (RF) interference at the site? d) active ionosphere above the site at this time?

Page 96: TEQC: (Translate, Edit, Quality Check) History Functional Overview Future?

Lou’s Top Ten Things to Look At

6) Slips in ASCII time plot: If ‘M’, ‘1’, ‘2’, ‘Z’, ‘5’, ‘T’, ‘7’, ‘S’, ‘8’, ‘Q’, ‘6’ – multipath slips. If ‘m’ (N-millisecond multipath slips) – Trimble NetR* tracking has more of these than expected at low elevations; reason: unknown.

Example: site gmpk with NetRS 54:211 (instead of 1:50)

Page 97: TEQC: (Translate, Edit, Quality Check) History Functional Overview Future?

Lou’s Top Ten Things to Look At

6) Slips in ASCII time plot: Quick “goodness”: look at ‘o/slps’ (observations per slip): For 15- or 30-second sampling, should be ~600 or larger

Page 98: TEQC: (Translate, Edit, Quality Check) History Functional Overview Future?

Lou’s Top Ten Things to Look At

6) Slips in ASCII time plot: Quick “goodness”: look at ‘o/slps’ (observations per slip): For 15- or 30-second sampling, should be ~600 or larger

Special note on an “observation”:

It’s what our community normally expects, e.g. for a GPS SV we want phase and pseudorange measurements on both an L1 and an L2 signal, e.g. L1C/A and L2P(Y).

Page 99: TEQC: (Translate, Edit, Quality Check) History Functional Overview Future?

Lou’s Top Ten Things to Look At

7) Look for missing data or modernized data, e.g. GPS or GLONASS SV with ‘c’ or ‘.’ or ‘:’ missing L2 data SBAS with ‘c’ exactly what you would expect (for most SBAS SVs) GPS SV with ‘s’ has L5 data GPS SV with ‘z’ missing L5 phase data Galileo SV with ‘s’ exactly what you would expect

In the GNSS arena, you have to know what is going on for each constellation and perhaps know about differences between different generations of SVs.

Page 100: TEQC: (Translate, Edit, Quality Check) History Functional Overview Future?

Lou’s Top Ten Things to Look At

8) If lots of ‘-’: Missing epochs, but really how many?

site gv03 for 2015-006:

Page 101: TEQC: (Translate, Edit, Quality Check) History Functional Overview Future?

Lou’s Top Ten Things to Look At

8) If lots of ‘-’: Missing epochs, but really how many?

site gv03 for 2015-006:

Page 102: TEQC: (Translate, Edit, Quality Check) History Functional Overview Future?

Lou’s Top Ten Things to Look At

8) If lots of ‘-’:

You have to be careful because a small number of random missed epochs can yield many ‘-’ indicators because it only takes one missed epoch per time bin column to result in a ‘-’.

Reminder: You can use time windowing and/or the –w option to get down toone epoch per column to finely examine missing epochs or other structure.

Page 103: TEQC: (Translate, Edit, Quality Check) History Functional Overview Future?

Lou’s Top Ten Things to Look At

9) All tables and histograms in the qc report:

• Patterns might be indicative of a specific GNSS constellation. GPS vs. GLONASS vs. Galileo …

Page 104: TEQC: (Translate, Edit, Quality Check) History Functional Overview Future?

Lou’s Top Ten Things to Look At

9) All tables and histograms in the qc report:

• Patterns might be indicative of a specific GNSS constellation. GPS vs. GLONASS vs. Galileo …

• Patterns might be indicative of specific blocks of SVs within a GNSS constellation …

GPS I (no Y-code), II (Y-code starts), IIA, IIR, IIR-M (L2C starts at IIR-14M), IIF (L5 starts), III (L1C starts)

GLONASS M vs. K

Galileo: GIOVE vs. IOV vs. FOC

Page 105: TEQC: (Translate, Edit, Quality Check) History Functional Overview Future?

Lou’s Top Ten Things to Look At

9) All tables and histograms in the qc report:

• Patterns might be indicative of a specific GNSS constellation. GPS vs. GLONASS vs. Galileo …

• Patterns might be indicative of specific blocks of SVs within a GNSS constellation …

GPS I (no Y-code), II (Y-code starts), IIA, IIR, IIR-M (L2C starts at IIR-14M), IIF (L5 starts), III (L1C starts)

GLONASS M vs. K

Galileo: GIOVE vs. IOV vs. FOC

• Patterns might be indicative of the receiver type or data format

or RINEX translation or …

Page 106: TEQC: (Translate, Edit, Quality Check) History Functional Overview Future?

Lou’s Top Ten Things to Look At

10) The “good news” and “bad news” lines, -dn and +dn, qc full only

“dn” means “discrepancy SV count from optimal”, based on receiver tracking capability and the SV ephemerides being used

-dn and +dn are the least and greatest epoch counts for that column – together they form a small histogram:

Page 107: TEQC: (Translate, Edit, Quality Check) History Functional Overview Future?

Lou’s Top Ten Things to Look At

10) The “good news” and “bad news” lines, -dn and +dn, qc full only

“dn” means “discrepancy SV count from optimal”, based on receiver tracking capability and the SV ephemerides being used

-dn and +dn are the least and greatest epoch counts for that column – together they form a small histogram:

A value of N on the –dn line means that every epoch in that column (time bin) is missing N SVs that could have been tracked.

Page 108: TEQC: (Translate, Edit, Quality Check) History Functional Overview Future?

Lou’s Top Ten Things to Look At

10) The “good news” and “bad news” lines, -dn and +dn, qc full only

“dn” means “discrepancy SV count from optimal”, based on receiver tracking capability and the SV ephemerides being used

-dn and +dn are the least and greatest epoch counts for that column – together they form a small histogram:

A value of N on the –dn line means that every epoch in that column (time bin) is missing N SVs that could have been tracked.

A value of N on the +dn line means that there is at least one epoch in thatcolumn (time bin) that is missing N SVs that could have been tracked.

Page 109: TEQC: (Translate, Edit, Quality Check) History Functional Overview Future?

Lou’s Top Ten Things to Look At

10) The “good news” and “bad news” lines, -dn and +dn, qc full only

“dn” means “discrepancy SV count from optimal”, based on receiver tracking capability and the SV ephemerides being used

-dn and +dn are the least and greatest epoch counts for that column – together they form a small histogram:

But if the receiver is tracking below teqc’s elevation cutoff:

You can use +set_mask option and repeated qc operations to zero in on the effective elevation cutoff of the receiver/antenna/site!

Page 110: TEQC: (Translate, Edit, Quality Check) History Functional Overview Future?

Lou’s Top Ten Things to Look At

10) The “good news” and “bad news” lines, -dn and +dn, qc full only

To get a 100% accurate count of missing SVs, one would need:

– accurate SV tracking capability of receiver … can adjust this value with –max_rx_SVs option

– full broadcast navigation messages

– teqc to be able to compute orbits for all SVs in view

Page 111: TEQC: (Translate, Edit, Quality Check) History Functional Overview Future?

Lou’s Top Ten Things to Look At

10) The “good news” and “bad news” lines, -dn and +dn, qc full only

To get a 100% accurate count of missing SVs, one would need:

– accurate SV tracking capability of receiver … can adjust this value with –max_rx_SVs option

– full broadcast navigation messages

– teqc to be able to compute orbits for all SVs in view

But these are still immensely useful even when all the above conditions are not met 100%.

Page 112: TEQC: (Translate, Edit, Quality Check) History Functional Overview Future?

Lou’s Top Ten Things to Look At

10) Use of +dn for p319 (late Dec 2013 – mid-Jan 2014):

Extraction of +dn lines from .YYS files for consecutive days:

Page 113: TEQC: (Translate, Edit, Quality Check) History Functional Overview Future?

Lou’s Top Ten Things to Look At

10) Use of +dn for p319 (late Dec 2013 – mid-Jan 2014):

Action taken by a UNAVO field engineer:

Page 114: TEQC: (Translate, Edit, Quality Check) History Functional Overview Future?

Lou’s Top Ten+ Things to Look At

11) Other useful options:

+eep, +eepx, +eepg – “every epoch position”

+svo – “SV ordering”

-O.obs list, -O.-obs list – might want to remove some observables

-ion_bins #, -mp_bins #, -sn_bins # – for those ASCII histograms

+ssv – qc summary for each SV

-slips file – write out slip details to file

-ion_jump 8 –iod_jump 1e38 – for high-rate data

-O.dec 15 –O.int 0.2 – or decimate high-rate data, e.g. 5-Hz data

Page 115: TEQC: (Translate, Edit, Quality Check) History Functional Overview Future?

Lou’s Top Ten++ Things to Look At

12) Be familiar with normal good data.

ftp://data-out.unavco.org/pub/rinex/qc/2015/006/

Start with site gdec and then some other sites.

Page 116: TEQC: (Translate, Edit, Quality Check) History Functional Overview Future?

The Future?

Support for RINEX 3? Probably not … at least not any time soon

Page 117: TEQC: (Translate, Edit, Quality Check) History Functional Overview Future?

The Future?

Support for RINEX 3? Probably not … at least not any time soon

Support for reading other manufacturer formats, e.g. Magellan, NovAtel? Probably only if one became a receiver type used by NSF PIs that UNAVCO supports.

Page 118: TEQC: (Translate, Edit, Quality Check) History Functional Overview Future?

The Future?

Support for RINEX 3? Probably not … at least not any time soon

Support for reading other manufacturer formats, e.g. Magellan, NovAtel? Probably only if one became a receiver type used by NSF PIs that UNAVCO supports.

Long-term support (post-2018)? No idea.

Page 119: TEQC: (Translate, Edit, Quality Check) History Functional Overview Future?

However, until then:

• Will be providing periodic updates of teqc, as usual

• On-line tutorial and (soon, hopefully) a new user’s guide

• You can join the teqc email forum: http://postal.unavco.org/mailman/listinfo/teqc