Skip to content
This repository has been archived by the owner on May 5, 2022. It is now read-only.

Commit

Permalink
Merge pull request #18 from kberryUSGS/move_messenger_wac_dem
Browse files Browse the repository at this point in the history 
Move messenger wac dem creation scripts from ISISDATA/base/templates/…
  • Loading branch information
@scsides
scsides authored Mar 30, 2020
2 parents 4daf38b + 13b690a commit 979ccac
Show file tree
Hide file tree
Showing 7 changed files with 1,155 additions and 0 deletions.
191 changes: 191 additions & 0 deletions messenger_wac_dem/POINTCLOUDTAB.FMT
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,191 @@
OBJECT = COLUMN
COLUMN_NUMBER = 1
NAME = POINT_ID
DATA_TYPE = CHARACTER
UNIT = NONE
START_BYTE = 2
BYTES = 32
DESCRIPTION = "Unique point identifier for each jigsaw control point."
END_OBJECT = COLUMN

OBJECT = COLUMN
COLUMN_NUMBER = 2
NAME = STATUS
DATA_TYPE = CHARACTER
UNIT = NONE
START_BYTE = 37
BYTES = 12
DESCRIPTION = "Status of point. It can be FREE, FIXED or CONSTRAINED."
END_OBJECT = COLUMN

OBJECT = COLUMN
COLUMN_NUMBER = 3
NAME = ACCEPTED_MEASURES
DATA_TYPE = ASCII_INTEGER
UNIT = NONE
START_BYTE = 51
BYTES = 4
DESCRIPTION = "Number of accepted (non-rejected) measures for the control
point. This is actually the number of images that contain
the control point (also referred to as the point depth)."
END_OBJECT = COLUMN

OBJECT = COLUMN
COLUMN_NUMBER = 4
NAME = REJECTED_MEASURES
DATA_TYPE = ASCII_INTEGER
UNIT = NONE
START_BYTE = 56
BYTES = 4
DESCRIPTION = "Number of measures rejected by jigsaw during the bundle
adjustment (if outlier rejection is active)"
END_OBJECT = COLUMN

OBJECT = COLUMN
COLUMN_NUMBER = 5
NAME = RESIDUAL_RMS
DATA_TYPE = ASCII_REAL
UNIT = PIXELS
START_BYTE = 61
BYTES = 8
DESCRIPTION = "Root mean square error of residuals for a point.
Indicates the statistical measure of variation in the
difference of each measure within a control point."
END_OBJECT = COLUMN

OBJECT = COLUMN
COLUMN_NUMBER = 6
NAME = ADJUSTED_LATITUDE
DATA_TYPE = ASCII_REAL
UNIT = DEGREES
START_BYTE = 70
BYTES = 16
DESCRIPTION = "Latitude coordinate of the control point. This is the
adjusted location of the control point after jigsaw bundle
adjustment."
END_OBJECT = COLUMN

OBJECT = COLUMN
COLUMN_NUMBER = 7
NAME = ADJUSTED_LONGITUDE
DATA_TYPE = ASCII_REAL
UNIT = DEGREES
START_BYTE = 87
BYTES = 16
DESCRIPTION = "Longitude coordinate of control point. This is the
adjusted location of the control point after jigsaw bundle
adjustment."
END_OBJECT = COLUMN

OBJECT = COLUMN
COLUMN_NUMBER = 8
NAME = ADJUSTED_RADIUS
DATA_TYPE = ASCII_REAL
UNIT = KILOMETERS
START_BYTE = 104
BYTES = 16
DESCRIPTION = "Radius of control point. This is the adjusted radius of
the control point after jigsaw bundle adjustment."
END_OBJECT = COLUMN

OBJECT = COLUMN
COLUMN_NUMBER = 9
NAME = SIGMA_LATITUDE
DATA_TYPE = ASCII_REAL
UNIT = METERS
START_BYTE = 121
BYTES = 16
DESCRIPTION = "Adjusted uncertainty of Latitude coordinate of the control
point after jigsaw bundle adjustment."
END_OBJECT = COLUMN

OBJECT = COLUMN
COLUMN_NUMBER = 10
NAME = SIGMA_LONGITUDE
DATA_TYPE = ASCII_REAL
UNIT = METERS
START_BYTE = 138
BYTES = 16
DESCRIPTION = "Adjusted uncertainty of Longitude coordinate of the
control point after jigsaw bundle adjustment."
END_OBJECT = COLUMN

OBJECT = COLUMN
COLUMN_NUMBER = 11
NAME = SIGMA_RADIUS
DATA_TYPE = ASCII_REAL
UNIT = METERS
START_BYTE = 155
BYTES = 16
DESCRIPTION = "Adjusted uncertainty of Radius coordinate of the control
point after jigsaw bundle adjustment."
END_OBJECT = COLUMN

OBJECT = COLUMN
COLUMN_NUMBER = 12
NAME = DELTA_LATITUDE
DATA_TYPE = ASCII_REAL
UNIT = METERS
START_BYTE = 172
BYTES = 16
DESCRIPTION = "Latitude adjustment. The number of meters the latitude
coordinate has been adjusted by the jigsaw bundle
adjustment."
END_OBJECT = COLUMN

OBJECT = COLUMN
COLUMN_NUMBER = 13
NAME = DELTA_LONGITUDE
DATA_TYPE = ASCII_REAL
UNIT = METERS
START_BYTE = 189
BYTES = 16
DESCRIPTION = "Longitude adjustment. The number of meters the longitude
coordinate has been adjusted by the jigsaw bundle
adjustment."
END_OBJECT = COLUMN

OBJECT = COLUMN
COLUMN_NUMBER = 14
NAME = DELTA_RADIUS
DATA_TYPE = ASCII_REAL
UNIT = METERS
START_BYTE = 206
BYTES = 16
DESCRIPTION = "Radius adjustment. The number of meters the radius has
been adjusted by the jigsaw bundle adjustment."
END_OBJECT = COLUMN

OBJECT = COLUMN
COLUMN_NUMBER = 15
NAME = X
DATA_TYPE = ASCII_REAL
UNIT = KILOMETERS
START_BYTE = 223
BYTES = 16
DESCRIPTION = "Body-fixed X coordinate of the vector from the center of
the target body to the surface coordinate location."
END_OBJECT = COLUMN

OBJECT = COLUMN
COLUMN_NUMBER = 16
NAME = Y
DATA_TYPE = ASCII_REAL
UNIT = KILOMETERS
START_BYTE = 240
BYTES = 16
DESCRIPTION = "Body-fixed Y coordinate of the vector from the center of
the target body to the surface coordinate location."
END_OBJECT = COLUMN

OBJECT = COLUMN
COLUMN_NUMBER = 17
NAME = Z
DATA_TYPE = ASCII_REAL
UNIT = KILOMETERS
START_BYTE = 257
BYTES = 16
DESCRIPTION = "Body-fixed Z coordinate of the vector from the center of
the target body to the surface coordinate location."
END_OBJECT = COLUMN
END
41 changes: 41 additions & 0 deletions messenger_wac_dem/README
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,41 @@
This directory contains the scripts used to create a WAC only DEM from all
MESSENGER orbital images. The first isisminer run is with the
mdis_image_overlaps.conf CONFIG file. It reads from an SQLite database
that is not generally available, but can (and was) created using output
from the caminfo application. The output of this is a 10x10 region based
set of CSV files of image overlap information. These files are then input
into the next isisminer run of mdis_wac_matcher.conf. After all these
files have been run, here is the list of the general steps to create a DEM:


1) Run all EW*_overlap.csv files in /scratch/kbecker/MESSENGER/Control/Overlaps
(i.e., find /scratch/kbecker/MESSENGER/Control/Overlaps -name 'EW*_overlaps.csv')
with mdis_wac_matcher.conf to create image-based control networks (from 118,267
WAC image overlap sets).
2) Combine all these image-based files into a single global control network
using cnetcombinept.
3) Run cnetcheck of the global network to identify issues.
4) Run jigsaw to control images (will likely take a few days without error
propagation).
5) Run cnet2dem to create interpolated DEM.
6) Run series of noise and smoothing filters on DEM from previous step.
7) Run ckwriter on the updated images to generate CK kernel.

Note, the control network is likely to be ginormous. You may need up to 512GB
of memory to complete these steps. Also, use of a compute cluster is highly
recommended!

-------------

To create a PDS table of the control point cloud, you can use the CONFIG
jigsaw_bundleout_sigmas_pdsfmt.conf. This requires the bundleout points file
created by jigsaw that *MUST* have error propagation turned on (without error
propagation, the format of the outfile is missing columns which will result
in very bad things!). The config file shows how POINTCLOUDTAB.FMT is created
but is disabled (by adding "_DNE" to the Strategy Object keyword) - this can
be time consuming. Here is the command used to create the MESSENGER PDS
pointcloud table:

isisminer config=$ISIS3DATA/base/templates/isisminer/jigsaw_bundleout_sigmas_pdsfmt.conf \
parameters="bundleout:RadAngTwist_global_jigsaw_errorprop_bundleout_points.csv \
@tofmt:$ISIS3DATA/base/templates/isisminer/POINTCLOUDTAB.FMT@totab:MSGR_DEM_USG_EQ_C_V01.TAB"
137 changes: 137 additions & 0 deletions messenger_wac_dem/jigsaw_bundleout_sigmas_pdsfmt.conf
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,137 @@
Object = IsisMiner

Name = Jigsaw_Bundleout_Formatter
RequiredParameters = ( bundleout, tofmt, totab )

Object = Strategy
Name = CsvReader
Type = CsvReader

# Here are the possible parameter
CsvFileArgs = BundleOut
CsvFile = "%1"
HasHeader = False
SkipLines = 3
IgnoreComments = False
Delimiter = ","
Header = ("POINT_ID", "STATUS",
"ACCEPTED_MEASURES","REJECTED_MEASURES", "RESIDUAL_RMS",
"ADJUSTED_LATITUDE","ADJUSTED_LONGITUDE","ADJUSTED_RADIUS",
"SIGMA_LATITUDE","SIGMA_LONGITUDE","SIGMA_RADIUS",
"DELTA_LATITUDE", "DELTA_LONGITUDE", "DELTA_RADIUS",
"X", "Y", "Z")
Identity = Point_Id

EndObject

Object = Strategy_DNE
Name = JigsawBundleFormat
Type = PdsTableFormat

# Here are the possible parameter
PdsFormatFileArgs = tofmt
PdsFormatFile = "%1"
Delimiter = ","

# Specify the PDS table format COLUMNB objects
Column = ("POINT_ID", "STATUS",
"ACCEPTED_MEASURES","REJECTED_MEASURES", "RESIDUAL_RMS",
"ADJUSTED_LATITUDE","ADJUSTED_LONGITUDE","ADJUSTED_RADIUS",
"SIGMA_LATITUDE", "SIGMA_LONGITUDE", "SIGMA_RADIUS",
"DELTA_LATITUDE", "DELTA_LONGITUDE", "DELTA_RADIUS",
"X", "Y", "Z")
DataType = ("CHARACTER", "CHARACTER",
"ASCII_INTEGER", "ASCII_INTEGER", "ASCII_REAL",
"ASCII_REAL", "ASCII_REAL", "ASCII_REAL",
"ASCII_REAL", "ASCII_REAL", "ASCII_REAL",
"ASCII_REAL", "ASCII_REAL", "ASCII_REAL",
"ASCII_REAL", "ASCII_REAL", "ASCII_REAL")
Unit = ("NONE", "NONE",
"NONE", "NONE", "PIXELS",
"DEGREES", "DEGREES", "KILOMETERS",
"METERS", "METERS", "METERS",
"METERS", "METERS", "METERS",
"KILOMETERS", "KILOMETERS", "KILOMETERS")
DataDescription = ("Unique point identifier for each jigsaw control point.",
"Status of point. It can be FREE, FIXED or CONSTRAINED.",
"Number of accepted (non-rejected) measures for the control point.
This is actually the number of images that contain the control point
(also referred to as the point depth).",
"Number of measures rejected by jigsaw during the bundle
adjustment (if outlier rejection is active)",
"Root mean square error of residuals for a point. Indicates the
statistical measure of variation in the difference of each measure
within a control point.",
"Latitude coordinate of the control point. This is the
adjusted location of the control point after jigsaw bundle
adjustment.",
"Longitude coordinate of control point. This is the
adjusted location of the control point after jigsaw bundle
adjustment.",
"Radius of control point. This is the adjusted radius of
the control point after jigsaw bundle adjustment.",
"Adjusted uncertainty of Latitude coordinate of the control point
after jigsaw bundle adjustment.",
"Adjusted uncertainty of Longitude coordinate of the control point
after jigsaw bundle adjustment.",
"Adjusted uncertainty of Radius coordinate of the control point
after jigsaw bundle adjustment.",
"Latitude adjustment. The number of meters the latitude
coordinate has been adjusted by the jigsaw bundle
adjustment.",
"Longitude adjustment. The number of meters the longitude
coordinate has been adjusted by the jigsaw bundle
adjustment.",
"Radius adjustment. The number of meters the radius has been
adjusted by the jigsaw bundle adjustment.",
"Body-fixed X coordinate of the vector from the center of the
target body to the surface coordinate location.",
"Body-fixed Y coordinate of the vector from the center of the
target body to the surface coordinate location.",
"Body-fixed Z coordinate of the vector from the center of the
target body to the surface coordinate location.")


# Declare all column byte sizes
POINT_ID_BYTES = 32
STATUS_BYTES = 12
ACCEPTED_MEASURES_BYTES = 4
REJECTED_MEASURES_BYTES = 4
RESIDUAL_RMS_BYTES = 8
ADJUSTED_LATITUDE_BYTES = 16
ADJUSTED_LONGITUDE_BYTES = 16
ADJUSTED_RADIUS_BYTES = 16
SIGMA_LATITUDE_BYTES = 16
SIGMA_LONGITUDE_BYTES = 16
SIGMA_RADIUS_BYTES = 16
DELTA_LATITUDE_BYTES = 16
DELTA_LONGITUDE_BYTES = 16
DELTA_RADIUS_BYTES = 16
X_BYTES = 16
Y_BYTES = 16
Z_BYTES = 16

# Specify the column name that is unique to use as the Resource name
# (identity). If this is not present, an name is generated as "RowX"
# where X is input row starting at 0.
IdentityKey = POINT_ID

EndObject

Object=Strategy
Name = JigsawPointcloudWriter
Type = PdsTableCreator

PdsFormatFileArgs = tofmt
PdsFormatFile = "%1"

PdsTableFileArgs = totab
PdsTableFile = "%1"

Mode = Create
Delimiter = ","
DefaultValue = "NULL"
EndObject
EndObject
End

Loading

0 comments on commit 979ccac

Please sign in to comment.