Skip to content
Open

Lroc #46

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
59 commits
Select commit Hold shift + click to select a range
39eedca
hirise run successful
Emmy-D Feb 17, 2022
a55d044
starting lroc adding
Emmy-D Feb 17, 2022
d0a5f97
info on how to create mini files
Emmy-D Feb 21, 2022
961204a
mini tables
Emmy-D Feb 21, 2022
26023ec
working on lroc
Emmy-D Feb 22, 2022
8225480
got lroc metadata working
Emmy-D Feb 22, 2022
d708761
lroc loc
Emmy-D Feb 22, 2022
e93a8d3
runs on lroc now using the moon radius
Emmy-D Feb 22, 2022
7bc927f
add localization for browse imagery
Emmy-D Feb 23, 2022
f17663b
readme update
Emmy-D Feb 23, 2022
ca89c61
updated config file w/ obs i
Emmy-D Feb 23, 2022
a7c3566
added browse version
Emmy-D Feb 28, 2022
0f3fb36
list comprehension for a given row
Emmy-D Mar 1, 2022
bdd97e4
script for distributed samples around the lunar glob
Emmy-D Mar 2, 2022
582c69b
bug fix in selected indices
Emmy-D Mar 2, 2022
ccf8593
display more output
Emmy-D Mar 2, 2022
0b31673
more printing out
Emmy-D Mar 2, 2022
ec71272
selected 10 images to start
Emmy-D Mar 2, 2022
3e140cf
update default number of files
Emmy-D Mar 2, 2022
55b58bd
initial 5 examples selected
Emmy-D Mar 3, 2022
f9dfa1d
increased number since some do not exist
Emmy-D Mar 3, 2022
3e61969
save the solar azimuth to file
Emmy-D Mar 14, 2022
46557fe
point to script on deep learning repo
Emmy-D Mar 23, 2022
8e2b847
readme update
Emmy-D Mar 29, 2022
63575d7
removing pdb
Emmy-D Mar 29, 2022
4cebd20
added selected
Emmy-D Mar 30, 2022
41e1872
updates to ingest lroc metadata
Emmy-D Sep 20, 2022
a98cf0e
Merge pull request #45 from Emmy-D/master
Emmy-D Sep 20, 2022
0ab0e42
update default resolution to be 1/3 that of HiRISE
Emmy-D Sep 20, 2022
8e45709
updated LROC default resolution, should be ~1/3 that of HiRISE, but n…
Emmy-D Sep 21, 2022
75ac427
readme update
Emmy-D Oct 6, 2022
9e58d57
have mars radius as default, now passes all but 2 unit tests
Emmy-D Oct 6, 2022
a95661c
added mars radius default now all pytests pass
Emmy-D Oct 6, 2022
1d7fe20
update body_radius to be passed in after localizer_kwargs
Emmy-D Oct 6, 2022
9b05d60
lroc localizer using geodesic for non-map proj images
Emmy-D Oct 7, 2022
5343ed1
Merge branch 'lroc' of https://github.com/JPLMLIA/pdsc into lroc
Emmy-D Jan 11, 2023
232ca0c
updating readme
Emmy-D Jan 11, 2023
3acefbc
remove lroc localizers, we will just use the four courner localizer i…
Emmy-D Mar 15, 2023
3eb7e83
added lroc localizer using 4 point localizer, looks like upperright a…
Emmy-D Mar 16, 2023
6757e2f
corner pixels and lat/lon now line up
Emmy-D Mar 16, 2023
b2df394
readme update
Emmy-D Mar 18, 2023
041f69f
added scripts used to test pdsc localizer
Emmy-D Mar 30, 2023
18a7940
updated readme
Emmy-D Mar 30, 2023
012366f
readme update
Emmy-D Mar 30, 2023
7802a01
format
Emmy-D Mar 30, 2023
3c74bde
unit tests for lroc
Emmy-D Mar 30, 2023
58b7fce
updated documentation
Emmy-D Mar 30, 2023
d1bf8b5
moved inputs_mini to PDSIMG/deep-learning repo
Emmy-D Jun 9, 2023
76ab1bd
moved scripts over to PDSIMG/deep-learning repo at JPL
Emmy-D Jun 9, 2023
ef89710
revert readme back to Garys
Emmy-D Jun 9, 2023
db8a583
moved input folder to JPL repo
Emmy-D Jun 9, 2023
038c46d
move bare bones env to JPL repo
Emmy-D Jun 9, 2023
5f0bee5
removed no longer needed lines from gitignore
Emmy-D Jun 9, 2023
06e0bff
removed debug import
Emmy-D Jun 9, 2023
c6c6231
removed print
Emmy-D Jun 9, 2023
47b9a1b
always pass in body_radius for store_segments and trisegmentedfootprint
Emmy-D Jun 9, 2023
b81ef32
removed debug statements and get assume radius passed in
Emmy-D Jun 9, 2023
8e910c0
added ability for user to pass in downsample amount, with a default o…
Emmy-D Jun 10, 2023
4a41b47
Changed LrocCdrBrowseLocalizer so it inherits from LrocCdrLocalizer. …
Emmy-D Jun 15, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 3 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,11 +10,12 @@ several instruments from Mars orbiters are supported, but the system is
designed to be extensible to other instruments and bodies.

Please refer to the [documentation](https://jplmlia.github.io/pdsc/) for
instructions on installation, setup, and usage.
instructions on installation, setup, and usage.

---

Copyright 2019, by the California Institute of Technology. ALL RIGHTS RESERVED.
Copyright 2023, by the California Institute of Technology. ALL RIGHTS RESERVED.
United States Government Sponsorship acknowledged. Any commercial use must be
negotiated with the Office of Technology Transfer at the California Institute
of Technology.

49 changes: 49 additions & 0 deletions pdsc/config/lroc_cdr_metadata.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,49 @@
columns:
- [PRODUCT_ID, observation_id, text]
- [LINE_SAMPLES, samples, integer]
- [IMAGE_LINES, lines, integer]
- [CENTER_LATITUDE, center_latitude, real]
- [CENTER_LONGITUDE, center_longitude, real]
- [NORTH_AZIMUTH, north_azimuth, real]
- [SCALED_PIXEL_WIDTH, pixel_width, real]
- [VOLUME_ID, volume_id, text]
- [FILE_SPECIFICATION_NAME, file_specification_name, text]
- [PRODUCT_VERSION_ID, product_version_id, text]
- [TARGET_NAME, target_name, text]
- [ORBIT_NUMBER, orbit_number, integer]
- [MISSION_PHASE_NAME, mission_phase_name, text]
- [RATIONALE_DESC, description, text]
- [START_TIME, start_time, text]
- [STOP_TIME, stop_time, text]
- [SPACECRAFT_CLOCK_START_COUNT, sclk_start, real]
- [SPACECRAFT_CLOCK_STOP_COUNT, sclk_stop, real]
- [EMISSION_ANGLE, emission_angle, real]
- [INCIDENCE_ANGLE, incidence_angle, real]
- [PHASE_ANGLE, phase_angle, real]
- [SPACECRAFT_ALTITUDE, spacecraft_altitude, real]
- [TARGET_CENTER_DISTANCE, target_center_distance, real]
- [SUB_SOLAR_AZIMUTH, sub_solar_azimuth, real]
- [SUB_SOLAR_LATITUDE, sub_solar_latitude, real]
- [SUB_SOLAR_LONGITUDE, sub_solar_longitude, real]
- [SUB_SPACECRAFT_LATITUDE, sub_spacecraft_latitude, real]
- [SUB_SPACECRAFT_LONGITUDE, sub_spacecraft_longitude, real]
- [SOLAR_DISTANCE, solar_distance, real]
- [SOLAR_LONGITUDE, solar_longitude, real]
- [UPPER_RIGHT_LATITUDE, upper_right_latitude, real]
- [UPPER_RIGHT_LONGITUDE, upper_right_longitude, real]
- [LOWER_RIGHT_LATITUDE, lower_right_latitude, real]
- [LOWER_RIGHT_LONGITUDE, lower_right_longitude, real]
- [UPPER_LEFT_LATITUDE, upper_left_latitude, real]
- [UPPER_LEFT_LONGITUDE, upper_left_longitude, real]
- [LOWER_LEFT_LATITUDE, lower_left_latitude, real]
- [LOWER_LEFT_LONGITUDE, lower_left_longitude, real]

scale_factors:
SPACECRAFT_ALTITUDE: 1000 # km to m
SOLAR_DISTANCE: 1000 # km to m

index:
- observation_id

segmentation:
resolution: 50000
45 changes: 37 additions & 8 deletions pdsc/ingest.py
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,11 @@
TriSegmentedFootprint, SegmentTree)
from .util import standard_progress_bar

# https://tharsis.gsfc.nasa.gov/geodesy.html
MARS_RADIUS_M = 3396200.
#https://nssdc.gsfc.nasa.gov/planetary/factsheet/moonfact.html
MOON_RADIUS_M = 1736000

Comment on lines +16 to +20
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Might be preferable to import from localization.py rather than duplicate

DEFAULT_CONFIG_DIR = os.path.join(
os.path.dirname(os.path.realpath(__file__)),
'config',
Expand Down Expand Up @@ -134,7 +139,7 @@ def store_metadata(outputfile, instrument, table, config):
for v in progress(cur.fetchall())
]

def store_segments(outputfile, metadata, config):
def store_segments(outputfile, metadata, config, body_radius=MARS_RADIUS_M):
"""
Segments observations corresponding to each entry in ``metadata``, and
stores these segments in a SQL database
Expand All @@ -159,6 +164,8 @@ def store_segments(outputfile, metadata, config):
- ``localizer_kwargs``: the ``kwargs`` that will be supplied to
the :py:meth:`~pdsc.localization.get_localizer` function for
determining observation footprints
:param body_radius:
radius of celestial body, default is mars

:return: a list of :py:class:`~pdsc.segment.TriSegment` objects for segments
across all observations
Expand All @@ -172,11 +179,16 @@ def store_segments(outputfile, metadata, config):
progress = standard_progress_bar('Segmenting footprints')
for m in progress(metadata):
try:
s = TriSegmentedFootprint(m, resolution, localizer_kwargs)
s = TriSegmentedFootprint(m, resolution, localizer_kwargs, body_radius)
for si in s.segments:
segments.append(si)
observation_ids.append(s.metadata.observation_id)

# erd: check if observation_id attribute exists
if hasattr(s.metadata, 'observation_id'):
observation_ids.append(s.metadata.observation_id)
else:
raise ValueError('Please rename your index col to observation_id in the config file')
# erd: lroc does not have observation_id
#observation_ids.append(s.metadata.file_specification_name)
Comment on lines +185 to +191
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It might be preferable to check for this in the store_metadata function. We just want to check that some column is named observation_id. It should be a unique identifier, but does not need to be an indexed column.

except (TypeError, ValueError):
continue

Expand Down Expand Up @@ -208,7 +220,7 @@ def store_segments(outputfile, metadata, config):

return segments

def store_segment_tree(outputfile, segments):
def store_segment_tree(outputfile, segments, body_radius=MARS_RADIUS_M):
"""
Constructs a ball tree index for segmented observations and saves the
resulting data structure to the specified output file.
Expand All @@ -218,8 +230,14 @@ def store_segment_tree(outputfile, segments):

:param segments:
a collection of :py:class:`~pdsc.segment.TriSegment` objects

:body_radius:
celestial body radius, default is Mars
"""
tree = SegmentTree(segments)
if body_radius == MARS_RADIUS_M:
tree = SegmentTree(segments)
else:
tree = SegmentTree(segments, True, body_radius)
Comment on lines +237 to +240
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can remove the if/else block and just always pass in body_radius=body_radius argument.

tree.save(outputfile)

def ingest_idx(label_file, table_file, configpath, outputdir):
Expand All @@ -241,6 +259,14 @@ def ingest_idx(label_file, table_file, configpath, outputdir):
will be stored
"""
instrument, table = parse_table(label_file, table_file)

if 'lroc' in instrument:
# use moon radius
body_radius = MOON_RADIUS_M
else:
# default radius is Mars
body_radius = MARS_RADIUS_M

if os.path.isdir(configpath):
configfile = os.path.join(configpath, '%s_metadata.yaml' % instrument)
else:
Expand All @@ -266,10 +292,13 @@ def ingest_idx(label_file, table_file, configpath, outputdir):
outputdir,
'%s%s' % (instrument, SEGMENT_DB_SUFFIX)
)
segments = store_segments(outputfile, metadata, config)

segments = store_segments(outputfile, metadata, config, body_radius)

outputfile = os.path.join(
outputdir,
'%s%s' % (instrument, SEGMENT_TREE_SUFFIX)
)
store_segment_tree(outputfile, segments)

store_segment_tree(outputfile, segments, body_radius)

98 changes: 94 additions & 4 deletions pdsc/localization.py
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,11 @@
MARS_RADIUS_M = 3396200.
MARS_FLATTENING = 1.0 / 169.8

# erd: added moon info (before, assumed Mars only)
#https://nssdc.gsfc.nasa.gov/planetary/factsheet/moonfact.html
MOON_RADIUS_M = 1736000
MOON_FLATTENING = 0.0012

LOCALIZERS = {}

register_localizer = registerer(LOCALIZERS)
Expand Down Expand Up @@ -63,6 +68,10 @@ def geodesic_distance(latlon1, latlon2, radius=MARS_RADIUS_M):
>>> geodesic_distance((0, 0), (0, np.pi))
10669476.970121656
"""
#print('The moons radius is: 1736000')
#print('Radius used is: ')
#print(radius)
#print('------')
Comment on lines +71 to +74
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can remove comments

haversine = DistanceMetric.get_metric('haversine')
return float(radius*haversine.pairwise([latlon1], [latlon2]))

Expand Down Expand Up @@ -198,6 +207,7 @@ def latlon_to_pixel(self, lat, lon, resolution_m=None, resolution_pix=0.1):

loc = np.deg2rad([lat, lon])

#print(self.BODY_RADIUS)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can remove comment

def f(u):
loc_u = np.deg2rad(self.pixel_to_latlon(*u))
return geodesic_distance(loc, loc_u, self.BODY_RADIUS)
Expand All @@ -217,6 +227,7 @@ class GeodesicLocalizer(Localizer):
"""

BODY = Geodesic(MARS_RADIUS_M, MARS_FLATTENING)

"""
A :py:class:`~geographiclib.geodesic.Geodesic` object describing the target
body
Expand Down Expand Up @@ -477,8 +488,9 @@ def __init__(self, proj_type, proj_latitude, proj_longitude,
np.sqrt(a**2 + b**2)
)
self.cos_proj_lat = np.cos(self.proj_latitude)

def _equirect_pixel_to_latlon(self, row, col):
# equations come from section 3.5.1
# https://hirise.lpl.arizona.edu/pdf/HiRISE_RDR_v12_DTM_11_25_2009.pdf
x = (col - self.col_offset)*self.map_scale
y = -(row - self.row_offset)*self.map_scale
return (
Expand Down Expand Up @@ -513,6 +525,8 @@ def _polar_pixel_to_latlon(self, row, col):
return lat, lon

def _polar_latlon_to_pixel(self, lat, lon):
# equations come from section 3.5.2
# https://hirise.lpl.arizona.edu/pdf/HiRISE_RDR_v12_DTM_11_25_2009.pdf
lat_rad = np.deg2rad(lat)
lon_rad = np.deg2rad(lon % 360.)
T = np.tan((np.pi / 4.0) - np.abs(lat_rad / 2.0))
Expand Down Expand Up @@ -726,6 +740,64 @@ def __init__(self, metadata):
corners, 1.0, 1.0, 1
)

class LrocCdrLocalizer(FourCornerLocalizer):
"""
A localizer for the LROC CDR observations (subclass of
:py:class:`FourCornerLocalizer`)
"""

DEFAULT_RESOLUTION_M = 1/3*(1e-6)
"""
Sets the default resolution for lroc CDR localization
1/3 of that of HiRISE
"""

def __init__(self, metadata, downsamp):
"""
:param metadata:
"lroc_cdr" :py:class:`~pdsc.metadata.PdsMetadata` object
:param downsamp:
value of 1.0, no downsampling for LrocCdrLocalizer
"""
corners = np.array([
[metadata.upper_left_latitude, metadata.upper_left_longitude],
[metadata.upper_right_latitude, metadata.upper_right_longitude],
[metadata.lower_right_latitude, metadata.lower_right_longitude],
[metadata.lower_left_latitude, metadata.lower_left_longitude],
])
super(LrocCdrLocalizer, self).__init__(
corners, metadata.lines/downsamp, metadata.samples/downsamp, 1
)

class LrocCdrBrowseLocalizer(LrocCdrLocalizer):
"""
A localizer for the LROC CDR observations (subclass of
:py:class:`FourCornerLocalizer`)
"""

DEFAULT_RESOLUTION_M = 1/3*(1e-6)*2
"""
Sets the default resolution for lroc CDR localization
1/3 of that of HiRISE, plus factor of 2 for browse
"""

LROC_DOWNSAMP = 2.0
"""
The default downsample amount for browse imagery
"""

def __init__(self, metadata, downsamp):
"""
:param metadata:
"lroc_cdr" :py:class:`~pdsc.metadata.PdsMetadata` object
:param downsamp:
the downsample amount of lroc browse image (if it varies from the default
value)
"""
if downsamp < 1:
raise ValueError('Invalid downsample: %f' % downsamp)
super(LrocCdrBrowseLocalizer, self).__init__(metadata, downsamp)

class HiRiseRdrLocalizer(MapLocalizer):
"""
A localizer for the HiRISE RDR (map-projected) observations (subclass of
Expand Down Expand Up @@ -817,6 +889,25 @@ def hirise_rdr_localizer(metadata, nomap=False, browse=False,
else:
return HiRiseRdrLocalizer(metadata)

@register_localizer('lroc_cdr')
def lroc_cdr_localizer(metadata, browse=False, downsamp=LrocCdrBrowseLocalizer.LROC_DOWNSAMP):
"""
Constructs the LROC CDR localizer (data is not map projected)

:param metadata:
"lroc_cdr" :py:class:`~pdsc.metadata.PdsMetadata` object
:param browse:
construct localizer for the BROWSE data product
:param downsamp:
if ``browse=True``, use this value for the downsample amount

:return: a :py:class:`Localizer` for the appropriate data product
"""
if browse:
return LrocCdrBrowseLocalizer(metadata, downsamp)
else:
return LrocCdrLocalizer(metadata, 1.0)

@register_localizer('moc')
class MocLocalizer(GeodesicLocalizer):
"""
Expand Down Expand Up @@ -855,9 +946,9 @@ def get_localizer(metadata, *args, **kwargs):

:param metadata:
a :py:class:`~pdsc.metadata.PdsMetadata` object for an observation
:param \*args:
:param *args:
additional args provided to the localizer constructor
:param \**kwargs:
:param **kwargs:
additional kwargs provided to the localizer constructor

:return: a :py:class:`Localizer` for the observation
Expand All @@ -871,5 +962,4 @@ def get_localizer(metadata, *args, **kwargs):
if metadata.instrument not in LOCALIZERS:
raise IndexError(
'No localizer implemented for %s' % metadata.instrument)

return LOCALIZERS[metadata.instrument](metadata, *args, **kwargs)
Loading