Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ISISDATA: MRO NAV spacecraft kernels are being downloaded and contributing to data bloat #5143

Closed
dpmayerUSGS opened this issue Mar 3, 2023 · 3 comments
Labels
bug Something isn't working

Comments

@dpmayerUSGS
Copy link

ISIS version(s) affected: >=4.x

Description
The current config for downloading SPICE kernels to the ISIS data area grabs many irrelevant files, leading to bloat in the data area. See #5110 for a summary of the broader issue.

More narrowly, the rclone config is downloading MRO spacecraft trajectory kernels (SPKs) produced by the MRO NAV team. These kernels either cover future time periods, which means that they serve no practical purpose for ISIS users, or they are early version of reconstructed kernels which are aggregated into the SPKs named like mro_psp??.bsp that are already being downloaded or superseded by refined reconstructed kernels named like mro_psp??_ssd_*.bsp that are also already being downloaded.

Historically, these kernels were not included in the ISIS data area. Downloading the original NAV kernels to the data area is contributing to the general problem of data bloat.

How to reproduce

Run downloadIsisData mro $ISISDATA and notice that $ISISDATA/mro/kernels/spk includes (dozens of) NAV kernels and their labels. These kernels are named like spk_psp_recJJJJJ_IIIII_MMMMM_K-vV. From documentation at NAIF:

Each of these files provides reconstructed trajectory for orbits "JJJJJ" through "IIIII" and predicted trajectory for orbits "IIIII" through "MMMMM". If the file contains only reconstructed data, then K is set to "r", otherwise it is set to "p". "V" is the file version.

Possible Solution
Update rclone.conf with exclude flags to prevent the NAV kernels from being downloaded as part of the MRO data kernels.

Additional context
Related to pull request #5109

@dpmayerUSGS dpmayerUSGS added the bug Something isn't working label Mar 3, 2023
@antonhibl
Copy link
Contributor

@dpmayerUSGS I believe we are aiming to add exclude patterns for rclone to the exclude_list that #5109 adds which would allow filtering of these files by default so I can look into adding this pattern(If I understand right the pattern to filter would be kernels/spk/spk_psp_* or something similar) into that PR, which would address this.

@dpmayerUSGS
Copy link
Author

@antonhibl Sounds good. The pattern to exclude would be kernels/spk/spk_psp_rec*

@AustinSanders
Copy link
Contributor

Closed via #5109

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

3 participants