downloadIsisData cannot exclude kernels

[marlens123]

hi, I am new to ISIS and appreciate any help!
I am currently trying to partially download mro data with the excluding kernels option. This is the command I am using:

downloadIsisData mro $ISISDATA --exclude="kernels/**"

However, this still downloads all the data including the kernels, which leads to more than 300GB of disk space required. Am I missing something?

[Kelvinrr]

hm, cant seem to replicate.

(isisdev) krodriguez$ downloadIsisData mro /tmp/isisdata --exclude="kernels/**"
2023-08-16 12:12:09 ERROR : Local file system at /tmp/isisdata/mro: Ignoring --track-renames as the source and destination do not have a common hash
2023-08-16 12:12:09 ERROR : Local file system at /tmp/isisdata/mro: Ignoring --track-renames as it doesn't work with copy or move, only sync
Transferred:       39.404 MiB / 39.404 MiB, 100%, 12.835 MiB/s, ETA 0s
Transferred:          195 / 195, 100%
Elapsed time:         3.5s
(isisdev) krodriguez$ ls /tmp/isisdata/mro/
calibration

What version of ISIS are you using? If you copied it via wget/curl, from what version of ISIS did you get the script.

[marlens123]

thanks for the reply! I am using isis 8.0.0_RC2 (with trying to install the latest public release, I ran into solving environment issues as reported in #5244 ).

[AustinSanders]

@marlens123

Sorry for the delay, but I've started digging into this ticket. It looks like the documentation may have been misleading, and I'm looking into adjusting it. The "exclude" flag is only able to exclude portions of the filename that are within the rclone remote, but it can't exclude portions of the filename that are actually specified as part of the remote. Inspecting the rclone.conf file that's shipped with ISIS (at $CONDA_PREFIX/etc/isis/rclone.conf) shows that the MRO targets are listed as:

[mro_naifKernels]
type = alias
remote = naif:/pub/naif/MRO/kernels/

[mro_usgs]
type = alias
remote = asc_s3:asc-isisdata/usgs_data/mro/

Since the "kernels" directory is directly listed as part of the remote, it can't be excluded. The files are just listed as :ik/something.ik rather than :kernels/ik/something.ik, so the exclusion filter doesn't view it as a matching file.

Two possible options are:

1. Edit your rclone.conf to remove the explicit "kernels" specification
2. Edit your downloadIsisData command to exclude the specific kernel directories, i.e. downloadIsisData mro . --exclude="{ck/**,ek/**,fk/**,ik/**,iak/**,lsk/**,mk/**,pck/**,sclk/**,spk/**}"

I would suggest the second option, since I'm not sure what all lies within the "MRO" directory outside of /kernels. Removing "/kernels" from the remote will allow you to excluded the kernels directory, but it will also lead you to download anything and everything at naif:/pub/naif/MRO/.

I'll leave this issue open until the documentation has been updated.