Some way to generate a table of kernel sizes for the current ISIS Data Area.

[Kelvinrr]

Description

There seems to be some stuff being downloaded nightly that are either not kernels, in order to find every instance and make this easier to diagnose in the future, might be worth creating some summary of the sizes of every mission category to get an idea of what is happening at a glance that can be generated with the nightly downloads. This is something that can be documented internally in the isis3mgr scripts repo.

Suggested Edit

Before trying to write some custom script to do this, AWS probably has something we can leverage via cloud watch and/or log groups to get an idea of what is happening nightly and what the changes are over time (https://docs.aws.amazon.com/efs/latest/ug/logging-monitoring.html). If it's "go to the AWS and press these buttons", it's still worth writing down.

[foobarbecue]

I found rclone ncdu works really well for this. You can get something like this, and when you press enter on a row it goes into it and gives you sizes:

To use, set up your rclone config file so it looks like this:

(isis) aacurtis@MT-405357:~$ cat /home/aacurtis/.config/rclone/rclone.conf
[isis]
type = s3
provider = AWS
region = us-west-2
location_constraint = us-west-2

and then run rclone ncdu isis:asc-isisdata/usgs_data/

[github-actions]

Thank you for your contribution!

Unfortunately, this issue hasn't received much attention lately, so it is labeled as 'stale.'

If no additional action is taken, this issue will be automatically closed in 180 days.

[Kelvinrr]

I have some work to redo the way S3 files are synced being done, we can incorporate that rclone command into those and publish them somehow.

[github-actions]

Thank you for your contribution!

Unfortunately, this issue hasn't received much attention lately, so it is labeled as 'stale.'

If no additional action is taken, this issue will be automatically closed in 180 days.

If you want to participate in our support prioritization meetings or be notified when support sprints are happening, you can sign up the support sprint notification emails here.

Read more about our support processs here