Make cronjobs to vacuum full PuppetDB tables #12
Conversation
npwalker
force-pushed
the
update_maintenance_strategy
branch
from
3e061b2 to
d6cfaf2
Compare
|
Should we check to make sure there is available disk space prior to running the vacuum full? Checking to make sure there is at least twice the size of the target table size will ensure customers do not inadvertently fill their disk in the middle of the night. A similar check is done in the support script: https://github.com/puppetlabs/puppetlabs-pe_support_script/blob/2016.4.x/files/puppet-enterprise-support#L1403-L1431. |
|
@abottchen the existing script doesn't check and it does a REINDEX on reports which probably takes a reasonable amount of space. So, I'm not sure how necessary that is. I think people tend to leave more space in Not opposed to it but I don't really want to figure out how to test it. I'm also not sure what the lower bound of space should be, unless we find a query to do that work for us we'd just use an arbitrary number. Also, depending on where users mount volumes where do we check? If you think it's worthwhile I'd take a PR to add the check or a PR that adds a different cron job that just screams loudly when disk space is lower than an adjustable threshold. |
npwalker
force-pushed
the
update_maintenance_strategy
branch
5 times, most recently
from
e8cfe71 to
126d730
Compare
We want to VACUUM FULL all the tables except reports and resource events. We do this across days and for some lesser used tables we do this only once per month. By default the script will pause for 5 minutes between VACUUM FULL of each table. This allows the PuppetDB queue to work down from the last VACUUM FULL. Note that cronjob default times are chosen to note conflict with the database backup times in the backup manifest.
npwalker
force-pushed
the
update_maintenance_strategy
branch
from
126d730 to
4381154
Compare
Prior to this commit, we included --verbose on the vacuumdb command so that we could see the command was working. It turns out that verbose output goes to STDERR and thus can't be redirected to a file seperate from actual error messages. After this commit, we no longer try to redirect output to any files. Instead we remove the verbose output and hope that when an error occurs it'll actually fire off a cron email.
npwalker
force-pushed
the
update_maintenance_strategy
branch
from
2fd8a73 to
762d4b8
Compare
npwalker
force-pushed
the
update_maintenance_strategy
branch
from
9f17778 to
a7ee337
Compare
We want to VACUUM FULL all the tables except reports and
resource events. We do this across days and for some lesser used
tables we do this only once per month.