cloudtrail_supported_actions contains actions that are not actually logged to cloudtrail

[danielpops]

At least, sqs:ReceiveMessage (and various other sqs APIs) do not actually appear in cloudtrail. The only sqs related ones I actually see in cloudtrail across several accounts over the past 30 days are sqs:CreateQueue, sqs:DeleteQueue, sqs:PurgeQueue, and sqs:SetQueueAttributes.

I imagine there are others that are incorrectly present in this list.

I know that this list was updated ~3 months ago. Maybe the procedure for generating/verifying that list's accuracy could be improved or revisited?

[0xdabbad00]

It definitely could be improved, as right now there hasn't been a process. :) All I had done previously was list all the actions from botocore and then manually look through the docs to remove actions that were mentioned as not used. :(

In theory, the concept behind Will Bengston's trailblazer project could be used: https://github.com/willbengtson/trailblazer-aws
He tries to call every single API and then looks in the CloudTrail logs to figure out which calls actually worked. Unfortunately that concept ran into a lot of issues as documented in willbengtson/trailblazer-aws#2 and there is an associated PR. To summarize that thread, AWS doesn't record the calls if you don't call them with the correct formatting. Ie. you need to pass certain arguments for each call, and those often have to match a regex. So my PR tries to automatically determine what those arguments are and uses a reverse regex to generate arguments that should pass through AWS. Unfortunately it still failed for a lot of calls, and because I was essentially fuzzing every API, it crashed the Organization service and an AWS on-call tech blew up my phone trying to get ahold of me to stop. That resulted in the on-call tech setting up a phone call immediately between myself and the AWS Organization PM to discuss the situation. I promised to stop, and that PM spoke with the CloudTrail PM who promised to document this stuff. As I later learned in a private conversation with someone, the CloudTrail PM made the promise, knowing he wouldn't keep it, just to get me to stop. Not cool! So anyway, I stopped, and moved on to some other things waiting for this to happen. In the meantime, someone at AWS has been working on getting these actions documented. Apparently, it involves working with every single one of the 150+ service teams at AWS, so it has been months since I last heard anything about that work. :(

[0xdabbad00]

Perhaps the best way forward is for some folks with active AWS environments to provide a list of known APIs that are logged, by collecting that data from their own CloudTrail logs. Just providing a list of action names is all that is needed.

Then for anything not in that list, we'd just assume it is not documented unless we hear otherwise from someone.