Granting access to github-bot server to github bot admins

[joyeecheung]

Today the github bot stopped working possibly because the access token of the bot got revoked. It is not able to access the GitHub v3 API right now. To fix that we'll need to generate a new token and configure it on the server, but the server is in the infra group so we'll need someone from infra to do that, but no one from infra is around.

I propose we add a new SSH key to the github-bot server and make it accessible to all github bot admins. The server is not used for release or any critical service so it should be safe to do so.

[Trott]

@rvagg

[richardlau]

@nodejs/collaborators AIUI until this is fixed the status of CI runs is not being reported to the PR's so you will need to manually check the status of the runs instead of relying on the PR's check status.

[refack]

Two security related questions that came to mind.

1. Who revoked the token (you could check in https://github.com/settings/security)
2. Is the bot's account behind 2FA?

[joyeecheung]

@refack The token was destroyed at 2018-06-16 05:42:15 +0800 by someone from Asker, Akershus, Norway (Safari on OS X 10.13.5). The bot account is behind 2FA.

[joyeecheung]

Hmm, wait, looks like it's not deleted, but leaked and deleted automatically by github? I matched the action with the login to identify #1353 (comment), but I noticed the action is named:

oauth_authorization.destroy: Personal access token (Bot Node.js infra deploy) deleted automatically (found in a public repository)

I googled the message and it looks like a security feature of github.

[phillipj]

.. by someone from Asker, Akershus, Norway

That's my location.

I logged into the bot account yesterday to accept #1341. I've recently got a new phone, so I had to setup Authy to be able to log in to the bot's account, I used the 2FA credentials found in the secrets repo. I didn't change the 2FA settings GitHub's side.

Could that be related? 🤷‍♀️

[ryzokuken]

I hope it was somehow triggered by @phillipj trying to log in from a new device and revoked the token as a security measure and the credentials of the bot weren't somehow leaked into the open...

[refack]

I hope it was somehow triggered by phillipj trying to log in from a new device and revoked the token as a security measure and the credentials of the bot weren't somehow leaked into the open...

That's what's nice about tokens, they can be revoked, then they are useless.

[joyeecheung]

@ryzokuken I don't think that's related, otherwise the other token for docker-node should be revoked as well. The action oauth_authorization.destroy has token_scan as explanation and explicitly says found in a public repository. It seems GitHub has a security measure to scan public repos for tokens and automatically revoke the ones leaked. The login from Norway should be irrelevant, it just happened to be on that day when GitHub found the token.

[ryzokuken]

@refack we can all sleep a bit better now that it's revoked, true, but still the reason how it was leaked in the first place remains.

[joyeecheung]

@phillipj Can you check the email inbox of the bot and see if there are any notifications from github about the token scan?

[phillipj]

@joyeecheung great suggestion, I didn't think of that.

I found a recent email to that points to this specific commit: https://github.com/maclover7/github-bot/blob/6a1f749e78fbf182d4a764e54758b5c4cbdad339/testteam.js

[phillipj]

Gotta say all creds to GitHub for this one! 👍

[rvagg]

Sorry for being responsive enough to be the one to help get this fixed, was weekending and all that. I'm pretty sure this is resolved and can be closed now right?

[joyeecheung]

@rvagg The action item proposed in the OP was

I propose we add a new SSH key to the github-bot server and make it accessible to all github bot admins. The server is not used for release or any critical service so it should be safe to do so.

The revocation of the token is just the cause of the proposal. @maclover7 and I are github bot admins but when we discovered that the bot was down, none of us could access the server (because it is in the infra group) to get it back on track, which was pretty weird given that we are the admins.

[rvagg]

Oh, ok then, missed that and I was unaware that it was tightly restricted. Or perhaps I knew and didn't think about it! I'll sort it out now and get something into secrets.

[rvagg]

Well, thinking on it more, there probably was a reason it was restricted. The original intent of the bot, from what I recall, was to offload privileged org admin type functions to it. So the bot GitHub account, webhook secret and SSH key should therefore probably have an appropriate level of security, similar to an org-owner. Can you confirm that this isn't the case and giving access to these things won't be providing elevated org access to any user that is able to access the machine, webhook secret and account credentials?

[refack]

Can you confirm that this isn't the case and giving access to these things won't be providing elevated org access to any user that is able to access the machine, webhook secret and account credentials?

I actually think it does, the bot token should have org admin powers. We actually need to make sure it's harder for it to leak.
But AFAICT that is similar to TSC members' level (accept for the anonymity) so it still gives us some room for expanding access to the machine.
NM, that group already has direct access to the bot account.

[joyeecheung]

@rvagg The bot admins already have access to the bot account, the bot is only a member of the org with write access to a bunch of repos (among them the most sensitive one is the core repo, others can be revoked once we merge nodejs/github-bot#183). See https://github.com/orgs/nodejs/people/nodejs-github-bot for the permissions that it has. Among the admins only @Starefossen and @williamkapke are not core collaborators but write access to the core repo should probably be fine given that we can always revert important actions (we have many inactive collaborator as well anyway).

I do not know where the Jenkins secret is coming from, but if it's just issued with the bot account then there should not be any elevated org access.

I think access to the server is no higher than access to the bot account and the admins already have the latter. Everything sensitive on the server is already accessible to bot admins anyway.

[maclover7]

Oh dear, sorry for accidentally pushing the key... and thank you GitHub for automatically revoking!!

[rvagg]

OK, I've added a new key for that server and put it, along with instructions in the github_bot_server.md file, into secrets/build/github-bot/ directory.

Let's just keep track of the access level this thing has. If we go back to the original intent of raising privileges to org-admin (or similar) then we should scale this back. I guess we need to just generally keep track of any elevated privileges that this provides anyone with access to these credentials.

[phillipj]

Thanks @rvagg!

FYI here's the actual the email sent to the github-bot account:

Hello nodejs-github-bot,

We noticed that a valid OAuth access token of yours was committed to a public GitHub repository. Publicly disclosing a valid access token would allow other people to interact with GitHub on your behalf, potentially altering data, your contact information, and billing data.

As a precautionary measure, we have revoked the OAuth token. A new token will need to be generated in order to continue using OAuth to authenticate to GitHub. Here are a couple of steps you can do to ensure your account security has not been compromised:

* Review any unauthorized OAuth applications, and remove any you don’t recognize: https://help.github.com/articles/reviewing-your-authorized-integrations/#reviewing-your-authorized-oauth-apps

* Review you security log to ensure that no malicious activity has occurred: https://help.github.com/articles/reviewing-your-security-log/

The commit in question is at https://github.com/maclover7/github-bot/blob/6a1f749e78fbf182d4a764e54758b5c4cbdad339/testteam.js

Please feel free to contact us at https://github.com/contact if you have any questions or concerns.

Thanks,
GitHub.com

[maclover7]

This is done via https://github.com/nodejs-private/secrets/commit/c8aea72cb54013c95c967d8e9578d9abb158876a, https://github.com/nodejs-private/secrets/commit/ef31bac6a74e00d5ab9d34e5125a8d4071b3b878