run Gratipay 155

[chadwhitacre]

154

[chadwhitacre]

Alright, we want to get #3419 and #3287 out the door today before running this if we can.

[chadwhitacre]

@rohitpaulk Wanna get a PR going for #3287? I'm going to try to knock out #3419 asap.

[chadwhitacre]

I've got PRs open for the blockers on #3419. I'm gonna look at #3287.

[chadwhitacre]

Okay @rohitpaulk, we're getting down to it on this. Do you think we can land #3470 in the next few minutes, or should we bump that to next week?

[rohitpaulk]

We should be able to run on Braintree today. The only part I'm not sure about is how we handle cancelling card holds - I can revisit that after today's Payday.

I'll push the final commits to #3470 in a few mins

[chadwhitacre]

Okay. We only have 35 minutes if we want payouts to be deposited by tomorrow.

[chadwhitacre]

I reviewed one new team: https://gratipay.com/mojolicious/. I'm not bothering with our old review system because it doesn't really impact today's payday since we haven't allowed new subscriptions to the new teams yet.

[chadwhitacre]

I'm going to run locally instead of on a DO droplet since we had problems with DO last week and we don't have much time this week.

[rohitpaulk]

How far along are we with the braintree_customer_id migration?

[chadwhitacre]

Not done. We have 20 minutes. I think we should run on Balanced.

[rohitpaulk]

I think we should run on Balanced.

Hmm, yh me too. Braintree today would be too risky.

[chadwhitacre]

Okay, proceeding on Balanced ...

[rohitpaulk]

I'm going to run locally instead of on a DO droplet since we had problems with DO last week and we don't have much time this week.

The number of API calls this time will be far more (CC charges are included), think it'll be fine?

[chadwhitacre]

Hmm, good point ... okay, let's start on DO and keep local as a fall-back.

[chadwhitacre]

Backup taken and verified.

[chadwhitacre]

Droplet spun up and being updated ...

[chadwhitacre]

Updated and script started.

[chadwhitacre]

Fetching card holds ...

[chadwhitacre]

HTTPError: 504 Server Error: GATEWAY_TIMEOUT

[rohitpaulk]

🔁

[chadwhitacre]

Indeed.

[chadwhitacre]

Lots of Set state to failed on a $10.00 hold for nnn.

[chadwhitacre]

Yay logging! :-)

[chadwhitacre]

Now into create_card_holds ...

[chadwhitacre]

Capturing card holds ...

[chadwhitacre]

Captured 70 card holds.
Canceling card holds.

[chadwhitacre]

Blocking ...

[chadwhitacre]

Same place as last time. :-/

#3415 (comment) (+ new logging)

[chadwhitacre]

Alright, so we run the rest locally. Do we need to comment anything out?

[chadwhitacre]

Rerunning locally as-is (no patch).

[chadwhitacre]

Lots more Set state to failed ...

[chadwhitacre]

pid-43138 thread-140735252603648 (MainThread) Processing subscriptions.
pid-43138 thread-140735252603648 (MainThread) Processing draws.
pid-43138 thread-140735252603648 (MainThread) Settling card holds.
pid-43138 thread-140735252603648 (MainThread) Capturing card holds.
pid-43138 thread-140735252603648 (MainThread) Captured 0 card holds.
pid-43138 thread-140735252603648 (MainThread) Canceling card holds.
pid-43138 thread-140735252603648 (MainThread) Canceled 0 card holds.
pid-43138 thread-140735252603648 (MainThread) Updating balances.
pid-43138 thread-140735252603648 (MainThread) Updated the balances of 615 participants.
pid-43138 thread-140735252603648 (MainThread) Taking over balances.
pid-43138 thread-140735252603648 (MainThread) Starting payout loop.
[...]
pid-43138 thread-140735252603648 (MainThread) Did payout for 4 participants.
pid-43138 thread-140735252603648 (MainThread) Checked the DB.
pid-43138 thread-140735252603648 (MainThread) Updating stats.
pid-43138 thread-140735252603648 (MainThread) Updated payday stats.
pid-43138 thread-140735252603648 (MainThread) Updated receiving amounts.
pid-43138 thread-140735252603648 (MainThread) Notifying participants.

[chadwhitacre]

6:32. We'll see if we made the cut.

[chadwhitacre]

Notifying participants could take a while ...

[chadwhitacre]

Script ran for three minutes (0:03:11.015422).

[rohitpaulk]

Notifying participants could take a while

That's weird, we just queue emails don't we?

[chadwhitacre]

Yeah, but one at a time for ... well, I guess only 70 users (70 card holds). Didn't take that long. :-)

[rohitpaulk]

:)

[chadwhitacre]

Droplet destroyed (log downloaded previously in order to continue running locally).

[chadwhitacre]

!m @rohitpaulk

Thanks for all your great work on this stuff, lot more fun to not be hacking alone through all this. :)

[chadwhitacre]

Again, not bothering with escrow shuffle.

[chadwhitacre]

MassPay done and posted back for 2 users.

[chadwhitacre]

Logs uploaded in https://github.com/gratipay/logs/commit/0452cee3155fbcb0e47f0684fd956a52f2ebf89b.