run Gratipay 156

[chadwhitacre]

155

[chadwhitacre]

Don't forget to migrate tips to subscriptions for newly accepted teams!

[chadwhitacre]

Teams reviewed (gratipay/inside.gratipay.com#210), still need to migrate tips to subscriptions.

[chadwhitacre]

I decided to go back and review users according to our old pattern, now that we're more confident that we don't have any more payout leaks (cf. #3453). However, there is a bug in the user review dashboard, and I've been setting everyone to suspicious instead of not suspicious. 😞

[chadwhitacre]

I'm going to break for lunch and coffee, and return to fix that bug and repair the database. Then I'll look at #3473.

[chadwhitacre]

Reticketed review bug as #3496. We are only looking at balanced_customer_href on the user dashboard. We should be looking at Braintree as well as PayPal. Let's not block this payday on improvements to user review. We can let user review slide for another week.

[chadwhitacre]

I'm thinking about how to repair the database, at least.

[chadwhitacre]

This selects all 34 relevant events:

  select ts, payload, payload->'id'
    from events
   where ts > '2015-05-28 17:00'::timestamptz and ts < '2015-05-28 18:00'::timestamptz
order by ts desc
   limit 100;

[chadwhitacre]

I'm going to take the database backup, and then run:

update participants
   set is_suspicious=null
 where id in (

      select payload->'id'
        from events
       where ts > '2015-05-28 17:00'::timestamptz and ts < '2015-05-28 18:00'::timestamptz

);

delete
  from events
 where ts > '2015-05-28 17:00'::timestamptz and ts < '2015-05-28 18:00'::timestamptz

[chadwhitacre]

Is that so wrong?

[chadwhitacre]

T-3 hours

[chadwhitacre]

I grabbed usernames with:

\copy (select username from participants where id in (select (payload->>'id')::int 
from events
where ts > '2015-05-28 17:00'::timestamptz and ts < '2015-05-28 18:00'::timestamptz)) 
to 'is_NOT_suspicious.csv' with csv

[chadwhitacre]

I set is_suspicious to false through the API:

#!/usr/bin/env python
import os, requests, sys

user_id = os.environ['GRATIPAY_USER_ID']
api_key = os.environ['GRATIPAY_API_KEY']

for line in sys.stdin:
    username = line.strip()
    url = 'https://gratipay.com/{}/toggle-is-suspicious.json'.format(username)
    response = requests.post(url, data={'to': 'false'}, auth=(user_id, api_key))
    print response.status_code, url

Two were toggled to true. I believe I had fixed them up in the UI before running the script, but I still don't understand why the script set them to true. I fixed them back up through the UI, and confirmed that all are now false with:

select username, is_suspicious
  from participants
 where id in (

    select (payload->>'id')::int
      from events
     where ts > '2015-05-28 17:00'::timestamptz and ts < '2015-05-28 18:00'::timestamptz

);

[chadwhitacre]

Okay! Back to payday ...

[chadwhitacre]

T-2.5 hours

[chadwhitacre]

Don't forget to migrate tips!

Last time: #3399 (comment).

[chadwhitacre]

T-1 hour.

@rohitpaulk I think we should bump payroll (#3433) to next week, and focus on Braintree (#3287) for today.

[chadwhitacre]

I'm going to review and probably merge #3470, then land #3473 and ... go for it ... hopefully ... 💐

[chadwhitacre]

Not forgetting to merge tips!

[rohitpaulk]

Sorry I was late -

@rohitpaulk I think we should bump payroll (#3433) to next week, and focus on Braintree (#3287) for today.

Yep. Braintree is an external deadline, payroll is in our control :)

[chadwhitacre]

@rohitpaulk Want to work on converting tips to subscriptions for new teams since last week?

[rohitpaulk]

Sure, on it.

[chadwhitacre]

!m @rohitpaulk

We've missed the bank payout deadline for today. We have so few users and everything is so disrupted, we can probably get away with slipping a few payouts until Monday. :-(

Let's land Braintree!

[chadwhitacre]

@rohitpaulk Can you paste your script for migrating tips for the new teams since last week?

[rohitpaulk]

from gratipay import wireup
from gratipay.models.team import Team, AlreadyMigrated

db = wireup.db(wireup.env())

new_teams = db.all("""
    SELECT slug
      FROM teams
     WHERE is_approved IS TRUE
""")

for slug in new_teams:
    team = Team.from_slug(slug)
    try:
        team.migrate_tips()
        print("Migrated tips for '%s'" % slug)
    except AlreadyMigrated:
        print("'%s' already migrated." % slug)

print("Done.")

[rohitpaulk]

'introtopython' already migrated.
'duo' already migrated.
'Gratipay' already migrated.
'sudo-room' already migrated.
'nuvola-player' already migrated.
'cyberpipe' already migrated.
'mojolicious' already migrated.
Migrated tips for 'pageres'
Migrated tips for 'sublimelinter'
Migrated tips for 'citizens-cooperative'
'catapultpgh' already migrated.
Migrated tips for 'sudomesh'
Migrated tips for 'perl-maven'
Migrated tips for 'exercism'
Done.

[chadwhitacre]

Cool. Is that in production?

(P.S. A couple minor style suggestions ... )

from gratipay.wireup import db, env
from gratipay.models.team import Team, AlreadyMigrated

db = db(env())

new_teams = db.all("""
    SELECT teams.*::teams
      FROM teams
     WHERE is_approved IS TRUE
""")

for team in new_teams:
    try:
        team.migrate_tips()
        print("Migrated tips for '%s'" % slug)
    except AlreadyMigrated:
        print("'%s' already migrated." % slug)

print("Done.")

[chadwhitacre]

Did payout for 7 participants.

[chadwhitacre]

This is going quickly!

[chadwhitacre]

Script ran for a minute (0:01:43.621884).

[chadwhitacre]

O.O

[chadwhitacre]

[chadwhitacre]

[chadwhitacre]

Good effing work, @rohitpaulk. Looks like you nailed it. 🔨 💅

[chadwhitacre]

Now next week we gotta get you paid again. :)

[chadwhitacre]

Payouts look good in Balanced, no charges there.

[rohitpaulk]

Already? Man that's quick. 😃

[rohitpaulk]

!m @whit537

[chadwhitacre]

Hit a bug in MassPay ...

Traceback (most recent call last):
  File "./bin/masspay.py", line 304, in <module>
    main()
  File "./bin/masspay.py", line 297, in main
    compute_output_csvs()
  File "./bin/masspay.py", line 174, in compute_output_csvs
    payees = [Payee(rec) for rec in csv.reader(open(INPUT_CSV))]
  File "./bin/masspay.py", line 62, in __init__
    self.gross = D(amount)
  File "/Users/whit537/lib/python2.7/decimal.py", line 547, in __new__
    "Invalid literal for Decimal: %r" % value)
  File "/Users/whit537/lib/python2.7/decimal.py", line 3873, in _raise_error
    raise error(explanation)
decimal.InvalidOperation: Invalid literal for Decimal: ''

[chadwhitacre]

I think it's because we're allowing 'null' fee_cap for PayPal?

[chadwhitacre]

Repaired with:

update exchange_routes set fee_cap=20 where network='paypal' and fee_cap is null;

[chadwhitacre]

42 results

[chadwhitacre]

Log downloaded and droplet destroyed.

[chadwhitacre]

MassPay done and posted back for 7 users.

[chadwhitacre]

Escrow shuffled.

[chadwhitacre]

Logs uploaded in https://github.com/gratipay/logs/commit/91d80c95a8ed39c40bff067f23eaa465822d14e9.

[chadwhitacre]

Thaaaaaaaaat is satisfying. Awesome work, @rohitpaulk. I will catch you flip side. 🌝