migrate cards from Balanced to Braintree

[chadwhitacre]

To: Balanced, Braintree

Greetings, teams. What's our process for migrating Gratipay's card vault from Balanced to Braintree?

From: Braintree

The process is pretty simple. Just reach out to [email protected] and let them know you’d like to export to us. We’ll receive a file from them and run the import on our side. The process takes about a week from beginning to end. We cannot process ACH transactions so unfortunately those customers will not be imported. Let me know if there are any questions.

[chadwhitacre]

From: Balanced:

Send this encrypted blob to your guys at Braintree.

-----BEGIN PGP MESSAGE-----
[...]
-----END PGP MESSAGE-----

It contains the data they need to import all your credit cards and bank accounts into Braintree. It's encrypted with their public key.

Please note that the URL within expires in 24 hours so if they don't get to this today they'll likely need to reach out for a new URL. They can email the support team directly and we'll assist them.

Forwarded to Braintree!

[chadwhitacre]

Okay, we've initiated the migration, though we may need to reinitiate it (though Braintree might take that initiative themselves per Marshall's email?).

That means that anyone who updates their card between now and when we start vaulting on Braintree will not have their new info saved on Braintree, unless we make another export. However, anyone who changes their card on Braintree before the second export lands could potentially have their change clobbered.

One option would be to simply disable the credit card form until cards are migrated.

[chadwhitacre]

From: Braintree
To: Balanced
CC: me

Hi Balanced Support,

Unfortunately I missed the window for this export. May I request it get run again? Thank you.

[chadwhitacre]

From: Balanced
To: Braintree
CC: Me

No worries, []. Here's a new link.

[chadwhitacre]

From: Braintree
To: Balanced
Cc: Me

Thanks for resending Marshall, we were able to pull down the file successfully. Chad, I have scheduled Gratispay for the first available opening on our import calendar, which is this Thursday (5/14). Let me know if you have any questions!

To: Balanced
Cc (I think?): Braintree

Awesome, thank you! :D

[chadwhitacre]

https://gratipay.freshdesk.com/helpdesk/tickets/2121

[chadwhitacre]

Thanks for your patience. We are on track for Thursday. We’ll let you know if there are any last-minute issues.

https://gratipay.freshdesk.com/helpdesk/tickets/2129

Please see attached for the logs. The import log and the error log should be clear but you had around 300 tokens with a customer ID of none. We imported those as well and had our gateway generate the tokens for them. Please take a look and let us know if there are any questions.

https://gratipay.freshdesk.com/helpdesk/tickets/2145

[chadwhitacre]

Import logs uploaded in https://github.com/gratipay/logs/commit/61bc4117f76b7e269d9937ea3c19ef41dea8f2b9.

[chadwhitacre]

Picking up from #3470 (comment) ...

[chadwhitacre]

you had around 300 tokens with a customer ID of none.

Hrm ... old records on Balanced, from before they introduced the customer type? Hunch it relates to the problems we've had in the past with Balanced URLs (#2085).

[chadwhitacre]

So the work here is to write a script to populate the Participant ID custom field for all of the new records from Balanced.

[chadwhitacre]

Vault search in the web UI doesn't support custom fields. :-/

[chadwhitacre]

Looking at the logs, I count 26 names of customers in the bad import log that aren't present in the successful import log.

[chadwhitacre]

Do we just let those people go? What's our option? Contact Balanced and ask for a re-export?

[chadwhitacre]

I count 328 customers in the none_customer log and 4,567 in the good log, so that's about 7% of records affected.

[chadwhitacre]

To: Balanced

Braintree reports that 328 out of 4,895 customer records (7%) in the import file they received have no customer id. Any idea what that's about?

[chadwhitacre]

I count 3,061 unique names in the good import (not counting Nil and None). 26 out of 3,061 ain't bad.

[chadwhitacre]

How does the set of customer ids that Braintree now has compare with what we have for balanced_customer_href?

[chadwhitacre]

And how does the set of customer ids in the CSV logs compare with what's actually in Braintree?

[chadwhitacre]

We have 5,309 participants with a balanced_customer_id.

[chadwhitacre]

Alright, I exported customers from Braintree, and confirmed that the set of Balanced ids in the import log is identical to the set in Braintree.

[chadwhitacre]

From: Balanced

Please pass along a list of the customers in question and I'll have engineering take a look at this for you!

To: Balanced

In working on an answer to your question, I've discovered that the Card ID may suffice to make the connection between our user and the customer record at Balanced/Braintree. I'll proceed with that and let you know if I get stuck again. Thanks! :-)

[chadwhitacre]

Balanced Customer IDs

Source	IDs	Not in Balanced	Not in Gratipay	Not in Braintree
Balanced	5,310	-	13	743
Gratipay	5,309	12	-	749
Braintree	4,567	0	7	-

[chadwhitacre]

Balanced Card IDs

Source	IDs	Not in Balanced	Not in Gratipay	Not in Braintree
Balanced	4,227	-	329	5
Gratipay	4,012	114	-	5
Braintree	5,865	1,643	1,858	-

[chadwhitacre]

I can't say that I'm entirely surprised that the numbers don't seem to quite line up. :-/

[chadwhitacre]

The task is to link up Gratipay participants with Braintree customers, by setting braintree_customer_id in the Gratipay database, and participant_id in Braintree. A false match means that we end up charging one person for another's subscriptions: bad!

[chadwhitacre]

I'm surprised that there are so many card IDs in Braintree that aren't in Balanced or Gratipay. I only looked at card IDs matching the CC... format that Balanced used.

[chadwhitacre]

There are only 15 "payment method tokens" in this Braintree export that don't match the CC... pattern.

[chadwhitacre]

Maybe Balanced included invalidated cards in the export to Braintree, but not in the export to us?

[chadwhitacre]

We should also create new exchange_routes for the Braintree cards.

[rohitpaulk]

Note: We only want to update braintree_customer_id for people who don't have a route with the braintree-cc network already (the new cards that are vaulted with braintree).

[chadwhitacre]

Alright, I'm going to write this script based on the import log. I'm going to depend on the customer ID and card ID in there. I'll double-check to make sure that the mapping lines up with what we have for exchange routes in the db, and then add new exchange routes for braintree along with setting braintree_customer_id.

[chadwhitacre]

I'm not going to worry about the records with no customer id for today.

[rohitpaulk]

!m @whit537

[chadwhitacre]

Spot-checking, I'm seeing a card that is "status: removed" at Balanced that is merely "Expired" at Braintree. 😮

https://dashboard.balancedpayments.com/#/marketplaces/MP12Xw5lL6iaILtqImIoroDL/cards/CC9ZBPfVyWiXwRNKrwSaO4m

https://www.braintreegateway.com/merchants/msjyyvtsvvnc4ny6/payment_methods/any/CC9ZBPfVyWiXwRNKrwSaO4m

[chadwhitacre]

At this point it sure feels like we'd better cut our losses. Better to have some credit card attrition than fail to get off Balanced in time.

We have cards waiting for us to use in Braintree. Maybe it's not the cleanest set of data but it's what we've got: it's too late to migrate more/better data from Balanced to Braintree.

Let's use the cards we have waiting for us at Braintree! :)

[chadwhitacre]

The exchange route is what really matters. We do pass braintree_customer_id during create_card_hold, though from the docs it's not clear to me that that's strictly necessary.

[rohitpaulk]

We do pass braintree_customer_id during create_card_hold, though from the docs it's not clear to me that that's strictly necessary.

There was a reason for that... If I recall correctly, it had something to do with custom fields. I'll dig in deeper later.

[chadwhitacre]

The five at #3391 (comment) are probably explained by the import error log.

[chadwhitacre]

Done and done!