Credit Card Failures Re: Capture and Presentation

[kaguillera]

I know that this may be a rehash of other issues but I tried to consolidate them, so forgive me.

Issue:

The issue is about showing CC failures to receivers/tippee (and possibly givers/tipper) so that they can see why their income is suddenly dropping (and for givers, if, when, and possibly why their credit cards are failing)

Related Tickets:

#364 - keep track of credit card failures in database
#583 - Notify Users of Payment Errors
#915 - analyze credit card failures
#1124 - Email me whenever you bill my credit card
#1746 - display on-site notification re: credit card failure
#2372 - investigate 402s
#2441 - log request id when cc fails
#2443 - log additional information in exchanges
#2534 - Display number of card failures on history page
#2603 - show failing credit cards on patrons chart
#2640 - Notify users of credit card failures (PR for #1746)
#2666 - Track down policy/code change causing credit cards to fail
#2764 - #2534 display card failure qty (PR for #2534)
#2779 - backfill exchange.status column
#3011 - Record credit card failure in the DB (PR for #364)

Notes:

When I first looked at attempting to work on ticket #2534 the problem as far as I could see was that there is no definite way of identifying if that failed status is due to credit card failure or from lack of funds and no card/bank account associated with user. This is due to the fact that the failures are identified at the point when Payday is run but the information is not stored in the DB. Please correct me if I am wrong. The following is notes that I made after going through all of the tickets that I thought were relevant and related (there may be more) to this entire issue of credit card failure and the presentation of the information to the patrons of Gratipay.

Ticket #364 highlighted the need to record the credit card failures in the DB. At the time of the creation of the ticket the credit card failures were being stored in a logs in flat files. The moving of the data from the flat file to the DB was to allow for better analyst of credit card failures (#365).

From #915 it was noted that credit card failures were identified as a problem. At the time of creation of the ticket the increase of failures were due to the new fraud rules applied by Balanced. There were no automated notifications.

From #583 a request for email notifications on credit card failures by givers/tipper was made. It was identified that this would required mandatory valid email address (and thus indirectly the verification of those emails which was not stated in the ticket)

Even though #1124 does not directly relate to credit card failures but it was commented that the email could also be used to inform users of credit card charges that fail.

Ticket #1746 identified the need to display on-site notification of credit card failures. At the time of creation of the ticket according to the tread there were on-site notifications for impending credit card expirations (#2349)

Ticket #2372 is about credit card failures when credit card is added but also include failures when payments are attempted using a credit card.

In ticket #2441 which is a re-ticket of #2372 the need to capture request id of credit card failures and was considered a possible part of the implementation of #2443.

In ticket #2443 the need to capture additional information about exchanges was considered necessary in order to aid in debugging.

Ticket #2534 highlighted the need to show the number of credit card failures on the receiver’s history page.

Ticket #2603 is similar to the other tickets in that it is a request to show credit card failures on the patrons’ charts.

Ticket #2640 is the pull request to implement ticket #1746 and accompany #2629

From ticket #2666 the need to trigger a notification to that user to inform them of the denial until correct or complete information is provided where the credit card failure is due to change in policy or code is raised. At the creation of this ticket there was on-site notifications (#2640) but still no email notifications. This is mainly due to that fact that at the time of the creation of ticket #2666 there weren’t any email notifications for anything. Since that time the requirement and validation of email addresses of user of gratipay are being implemented (#2312).

Ticket #2764 is a failed pull request for the ticket #2534.

Ticket #2779 is raises the issue of needing to have the exchange.status column in the DB have non-NULL values. This is related to #2443 and #364

Ticket #3011 is a PR for ticket #364 and thus should allow for necessary information to implement the majority of the tickets made above.

Requirements:

There are a number of issues identified in the note.

1. Identification of the relevant credit card transaction information in order to properly and effectively inform both givers and receiver as to if, when, how much and possibly why credit card failures occur.
2. The capturing of the relevant credit card transaction information including but not limited to credit card failures. In some cases this would require the retrieval of back data from Balanced (if possible)
3. Use the relevant credit card transaction information to notify the giver and receiver of credit card failures (if, when, how much and possibly why credit card failures occur) on-site, and via email
4. Also if desired use this information for analysis activities including but not limited to fraud.

Questions:

• Are credit card failures captured and recorded in the DB for both receivers and givers? @Changaco correct me if I am wrong but this is partially if not completely done in Record credit card failures in the DB #3011?
• Is the giver/tipper notified on-site when their credit card fails? Again @Changaco I believe this was implemented by the PR in Notify users of credit card failures #2640
• Is the giver/tipper notified via email that their credit card failed? I have not seen evidence that this has been implemented since we only implemented the requirement of valid emails (Email Verification #2312) and tickets that have requested this are still open (Notify Users of Payment Errors #583, Email me whenever you bill my credit card #1124, etc)
• Is the receiver/tippee notified on-site the number of its tippers’ credit cards have failed? No. (Record failed transfers and modify the history page to show them correctly #2534, Show failed transfers on patrons chart #2603)
• Is the receiver/tippee notified via email the number of its tippers’ credit cards has failed and is this necessary?
• Should all of these issues that have not been resolved be down in one PR? My personal opinion is that it should not. It may be too much.

Steps:

• Identify relevant credit card transaction information required to properly inform Gratipay patrons (both givers and receivers) of credit card activities. This could include but not be limited to credit card failures and possibly pending expiring credit cards.
• Make changes to the DB structure (and thus schema) to accommodate the new data to be collected.
• Retrieve back data from Balanced where necessary if it possible.
• Make changes to Payday to capture future information from Balanced.
• Identify places to effectively present this information to the patrons (both givers and receivers). i.e. for example do we place it only on the History page or on the patrons chart or both.
• Design and implement the sending of emails to inform the patrons of the credit card activities.
• Identify useful analytics of the data that we would have as a result of the completion of all of the steps above.

This may not be complete or it may be overly complicated @Changaco, @whit537 and others please review and tell me what you think. I would like to start working on completing this. It would be good if the steps are vetted and fine tune so that I don’t “reinvent the wheel” or do unnecessary work. Additionally if this is not important please tell me.

Want to back this issue? Post a bounty on it! We accept bounties via Bountysource.

[chadwhitacre]

I love it. @kaguillera's monster note-taking skills escape from Evernote onto GitHub! 💃 🐘

[Changaco]

Whoa, that's a big comment.

Here's a checklist showing what's already done and what's left to do:

• show CC failure to givers as on-site notification (Notify users of credit card failures #2640)
• record CC failures in the exchanges table (Record credit card failures in the DB #3011)
• show CC failures on the giver's history page (simple consequence of Record credit card failures in the DB #3011)
• notify givers of CC charges via email (Email me whenever you bill my credit card #1124) Closed (via notify users of card charges #3301)
• notify givers of failed of CC credit card failures via email (notify users of card charges #3301)
• record failed transfers and modify the history page to show them correctly (Record failed transfers and modify the history page to show them correctly #2534)
• show failed transfers on patrons chart (Show failed transfers on patrons chart #2603)
• weekly email report, which would notify receivers of failed transfers (weekly email report #1022)

Here's the status of each of the open issues:

• Email me whenever you bill my credit card #1124 and weekly email report #1022 are blocked on Notify patrons via email when user opts-in #3136 in which we're building the email notification system
• Record failed transfers and modify the history page to show them correctly #2534 is ready to start, it consists of adding a status column to the transfers table and modifying payday to actually record failed transfers
• Show failed transfers on patrons chart #2603 is blocked on Record failed transfers and modify the history page to show them correctly #2534, you obviously can't show data that's not available

Should all of these issues that have not been resolved be down in one PR?

No, separate issues → separate PRs. Experience has shown that big PRs are problematic and should be avoided.

[kaguillera]

Thanks @Changaco I have a starting point. I will begin to work on this.

[Changaco]

#3136 has landed, thus #1124 and #1022 are no longer blocked.

[chadwhitacre]

@Changaco Huzzah! Great news! 💃