Last modified: 2013-12-15 12:17:57 UTC
Some contribution/contacts are entered twice into Civi - generally 1-3 hours apart from one another. This behavior appears to have begun after the upgrade, and is possibly related to credit card payment processing. See: http://wikimediafoundation.org/wiki/Special:ContributionHistory/en -or- do a search for contributions after august 1.
After some investigation, it turns out that the problem is due to a change in how PayPal/Payflow handle transactions. Originally, transactions that we received via PayPal would result in PayPal sending us a message about that transaction through a service we wrote to receive notification about transactions from PayPal (it's our 'IPN Listener'). Transactions received via Payflow were completely handled on our end - we only received transaction information by asking Payflow directly for it. Now, Payflow transactions as well as PayPal transactions are sent from PayPal to our IPN listener. As a result, we were pulling in transaction information for the same transaction from two different sources, resulting in double transaction entry. I am working on a stop-gap measure to deal with this while we sort out a better solution to these changes.
The duplicate transactions we're seeing in the queue are missing some of the data bits that we normally attach to transactions (like the essential 'contribution_tracking_id'). So until we figure out a better long-term solution of how to handle the changes to PayPal's infrastructure, I updated the code to ignore transaction messages in the queue that do not contain a 'contribution_tracking_id'. The payment pipeline is back up, all outstanding contributions/comments that were in the queue are now in Civi and things appear to be running normally. Marking as resolved.
[Moving from "CiviCRM/General" to "Wikimedia/CiviCRM; see bug 57738]