PayPal IPN, CodeIgniter and Encoding – a recipe for disaster!


Having experienced a couple of banging-my-head-on-the-desk months trying to integrate PayPal IPN and CodeIgniter so that it doesn’t do stupid things, I can share with you a few pitfalls that us mere mortals might skim over in the awful world of PayPal documentation. (Although thumbs up to PayPal technical support, they have put me on the right track more than once!).

The source of my problems are international users and the funny non-english characters they selfishly include in their names and addresses, or assume that having two lines in their address is ok – well i’m here to tell you that it is not ok because dammit, we are trying to write programs here and we are lazy!

When it comes to PayPal IPN:

In other words if you put one little character out of place in your IPN response, PayPal will take its ball and go home

When a New Line is not a New Line

CodeIgniter is going to take any posted new lines (like the second line in a postal address) and standardise them, and there is a few unappealing hacks nothing you can do about it.

// it's my way or the highway!
if (strpos($str, "\r") !== FALSE)
    $str = str_replace(array("\r\n", "\r", "\r\n\n"), PHP_EOL, $str);

See that? In system/core/Input.php of CodeIgniter, any new lines in your POST, GET or COOKIE get standardised to your PHP installation, which more than likely is going to be “\n”. Without going into too much detail (because I barely know what i’m talking about), different OS’s have historically had different ways to express new lines and carriage returns, and you need to send back to PayPal EXACTLY what they sent you.

While this may serve some purpose to CodeIgniter internals, it’s a massive problem for PayPal,  which sends a “\r\n” but gets “\n” back. PayPal has a simple, brutal response to describe this heresy – INVALID.

You have a couple of options:

a) Instead of using $_POST or $this->input->post(), use:

 $data = file_get_contents('php://input');

This gets the raw post before CodeIgniter or any other part of your application can mess with it.

b) Reverse what CodeIgniter did – slightly hackish, but if you know PayPal sent you “\r\n” then make sure it gets it back

str_replace("\n", "\r\n", $v);

Do you speak my l@nguage©?

More specifically, do you use UTF-8 character encoding EVERYWHERE?

See, a bunch of people have names and addresses that contain things like accented characters. One option might be to tell them to type in an incorrect variation of their address or name, replacing the weird characters with something that is on my standard US keyboard. Another option is to store it correctly, serve it correctly and handle it correctly.

This comes in a few parts:

1. Tell PayPal about UTF-8
Put this in your PayPal web payment standard form – this apparently takes higher precedence than the obscure “PayPal button language encoding” option on the PayPal website:

<input type="hidden" name="charset" value="utf-8">

2. Tell PHP about UTF-8
PHP has an option in php.ini called default_charset, but tragically it’s default value was not UTF-8 until PHP 5.6. You can set this value with:

ini_set('default_charset', 'UTF-8');

Or update php.ini default_charset value with “UTF-8”.

The best way to test this is something like:

$str = 'Pauzé';

echo $str;

If it doesn’t output the same, you are probably using a non UTF-8 charset in use.

3. Tell your database about UTF-8
Although not strictly related to IPN, there is little point accepting UTF-8 characters in PHP if you do not also store these characters correctly in the database. Unfortunately MySQL has a stupid weird implementation of UTF-8 that is not corrected until v5.5.

As to whether the omitted characters are likely to exist in names and addresses, time will tell (i’m guessing not), however to be on the safe side you should run MySQL 5.5+ where possible.

Leave a Reply

Your email address will not be published. Required fields are marked *