PayPal IPN, CodeIgniter and Encoding – a recipe for disaster!

head_bde6c3_991300

Having experienced a couple of banging-my-head-on-the-desk months trying to integrate PayPal IPN and CodeIgniter so that it doesn’t do stupid things, I can share with you a few pitfalls that us mere mortals might skim over in the awful world of PayPal documentation. (Although thumbs up to PayPal technical support, they have put me on the right track more than once!).

The source of my problems are international users and the funny non-english characters they selfishly include in their names and addresses, or assume that having two lines in their address is ok – well i’m here to tell you that it is not ok because dammit, we are trying to write programs here and we are lazy!

When it comes to PayPal IPN:

ski-south-park

In other words if you put one little character out of place in your IPN response, PayPal will take its ball and go home

When a New Line is not a New Line

CodeIgniter is going to take any posted new lines (like the second line in a postal address) and standardise them, and there is a few unappealing hacks nothing you can do about it.

// it's my way or the highway!
if (strpos($str, "\r") !== FALSE)
{
    $str = str_replace(array("\r\n", "\r", "\r\n\n"), PHP_EOL, $str);
}

See that? In system/core/Input.php of CodeIgniter, any new lines in your POST, GET or COOKIE get standardised to your PHP installation, which more than likely is going to be “\n”. Without going into too much detail (because I barely know what i’m talking about), different OS’s have historically had different ways to express new lines and carriage returns, and you need to send back to PayPal EXACTLY what they sent you.

While this may serve some purpose to CodeIgniter internals, it’s a massive problem for PayPal,  which sends a “\r\n” but gets “\n” back. PayPal has a simple, brutal response to describe this heresy – INVALID.

You have a couple of options:

a) Instead of using $_POST or $this->input->post(), use:

 $data = file_get_contents('php://input');

This gets the raw post before CodeIgniter or any other part of your application can mess with it.

b) Reverse what CodeIgniter did – slightly hackish, but if you know PayPal sent you “\r\n” then make sure it gets it back

str_replace("\n", "\r\n", $v);

Do you speak my l@nguage©?

More specifically, do you use UTF-8 character encoding EVERYWHERE?

See, a bunch of people have names and addresses that contain things like accented characters. One option might be to tell them to type in an incorrect variation of their address or name, replacing the weird characters with something that is on my standard US keyboard. Another option is to store it correctly, serve it correctly and handle it correctly.

This comes in a few parts:

1. Tell PayPal about UTF-8
Put this in your PayPal web payment standard form – this apparently takes higher precedence than the obscure “PayPal button language encoding” option on the PayPal website:

<input type="hidden" name="charset" value="utf-8">

2. Tell PHP about UTF-8
PHP has an option in php.ini called default_charset, but tragically it’s default value was not UTF-8 until PHP 5.6. You can set this value with:

ini_set('default_charset', 'UTF-8');

Or update php.ini default_charset value with “UTF-8″.

The best way to test this is something like:


$str = 'Pauzé';

echo $str;

If it doesn’t output the same, you are probably using a non UTF-8 charset in use.

3. Tell your database about UTF-8
Although not strictly related to IPN, there is little point accepting UTF-8 characters in PHP if you do not also store these characters correctly in the database. Unfortunately MySQL has a stupid weird implementation of UTF-8 that is not corrected until v5.5.

As to whether the omitted characters are likely to exist in names and addresses, time will tell (i’m guessing not), however to be on the safe side you should run MySQL 5.5+ where possible.

CodeIgniter – Thanks For The Ride

Wow, how time flies!

It’s been about 2.5 years since I started using CodeIgniter, and 18 months since my last progress report. Even though I was clearly wrong on some things, I still get a laugh out of how pleasantly surprised I was at my first exposure to a real framework, and how (or more importantly why) I ever worked without one.

But alas; now seems like as good a time as ever to make what could be my final post on CodeIgniter :(.

worlds-smallest-violin 

I do not claim to be a PHP ninja by any stretch, but I feel obliged to try and help any up and coming developers who are asking the immortal question – which framework should I choose?  

2.5 years ago I was asking that question and my choice was CodeIgniter. Here is my interpretation of the bad, the good and the future for CodeIgniter.

The Bad

The Library and the API are stale and have cobwebs
You can certainly find some good CodeIgniter libraries around, but the fundamental problem is in the way they are implemented in CI. While all the cool kids and frameworks are using PHP 5.3+, framework agnostic composer packages and namespacing, CI is stuck with a library system (ignoring the getsparks initiative which never really  took off) that requires a lot of manual work to get rolling with. And nobody likes manual work.

Writing a native CodeIgniter library which significantly hooks into CI functionality in no way passes the ever-desirable “code reusability” test.

Stagnant development, tough decisions ahead
As pointed out in a post on Phil Sturgeon’s blog (a guy who seems to know his shit), CodeIgniter has some serious, if not insurmountable obstacles to overcome should it ever wish to catch up to the emerging PHP standards.

Significantly, to catch up to the cutting edge frameworks, CI would be unrecognisable from its current state (and hence, not backwards compatible).

Unit Testing
Doesn’t work. If you don’t realise the significance right now, in time you will – and if your vast code base is not readily testable, you will be doing these ones:

facepalm

The Good

It still does a decent job
CI still largely does a decent job of solving many common problems, and it’s well documented and supported. For plenty of developers, this is all that matters.

You can start using composer now
Whether you are building new apps with CI or you are maintaining old apps, start using composer now (I outline how easy this is here).  At least you can start getting into the practice of dependency management and using and developing project dependencies.

By studying many of the popular packages available you will also start to realise how top notch modern PHP code is written, and start to see some of the shortcomings in CodeIgniter.

The Future

CodeIgniter is here to stay – at least for now – and who knows, it might even get on the comeback trail if version 3.0 can really deliver. At the very least, it’s going to continue to solve problems for a lot of developers and be used in existing applications for years to come.

The problem for me is that it is a very long way from the cutting edge, and I see it as a slowly sinking ship. You have to scroll all the way to 24th August 2006 on the PHP release page to get to 5.1.6, which is the minimum requirement for the last stable release of CodeIgniter, 2.1.4.

Depending on your requirements, CI may very well be ok, but for the complex application I would definitely suggest you look at other frameworks, and like many have suggested, Laravel is a worthy modern successor to CodeIgniter and one that I am currently sinking my teeth into.

Perhaps “Developer13″ comment summed it up best in EllisLab’s post which proposes offloading the ownership of CodeIgniter – I couldn’t agree more.

Just put it to rest peacefully! It’s lived a good life.