Just yesterday, we were notified by a wonderful user (thank you Mark N.) that we had a problem with our purchasing code.  After digging into it a bit, it wasn’t just one problem, it was a comedy of issues–damn near three act play of them, to be honest.  The last time we’d tested the Facebook Credits system was in mid-December, and a few minor changes here and there had been just enough to break things completely.

You want an example?  You asked for it…

• I dug in and found that the images we thought had pixels in them had recently turned into a 1×1 pixel fully transparent .PNG files, which were supposed to indicate to a player what it was they were buying.  Oops, art process bug.
• I found that I’d “improved” the legibility of some includes at the top of the Facebook callback, which in layman’s terms means I was trying to load a file that was not there and it blew up immediately, even before we could write out a metric to our database saying the file had loaded up.  Ouch.  More comments were added warning not to make it pretty in the future.
• Further, I had slightly changed the database definition so that a piece of data was no longer in the place it had been for months.  Although I did make a sweep through the code, I missed this one place, so the final stage of the purchase would fail with an invisible database error.  This time I call foul on the language, the database, and will point the finger in more places if you aren’t careful!

But really, it’s part of the job to test things carefully and make sure that sweeping changes force the game to go through all the paces that a full test plan should.  In fact, before every release, that should happen.  Sadly, with a team our size with as many tasks on our plates, that hasn’t always happened to the degree that it should.  For that, I apologize to our players.  We aim to offer the highest possible quality title at all times, and it hasn’t always been smooth sailing.  But we are working on both Trivia Adventure and our delivery of it.  This was the first A-ha! moment: realizing that we hadn’t received any life-affirming purchases from any users at all since we started showing the game publicly wasn’t due to a bad game, but to a single page of busted code that could have been detected by a part-time QA person following a script of tests to run every day.

The most recent surprise comes tonight, when I’m poring over the details of some logs we’re writing out to help us understand user behavior.  These are called game metrics, in the parlance of social game developers.  I noticed something odd in our logs while reviewing a couple of users’ play: almost everyone has the same PHPSESSID cookie for their entire session–this value is assigned by the server and kind of helps track the user (although we don’t really use it much, we store it out to help make sense of the login process).  One user’s changed every time he called our server.  Every.  Time.  I was blown away.  The code is the same for all our users, so we weren’t telling his browser to do that.  I looked everywhere online for an answer but came up empty.  Nobody seemed to have reported this issue, and thus had no solution for it.  Luckily, it doesn’t matter to us, but I was worried because it shook everything I thought I understood about how browser cookies work.  Until I had that A-ha! moment: that player’s date is wrong.  Not just wrong, but off by at least a month, but maybe as much as a year!  I have to assume this is why the cookie expired literally the second it reached his computer.

In all, not a bad couple of days.  Learning something is always fun, but when it’s something as important as the vitality of the company hangs in the balance over (what is often considered) the least desirable, lowest paid, and sometimes even reviled position in the a game company… steps must be taken, because that job is important, dammit.

JH

spread the word:

You must be logged in to post a comment.