Monday, September 15, 2008

Pet Peeve of the Week: Commented Code

My fellow engineers at Amie Street can attest that I am full of almost inexplicable hatred for some very specific things. Today I would like to rant about my hatred for commented code.

By commented code, I don't mean the good kind of commented code. You know, the kind where it answers the questions you have about how something works but doesn't ramble on. I love this kind of commented code and try to write it wherever possible. Good commented code looks something like:

/**
* Parse a color parameter in the request. Expected format is
* six digit hex (like in HTML but without the #)
*/
private function getColorParam($request, $param, $default='FFFFFF') {
...

or maybe:

// Copy topleft corner
imagecopy(
$targetIm, $templateIm,
0, 0,
0, 0,
$slice_l, $slice_t);

Today I'm ranting about the bad kind of commented code. You know, the kind of commented code that is actually code that has been commented out. It looks like this:
public function newReleasesAction($request) {
/*
$n = date('N', time());
if($n==2){
// Its tuesday
$dateString = date("F jS", time());
}
else{
$dateString = date("F jS", strtotime('last Tuesday'));
}
*/
$dateString = date("F jS", Amie_DB::getUnixTimeForNow());
$this->stash('newReleasesDate', $dateString);

Note that the majority of the above code has been commented out with a block comment. Rather than answering questions, this kind of insidious "comment" leaves you with more. Mainly, why is it commented out? The developer trying to read this code can make several different guesses:

  • The code isn't necessary anymore - Maybe this code used to be important, but the variable it's setting wasn't used anywhere so someone commented out.

  • The code didn't work properly - the commented code was causing problems, so someone replaced it with the line below the comment block. The replacement code may or may not do the same thing as the original, but the developer needed to patch the issue to get the page to load, or whatever.

  • The code was commented out for debugging purposes and someone forgot to uncomment it. This kind of thing happens all the time - how often have you made a commit with random error logs like "Got here" in your codebase?

  • The code was replaced with a better version (or refactored) but someone had an emotional attachment- for some reason some developers are afraid of the delete key and the rm command.



Frankly, every one of these reasons is stupid. One by one:

The code isn't necessary anymore


Delete it! If you ever find that it becomes necessary again, this is why you use version control.

The code didn't work properly


Commenting it out just risks that someone else will come and uncomment it again at a later date. If you don't know how to fix it, you should replace the broken code with something like "/* TODO: the algorithm that used to be here had a flaw. See bug #234291. Replaced algorithm with hardcoded data below */" or something of that nature.

The code was commented out for debugging


Can't say I haven't done this plenty of times myself, but before you commit it's good practice to do a quick diff to make sure you haven't inadvertently added useless log messages or commented out code that should be working.

The emotional attachment


Get over it! If you ever want to admire your old work you can always check out an old version of the code from the repository. It's pretty easy to pull back old code, but being a code packrat is kind of like taking your leftovers at dinner, putting them in a shoebox, and shoving them under your bed in case you're hungry in a couple of weeks. When you come back to the dead code it will be full of bugs that have grown since it was last used.

Friday, September 12, 2008

Getting Click Events from the Slider Track in Flex

In the past few days I've run into two separate situations in which I needed to be able to get an event when the track of a slider was clicked. In one case, I was using an HSlider control to set the volume of a music player. When the track was clicked the thumb of the slider would nicely tween to its new position, but the volume of the sound would abruptly jump when the thumb was done moving.

Ideally you would be able to tween the volume at the same rate that the slider thumb is moving, but to do this you need to be able to know when the track is clicked, before the slider thumb is done moving. Furthermore, you need to know the values the slider is moving from and to.

Ordinarily, the only event the slider component will dispatch when the track is clicked is the "change" event, and this happens after the slider thumb is done moving to the new location on the track. It is possible, however, to receive the track click event by extending the HSlider or VSlider classes. Below is some example code. Though we shouldn't really be accessing mx_internal members, I can't see any other way to do it. I'm open to suggestions as to another way to get access to this event, and the from and to values of the slider.


package {
import flash.events.MouseEvent;
import mx.controls.HSlider;
import mx.core.mx_internal;

use namespace mx_internal;

public class ExtendedHSlider extends HSlider {
override protected function createChildren():void {
super.createChildren();

super.mx_internal::getTrackHitArea().addEventListener(MouseEvent.MOUSE_DOWN,
function(event:MouseEvent):void {
var fromValue:Number = value;
var toValue:Number = mx_internal::getValueFromX(event.localX);
trace("Hooray! We're getting events when the track is clicked!");
trace("The slider is going to move from " + fromValue + " to " + toValue);
});
}
}
}

Thursday, September 11, 2008

big sort_buffer_size causes slow filesort

I spent several hours tonight trying to track down why a 12-row filesort was taking upwards of 30ms on our production DBs. After installing 4 different versions of mysql and not figuring anything out, I finally took a guess and decreased sort_buffer_size from 64M to 256K.

I'm not entirely sure why I thought a 64M sort buffer was a great idea, but it turns out it's a terrible one! After setting it to 256K the query returns in 0.00sec as expected.

Looks like the guys at Percona have noticed this too.

Monday, September 08, 2008

array_intersect_key is terrible

I always had a hunch that array_intersect_key was slow in PHP, but I didn't quite realize how slow until tonight when I decided to benchmark it. Here's a quick test script:


$big = array();
for ($i = 0; $i < 10000; $i++) {
$big[$i] = 234;
}

$small = array();
for ($i = 0; $i < 10; $i++) {
$big[$i] = 2435;
}

////////////////////////////////////////////////////////////

print "Testing big first\n";

$st = microtime(true);
for ($i = 0; $i < 100; $i++) {
$junk = array_intersect_key($big, $small);
}
$bigfirst = microtime(true) - $st;

////////////////////////////////////////////////////////////

print "Testing small first\n";

$st = microtime(true);
for ($i = 0; $i < 100; $i++) {
$junk = array_intersect_key($small, $big);
}
$smallfirst = microtime(true) - $st;


print "array_intersect_key\n";
print "========================================\n";
print "Big first: $bigfirst\n";
print "Small first: $smallfirst\n";

////////////////////////////////////////////////////////////

function common_keys($a, $b) {
$res = array();
foreach ($a as $key => $val) {
if (isset($b[$key]))
$res[] = $key;
}
return $res;
}

print "Testing big first\n";

$st = microtime(true);
for ($i = 0; $i < 100; $i++) {
$junk = common_keys($big, $small);
}
$bigfirst = microtime(true) - $st;

////////////////////////////////////////////////////////////

print "Testing small first\n";

$st = microtime(true);
for ($i = 0; $i < 100; $i++) {
$junk = common_keys($small, $big);
}
$smallfirst = microtime(true) - $st;

////////////////////////////////////////////////////////////

print "common_keys\n";
print "========================================\n";
print "Big first: $bigfirst\n";
print "Small first: $smallfirst\n";
?>


And the results:


Testing big first
Testing small first
array_intersect_key
========================================
Big first: 13.1110720634
Small first: 12.5442099571
Testing big first
Testing small first
common_keys
========================================
Big first: 0.363513946533
Small first: 0.000243902206421


By implementing this in pure PHP you get a 50000x speedup over using the built-in in some cases. Wow...

Update: this appears to be fixed in PHP 5.2.5. See the Release Notes.

Thursday, September 04, 2008

Flexdoc for Ubiquity

Mike Chambers whipped up an API for AS3 reference on Google App Engine yesterday. The API allows you to send a class name or fragment of a class name a get the links to the livedocs pages for any matches. Combine this with Ubiquity and you have a super simple way to get to livedocs for any AS3 class. To install Ubiquity, you can grab the xpi from here. To add the command, go to this page and just click the subscribe button in the top right.

ubiquity_flexdoc gist

Saturday, August 30, 2008

Custom Drag and Drop for Lists in Flex 3

Yesterday I spent far too long trying to figure out why my custom dragDrop handler for the Flex 3 mx.controls.Tree class wasn't being called.

I had all the relevant callbacks (dragDrop, dragEnter, dragOver, dragExit) registered, had trace() statements in place so I could watch their behavior, and had dropEnabled set to true on the Tree. I could see trace() output coming from my dragEnter, dragOver and dragExit handlers. Still, I was getting nothing from the only one which really mattered: dragDrop. I tried playing around with DragManager.acceptDragDrop, tried calling preventDefault() as the first call in the event callbacks, and various other voodoo, but all to no avail. What could possibly be wrong?

The answer: in order for the dragDrop callback you set to actually get called, you need to set dropEnabled to false! The property described by Adobe as "A flag that indicates whether dragged items can be dropped onto the control" has to be set to "false" in order to actually receive the dragDrop event. Go figure.

The semantics of this property are, in reality, "A flag that indicates whether to use the default behavior for accepting dragged items, which is to assume that the object being dropped here is the object we expect, and to accept all. Set to false if you want to override this behavior."

This isn't just true for the Tree class, either. This is true for all controls which inherit from ListBase, which is quite a few, including HorizontalList, TileList, FileSystemList, Menu, Tree and DataGrid. Hopefully in Flex 4 sanity will prevail and they'll rename this property something a little more intuitive.

Side note: I also considered the title "Worst Property Name Ever" for this post.

Monday, August 11, 2008

How to crop images with opencv in python

I just spent way too long figuring this out, so I figured I'd contribute to the google knowledge:

cropped = cvCreateImage( cvSize(new_width, new_height), 8, 3)
src_region = cvGetSubRect(image, opencv.cvRect(left, top, new_width, new_height) )
cvCopy(src_region, cropped)


You'd think this would be easy and/or documented, but nope!

Thursday, August 07, 2008

Advice for Startup Marketing Advice - Learn Statistics

An interesting blog post, entitled Startup Marketing Advice came across my delicious network this past week. In this post, the author espouses the merits of the scientific method to determine catchy and effective marketing messaging in a cheap and effective way.

I have no problem with the thesis of the post. He's completely right that A/B testing is a terrific way to determine whether one message is better than another. Every elementary schooler knows that a good experiment has a control group and a test group, and the way to show that the variable under test is significant is to demonstrate that the test group responds significantly differently from the control.

The problem, though, is that the example given in the post doesn't prove anything! We'll look at the top two messages in his adwords result list ("Startup Marketing Advice" and "Marketing with Adwords") and show that there is no significant clickthrough rate difference.

The first problem that should stand out is that the number of clicks is so small that the proportions cannot be accurately estimated. The general rule of thumb for estimating a proportion is that you need at least 5 positive and 5 negative examples. The trial with the most clicks in his experiment only received 4 clicks. Therefore the CTR%s given by Google hardly mean anything - the standard error (calculated sqrt(p(1-p)/n) for a binomial proportion like CTR) can't be accurately determined, but is sure to be nearly as large as the CTR itself.

The second problem is the assessment of statistical significance between the different choices. The standard error of the difference between two estimators is sqrt(S_1^2/n_1 + S_2^2/n_2). Calculating this for the top two adwords campaigns, we see that the standard error for the difference between their clickthrough rates is somewhere in the vicinity of 0.35%. [Again, the statistics here are all inaccurate because we've already failed the "minimum of 5 clicks" rule of thumb, but we'll ignore that.] Because the measured CTR difference is 0.55-0.30 = 0.25, we calculate a Z score of (0.25 / 0.35) = 0.71. Looking this up in a table of the normal distribution we can find out that this is equivalent to about 75% significance that campaign 1 is better than campaign 2. This doesn't sound too terrible, but it's hardly scientific, especially when you consider the small sample sizes as mentioned earlier.

Note: I am not a statistician. I just play one on a blog.

Thursday, July 03, 2008

Cassandra source on Google Code

As predicted, Facebook has open sourced Cassandra. It's available on Google Code with very little fanfare.

Some notes after my browsing through the source:

  • It uses hinted handoff and bootstrapping just like Amazon's Dynamo

  • Its consistency model doesn't seem to be quite the same - Dynamo uses vector clocks to determine causal relationships, whereas Cassandra seems to be just based on timestamps and "majority rules" semantics when timestamps are tied.

  • Membership is communicated by a gossip protocol as described in the Dynamo paper.

  • Requests are made to the system by sending thrift calls to any node. The thrift interface is included in the source.



Some further thoughts:

  • It doesn't seem like there's a lock on the table during bootstrapping. What happens to mutations made on the source node while it is bootstrapping the destination? Are they marked for later hinted handoff?

  • Would system performance be improved by using the new Thrift TNonblockingServer (see THRIFT-5 on JIRA)? It should be more scalable than the TThreadPoolServer they're using now.

  • Cassandra is around 40K lines of Java. How many lines would an equivalent Erlang program be, and what would be the performance difference?



All in all, it's a very interesting project sure to attract much attention. Now that Powerset has been acquired by Microsoft, I'm a little worried for Hbase's future -- two of the three main developers are Powerset employees. Maybe Cassandra can help fill the open source scalable database niche.

Monday, June 30, 2008

Facebook's next open source projects: Hive and Cassandra

A couple weeks ago, Jeff Hammerbacher from Facebook presented some details on Cassandra (see later slides), a structured p2p storage system similar to Google's Bigtable or Amazon's Dynamo. What is most interesting about Cassandra is that they seem to be preparing to open source it imminently. Jeff bookmarked two things on delicious last night:
  1. Cassandra: Welcome to your new Wikidot site

  2. Cassandra: A Structured Storage System on a P2P Network in Launchpad

Both sites are empty as of now, but it looks like they're planning on releasing the source some time soon using bzr for version control.

Another interesting Facebook project is Hive, a sort of data warehousing solution built on Hadoop. They've been discussing open sourcing this for several months now, but it looks like things are starting to happen with HADOOP-3601: Hive as contrib project.

On the non-Facebook open source front, we've got some news coming soon as well. We've made the decision to open source several of our internal tools under an MIT license - hold tight for more info.

Monday, June 23, 2008

How to work around an MSIE bug with Scriptaculous

For several months we've noticed a lot of 404s in our access logs that look like this:
x.x.x.x - - [2008-06-23 13:14:30] "GET /static/r/toLRhQ/js/'+libraryName+' HTTP/1.1" 404 549 - "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1;1813)"
x.x.x.x - - [2008-06-23 13:14:30] "GET /static/r/toLRhQ/js/'+libraryName+' HTTP/1.1" 404 568 - "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1;1813)"
x.x.x.x - - [2008-06-23 13:17:06] "GET /static/r/toLRhQ/js/'+libraryName+' HTTP/1.1" 404 568 - "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1;1813)"
x.x.x.x - - [2008-06-23 13:17:06] "GET /static/r/toLRhQ/js/'+libraryName+' HTTP/1.1" 404 549 - "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1;1813)"
x.x.x.x - - [2008-06-23 13:18:50] "GET /static/r/toLRhQ/js/'+libraryName+' HTTP/1.1" 404 549 - "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1;1813)"

For a long time we just ignored them, but I finally decided to track the issue down yesterday, and found the following line in scriptaculous.js:

document.write('<script type="text/javascript" src="'+libraryName+'"></script>');

For whatever reason, this particular version of MSIE decides to interpret this script tag in the middle of the javascript and go trying to find a JS named "'+libraryName+'". This is clearly incorrect behavior, and it seems like the bug was fixed in a later version of IE.

Eric came up with the following clever fix:
    document.write('<' + 'script type="text/javascript" src="'+libraryName+'"></script>');

Splitting the '<' and the 'script' tricks the broken version of IE into not seeing the script tag and solves the issue.

Hope this is helpful to others who find these mysterious requests in their logs.

Tuesday, June 17, 2008

Automatically parallelizing list comprehensions in Erlang

This evening I got to chatting with some people on IRC about how it would be neat if Erlang had syntax to automatically parallelize a list comprehension. For those who aren't familiar with Erlang yet, here's a brief introduction:

A list comprehension is a quick way of performing an operation over a list of values. For example, at the erlang prompt:
1> MyList = lists:seq(1,10).
[1,2,3,4,5,6,7,8,9,10]
2> [X * 2 || X <- MyList].
[2,4,6,8,10,12,14,16,18,20]

For those familiar with Python list comprehensions, this is equivalent to [x * 2 for x in range(1,11)]. It's also just like the map operator common in pretty much every functional language. In both Python and Erlang you can get a bit more complicated:
3> [X * 2 || X <- MyList, X rem 2 == 1]. %% Filter out the odd numbers from the list
[2,6,10,14,18]

or even take the cartesian product of several lists in the same list comprehension:
4> [{X, Y, X * Y} || X <- lists:seq(1,3), Y <- lists:seq(1,3)].
[{1,1,1},
{1,2,2},
{1,3,3},
{2,1,2},
{2,2,4},
{2,3,6},
{3,1,3},
{3,2,6},
{3,3,9}]

Now, back to the interesting part. Everyone talks about how Erlang is so easy to make concurrent. But if we add the self() function, which returns the current running process id, to the output of the list comprehension we see that it is not parallelized:
5> [{self(), X, Y, X * Y} || X <- lists:seq(1,3), Y <- lists:seq(1,3)].
[{<0.31.0>,1,1,1},
{<0.31.0>,1,2,2},
{<0.31.0>,1,3,3},
{<0.31.0>,2,1,2},
{<0.31.0>,2,2,4},
{<0.31.0>,2,3,6},
{<0.31.0>,3,1,3},
{<0.31.0>,3,2,6},
{<0.31.0>,3,3,9}]

All of the results were computed in the same process.

Handily, Erlang provides a way of inserting user code in between the parser and the compiler in the form of parse transforms. A parse transform is specified in the compile options for a module and can make arbitrary modifications to the abstract syntax tree of an Erlang program before the compiler gets to it, with the important caveat: "programmers are strongly advised not to engage in parse transformations and no support is offered for problems encountered." Everyone knows caveats are lame, so I went ahead and built a parse transform. Here's example usage:
-module(test).
-compile({parse_transform, plc}).
-export([test/0]).

test() ->
Result = plc:lc([{self(), A, B, C} || A <- lists:seq(1,2),
B <- lists:seq(3,4),
C <- lists:seq(5,6),
A * B < 5
]),
io:format("result: ~p~n", [Result]).

Output:

result: [{<0.3145.0>,1,3,5},
{<0.3146.0>,1,4,5},
{<0.3147.0>,1,3,6},
{<0.3148.0>,1,4,6}]

We can see that the list comprehension has been performed in parallel this time - each result element has a different value of self().

Want to check it out? Clone the git repo on github.

This probably doesn't work that well. Don't use it on production software. Unless you want to test it thoroughly first.

Thursday, May 29, 2008

Confidence intervals for Jaccard Similarity?

Hoping that someone is googling for the right terms here:

Anyone out there know how to calculate a confidence interval around an estimate of the Jaccard similarity coefficient?

For Pearson correlation you can use Fisher's Z-prime Transformation, but I can't quite figure a principled way of doing the same for Jaccard similarity.

Friday, May 16, 2008

FB Engineering blog post on Facebook Chat

Eugene at Facebook posted an interesting article about the technology behind the new Facebook Chat. This new service has large parts written in Erlang and communicates with the rest of the system using the Thrift bindings Amie Street and Facebook have been collaborating on for the last couple of months.

The good news for us: our thrift bindings are pretty much guaranteed to be stable and leak/bug free now that they're used for millions of messages/second over at FB.

If you're interested, check out over at the thrift git repository

Tuesday, May 13, 2008

Forcing a process to garbage collect in Erlang

We upgraded our dynamic pricing service tonight with a new version of thrift, so I was checking top to make sure everything was cool a few hours later. I noticed that one of the pricers was using 1.1G of RAM - significantly more than I'd ever seen it using before. Figuring it was a memory leak, I started a console node and connected it to the erlang cluster:

amiest@app2:~$ erl -name console
Erlang (BEAM) emulator version 5.5.2 [source] [64-bit] [async-threads:0] [hipe] [kernel-poll:false]

Eshell V5.5.2 (abort with ^G)
(console@app2.prod.amiestreet.com)1> P = pricer@app2.prod.amiestreet.com.
'pricer@app2.prod.amiestreet.com'
(console@app2.prod.amiestreet.com)2> net_adm:ping(P).
pong

First step of diagnostics was to fire up etop with etop:start([{node, P}]). This showed that the process count was remaining stable at an appropriate number -- we'd had a bug with a process leak once before but it didn't seem to be the cause this time. The amount of RAM used by processes seemed pretty high though. Next step:

(console@app2.prod.amiestreet.com)4> rpc:call(P, shell_default, i, []).

This is equivalent to running the i() command on the remote node, and shows all the running processes along with some info.

This printed out something useful - the rex process was using almost 900MB of RAM for no apparent reason. I'd never heard of rex, but evidently it handles remote execution from other languages, and possibly RPC as well. Checking on some other erlang nodes I saw that rex usually only used a few hundred KB.

After much googling, I came across this article which is my only plausible explanation for how rex got so big -- the Erlang GC doesn't run on a process if the process isn't doing any work.

The solution? rpc:call(P, erlang, garbage_collect, [pid(5038,10,0)]). (5038.10.0 was the pid shown by i()). This kicked the memory usage back down where it should be.

Friday, May 02, 2008

io_lib_pretty - a nice secret module

There are some modules in the erlang stdlib that aren't exactly advertised, but are quite useful. My newest discovery is io_lib_pretty. It hasn't got a manpage, but there are some docs if you less `locate io_lib_pretty.erl`.

io_lib_pretty is the module used by the shell to print records in a nicely formatted way. This isn't possible using plain io:format but can make program output a lot nicer.

Take for example a logging program that deals with records that look like this:

5> L = #logMessage{actor=23507, server_ip = <<123,234,123,234>>}.
#logMessage{actor = 23507,
server_ip = <<"{\352{\352">>,
timestamp = undefined,
level = undefined,
log_filename = undefined,
message = undefined}

If you just try to print it out, you get:


7> io:format("Logged: ~p", [L]).
Logged: {logMessage,23507,<<"{\352{\352">>,undefined,undefined,undefined,undefined}ok

Pretty useless output.

Using io_lib_pretty you can get:

9> io:format(io_lib_pretty:print(L, fun(logMessage, 6) -> [actor, server_ip, timestamp, level, log_filename, message] end)).
#logMessage{actor = 23507,
server_ip = <<"{\352{\352">>,
timestamp = undefined,
level = undefined,
log_filename = undefined,
message = undefined}ok

Just like the shell. I listed the record information manually in the function above, but you can easily use the record_info macro to accomplish the same without code duplication. Or even easier, use the exprecs parse transform (pretty printing example available there).

Next time: how to load record definitions dynamically at runtime.

Monday, April 21, 2008

Now using a CDN

I just pushed a change that enables CDN delivery of most of our static content (except for the music itself... for now). In theory it should make browsing the site a bit quicker, especially for those users not in the northeastern US.

If you happen to read this and notice any difference, comment below!

Saturday, April 12, 2008

EXPLAIN everything, part 2

I previously wrote a post urging people trying to optimize their database usage to run the "EXPLAIN" command on everything. This is easy advice to ignore in the case of simple queries, but even the simple queries can be big performance problems if you miss an important detail.

The other day, a friend was complaining of slow queries and was looking for some help fixing them. The first step we took was to take the "problem page" and enable debug output that printed each query that was run along with the number of milliseconds each took. If you don't have a facility for easily doing this (at the very least on a staging or development server) then you are essentially wandering in the dark for optimization.

Looking at the debug output, we found something surprising. The slowest query on the page was the seemingly innocuous:

SELECT `User`.`id` FROM `users` AS `User` WHERE `opensocial_id` = 219771253 LIMIT 1;

My friend was sure there was an index on User.opensocial_id yet the debug output showed that it took more than a second. With an index, this should take less than a millisecond. I asked him to run an EXPLAIN on it, and we saw:

mysql> EXPLAIN SELECT `User`.`id` FROM `users` AS `User` WHERE `opensocial_id` = 2778153 LIMIT 1;
+----+-------------+-------+------+---------------+------+---------+------+--------+-------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+----+-------------+-------+------+---------------+------+---------+------+--------+-------------+
| 1 | SIMPLE | User | ALL | opensocial_id | NULL | NULL | NULL | 272877 | Using where |
+----+-------------+-------+------+---------------+------+---------+------+--------+-------------+
1 row in set (0.00 sec)

The entry in the possible_keys column confirmed that there indeed was an index on opensocial_id. However, NULL in the key column meant that mysql was deciding the opensocial_id key was either unusable or didn't have good enough selectivity to choose over a full table scan. The high value in the rows column along with Using where under Extra indicated that mysql was reading through every row in the table, running a comparison, and outputting those rows for which the condition matched.

So, the next question was why mysql was deciding not to use the index for this query. Doing a SHOW CREATE TABLE User revealed that the opensocial_id column was of type varchar(30). Because the column was a varchar, mysql couldn't use an index on it to do a comparison against a numeric query. You can see the reason for
this:

mysql> select "0234" = 234\G
*************************** 1. row ***************************
"0234" = 234: 1

mysql> select "234.0" = 234\G
*************************** 1. row ***************************
"234.0" = 234: 1
1 row in set (0.00 sec)

For any given numeric value in the query, there are several varchar representations that are "equal" to it (shown as value 1 above). As such, there is no way for mysql to do an index lookup on a varchar column to satisfy an equality condition against a numeric constant.

One easy solution would be to change the varchar(30) to a numeric type. Unfortunately, it turns out that opensocial user identifiers are not simple 64-bit numeric values -- depending on the opensocial platform provider they are sometimes zero-padded, sometimes not, etc. So, using a bigint unsigned was not an option.

What was the solution, then? We simply added quotes to the query:

SELECT `User`.`id` from `users` as `User` WHERE `opensocial_id` = '23424234' LIMIT 1;


This made the equality comparison between two varchars, and the index was used. The query now runs consistently in less than a few milliseconds.

In summary, EXPLAIN everything! Even the queries that look incredibly simple can be incredibly slow if they're not using the index you think they are.

Wednesday, April 09, 2008

PHP Pop Quiz

What would you expect the following code to do?

$foo = 0;
if ($foo == "DB") {
die("hello world");
}
die("goodbye world");
?>


I think most rational people would expect the program to exit with the message "goodbye world."

It turns out that PHP is not so rational. Because $foo is a numeric value, it's compared using a numeric comparator with "DB". This typecast coerces "DB" into an integer value 0, so the equality matches.

The solution, of course, is to use the odd === operator which is unique to PHP. This means something along the lines of "I really mean it. No, seriously!" and disables the coercion of the dissimilar types.

The problem with that method, though, is that "0" === 0 evaluates to false.

Is there a reasonable solution to this?

Wednesday, March 05, 2008

Optimizing JPEGs with jpegoptim

Over the weekend I deployed a new plugin I wrote for Perlbal that tracks bandwidth usage of certain features on Amie Street. The plugin is only running on 25% of our traffic for now, but after a couple of days I had collected plenty of data to start analyzing it this morning.

The first thing I noticed is that a single one of our images was accounting for 10x as much bandwidth use as the next most transferred image on the site. I loaded it up to find that it was only a 128x128 album thumbnail, yet was a 560KB file. I downloaded it and tried to figure out why it was so big, but without much luck. I soon figured out that I could use convert -scale 1x1 to scale it to a single pixel and it still took over 500KB.

After a bit of googling I came upon jpegoptim, a utility that performs lossless compression on JPEGs by reconstructing the huffman encoding without changing the image contents. Simply running jpegoptim image.jpg I cut the image size down to 28KB and reuploaded it to production.

After scanning the top 1000 largest thumbnail images on the site, I found that the majority of them could be decreased by a good 50% or more by simply using this tool. So, if you've got a site that has a lot of JPEGs (particularly user-uploaded content that's automatically scaled using ImageMagick) give jpegoptim a try. The worst that happens is that it will report that the file is already optimal; the best case is a 95% reduction in load time for users with slow connections.