Obvious Ideas

We started using MixPanel to get realtime statistics on user engagement in our application. We’ve been using MixPanel before for funnel tracking, but you can’t really appreciate their service until you start using their event tracking. It is truly realtime (as advertised) – the second the user performs an action on your application, you see it on your dashboard.

Besides the realtime reporting what I really like about MixPanel is the fact that they allow easily reporting from your backend and not only from frontend/Javascript (as opposed to Google Analytics).

Within their documentation they have code samples in many languages, including Python. But their Python sample requires ability to start new processes on the server you’re running – something not possible on App Engine. Therefore I changed it to use URLFetch in RPC mode (so that calls to Mixpanel won’t block the call):

It could be re-factored more, like allowing setting the project token outside the function call, but that I will leave for you to do (and if you do, please share with us)

Arik

I recently started using deferred tasks in my App Engine application. I’m using them mainly for two things:

1. splitting user requests into two pieces – one that being done immediately and one that being done in the background, to speed up the response time.
2. mapping (iterating over) all my entities to calculate various statistics.

I won’t provide here details on how to use deferred tasks, because this is described in detail in a great article by Nick Johnson, which I highly recommend reading. One thing I do want to share from my experience, is my implementation of the Mapper base class:

This implementation is taken from Nick’s article mentioned above, but I made some changes to it. The first change is giving subclasses the ability to set the property to order the entities by. I needed this, because I found out that when your entities have custom key names ordering by key breaks for some reason.

The other change is re-factoring the way the next batch starts. It seems to me that in the sample provided in the article there’s an error in the indentation, resulting in the code not doing batching properly and can even break in some situations. In my version every task run maps only one batch (or less). This is sub optimal in terms of # of tasks it takes to map all your entities, but as long as you don’t have really a lot of entities this is not such an issue.

I really would love to hear others’ comments no my changes and about how you use deferred tasks or map your entities.

Arik

I’ve been doing quite a lot of Facebook development in the last few months. At first I was using my own account to do most of the testings for the apps I was working on. I always have (at least) two versions of the app – production and development. The development version is marked as sandboxed, so any action I do in it (including posting to the newsfeed) isn’t shown to my friends.

At first I felt that this is enough, and didn’t feel the need for a test account. But after some time I realized that while developing, anytime I get to my profile or newsfeed, I get distracted by my friends’ posts. So I started using another account to do the testings. This way, whenever I get to the profile or newsfeed I practically see nothing there that can distract me from what I’m working on.

Another benefit of using a test account is that whenever you test something on the production (un-sandboxed) version of your app and need publish a lot to your newsfeed, you don’t spam your friends (or reveal upcoming features).

Of course, we can rely on our self discipline to avoid this distractions, but who are we kidding?

Arik

Recently for Topify I needed to shorten some long urls (our invite links) and my obvious choice of shortener was bit.ly. I tried looking for PHP wrapper for their API, but only found this project which was way more than I needed. So I’ve decided to write my own simple client using the Zend Framework HTTP Client. Here’s the result:

<?php
function bitly_shorten ($url) {
    $client = new Zend_Http_Client('http://api.bit.ly/shorten');
    $client->setParameterGet(array(
        'version' => '2.0.1',
        'longUrl' => $url,
        'login' => 'arikfr',
        'apiKey' => 'R_03feadf27c1c7c1a1ac3c3e7d6d41a27'
    ));    

    $response = $client->request();
    if ($response->isSuccessful() ) {
        $phpNative = Zend_Json::decode($response->getBody());
        if ($phpNative['errorCode']==0) {
            return $phpNative['results'][$url]['shortUrl'];
        }
    }
    return "";
}
?>

As I said – it’s simple. There’s a lot more to do with it, but it serves the purpose I needed it for and I think it might be useful for others who only want to shorten some urls. If you need more than that, you might want to take a look at the above mentioned project.

Arik

The Unread Items counter in Google Reader is putting you under pressure? Can’t sleep because of it? Well, you don’t have to see it any more. I’ve hacked a simple Greasemonkey script that hides the unread items counter in Google Reader. There’re two flavours of the script: one that hides only the all items counter and other one that hides all the counters.

To those of you who don’t know what Greasemonkey is:

Greasemonkey is a Mozilla Firefox extension that allows users to install scripts that make on-the-fly changes to most HTML-based web pages. [..]

Greasemonkey can be used for adding new functionality to web pages (for example, embedding price comparison in Amazon.com web pages), fixing rendering bugs, combining data from multiple webpages, and numerous other purposes.

(From Wikipedia entry on Greasemonkey)

You can find a lot more scripts on userscripts.org . Beware that some scripts become unstable/not working due to constant updates of the web sites (most of Gmail scripts break every version update).

I’ve also created three bookmarklets that allow you to unhide and hide the counters, but had trouble to embed them in the post. So if anyone interested in them, just leave here a comment.

Comments are mostly welcomed.

Enjoy.

Arik

Today Michael Arrington announced that Facebook lifted their 5000 friends limit. As a tech blogger, I bring you the “behind the scenes” view: the code change that was needed to be done to increase this limit:

For the non techies:

• line 3: commented out – the previous friends limit defined
• line 4 – the new friends limit

Shocking.

Arik

Jeff Atwood writes about WordPress performance. Quite astonishing – it appears to be that if your blog is medium sized (or bigger) you must install WP-Cache in order it will survive traffic peaks and doesn’t kill your server.

When I read it, my first though was: “too bad they though that fancy control panel is more important the performance.”, but then I though of something else – maybe avoiding performance issues and dealing with other stuff before that, that’s more user targeted thinking of features: most Wordpress users are small bloggers (like me (-:) and therefore they need better UI experience than caching, while the big fish can handle the extra hassle of installing WP-Cache or paying for a dedicated server.

Think of that next time you decide which feature to implement first. Remmeber – listen to your users, not your engineers

Arik

Amazon Elastic Compute Cloud, also known as “EC2″, allows scalable deployment of applications.^[1] Current users are able to create, launch and terminate server instances on demand, hence the term “elastic”.(Wikipedia)

I believe that the Amazon Web Services (and especially EC2) are one of the most influencing technologies that shape the future of the web. While for me it’s pretty obvious what is the benefits of such a service, it sometimes difficult to explain to others. Yesterday, I stumbled over the most convincing example of them all.

Enter Animoto. Animoto is a cool web application that you feed it with your photos and it creates you a video of it (see example above). The geeks among you readers, know that rendering video can be CPU consuming. So how they do that? Using Amazon EC2, of course.

During last week Animoto userbase grew from 25,000 users on Monday to 250,000 users on Thursday (!). But the graph the you see above isn’t the graph of their user count, it’s the graph of the EC2 instances they used to handle that traffic. They started the week with 50 EC2 instances, grew to 100, 900 and eventually 3400 instances of EC2. Later on when the demand lowered, you can see the the count lowered to something like 1200 EC2 instances.

Now imagine if they have been using regular servers. Just thinking of the meaning of managing all this amount of hardware, of storing it, DRP plans, etc.. gives me the creeps!

This example shows perfectly the benefits of EC2 in particular and cloud computing in general:

• Pay as you go – you need 50 servers today? No problem. You need 3000 tomorrow? That’s no problem either. And the day after tomorrow you want only 1000? Just do what ever you want. No need to plan ahead of how much computing power you will need, just build the infrastructure that can grow.
• No system management overhead – I guess that most of the EC2 instances used by Animoto are servers that do the video rendering. So when the usage grows that just duplicate the same server image they already have. When one of them stuck or fails, they just shut it down and start a new one. No need to have people maintaining huge server sites, buying hardware and stuff. Simple example for such an architecture using EC2 is explained in this article.
• Simple! Everything is API based, and you’re not the first one to use it, so there plenty of implementions of scalable architectures over EC2 .
• DRP? In a few mouse clicks (or automatically using a watchdog script that monitors their health dashboard) you can turn on a new instance at the European data center, and forward all you traffic from the US one there. More DRP than that?

I’m a total believer in cloud computing. I was really happy to see that Google entered the game, and I hope that other major players will join too (Microsoft, what are you waiting for?).

Arik

For those of you who don’t know who Guido is, he’s the creator of Python and now works for Google. Recently he moved to the Google Apps Engine team. If you want a quick introduction to Google Apps Engine, just read this blog post and then watch the following two videos: (the first one shows creation of a sample application and the other one is Guido’s recap of Google Apps Engine):

Currently the Google Apps Engine is in a preview beta, and to the moment of writing this post, there’re no more inventions left. If you still want to experience the Google Apps Engine, you can download the SDK. Currently they support only Python, but they plan to support more languages as the product will mature. It really makes me curious to see what will be next language that they will support.

One important thing to understand is that Apps Engine isn’t Amazon Web Services. They both offer virtualization services, but in a different style:

Will it replace Amazon?  It sure doesn’t look like it from where I sit.  In fact, I don’t see this as much of a competitor to Amazon Web Services.  There’s some overlap in some small area (hosted web apps on EC2), but I doubt that’s the bulk of Amazon’s business.  As I said, we’ll likely end up using both (and other providers as they come along, too).

(Don MacAskill on Google Apps Engine)

In my opinion, in current stage, Google Apps Engine is aimed at and more suitable for the most standard web applications. What’s not a standard web application? YouTube for instance – because it requires not only serving loads of pages (data), but also some computation (like converting video formats).

One things is sure – now that there’s competition to AWS, it will make things much more interesting.

Arik