In various presentations throughout 2007, the Twitter team has made reference to a pure Ruby message queue server called Starling, written by Twitter's own engineer Blaine Cook. Starling is a light-weight persistent queue server that speaks the MemCache protocol. It was built to drive Twitter's backend, and is in production across Twitter's cluster.
Starling is at the core of what we do at Twitter; it moves small messages around to daemons that work on jobs like processing updates, delivering messages, archiving user accounts, and so forth. An asynchronous messaging solution is becoming a necessity for big web applications, and Starling fits the particular needs we have at Twitter. It's fast, it's stable, it speaks the memcache protocol so it doesn't need a new client library, and it's disk-backed for persistence. When other parts of the Twitter site go down, Starling stays up. It's a champ, and we love it.
Until now, Starling has lived a sheltered life in the Twitter code base. Starling is now open source and freely available for anyone to use, modify, and improve.
To give Starling a try today, just sudo gem install starling on your favorite Ruby development box.
Here is the Starling page at RubyForge.
No comments:
Post a Comment