Scaling Twitter: Making Twitter 10000 Percent Faster | High S...
Popularity Report
![]() |
|||
![]() |
|||
![]() |
|||
![]() |
|||
![]() |
|||
![]() |
URL Tag Cloud
Bookmark History
Saved by 27 people (5 private), first by anonymouse user on 2007-09-14
- Mrklein on 2008-07-21 - Tags architecture , development , web
- Wibblefarmer on 2008-06-29 - Tags todo , reading , scalability , rails , ruby , architecture , twitter
- Progprog on 2008-06-24 - Tags scalability , architecture , internals
- Harjeet on 2008-06-10 - Tags scalability , architecture , ruby , rails , frameworks , grid
- Arrix_ on 2008-03-03 - Tags performance , scalability , scaling , twitter
Public Sticky notes
Twitter started as a side project and blew up fast, going from 0 to millions of page views within a few terrifying months. Early design decisions that worked well in the small melted under the crush of new users chirping tweets to all their friends. Web darling Ruby on Rails was fingered early for the scaling problems, but Blaine Cook, Twitter's lead architect, held Ruby blameless:
For us, it’s really about scaling horizontally - to that end, Rails and Ruby haven’t been stumbling blocks, compared to any other language or framework. The performance boosts associated with a “faster” language would give us a 10-20% improvement, but thanks to architectural changes that Ruby and Rails happily accommodated, Twitter is 10000% faster than it was in January.
If Ruby on Rails wasn't to blame, how did Twitter learn to scale ever higher and higher?
Update: added slides Small Talk on Getting Big. Scaling a Rails App & all that Jazz
Highlighted by missiontobe
Highlighted by vincent
Highlighted by fulvius
Highlighted by inouemak
Highlighted by inouemak
Highlighted by inouemak
Highlighted by inouemak
- For example, if getting a count is slow, you can memoize the count into memcache in a millisecond.
- Getting your friends status is complicated. There are security and other issues. So rather than doing a query, a friend's status is updated in cache instead. It never touches the database. This gives a predictable response time frame (upper bound 20 msecs).
- ActiveRecord objects are huge so that's why they aren't cached. So they want to store critical attributes in a hash and lazy load the other attributes on access.
Highlighted by wade
Highlighted by jerobins
Highlighted by wade
- Distributed queues were made to survive system crashes by writing them to disk. Other big websites take this simple approach as well.
Highlighted by wade
- A lot of down time because people crawl the site and add everyone as friends. 9000 friends in 24 hours. It would take down the site.
Highlighted by wade
Highlighted by wade
- A lot of down time because people crawl the site and add everyone as friends. 9000 friends in 24 hours. It would take down the site.
- Build tools to detect these problems so you can pinpoint when and where they are happening.
- Be ruthless. Delete them as users.
Highlighted by fulvius
Highlighted by wade
Highlighted by wade
Highlighted by fulvius
Highlighted by fulvius


Public Comment
on 2007-09-17 by annabay21