Friday, May 2, 2008

Scaling out to 200million hits per day

Today I happened to have a discussion with one of the "core team" members of a startup in USA (don't ask me what that is up to -- I'm never gonna tell that).

Anyway.. the interesting part of the discussion was how to go about designing and implementing a system to support an average of 200 million hits per day. A big number? Well, if we translate it to lower base, it turns out to be lower than 2500 hits per second.

Of course, you need a cluster - web-server cluster, load-balancer, sticky-sessions (to be avoided completely), distributed database, machine configuration etc etc etc.

The real questions are:
  • Which programming language to choose?
  • Which database to go for?
  • Which file-system to use?
  • What about the caching?
  • What should the load-balancer be and how should it be configured?
I have got answers to almost all - based on whatever systems I have studies on the basis of case studies on the web, feedback from my friends who have worked in those "big" companies with such implementations.

Have a look at the discussion on Facebook here. And then I found some great for MySQL here. And finally, I linked PHP and MySQL with memcached.

Finally, I came down to a conclusion... let me take a nap. I will think over this, analyze this and do some experimentation... I hope to publish at least some of the useful findings.

No comments:

Post a Comment