Ask HN: What technologies do you use in your startup? | Hacker News Reader

Ask HN: What technologies do you use in your startup?

22 pointsrsa22 comments

I'm interested in all startups using linux and windows environments. What do you use to scale to app to millions of hits per day ?

View on HN

Scaling is more about sound architectural decisions than it is about stack.

(And it is more about "not a problem" than it is about architecture, because a gigantic proportion of all startups will never have any scaling problem to speak of, and the majority of the remainder will be able to solve theirs in fairly boring fashions.)

Almost all startups would be better served by playing around in their technology of choice looking for something that could possibly cause a scaling problem rather than planning for scaling problems.

Engineers sometimes remind me of the two guys that used to live next to me that spent every day tinkering with the engine of their Camaro out in the garage. I'm sure it was pretty tricked out but it also spent 9/10 of its life up on blocks instead of actually driving somewhere interesting.

Precisely why I stopped caring and focused on forward movement.

Can you name a single startup that failed because of their inability to scale?

Development speed is way more important than scalability. Like Paul Graham says, the vast majority of startups fail because they make a product no one wants or because they never launch a product in the first place. Faster software development means you figure out if your product is a dud faster, and you're less likely to run out of morale before you've got something substantial.

I've been working on my current (unreleased) project with Django and jQuery on Linux for about a month. Here's all of the stuff I've made use of from the Django, jQuery, and Python communities:

http://code.google.com/p/django-command-extensions/

http://south.aeracode.org/

http://github.com/dcramer/django-ratings

http://github.com/pydanny/django-uni-form

http://github.com/robhudson/django-debug-toolbar

http://github.com/alex/django-ajax-validation

http://fabfile.org/

http://orkans-tmp.22web.net/star_rating/

http://jquery.malsup.com/form/

http://plugins.jquery.com/project/ScrollTo

http://jashkenas.github.com/coffee-script/

http://code.google.com/p/modwsgi/

There's no way I could ever use a niche framework after having drunk this reusable code kool-aid.

Even if Django weren't scalable, if I really was on the road to being the next Facebook or Youtube, there'd be plenty of time and money to rewrite in something that could scale.

Can you name a single startup that failed because of their inability to scale?

yeah - friendster.

I remember the early days when you couldn't log in to Friendster for hours because the servers were overloaded!

Did they attribute their performance problems to tech stack or architecture? Or did they ever say?

Friendster is actually in decent shape today. So inability to scale didn't kill them--it just dealt them a very substantial blow.

Twitter would be an example of a startup that experienced severe scaling issues and still ended up on top.

Know any others?

I wonder, could the following be true: any startup that faces the problem of unscalable implementation will by then have the necessary resources to resolve the scaling problem? Because, by the time you run into scalability problems, you'll probably have enough investor|customer money to invest in re-architecturing your code?

That implies that startups charge money for their product/service. More often than not, sites that need to scale for millions of users are "free to use".

He also mentioned investor money. Investors tend to invest in sites that have tons of users even if the site is unprofitable at the time of investment (see youtube, digg, facebook).

1. caching

2. prefer static content over dynamic, as much as possible

3. lots of machines, with the requests distributed across all of them

these 3 strategies alone will buy lots of scale

That's the magic formula we use at work (200+ sites, many with 1 million+ unique visitors per month). I apply the same techniques to my own projects, even though it's unnecessary at this point.

Our sites run on a standard LAMP stack (load balanced across between 2 and 25 machines using an F5), memcached for object and query caching, caching at the application level, varnish and nginx for file caching, and limelight CDN for serving images, css, and javascript.

Rails, node.js, redis, resque, mysql, nginx, unicorn, monit, solr, ubuntu.

I don't think that the technology stack matters that much, though.

while I agree that some of the strongest techniques for scaling are available on many platforms/stacks, you will generally have the easiest time of it on a Linux stack rather than say Windows. Only at the thousand-foot perspective does the stack not matter. But on the ground, where the devil is in the details, the stack matters quite a bit when it comes to what toys are available to you, what resources, what options.

I would say your familiarity with the stack you're using is what matters, both Linux and Microsoft platforms have been shown to handle massive loads with enough massaging.

I would also say it's a massive and on-going trial and error process to see what actually works, the biggest sites are still learning and falling over.

Stack Overflow claims that by running a Windows stack they save money compared to a LAMP stack due to its better performance (MVC2,SQL2008) outweighing the initial cost. I think they talked about this in their Google Tech Talk (Joel Spolsky).

java (jee, spring, struts), hadoop, hbase, nginx, fedora, solr, activemq, ehcache/terracotta and a bunch of other OSS.

We use Django, Linux (Ubuntu for both servers and developer machies), PostGIS database, memcache, Apache, etc. and of course Eclipse, Trac, etc. etc.

Haven't reached 1M hits per day yet, but I'm about to experience a bit of preemptive load testing to find out where the initial problem spots may be.

Rails, distributed erlang with a BERTRPC endpoint, memcache, redis, thin/sinatra/eventmachine. The key is the less work you have to do in the request response cycle and the more you can offload to better suited environments/servers, the easier time you will have adapting to new challenges.

Rails on Heroku (Meaning, Postgresql database)

grails: java, groovy, ajax, lucene, sitemesh, gorm, etc.

Reply on news.ycombinator.com