fendale

What I don't understand is why Facebook feels slow each time I logon with these 10K servers - its often one of the slowest sites I visit!

If they are intending to have 60K servers, they better keep a slug of that cash around to pay the electric bill!

Yea, 10,000 servers is a serious number - is that actually an accurate figure?

The fear factor is certainly one thing holding me back. That any not having an idea that is good enough.

I have plenty of ideas all the time, but I don't think any are good enough to put all that work into. I have also found it is tough to think of ideas outside the field of IT - thats where there are all these potential problems to be solved, but as I have never worked there so I don't know about them!

Well said - I remember reading somewhere before that the database was mentioned (as usual) as the problem area.

Someone else in the comments mentioned 400K messages a day, plus personal messages, say thats even another 400K.

I have zero experience working on a high traffic web application, but I work on a serious database application were we can push somewhere in the region of 1M transactions in a 6-8 hour window. Each of these transactions comes in the form of an XML string and results in as many as 20 plus selects and maybe 10 - 20 inserts.

Our application is build on top of Oracle and implemented in PLSQL - not too sexy, but it seems to get the job done.

The point I am trying to make is that with a bit of a caching layer it takes some serious throughput to reach the limits of database scalability (certainly with Oracle, where is where my experience lies).

I am pretty certain it was mentioned in a blog post recently (I cannot find it now) that Ruby on Rails is only used for a small part of what Twitter is doing - ie the front end, and there is a lot of non Rails or Ruby code in the background.

The article mentions the cost of gas reaching $3.60 a gallon - here in the UK, based on 3.78 litres per gallon, the cost is about £4.35 - thats over twice the price! I always knew gas was cheaper in the USA, but didn't realise how much.

I am lucky enough to commute to work in 10 minutes on a bike, but the cost of fuel here is starting to get a bit ridiculous bearing in mind 70% or so of it is taxes.

Sounds like NBC are creating a Meatball Sundae (to steal Seth Godin's term) - that is what his latest book was all about.

This is something I have been struggling with too. I would love to make a useful web app that businesses would pay for, but all my ideas are something to do with software development, as that is all I know ...

I do try to speak to people in other industries from time to time to get ideas, but have got none so far :(

All we need is a 'social site' that non IT types can post their frustrations with the tools of their trade on to give us ideas - only problem is how we get non IT types to use it :-)

Not so much a distribution problem, but a platform problem - I cannot get Ruby to compile on an Itanium HP-UX box - even the HPUX ports site says it won't work :-(

If anyone has got it to work please let me know!

Wall street types are used to working on serious hardware though (I am not a wall street person, but work for a fortune 500 type place).

One of our Oracle databases sits on a 24 core itanium box

That sort of hardware + Oracle can munch through some serious work - perhaps that is the database solution, but I believe it doesn't come cheap!

For me, Facebook often seems very sluggish and slow - I have a pretty useful 4Meg connection and others sites are fine - maybe 10,000 servers is part of the problem - it seems like a ridiculously high number. I wonder how many transactions/pasge views per second they get ...

You may well be correct - perhaps the distributed tools are easier - I need to get one of them installed so I can play around with it to figure it out.

That said, if people don't get version control at all (yip, there are plenty of them) then its probably going to be harder to convince them of the merits of both version control and a distributed model at the same time.

I have certainly learned in the last few month that changing the direction of a large team is no easy task, and convincing people to use any tool that is even slightly off the beaten track is going to make that task even harder.

Its sad - I wish there were more curious, keen to learn people when I work!

I have just got a rather large team ( > 100 people) using Subversion - from my experience, I noticed two things.

Firstly, most developers I have encountered in big corporations (and India) don't even give version control a thought - I thought it was programming 101, but I was shocked about the resistance I got when trying to persuade the team to use anything.

Second, the people I was trying to get to use version control could barely grasp the basics - branching, merging, frequent checkin's, updating your working copy - alien concepts - and that was using Subversion. I cannot even imagine trying to convince them to use a distributed model!

So my point is - distributed version control is probably great (I have never used it, but I have read a bit) for the clued in 20%, who work with others from the 20%, but if your team involves people mostly in the 80% don't even think about it - Subversion will have a market there for a long time to come ...

I had the same problem, but that url isn't working for me - guess I will have to wait until I get home to read it!!

Yea that would be correct I reckon - the more adventurous you try to be, the bigger the reward, but also the less chance of getting any reward.

"Also, will people be more employable after quitting their jobs? It depends what employers are looking for. Some of them want "stable" employees who won't go leaving soon again, when the next best opportunity shows up."

Yea I guess that is true. It gives you a lot of things to talk about in an interview though, and you will no doubt have polished your dev skills nicely.

I think their model carries a lot of weight, but its not the only way.

Besides, show me 1 successful business and I will be able to show you 9 that failed (if the 1 in 10 success statistic is correct) - most businesses will fail, not matter which way you do it.

And then they will go out and get a new job and probably be more employable as a result of their experiences ...

Clearly a few will make millions, some will make enough and most will go broke, but that's the way business is.

Now, to some how figure out a product that people want and get the courage to quit my job ... not easy!

Has anyone figured out how much less memory mod_rails uses than say Mongrel? I haven't gotten round to experimenting with it myself yet.

I work for a big company where they believe in the cog-in-a-machine idea so much they actually implemented a process they coined 'the factory process' with the emphasis on being able to swap in team of developers at the drop of a hat.

Noone had read the mythical man month, or Spolsky's 'Hitting the highlights' (or named something similar to that).

I can testify first hand that treating software engineer like cogs is a road to slippage, bugs and all the good engineers leaving!

Sigh - I am with Virgin and have been generally happy for years now. I don't like the sounds of this one bit.

Its getting bad enough in the evenings, as if you download more than a few 100MB they throttle your connection down to 1MB from 10MB for 4 hours.

And then there is the business with Phorm. Its just downright sleezey - us tech types will be sure to opt out, but the 99% will have no idea what is happening.

This is great news good to see it!

What I have been wondering is what happens to the memory footprint? It says somewhere on the site it is reduced by 33% - is that a typical reduction?

Also, if this is like mod_perl, deployment wasn't really as simple as it seemed. The general advice on mod_perl was to proxy requests from your front end 'static' webserver to the mod_perl one, because each apache process had to effectively run a Perl interpreter, giving them a big memory footprint. You don't want to use your expensive mod_perl/ruby processes to serve images etc.

I also think there are other advantages to running expensive processes behind a proxy (I may be wrong on this one), but I think that slow clients tend to tied up the front end process as it streams the data to them.

If you have appServer -> proxy -> slow client, the app server sends the data to the proxy at LAN speed, freeing it to serve another request, and the proxy ties up a process streaming the data to the client.

I am not trying to criticize, I am interested in whether they solved these problems (if they even are problems anymore) and how.

Yea, I had a J1 visa once and spent a summer working in a beach club on Cape Code - good times!

They expect evidence that you have at least one more year of university left to get it (they don't want people not going back home and staying in the US illegally).

I believe it only lets you work for 4 months however, with 30 days 'travel' allowed at the end, so it may be no good for a startup.

First its bad for you, then its good - I wish they would make up their mind!

I actually don't like the taste of coffee, which makes me a bit of a weirdo apparently, but I believe you can get your caffeine hit from tea instead (or red bull if your really brave!) - but I tend to avoid those too. I would hate to think that I need a hit of caffeine through the day just to keep me awake.

Feel sleepy in the mornings? Goto the gym before work.

Feel sleepy in the afternoon - go for a walk or get away from your desk for a while. I always find those two things wake me up better than anything else.

Yes - and second that with a usable white board interface - Livemeeting sucks so badly!

For sure - thats one of those hard problems that has not be well solved yet ...

I am very interested to see the memory usage. With mod_perl, you pre-loaded a lot of your modules in the httpd.conf file, and Apache loads them once and shares them across all processes saving memory. With Rails that could give large memory savings, and memory is the major pain point of getting more than one or two Rails apps up and running on the same server as I understand it.

I remember that article - I quickly reassured my self by writing fizz-buzz in Perl, Ruby and PLSQL (my three most used languages) - no great achievement, but I am better than the majority of comp-sci grads apparently!

Are things really getting that bad here? I have been lurking and occasionally commenting here for quite a while and I haven't really noticed much change recently - the only thing I did notice was a slight slowdown in posts over the Easter period, which I might have expected anyway (but it may have been because I wasn't at my day job and spent more time here than normal!)

Not being from the USA, I am not sure how this works. In the UK, generally the person selling a house finds an agent to sell it for them - they normally charge a fee based on the sale price that the seller has to pay.

The buy doesn't require an agent, or pay anything (apart from the price agreed for the house).

After the sale is agreed, the solicitors get involved, and both the buyer and seller will have to pay his own solicitor.

In the USA do you need to pay a fee to an agent to buy a house too? Do they perform the duties of a solicitor or provide any useful function at all for this fee?