bmurphy

Just because you have 16gb of memory doesn't mean you won't ever swap. Firefox is not the only application running on a computer. My working set is routinely over 7gb and I have 8gb on my laptop. A goldfish will grow to full its bowl. If I had 16gb, I know I could use it. Every little bit counts.

I used MSSQL up to MSSQL 2005 and MySQL up until about the same time. Around then I switched over to PostgreSQL completely and have not had a reason to go back. My biggest complaints had to do with simple replication and that was mostly solved in Postgres 9.0. Postgres is a damn fine database.

I agree with the parent. Something very simplistic in json or yaml would be preferred.

I can't play it. I've played too many FPS and Flight Simulators in my time. The controls need to be inverted so i can fly it like a real plane. Too bad.

Absolutely. You should always be spreading your data across multiple availability zones and where feasible across multiple data centers and S3 is a great place to store your wal logs. We do the same thing.

EBS is a shared networked SAN. The performance characteristics of it are not that great and even worse, highly variable. The last thing you want to be running your database on is a system where the performance characteristics vary greatly throughout the day and you have no control over it.

The ephemeral drives are drives directly attached to the server and to the best of my knowledge are not a shared resource. Their performance characteristics are highly consistent, but if your server goes down all data on those drivers are lost.

EBS sounds nice in theory, but by going to EBS RAID you throw away most of its benefits (such as snapshotting) and take on it's worst aspects.

Very nice, thank you. Mostly all of this is good advice and correlates nicely with my experience.

One thing though:

"EBS volumes and Software RAID is best but scary on AWS"

I've managed an EBS RAID10 database for a few years now. I wouldn't touch this with a 10 foot pole.

Do yourself a favor, set up an m1.xlarge (or bigger) instance, put the ephemeral drives in a RAID0 and mirror across multiple machines using hot-standby, slony, londiste, or some other tool. You'll be much happier, your system will perform much better, and you'll have a failover strategy in place.

Agreed.

I personally go through my RSS feed every few months and re-evaluate my feeds. If I have feeds that have low signal to noise ratios, or simply publishes articles that I never read, I remove them.

Also, I segregated high volume feeds (such as Flickr picture feeds and Gizmodo/NYTimes/BBC etc.) into a separate account. I have two accounts, one (low volume) where I try to take a closer more thorough look at everything, and another (high volume) where I simply do a quick scan then mark all as read.

Don't blame RSS for your inability to manage it.

Nope, he does not mean open a port. Ports is the name of the packaging system FreeBSD uses, and a port is the equivalent (more or less) of an RPM or deb source package.

http://www.freebsd.org/ports/

Nice. Much better.

As the Father of a 19 month old I can only say this is a complete rip-off.

Craig's list, friends and family, birthdays, holidays, garage sales, second hand stores, 50% off coupons, the list goes on and on.

$16/mo for two outfits? Really? Maybe $1/mo/outfit or something like that and I'll bite. Right now, one years subscription would be more than her entire wardrobe and we're pretty much done until age 4.

Argh. Same crap as with all the population predictions. Everybody always assumes the graph is exponential. It is not. It always levels off. The real trick is predicting where/when that will happen.

You could always replace it with an EBS volume.

I don't have solid numbers, just some experience using this. Ephemeral drives outright fail more often than EBS volumes, however, EBS volumes suffer performance degradation significantly more often than ephemeral drives. EBS volume performance is HIGHLY variable, at all times of day, no matter what load you throw at it. Ephemeral drives are very consistent most of the time.

Both types of drives CAN and DO fail, so RAID-10, fail over, and replication are a must have.

My experience has been that EBS handles concurrent loads better, ephemeral drives handle non-concurrent loads better.

Transferring 100gb+ of data on EBS (even with an 8x RAID) is a nightmare. Ephemeral drives, however, it's fairly fast.

Throw 100+ database connections at a few ephemeral drives (even in a RAID) and watch your web site slow to a crawl.

I tuned the hell out of our big postgresql instance a year ago, but I'll be damned if I can remember the rational for every change. I have a list of all the changes from default, but I've long since forgotten/lost the reason for making them.

That being said, we get more bang for our buck by spreading our data across many small databases that don't need much tuning beyond upping the memory defaults. The EC2 cloud isn't great for the uber-server, but it's halfway decent for many small servers.

Yes. Except, anybody who is doing RAID-0 over an EBS volume for perf reasons is ASKING for trouble.

You need to do RAID-10. EBS volumes CAN and DO fail.

Having been running a 200gb millions of transactions per day Postgres cluster on Amazon's EC2 cloud for two years now, I can attest to the fact that EBS performance and reliability SUCKS. It is our SINGLE biggest problem with EC2.

200gb really isn't all that big of a database. It shouldn't have to be this hard.

That song is terrible (and I love Devin Townsend, the singer).

Interesting, but I really don't think this belongs here.

Disk I/O performance that doesn't suck.

Let us add dev pay instances to an ELB.

More ram.

Elastic private ip addresses.

Change security groups of running instances.

I have a lot more, but those are my big ones.

You're right. But it's not happening now. I don't see it. Who are the alternatives everybody is flocking to? They don't exist.

Don't get me wrong. I'm hopeful for a new world of open protocols and you controlling your own data, but I just don't see it happening right now.

There are no Facebook alernatives. My friends and family are on Facebook. They are not on ANYTHING else. Until that changes, which I suspect it won't, nothing else will be viable.

I just ran the same test. Set up a 2 slave cluster, killed slave 2, then killed the master. Ran pg_last_xlog_receive_location on each slave, picked the one that was furthest along in the timeline, promoted it to be the new master and the other slave caught up as expected.

That's great. That's the first time I've been able to get this to work smoothly. I'm sure there are still ways to break it but this gives me hope.

Yes, that's exactly what I was referring to, you're getting a new timeline. You need to rebase the slaves and you have no real-time backup for a few hours. Sucks. You have to be very careful, and there are all sorts of ways you can break it. It's not something I'd want to rely on right now when under fire.

Close. Postgres has traditionally focused on letting the community provide replication solutions, but we've been screaming so long for one that "just works" that they're finally getting around to building it.

I wouldn't say it covers 95% of the use cases. In fact, I find it very limiting, but if all you need is to make sure your database is backed up it's a great step in the right direction.

That being said, the topology limitations suck. If you lose your master server, you are running in a degraded environment until you can rebuild ALL slaves from scratch. This really sucks if you have a very large database that takes a long time to rebuild.

Postgres 9.0 replication is a great first step, but it's only a step. There's still a lot of work to be done.

I was reading Heart of Darkness but took a break from it to read PostgreSQL 9.0 High Performance.

Seriously missing a lot of languages here. Where's Objective C? Visual Basic?