Ask HN: What is the best MongoDB AWS configuration for maximum performance? | Hacker News Reader

Ask HN: What is the best MongoDB AWS configuration for maximum performance?

4 pointswslh9 comments

We are struggling with performance issues in MongoDB on AWS using Node.js and the Mongoose module. Has anyone struggled with similar issues and know how to improve this?

We constantly perform inserts to a collection with 13 indices and 16 basic types fields. On a Dell notebook (XPS 9343 i7 2015 8gb with SSD) running Microsoft Windows it is 10x faster than in AWS running Linux. I don't think the OS is the main issue.

We have tried to improve the performance of the AWS instance using SSDs (io1 volume type) and a m4.2xlarge (8 vcpu and 32 GiB of memory) but we are not seeing significant improvements when scaling the VM or changing the storage type.

View on HN

> We constantly perform inserts to a collection with 13 indices and 16 basic types fields. On a Dell notebook (XPS 9343 i7 2015 8gb with SSD) running Microsoft Windows it is 10x faster than in AWS running Linux. I don't think the OS is the main issue.

Does your Dell machine has exactly the same data files as the aws instance ? As the size grows index grows as well. Also you have not mentioned how many mongo instances in the cluster. One easy way is, you could separate the index and data directories into separate volumes, think this is only possible in wired tiger though.

The date is exactly the same since it is retrieved from an internet db.

What's the size of the data directory on AWS and Dell workstation ?

We don't have MongoDB in production anymore BUT when we did...

We had 27 instances (9 shards) 3 routers and 3 configs

Main instances ran on c*.4xl machines with 5,000-10,000iops disks. SSD wasn't nearly enough for us.

Is it possible the data schema is poor? Honestly I may be misunderstanding your description, but 1 collection with 13 indices and 16 basic type fields seems out of balance to me. And misplaced, misused indices can be a huge issue for any database. You may not see the issue locally on a laptop because of the lack of contention based on the small number of concurrent requests.

No, the laptop and AWS instance are inserting exactly the same data.

The data may be the same but is the contention? In other words, if you have 100 clients accessing and writing in AWS but only 1 on the laptop it isn't realistic and the schema may still be at issue.

How large is the dataset too?

Honestly, I know people like to complain about Mongo a lot of times, but frankly I have had really large and small datasets use it and when setup properly with a well designed schema it never was the bottle neck in terms of performance etc. But on a couple of occasions we chose to move away from Mongo because the trade off in schema and machine config to make it work was more of a problem for us.

Maybe your local Mongo is running wiredTiger and the Amazon is running the old storage engine?

It seemed to me that there is some bottleneck for mongo on aws with Ubuntu. Hardware did not matter, mongo was slow for whatever reason. Why don't you run some perf tools?

Reply on news.ycombinator.com