http://news.ycombinator.com/item?id=130567
ended up hitting on a couple of things that ARE worth doing right away; things which are worth at least considering, and perhaps implementing, early.
I'd appreciate insights from the community about those "early" optimization which seem worthwhile.
If a software grows like a tree, then only some branches, no, some leaves will require optimisation. You never know from the start the route to those leaves because of the complexity of the software. Trying to optimise early is either like taking all possible routes but stopping after a short while on each. What I'm trying to say is optimisation is always a matter of _depth_, not of breadth. Finding the right direction and going all the way gives order of magnitudes better results then following best practices throughout the software.
1. Make sure every table has a user_id field
2. Make sure every record gets that field populated
3. Don't forget to index it
Watch out for queries that don't use the field as part of their 'where' clause.
if you know your loop might iterate up to thousands, then it is time to seek some better algorithms.
Also, MINIMIZE the amount of data that travels between client and server, either way, ever. If you don't now, you won't notice the hit in your test environment, but you will notice it later. The problem then may be a lot of restructuring and rewriting.
I think when your program, you should program thinking in a this way: What's the max usage groth I can realisticly predict of happening on the next year?
If in a year you have the good problem as too many people are siging up for your service, there is many ways to keep that groth and performance happening.
Concetrating on having millions of users, when you haven't written a line of code yet, is just an dellusion delaying your progress to even get any users.
I never said that.
I said to structure your data AS IF you had billions of records.
Are you going to normalize your data? How much? Are you going to index it? How? How are you going to structure your iterations, branches, and internal data structures? Put any thought into any of it?
With rare exceptions, what works well for billions of records also works well for hundreds of records. The converse, unfortunately, is painfully untrue. Think ahead and you won't have to worry about dealing with it.