A table that should exist in all projects with a database
blog.cherouvim.com
blog.cherouvim.com
What happens when developers create branches and use time stamps? Developer A working on trunk might check in db updates at 4pm and 5pm. Developer B working on a branch checks db updates into his branch at 4:30pm and 5:30pm. When working on the branch his db updates don't affect the "trunk" db. As soon as he merges does his 4:30pm update get run on the "trunk" db, or is it skipped over because the 5pm db update already ran, and only the 5:30pm change gets run?
Where I come from we use int values for our db update script names. When I merge from a branch I renumber my db update script file names starting at the tip int value +1 of the trunk db update.
fyi - I'm not coming from a Ruby/Rails perspective, if that matters.
So if the two branches are separate there should be no issue - all my changes get run in order, irrespective of what the other developers have been doing.
And yes, manually renumbering is exactly what you'd have to do. Timestamps obviate that.
Theoretically, you can still have collisions (two migrations alter the same column, eg) or violated prereqs this way, so some communication is still required, but not to the degree before.
Developer A creates a migration numbered 201. Developer B creates a migration numbered 201. They push their code, pull, and get conflicts.
So instead, they use a timestamp based system. Developer A created his and got 20100604181252. Developer B got 20100604182530. Now, everyone is happy, and no one has to change their migrations.
Developer A creates migration 20100604181252. Developer B creates migration 20100604181253. Developer B doesn't notice Developer A's migration: there's no way to know from the timestamps that there might be a conflict. Everything breaks.
vs
Developer A creates migration 201. Developer B creates migration 201. Developer B gets a handy notification that something has happened in the meantime, and gets a chance to patch the migration if necessary.
However, the second scenario would happen every single day in a team.
I do concede it is rare, but it happened, and we only discovered the bug too late.
As you say, it would rely on notifications that there was the possibility of clashes, but unless you monitor every other developer's local feature branch, you would only know when the branches are merged into master.
It's not necessary that schema versions be that rigid. All you really need is that they be monotonically increasing; a timestamp in UTC is sufficient for those purposes, although technically you could run into a situation where a UUID is necessary.
Even with a single server, not relying on IDs being sequental is useful when you want to do a database migration without temporarily shutting down the site. It is much simpler to merge the two databases together.
You may want to re-think that statement.
(and I'm not trolling, I use and like rails, but the database abstraction is amongst the weakest of all popular frameworks)
But assuming that you are sincere, your reasoning is flawed. See, the fact that Rails' orm is in some way badly designed (your words - not mine) doesn't diminish this particular feature. The system could be overall crap, but still have one good thing about it. Besides, the schema-migration has very little to do with the ORM, except for the fact that they both interact with a database.
What you're doing, is called a straw man argument.
Well, this particular feature is a bandaid for an underlying misfeature (manual schema management).
Proposing this as good practice is somewhat akin to proposing "If you design a new operating system then make sure the installation of virus scanners is as easy as on Microsoft Windows".
You may call it a strawman, I'd call it "putting something into perspective".
Besides, the schema-migration has very little to do with the ORM, except for the fact that they both interact with a database.
That's a nice theory but in practice most rails users do use the builtin migration mechanism, for the lack of a better alternative (at least to my knowledge, feel free to correct me).
Consequently I don't think it's unfair to compare that mechanism to the equivalent in similar frameworks - and in that comparison rails loses, big time.
Could you expand on that? Maybe I'm missing something here.
The explicit mention of rails just stuck out to me (probably more than the author intended), as if rails had achieved something extra-ordinarily smart there. - While to me it's just an implementation detail of an otherwise rather inferior approach.
Modern schema migration tools compute the difference between the current and the desired state on-the-fly, and generate the migration commands accordingly. Explicit version numbers/timestamps/hashes are mostly used for documentation and transparency purposes there, not for the sake of conflict resolution.
These algorithms are naturally not flawless, but they work well for the proverbial 90% of cases. In such an environment the user doesn't normally have to futz with migration scripts or migration version-numbers at all.
Please list those widely used modern schema migration tools as they occur in other popular languages like PHP, Python, Perl, and so on.
In rails, users don't futz with migration version numbers. Those are handled automatically by the system. The user simply specifies a migration, e.g. "I want to add this column to this table." The migration is then automatically run when doing the next deployment. That doesn't work in 90% of the cases - it works in 100% of the cases.
Woe be unto the developer whose deployment process only works 90% of the time.
I already did in another comment but I'll repeat: django south, hibernate schemaupgrade, datamapper automigrate, postfacto (standalone python/postgres), sqlalchemy-migrate
e.g. "I want to add this column to this table."
The difference is that in Rails the user has to update the model and create a matching migration. In most other frameworks the second part is strictly optional and by default implicit.
Woe be unto the developer whose deployment process only works 90% of the time.
That's not what happens. In practice there's always a review step.
However, I've already been accused of strawmanning, so rather than taking this further offtopic let me just point you to http://south.aeracode.org/docs/tutorial/part1.html
Actually most of them (except datamapper and postfacto) have existed before rails was created.
And I'll argue strongly against "matter of taste" when it comes to error-prone routine procedures.
Some small teams (including some I work with) find it more clear to have 001,002 etc, and just have the convention to get in touch before any schema change, which is overall often a good idea anyway.
Danny you autocorrect on my Android
Mango would be a good nickname for Django + MongoDB
I know the developer - she is awesome!
while(true) {
string current_version = <sha1 of the dictionary table contents>
if (!upgrade_scripts.containsKey(current_version)) {
complain_loudly()
exit(1)
} else {
string script = upgrade_scripts[current_version]
if (script.empty())
break
else
execute(script)
}
}And resist the temptation of hacking the schema directly in your servers, it ends being a larger effort :)
Nowadays I'm primarly working with non-relational datastores where this seems to be much less of an issue. There we version the entities that we store instead of a schema and that is done entirely in source control as well.
Each change is stored as a script which have names like "<timestamp>_<brief description>.sql", and the timestamps serve as IDs as well in the "changelog" table in the DB. Creating the changelog table is always the first change.
The "migrate" command of the tool then has options for "status", "up" "down" etc., and features to create scripts to go from v1 -> v2. Another nice concept they provide to the mix is adding an "Undo" section to each script so that you can rollback changes and downgrade schemas just as easily as upgrading them.
We use a UDF called VersionNumber that returns an integer. When the continuous build processes a new database change script, that function gets modified to increment the value it returns. So you can always call VersionNumber() on any of our databases and know which version you're looking at. The builds actually use it to decide which scripts they need to run to get from where they are to where they need to be.
You already have all the commentary you need in source control and in the change scripts themselves. I don't see a reason to duplicate it in the database as well.
Rails is surprisingly anachronistic here, which probably stems from that awful conceptual separation between model and schema.
Most other platforms have semi-automatic schema evolution that usually works very well (datamapper automigrate, django south, hibernate SchemaUpdate, etc.).
(This is far easier when using SQLite, which has its own tradeoffs.)
http://news.ycombinator.com/item?id=1984609
as I don't have to google around to find out what south is (I haven't used Django yet)
A few years back, I built an installer in Ruby to apply 'plsql modules' to an Oracle database. This was a massive project with > 100 developers at this point and probably heading toward 1M lines of code.
My migrations table worked on modules, so there were a set of migrations per application area, but it was really just an extension of this idea (and I borrowed the idea from Rails too)!
Nathan also does a good job of applying submitted patches quickly.
DbDeploy http://dbdeploy.com/ DbMaintain http://www.dbmaintain.org/
Here is an older article... http://www.davedevelopment.co.uk/2008/04/14/how-to-simple-da...
Further, consider if your company had three deployments of your product. One for bleeding edge testing, one for staging/QA, and live. You'd want the schema version to be consistent across all three schemas.