Mongrel2 Sqlite3 Config System Working
sheddingbikes.com
sheddingbikes.com
With a config file you have to hunt around for the right stanza, type in the right exact change, restart the server, and pray it works. With SQL you can actually query the database, and run updates (then pray that works) which are much easier to get right and do.
With apache, I make an edit, check the syntax with apachectl configtest, then restart if the syntax checks out. The kinds of errors that get by the syntax check (bad regex in rewrite rule, typo in a host name etc) are just as likely in SQL. Text files allow me to use the same version control tools, text editors etc I use for the rest of development. Deployment is just a matter of copying a simple text file around etc.
Putting this all in SQL is a solution in search of a problem.
Also, I still stand by that getting a SQL update correct is much easier than editing an apache config file. Only The Spaghetti Monster knows how Apache configs actually work (and he works for Microsoft).
For instance, how would you represent something like this in SQL? http://topfunky.net/svn/shovel/nginx/conf/nginx.conf
Seriously though, SQL's been proven capable of doing all of this, and for the half-assed turing tarpit if-statements in these files you'll get something better. I'm aiming for Lua or similar to let you really control it.
But if your worry is that there will be some monster data structure you have to understand and work with manually in order to do anything then have no fear, you will get good tools.
The problem I'm solving is one of operations automation and design. Go take a look at the nginx code for loading configurations and then go write an nginx module. Figuring out how to get configuration settings out of the thing is horrible. Then, come talk to me when you can change the nginx config on 100 hosts based on the hostname and three sections of the configuration file using sed and awk.
Hell, let me know if you can load a random nginx config and tell me what version of nginx it works for. That'd be a trick.
SQL is a DSL for dealing with Relations (n-tuples) and collections of Relations. In other words, its domain is Data in the most generic sense. It's not really Domain Specific to web server configuration per se. Maybe that doesn't matter, though. I know that Sinatra's config DSL is really doing nothing more than manipulating path data and associating that data with code. Ops people love manipulating stuff like that with SQL, so maybe it's a good fit.
If I have a section of a config file that's host-specific I'll bust it out into an include and generate just the relevant parts.
Go take a look at the nginx code for loading configurations and then go write an nginx module. Figuring out how to get configuration settings out of the thing is horrible.
Putting settings into SQL doesn't magically solve this problem. As an admin I'm still going to have to know what fields and values each module expects. Much worse, I'm also going to have to manage tree-structures in SQL for directives with subdirectives, which is something even full-time SQL people often struggle with.
So yes, putting them in SQLite (not IN AND RDBMS), does magically solve it, and does it for all the languages that know sqlite3, which is all of them.
As for having to know about the structure of the database and know trees and stanzas and things like that: Did you RTFA? I said there'd be handy tools for dealing with it so you don't need to know this stuff. Seriously, you think I'm going to foist a full on SQL schema on anyone?
That'd be about the stupidest thing ever. Of course I'm going to have tooling that makes this easier to work with, and in fact you can write your own tools to manage it.
So you are planning on wrapping this in some kind of reasonable DSL? If so, the fact that you're storing an intermediate representation in SQL seems uninteresting, although you also seem to suggest that the advantages of this approach are realized via direct manip via SQL.
I can see some room for improvement and standardization in config languages, but any server that doesn't allow me to do config purely via text files and that doesn't have a config language rich enough to encompass directives with subdirectives and conditionals isn't useful to me. Why not just make everything Lua?
If you're a modern system administrator who automates, automates, automates, then this is better.
For example, my challenge still stands, load an nginx config and tell me what version of nginx it is for. If you can't even do that then how can you automate it better than a sqlite3 file (remember, sqlite3 file, NOT RDBMS).
Regarding your question: I don't really know or care what version a nginx config is written for (I could specify this in a comment if this was an issue and I can always test the config by running "nginx -t"). For the record, I have used nginx for years and this has never been an issue.
Anyway, I am generally impressed by your other ideas in Mongrel2 and happy hacking :)
The version challenge points directly at what's wrong with config files: they aren't accessible to automation. If you can't even do this very simple thing from one programming language, then it means you can't do other things easily.
At least, that's what I heard...
Someone would have to up and write a Go web framework to use with that. They wouldn't have to, but it would be a shame not to.
Anyhow, that's easily solved. It wouldn't have to be in a process doing any web serving. The compiler would be available in a process that just does the starting up. The worker processes would then know which dynamically loadable module they need to load. So, no there wouldn't have to be a compiler in the web server.
I love this.
(I can't think of any other way of putting that :))
Let's take a real-world nginx conf for example (comments and blank lines elided to save space). I can't imagine how trying to shoehorn something like this into sql is a win. Even Java-style XML situps seem preferable:
http {
include /usr/local/nginx/conf/mime.types;
default_type application/octet-stream;
log_format main '$remote_addr - $remote_user [$time_local] '
'"$request" $status $body_bytes_sent "$http_referer" '
'"$http_user_agent" "$http_x_forwarded_for"';
access_log /var/log/nginx_access.log main;
error_log /var/log/nginx_error.log debug;
sendfile on;
tcp_nopush on;
tcp_nodelay off;
gzip on;
gzip_http_version 1.0;
gzip_comp_level 2;
gzip_proxied any;
gzip_types text/plain text/html text/css application/x-javascript text/xml application/xml application/xml+rss text/javascript;
upstream mongrel {
server 127.0.0.1:5000;
# server 127.0.0.1:5001;
# server 127.0.0.1:5002;
}
server {
listen 80;
client_max_body_size 50M;
root /var/www/apps/mysite.com/current/public;
access_log /var/www/apps/mysite.com/shared/log/nginx.vhost.access.log main;
if (-f $document_root/system/maintenance.html) {
rewrite ^(.*)$ /system/maintenance.html last;
break;
}
location / {
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Host $http_host;
proxy_redirect false;
proxy_max_temp_file_size 0;
if (-f $request_filename) {
break;
}
if (-f $request_filename/index.html) {
rewrite (.*) $1/index.html break;
}
if (-f $request_filename.html) {
rewrite (.*) $1.html break;
}
if (!-f $request_filename) {
proxy_pass http://mongrel;
break;
}
}
error_page 500 502 503 504 /500.html;
location = /500.html {
root /var/www/apps/mysite.com/current/public;
}
}And they are not much better :) (actually I prefer Nginx - better syntax)
Second, no, they're not heirarchical, you're just used to them being that way. They're data structures, which are organized however I like them. In nginx they're hierarchical, in Mongrel2 they'll mostly be relational.
Third, like I said before, the advantage is tooling. Managing the Mongrel2 configs from other languages, scripts, shells, and remotely with automation will be much easier than what you have in an nginx config.
But, here's a thought: What if you could feed an nginx config to a tool and it spit out the Mongrel2 config to replicate it? Wouldn't that be slick?