I do remember using that tool to publish the very first aMule packages for Debian. I think it was called watchinstall.
75 karma · joined November 15, 2012
Pros:
- Less typing, which means less bugs.
- Type-checked queries, although this is technically not part of the ORM. We have a custom tool which statically examines the code creating ORM queries, checks the types of the output and lets you know of potential errors (e.g. the field you're querying on does not exist anymore). Writing the same tool for SQL would be incredibly difficult.
- No need for long ugly SQL strings in code (well, except for the ORM implementation itself).
- Easier refactoring. If we make changes to a model, the ORM makes the schema modifications for us (with some limitations - e.g. if you rename a field you have to tell the ORM the old name, so it renames the field in the table instead of creating a new one). Coupled with type-checking this is god-send.
- Complete SQL database abstraction (database/sql does not provide that - placeholders, quotes, etc... vary by database driver).
- SQL / NoSQL / KV-store / whatever transparency. We have SQL drivers for sqlite, postgres and mysql, as well as drivers for mongodb, the GAE datastore and redis. Of course, some drivers have limitations, like the NoSQL ones can't perform JOINs, but we still get incredible flexibility. We had one app running on GAE with a data model which was a good fit for the datastore. Some months later, a few requirements were added and moving to a relation database was the best solution. So we spun off a Google Cloud SQL instance, migrated the data and started using JOINs for the new features, while the previous code continued to work unchanged.
- App Engine / vanilla Go cross platform (well, you'd also need our full web framework for that, but the ORM is one of the key components). We have some applications running in GAE. If they ever get too costly to run on GAE we can just execute three of commands and deploy it to our own servers (and by "three" I mean the exact number: one command to move the ORM data from the datastore to the new database, another one to move the stored blobs from the GAE blobstore to S3/GCS/filesystem and another one to deploy the app to the new server).
Cons:
- Overhead. Negligible in most cases, but when it matters we can drop down to raw queries.
At the end of the day, I think using an ORM or raw database queries is just an engineering decision you have to make depending on your project constraints, not on the language you're using.
Just to clarify, maps can be read concurrently without locking as long as you're not writing to them. As soon as any concurrent operation might make changes to the map (as the example does) you need locking, preferably a sync.RWMutex (a sync.Mutex will also work but will be way less efficient for no good reason).
Also, if you want to make the serialization faster you need to understand exactly what makes encoding/json slow (hint: is not only reflect) and remove all reasonable bottlenecks. You state that megajson does not support the MarshallJSON interface like that's a bad thing, but I'm pretty sure that's deliberate because it's indeed a feature. When encoding/json encounters a type which implements MarshalJSON it does the following:
1 - Call MarshallJSON to obtain its JSON representation as []byte 2 - Validate the produced JSON using a slower-than-the-bad's-guy-horse function based state machine which processes each character individually 3 - Copy the []byte returned by MarshallJSON to its own buffer
Unsurprisingly (after reading encoding/json's code, of course) having a MarshalJSON method is way slower than letting encoding/json use reflection if the JSON you're generating is anything but trivial and without almost any nesting, because it avoids extra allocations, copies and the validation step.
* A full ORM which generates tables from structs and allows relations between models, as well as a simple querying system based on Go functions which joins the tables automatically for you e.g.
memeId := base36.Decode(param)
var meme *Meme
var template *Template
err := ctx.Orm().One(orm.Eq("Meme|Id", memeId), &meme, &template)
* Inheritance based template system, with a declarative assets pipeline supporting compilation (e.g. it compiles coffescript to JS and less to CSS on demand - also pluggable). This is the base template from one of our apps:
{{/*
jquery|if=~ie-gte-9: 2.0.3
jquery|if=ie: 1.10.2
bootstrap|fontawesome=4.0.3: 3.0.0
styles|bundle: css/style.css, lightbox/css/lightbox.css
scripts|top,bundle: js/modernizr.js, js/detectizr.js
scripts|bundle: js/responsiveslides.js, lightbox/js/lightbox-2.6.min.js, js/site.js
analytics|nodebug: {{ $Config.Analytics }}
*/}}
<!DOCTYPE html>
...
{{ template "main" }}...
{{/*
extends: base.html
*/}}
{{ define "main" }}...
* Easy to use signed and encrypted cookies.
* Regular expression based URL routing.
* Functions for easily parsing input parameters.
* A development server which automatically rebuilds the source as it changes and reports compilation errors in the browser itself. While developing, runtime errors are also reported in the browser with complete backtraces and printing the values passed to each function. While running in production, those reports are sent by email to the project administrator(s).
* App-wide and low-level caching system with pluggable backends, currently supporting Redis and Memcache as well as a dummy backend for development.
* Form generator and validator from Go structs, with renderers for both Bootstrap and Foundation (renderers are also pluggable, so you could also write your own. We also support generating forms from multiple objects so you e.g. write an object which generates a captcha and include it in any form with a single line in code (in fact, we provide just that, but a simple numeric captcha a a full-blown recaptcha object). It also does automatic CSRF protection.
* A complete i18n system, which extracts strings from Go code and templates, generates .pot files which can be translated using any editor (or a specialized .po editor if you feel like it) and them compiles .po files back to Go code again. This makes packages with their translations "go-get-able".
* Periodic and background tasks which can either be scheduled or just started from any web request.
* A configuration package which takes a struct and fills it by parsing an optional configuration file as well as command line flags.
* Support for administrative commands, which can be executed from the command line or started remotely.
* Support for pluggable apps. e.g. we do have a "users" app which provides user registration, authentication and social integration. This can be included in any other app with a couple of lines of code (one line for the import, another line to tell our main app to include it). Included apps can do a lot of stuff, like inserting code in every template rendered by the parent app, including external assets like images, styles or scripts and even rendering their own templates contained inside your parent's app base template.
* Last but not least, a minimal overhead (in the range of 100ns per request) profiling framework which lets you know how much time a request spent in the cache, the orm, the template, e.g... as well as seeing how much each operation took (e.g. you can check how much time a SELECT took and also perform an EXPLAIN on it). Users can also define their own events and profile them by instrumenting their code (it usually requires just a line of code per resource, to indicate its name). You can also profile a live server by using a command line app included in the framework.
We also have helper and utility functions for the more mundane stuff and basically everything you could expect from a web framework. We took inspiration from Django, Tornado and Rails and tried to take the best of them while building this framework (I used to contribute to Tornado back in the day). We've been developing this internally for almost 2 years and it's now around 40KLOC of Go and a couple of K of coffescript, less and html.
Of course, you need to cache the thumbnails for a reasonable time once they're generated, otherwise the CPU usage will skyrocket, but that's pretty easy to achieve with nginx's proxy_cache directive.
I also have implemented this in Go, but using cgo and graphicksmagicks, which is way faster and decodes almost any image you throw at it (there are some issues with very optimized GIFs, but I fallback to gifsoup for "deoptimizing" them in those cases). In fact, I even added a function for cropping and resizing an image to a given size in the module itself (since I think is a very common need), while keeping the aspect ratio and also giving the option to just center the result or grab the part of the image with higher entropy (e.g. suppose you have an image with a person in the side and then a lot of blue sky, you probably don't want to crop and image and end up with just sky in the thumbnail). This is just one of a few benchmarks I wrote:
BenchmarkResizePngMagick 20 80665091 ns/op 689 B/op 3 allocs/op
BenchmarkResizePngNative 1 9689016519 ns/op 351200 B/op 27 allocs/op
(yup, that's 120x faster)
The bindings are mostly documented, but I haven't gotten around to releasing the code yet, although I do hope do publish it soon. If you're interested, send me an email and I'll let you know when I put the code in Github.
Disclaimer: I'm the developer