Hacker News Reader

If you have an image stored in an SQLite database file, you can extract it using the "sqlite3" command-line tool as described at https://www.sqlite.org/cli.html#file_i_o_functions

Or, you could do it several dozen other ways. For example, maybe you have an archiver tool for SQLite that works like "zip" - https://www.sqlite.org/sqlar/doc/trunk/README.md

About this much we agree: The objective is to make the content of the document more accessible and easier to use. The article intended to show that using an SQLite database rather than a pile-of-files in a ZIP archive achieves that goal. If my presentation were stored in an SQLite database, I might (depending on how the schema was structured of course) be able to ask interesting questions of the presentation, such as:

  *  How many pages are in this presentation?

  *  Which pages contain the word "discombobulated"?

  *  How many images are in the presentation?

  *  Does this presentation use a prepackaged background?

  *  Does the presentation begin with my companies standard "start" slide?

Can you write (say) a Python program that will answer any of the above about an OpenOffice .odp file? All I'm trying to say is that if OpenOffice .odp files were SQLite databases with a sensible schema, then probably you could answer any of the questions above with perhaps 5 lines line of code or less. Or just a shell script that uses the "sqlite3" command-line tool. As it stands now, if you want low-level information about an OpenOffice presentation, you have to open manually open the document on your disktop and click around - it cannot be (easily) automated. An ".odp" file is essentially an opaque BLOB that is not useful to me. Only the OpenOffice program (or its various forks) can access it.

I have 104 historical presentations in a folder and I want to find some slide I wrote years ago. Right now, I have to laboriously open each presentation and manually search for the slide I want. If the format were SQLite, I could perhaps do the same with a query, which if run from a shell script, would quickly search all 104 presentations for me.

So, Yes, the whole point is to make the content more easily accessible and usable. Relative to a pile-of-files in a ZIP archive, SQLite does exactly that.

> Can you write (say) a Python program that will answer any of the above about an OpenOffice .odp file?

Using the libraries or packages available from the OpenDocument home page? (http://opendocumentformat.org/developers/) Probably, yes. Four of your five questions questions are just: "Is there an API?" The answer is: Yes, yes there is. Did you try searching for an Open Document API? For the image file one, however, all you really need is ZipFile.infolist() because ZIP files already have an API.

> All I'm trying to say is that if OpenOffice .odp files were SQLite databases with a sensible schema, then probably you could answer any of the questions above with perhaps 5 lines line of code or less.

Most of the questions involve an XQuery against content.xml. You could iterate through the files, extract content.xml, and apply the XQuery. No, it's not easy to write the correct XQuery because XQuery sucks and the schema is complex, but it can definitely be done.

I don't know why you think it's going to be magically simplified by changing the storage method. You'd either have compressed XML BLOBs like the article's first suggestion -- which puts you right back where you already are -- or you'll have dozens of tables with a ton of metadata and document structure to dig through. Either way you will need a schema reference to understand what's going on. An RDBMS isn't going to make the content simpler. The schemas are complex because the documents are potentially very complex.

> I don't know why you think it's going to be magically simplified by changing the storage method.

Because ZIP is a key/value store and SQLite is relational.

(Later:) Here is a slide comparing SQLite to ZIP from a talk I'm scheduled to give on Monday: https://sqlite.org/tmp/sqlite-v-zip.jpg

Both SQLite and ZIP will store files and both are well-established open formats with a trillion instances in the wild.

SQLite does everything ZIP does but also gives you transaction, a rich query language, a schema, and the ability to store small (1-8 byte) objects and to translate objects into the appropriate byte-order for the reader.

> It's complaining that the Open Document Formats are bad because they don't use an optimized format.

That's the opposite of what the article states. From the introduction of the article:

> Note that this is only a thought experiment. We are not suggesting that OpenDocument be changed. Nor is this article a criticism of the current OpenDocument design.

Yeah, and if you finish the quote, the next line is:

> The point of this essay is to suggest ways to improve future file format designs.

That makes it sound like, "No offense, but < extremely rude comment >." Just because they say they're not trying to be critical doesn't mean they're not actually being critical.

> What does that mean you need to do? Connect to the file with SQLite, fetch the BLOB, hexdump it since it will return as a string, decompress it, and then save it to disk.

Or you could just use a SQLite GUI (eg sqlitebrowser.org). Select the field/BLOB you're interested in, then Export (saves it to a file). Done. :)