Show HN: Fieldbook: Lightweight Database with a Spreadsheet UI and a REST API | Hacker News Reader

Show HN: Fieldbook: Lightweight Database with a Spreadsheet UI and a REST API

fieldbook.com

335 pointsjasoncrawford92 comments

I would use an open source version of this immediately. I've written several hacked up projects using google spreadsheets and it works, but its unreliable. I would not want to use a smaller 3rd party service as I would be afraid of the service getting shutdown or neglected. I would be much more inclined to use and pay for this service if they also open sourced it. Maybe run it like wordpress, they make a lot of money :)

Same here. I'm really excited about what these people (and others) are doing in this space, but the lack of an open source code and the service lock-in (mostly) are huge turn offs for me.

I've started building a prototype of this idea using PouchDB/CouchDB (data is stored locally in PouchDB, synced to your own CouchDB server or Cloudant). As with most of my side projects, it remains abandoned waiting for some love.

Where is your PouchDB/CouchDB prototype? Can I see it? I want take some ideas from it, perhaps fork it.

Wordpress is kind of a "one-off" in the same way that Farmville and Minecraft were (i.e. right place, right time -- Newspro and Perl CGI was falling out of fashion, PHP4/MySQL3.23 were common everywhere, shared hosting was easy, and Wordpress came bundled with CPanel IIRC).

A more sustainable business model would be, I think, the safe middle-ground for both the consumer and the developer-- the Atlassian model. Offer an absurdly cheap .jar/.dmg/.exe out of the box which you can run locally for the end-user/small-business (what is it like a dollar a user a year for the self-hosted binaries?) and limit it to 5 or 10 seats. If the acquire-hire-kill by GoogAplTwitSquare happens, you still have a local copy of your data and the ability to migrate off the platform. If >(5||10) employees are using the application as a critical component of your business, making the seat-fee price-jump at that point is worth it (value provided > cost of seats, even at a few hundred dollars a month).

WordPress 4.4 comes with a new REST API that might change some things:

http://v2.wp-api.org/

I would use this, even license it, if it could handle forms and I could reliably run it for the foreseeable future if the hosted version went away. DabbleDB was real interesting and it sadly went away.

I too would like open source, hard to pitch this to peers when no open source option is available.

What if they offered a data export/backup option? Since the Api is there you could even write one yourself

Seems to me that unless they are lucky and get under the protecting wings of Red Hat or something, most Open Source developers do it for free even if the work is worth good money.

I'm hoping that sandstorm or another hosting provider that makes a business out of reselling open source anyway can try out one or more models where a fraction of the revenue is fed back to the creators / the community to encourage more development.

Don't know if it would work but I can hope.

I'm working on exactly that. However, I'm a bit fearful it will be received negatively by developers.

Of course it will, by some.

Hopefully most should see some value in getting their software in the hands of more users while also getting a little money without doing anything extra.

If they receive it negatively, perhaps they should have put more thought into licensing - or commercialised it themselves.

We (Cloudstitch) would love to help you migrate your GSheet-driven pages onto our platform. We provide all the middleware tooling to run a GSheet-driven page or web app at production scale.

Hey Folks!

I'm the CTO of Fieldbook, we're really proud of what we've built here. In particular, I really like our API explorer that allows you to run real node code right in the browser to explore our API. (And see our realtime updates in action on the same page). That feature is powered by Tonic (https://tonicdev.com)

Its mad fast to setup a database with a REST api with Fieldbook, and we'd love to hear what you think / what could be better / etc.

What does the premium plan offer that the free one doesn't?

And what are the usage limits?

We want you to use Fieldbook as your database, so we don't limit the number of api calls or books you can have in Fieldbook! (we hate when you run up against an artificial limit and have to scramble to get your product back online). :)

We don't scale as far as we want to right now. For a good experience, we're limited to about 10k rows in a book. Definitely something we want to improve a LOT in the future.

More info here: http://docs.fieldbook.com/docs/data-size-limits

Glad to hear you folks are looking at scaling. I love the idea, and for small stuff the app so far is great. My book with around 2500 rows and 20 columns I find to be significantly slower than I prefer, though. Unclear how much of that is backend versus frontend, but definitely a limiting factor for my use right now. In an ideal world it would feel as snappy as Google Sheets.

Definitely hear you! We are working on some performance improvements right now, if you contact us in app, we might be able to figure out what is happening with your data. 2500 rows and 20 columns should still be snappy!

This is front-end and should be fixed soon, literally working on it now. Thanks!

The paid tier right now offers a couple of things, mainly focused on business administration. In particular, the ability to manage accounts in your organization (remove access once someone has left the company for instance) and priority support.

More here: http://docs.fieldbook.com/docs/plans-and-pricing

This looks great! I've been on the hunt for a good CMS-as-a-service and have been a bit disappointed by the current options. Does fieldbook handle image uploads? Any other caveats for using this as a full-blown CMS?

We love using Fieldbook for CMS use cases. We think the public API coupled with public posts is a great fit for the API, too.

We don't handle image uploads yet. Definitely something we want to support in the future. Of course, you can put urls in the cells and build things that way... Definitely something we want to improve in the future :)

Are you familiar with recent work of Chris Granger?

Apparently so far he thinks that data structure you have in your app is most approachable for non-programmers:

https://youtu.be/VZQoAKJPbh8

Thanks, I agree with him on that point! We've developed the conceptual model and the UI for Fieldbook through hundreds of usability tests with users from a variety of roles and backgrounds. The model we have has been tuned to match how people naturally think of working with data and how they can best understand it.

While it's nice that you have your own spreadsheet interface, how hard would it be to hook Excel in as a front end?

I deal with large enterprises primarily and I've been looking for a nice relational backend for Excel for years.

So breaking out the roles of a consumer vs. a creator, I would like end users to consume without knowing that there is a different backend to the Excel sheet they are using while still freeing up the creator to do what is needed on the backend.

For example, we have extremely complex pricing spreadsheets that we use to estimate projects, it would be fantastic to be able to collate and manipulate data on the backend as well as data sources while being able to tell an end user to keep using Excel as the front end as before.

Maybe this is a fit, maybe it isn't but any insight/feedback would be welcome. Thanks.

Thanks! At some point I think we'll have an integration that allows data syncing to and/or from a spreadsheet like this.

The spreadsheet isn't ideal as the primary UI for a number of reasons. That's why we built our own UI, inspired by spreadsheets but fundamentally breaking from the model to be more like a relational database.

But a spreadsheet connection could be great for reporting, modeling scenarios, etc. – basically, all the things that spreadsheets are actually good for.

I've been shocked at the number of enterprises with workflows involving shared Excel docs (particularly in the eCommerce space). One of my (Magento) community members has a business which seems to tackle some of the issues around this: https://www.cobby.io/

Might be worth having a chat with him.

Excel-as-a-front-end hooks in so nicely and easily to SQL Server Standard via standard ODBC/ConnectionStrings. You get audit tables, integration with Active Directory, SSRS, concurrent modifications, the ability to dynamically perform queries (i.e. [1], if you enable a filter clause on your projection from one of those DropDownLists, it will send it out to SQL Server and then return the resultset), audit trails, permissions, and most importantly you can set all of those NON NULL, FK, restraints at the database level, and use SQL Server Reporting Studio (comes with Standard) to perform your reporting. Priced out per CPU you can get it for under 10k. I've done it legally for free too[3] in my early years, but if your client won't pay 10k for a RDBMS, you've got a tire-kicker who won't respect net/90 much less net/30. I've done literally dozens of times for every type of industry you can imagine and I can count on my left hand the number of times those clients needed to later supplement their infrastructure with an actual data-warehouse. (Also the newer Excel versions allow data constraints to a certain degree, so you can avoid that unnecessary transaction/lock acquisition in an anticipatory fashion.) Combine SSAS/SSRS, and the free downloads of PowerPivot and PowerBI and you have a _very_ robust, flexible system that can handle millions of rows easily.

[1]https://www.ablebits.com/_img-blog/find-duplicates-excel/sel... [2] https://www.microsoft.com/en-us/server-cloud/products/sql-se... [3] SQL Server Express + the Express version of the Business Intelligence Suite was free for production use up to 4 GB and 2(?) processors [things might have changed since then, SQL Server 2008]. A few hundred lines of SP's to dump stale data out and the client was golden.

How much of this is manual effort vs. built-in? Every example I've seen that actually updates data in the database winds up with a pile of VBA.

I've done it one of two ways - via VBA and via VSTO Add-ins. My VBA version has been carefully crafted to emulate a pseudo-ORM based off INFORMATION_SCHEMA and all, making it pretty portable. It works fairly well because what usually ends up happening is say Partner Foo will have his read-write to his schema and it will all be segmented up via catalog views and other rules. There are the standard concurrency issues (you have to discuss with your client whether you want him to take a mutex on the result set, or potentially allow stale data) so it's fairly naive in that regard. The upside of it it's all self-contained (i.e., not a pile) and portable because you effectively get "reflection" with any sufficiently decent RDBMS impl. Maybe 10-15% of that VBA has to be tweaked based on their business requirements (defining behavior like if pri_row gets deleted in table foo by user bar, do you offer a prompt to cascade delete[1]?)

In some situations [such as a cascade delete], the business requirements need the Stakeholder or PM or TL or whomever to authorize certain events due to internal corporate governance policies. At that point, the VSTO add-in route is taken because usually there has to be integration with other governance/workflow software (one of the subsidiaries for Blue Cross/Blue Shield I worked with was notoriously bad re: pushing the buck and over-complicating workflows). Anyways, the MSVS sln I use with that is pretty much the opposite of a VBA pile.

[1] There's almost never any destructive DELETE's taken, rather a boolean active flag is toggled, and a new row is inserted with a more recent timestamp so you can '...order by timestamp desc limit 1'. That way if auditors come in, not only do they have the full audit log but we can reconstruct state at any given time.

Either way, a naive VBA implementation that doesn't appeal to system tables for all the schema info can easily be written (in a maintainable fashion, as maintainable as VBA goes at least) in under a week.

Thanks for taking the time to fill in some of the details!

This is the most detailed explanation I found regarding the VBA method: http://www.toadworld.com/platforms/sql-server/w/wiki/10392.e...

And the company you linked originally gives away a generic VSTO add-in: https://www.ablebits.com/excel-addins.php#sql-server

The part I was missing was when you said 'audit tables [and] audit trails' (etc.)... it sounds like this would all be build-it-yourself.

I disagree strongly with an interdependence on Excel. Instead, it would make sense to ask what new functionality could be added - and start by looking at major applications of databases (you might be surprised to hear that scalibility and reliability are much more important than most think - and almost always beat other features for adoption by paying business customers). The Excel UI is a boat-anchor to innovation in the space (no reliability/scalibility or modern architecture). The Web is the new de-facto GUI, and it should be possible to start adding functionality that Excel would be hard-pressed to replicate.

I should add that I was once a Product Manager for core database at Oracle. We had constant requests for an Excel front-end to Oracle. I don't know if we had any major customers who didn't ask for a spreadsheet front-end at some point. But we were busy making money on other things, and it just wasn't a priority (though consulting routinely built hooks.) So I think that this has the potential to be a huge product - especially if it can leverage new technology in ways which Excel can't. Don't let Excel and Microsoft drag this product down.

Hi, sheetsu.com creator here. If you are looking for an Excel as DB for your front-end give it a try. Paste link to your Google Spreadsheet and then you get an URL to the API. You can CRUD your Spreadsheet using that API.

More info here - https://sheetsu.com/docs/beta

Any plans to support triggers & email notifications? I consult for several small businesses and this could be a quick & low budget alternative to the bigger SAAS' out there for HR, CRM & Asset Tracking.

Yes, all in the master plan!

Just a FYI, with uBlock on (mostly default settings except for a few specific sites of which yours isn't one) I couldn't sign up at all, the field for entering the email address just wasn't showing at all. As most others have said, it looks great! Looking forward to playing more with it

Love the product!

What about offering a whitelabel product where I am the account owner and can spin up an independent database for each of my clients with their own login credentials while I still have master access to each? I would literally sign up tonight and deploy this next week to my restaurant clients who want the ability to update their food menus at will and in complicated ways. Static site front end with a live menu API and Excel-style interface for them to edit in? That's the perfect solution for what these kinds of clients want/need.

Here's how you could do this in Fieldbook now:

* Create a template book for yourself * For each client, make a copy for them and share it with them so they can edit it in Fieldbook * Read from each book via the API for the live app

It's not whitelabel, but it would give you an independent database (book) for each client, they would each have their own login credentials (to Fieldbook), and you would have access to each (in the UI and in the API).

Here's one simple thing: really good SQL export.

I can imagine using this as a way to get users to build simple prototypes. They can chuck data at it for a few weeks, then hand the data off to developers to build a first cut of a web app. If said developers could easily click a button and get a gzipped SQL file compatible with, say, Postgres, that'd be a really useful feature.

Neat. How does this compare to Airtable? https://airtable.com/

Thanks!

I think we've focused a lot more on relational support. For instance, on a detail page, you can view and edit all linked records, and add new ones inline: http://docs.fieldbook.com/docs/go-beyond-the-grid

We have a full-fledged query UI: http://docs.fieldbook.com/docs/focus-on-relevant-items

And we have a grouped view (think Trello meets spreadsheets), which is great for any kind of pipeline/kanban view: http://docs.fieldbook.com/docs/track-a-workflow

Just an idea to share, from someone who's been building business apps using Knack's visual builder for the past year, I think a killer combination would be fieldbook's friendly UX for the database with a "marketplace" of front-end "plug-and-play" UI modules. So in fieldbook I would create and save views for the database with your current UI but then I could plug ready-made UI modules on top of that database view and then customize the front-end UI. Ideally these UI modules would be free or pay web-based or native.

I know it's extremely important to strike the balance of complexity, capability and UX so this area is notoriously difficult to tread but I think whatever solution does it right without sacrificing customization extensibility is going to win big!

Fieldbook looks great and I hope it keeps getting better!

Thanks! Good ideas, and yes, you nailed it: the right balance of complexity and functionality is key.

Eve's already been mentioned here, and Chris' lecture at Cal was really interesting from a UX perspective. I'm sure you've seen it but if not, go watch it.

The spreadsheet model is fairly intuitive for the end-user. He and Rich Hickey both are pretty convinced that a Datalog-esque query engine is the way to go. 20 years ago, Lotus' Improv's model (where you separated the data manipulation from the data type constraints) was way ahead of its time and guaranteed so much more internal/referential integrity than 1-2-3. Modern-day Quantrix is probably the closest you'll get to it, used pretty heavily in niche finance.

MSR puts out some interesting work on the PLT behind alternate spreadsheet grammars[1] which are quite interesting too. Other than the obvious issues (ETL[2], ACLs/RBAC, presentation/extensibility) MSR has put out quite a few interesting papers out there. This is a very interesting problem domain to say the least. I've developed up-to-the-Pareto-point for this in the past for clients and if you get it right, it's a license to print money.

[1]http://research.microsoft.com/en-us/um/people/sumitg/pubs/pl... [2][both conventional a la SSIS and Informatica, as well as a Pipes-esque query; a premium feature you could easily charge into-the-Oracle-rates would be a Pipes-esque service with an SLA. I've been in enterprise consulting long enough to know that whether you're Ingersoll Rand, Haas, an REIT or Cravath, sanitized, guaranteed data is worth a lot, integration is worth even more.

When you refer to Pipes-esque - are you talking about yahoo pipes?

Wow, both of these look really cool! I'm glad to see the database + friendly UI category is getting some reinvigorated attention.

Last time I researched this category, the best options I could find were MS Access and FileMaker Pro. I'm often annoyed with overly use-case-specific apps, as they do not have good flexibility in interacting with the data they use. It seems like many use-case-specific apps would be better suited as templates in a generic database UI tool.

Chris Granger's Eve is also tackling the database UI problem, and they are investigating wider-scoped UI patterns, beyond spreadsheets.

Yes, it's time for the FileMaker/Access category to be reborn for the age of webapps, mobile, and APIs.

Sadly it seems like whenever this happens they get acquired by someone(probably Salesforce) and you never hear about the product again...

My first thought as well - while Airtable isn't for me, it seems like somehing I'd recommend to tech-savvy non-developers. Also, as far as I can tell, the author of Airtable seems to come from a FileMaker background, which shows in the product: A database for normal people.

I've been using Fieldbook a little bit, and even playing around with their API.

It's a great product I'm going to use more. Big fan - there have been several side projects where I've wanted to use Google Sheets as a prototype DB - with the simpler API and better relationship model, I'm going to use Fieldbook from now on.

It looks very nice. But I can't integrate it into a project if I can't guarantee that it will run even if Fieldbook-the-company changes business models next year. When can I deploy a self-hosted version, and how much will that cost?

We definitely intend to be in this for the long haul! We will always let your data out of Fieldbook, we already support CSV downloads of any sheet or view, and we are thinking about a wide variety of export and sync options to other services / data formats.

I understand being leary of business model changes. While we can't make any guarantees in the wild world of startups, we can say that we want a free tier to remain forever, and would give plenty of warning if it were ever to change. A little more information available here: http://docs.fieldbook.com/docs/security-and-privacy

We don't currently have any short term plans for a self-hosted version.

The user interface looks beautiful and I like the innovation for the group/sorting/filtering. Allowing all three to be defined using drop downs within the single area is really neat. Fast to add and compact in space.

How does it handle data types? Can you put any type of data into any cell? If I want a column that is numbers is there anything that stops a user adding a string to one of the cells by mistake?

By default you can put any type of data into a cell, like spreadsheets. If you want some input validation, you can add an optional input type to a cell. So you could set a number type on a cell, and then if someone tries to type a non-number, they'll get an error. http://docs.fieldbook.com/docs/input-and-formatting#input-ty...

Can multiple people edit the same spreadsheet at the same time? If so, are their changes displayed live?

Yep! Definitely. We don't (yet) show cursors and who is connected like google docs does, but you do see changes replicated to all windows / users in real time. You can even see your own real-time changes from API calls in realtime while looking at your book.

This looks really awesome. Can you lock columns or rows to prevent people from changing them?

Unfortunately we don't yet have that particular feature... But we definitely want to do something like that in the future. What are you thinking you'd use it for (we love to collect information like that for when we do implement the feature).

Obviously, using the API you could build a restricted interface on top of fieldbook, but that probably isn't what your looking for, I understand :).

We're also considering doing a mode where you can't change any of the metadata (sheets/fields) but can add/remove/change rows.

Thanks! We're working on a feature that will let you share a book “metadata-readonly”: allow editing of rows/cells but not the structure of the book (sheets, fields, links, formulas, etc.)

Longer-term we might offer more fine-grained locking or permissions control.

This looks excellent! I recently started writing something similar for my phone since I needed something more flexible than the standard todo/gtd apps. Do you have plans for a 1st party mobile client?

Right now there is basic mobile web access. Definitely going to do a native mobile app!

Great to hear! I've been playing with the product for a few minutes and I really like it so far. Lots of very nice touches! I think I found a bug -- If I give 2 columns the same name, the resulting JSON field gets overridden: http://i.imgur.com/KA9mAeI.png

Maybe display an error if I name 2 columns the same? Or disambiguate somehow in the JSON?

Correct, we don't handle this case right now and one of the fields will get shadowed. We'll handle this more gracefully in the future; for now we recommend unique column names :-)

I've used Fieldbook a few times. It is incredibly neat.

It's an interesting idea, However if it's not self-hosted then I think their reach of customers is very limited. We all know most of the businesses run on spreadsheets and most of them don't want to share that information with anyone.Unless you handle this problem with run time encryption or provide a self-hosted version It's difficult to land lot of customers.

This is so awesome. Easy to use, set up in seconds, and is mobile friendly. Nice! Can't believe no else has something like this.

Its not that difficult to use Google Spreadsheets as a database - here's an app I put together in a couple of hours to allow people to rate players after a soccer game - currently used by a reddit sub:

http://www.teamrating.com

Nice!

One of the key advantages of Fieldbook is actual relational support. You can create a relational model by “linking” sheets, and then linked records are returned by the API.

Also, the Fieldbook UI is more suited to being used as a database than spreadsheets are. It's much easier to do queries, for instance, and sorting/filtering won't mess up your book.

Lots more on Fieldbook vs. spreadsheets here: http://docs.fieldbook.com/docs/what-is-fieldbook

Interesting. But I have no idea what it does.. that picture and bullet points don't really tell me much. I want screenshots!

Congratulations Jason. it looks awesome and has come a long way in the last year.

Looks like the beginnings of a QuickBase competitor, which would be welcome...Intuit's pricing model is terrible.

I had high hopes for DabbleDB before Twitter bought them and shut it all down.

I went down the road of working with QuickBase, meeting their implementation consultants. It was classic old school software company. Left hand not talking to the right, bloated, unresponsive, just trying to sell you.

TrackVia is another interesting alternative, they started as an Excel replacement (and very good one at that), but have since gone more towards mobile and simplicity in the new version.

Knack is in the conversation as well, but it seems like Fieldbook & AirTable are now the ones to watch.

How does this differ from DabbleDB, which seemed to have all this and more?

(Sadly went nowhere and subsumed by Twitter.)

Hopefully it differs in that it will continue to exist and not be acqui-sunsetted.

I didn't learn about Dabble until after it was gone, so I only know it from watching the demo video and talking to one of the founders. The spirit is similar and I think we have a lot of shared vision.

I think our UI for creating links (relations) and for doing queries is unique, both were developed through hundreds of usability tests to be both accessible for novices and satisfying for power users. Check it out.

The UI is just gorgeous IMHO.

Can you share which frameworks you used ? Did you use a template for the front page ?

Thanks!

Not much of a framework on the front end, but we do take advantage of Bootstrap and Backbone.

That looks like a really well made product, awesome UI!

What are some real world use case for using fieldbook?

Thanks! We actually have a few user success stories, and most of our templates are directly from use cases we've seen people use.

Here is a story of Data Analysis tracking sales leads: https://medium.com/@fieldbook/data-analysis-tracks-sales-lea...

And here is one about how Continuum Analytics uses us to track their professional service work: https://medium.com/@fieldbook/continuum-analytics-manages-pr...

Thanks! Real-world use cases include project tracking and sales/CRM. More here: http://docs.fieldbook.com/docs/what-is-fieldbook#what-is-fie...

Also check out our templates for more examples: https://fieldbook.com/templates

Anyone remembers similar product DabbleDB? Or I'm too old?

Yup, we've had a few conversations with Avi, one of the Dabble founders; he's been helpful with advice and feedback.

Yes this reminded me of DabbleDB. It's just a shame (for everyone except the founders/employees) that they were acquired and shut down.

Because of what happened with DabbleDB, I worry about these awesome startups getting acquired. Again, great for the founders, but not for customers.

is this a bit like sheetsu?

It's someway similar. The difference is that Sheetsu.com use Google Spreadsheet as a storage.

It would be nice to get something modern to replace what people used to do with FileMaker. This may be it.

Hi DHJSH, I just wanted to let you know that you have been "shadowbanned", which means all of your comments are dead on HN by default. I vouched for this comment to make it visible, but mainly in order to reply to you and let you know that you have been banned.

Disclaimer: early Kinvey engineer here.

Doesn't sound too far off from Kinvey and Parse except for the fact that MBaaS services have been around a lot longer and have much more robust mobile and web SDKs.

We don't think of it as BaaS. Instead, we think of it first as a spreadsheet-like UI, and it also has an API.

The use cases people have been most excited about need both. For instance, set up a configuration database or CMS for your app--an engineer can create the schema and read via the API, then anyone on the team can view and edit the configuration.

With the BaaS offerings I've seen, you get the database and API, but you have to build any UI you want yourself, including any web or admin UI.

>With the BaaS offerings I've seen, you get the database and API, but you have to build any UI you want yourself

Not true, you get a spreadsheet data grid like UI out of the box.

They are more general purpose - that is true.

Reply on news.ycombinator.com