Hacker News Reader

Comment by londons_explore | Hacker News Reader

People use excel sheets as databases all the time because it doesn't require any specialist knowledge to build or use.

Sure, we techy types prefer a proper database with a web frontend and an API, but that requires significantly more skill to build than an excel file.

No database has the flexibility and flat learning curve that excel has.

Until a database manages to meet these goals, it remains a good tool for certain usecases.

We as engineers should be working towards making a real database thats just as easy to use as excel, and only then can we complain that people are using excel sheets for this kind of stuff.

View on HN

> People use excel sheets as databases all the time because it doesn't require any specialist knowledge to build or use.

Test and Trace is a multi billion pound programme of work, involving huge companies (eg, Serco) interfacing with national governmental bodies (Public Health England).

We've had months to get something in place. One of the reasons organisations like Serco is used is their apparent expertise with IT.

The main reason Serco is used is their political connections; they really ought to have been banned from consideration after the fraud scandal: https://www.bbc.co.uk/news/uk-50806919

(my one-step programme for improving UK public contracting would be that after a significant failed contract the contracting company and its directors and any other companies they are directors of would be banned from tendering for a period; maybe a couple of years?)

Unfortunately the procurement rules currently in place prevent procurement personnel evaluating a tender from taking into account any prior performance. I'm unsure whether this is a result of EU legislation and common market rules, or whether it is just the UK.

italy has legislation to forbid people who engaged in corruption/embezzlement from working with public administrations.

It also has a required certification for companies that work with the public that they weren't condemned for certain crimes (trafficking in illegal waste, extorsion, organized crime etc).

So, it doesn't seem an EU/Single Market rule that you can't check this.

Please tell me that is not true. That is just...nuts! What is the rationale behind that beyond corruption?

One issue they have is that very few companies are willing to have the government as a customer. That's one reason why they keep going back to the same failed firms time and time again.

Reasons vary but the ones I've seen are generally:

1. Complex procurement regulations that take a lot of effort to comply with by contractors.

2. Low willingness to pay (both price-wise and time-wise).

3. Civil servants are nightmare customers who don't know what they want and are under no pressure to figure it out, so there's a lot of timewasting involved.

4. PR nightmare if/when things go wrong even if it's not your fault, as government is relatively open compared to other types of customers so easier to find out about problems, and lots of people automatically blame the private sector in preference to blaming government employees due to (misguided) assumptions of moral superiority of the public sector.

This is a very good point and a shame to see it being downvoted. I have been wishing there was an accessible database-ish software for doing my accounts, as the current mega-spreadsheet I use is pretty hairy and none of the specialised apps quite fit the bill. The closest thing seems to be Airtable, which while nice, is quite expensive.

Recently I was visiting hospital quite a bit (ante-natal) and witnessing the horror of the massive NHS form filling software the nurses and midwives have to use. It's clunky and sprawling but one of the main sins I think is it's lack of adaptability. In the old days of pen an paper, a new form could be written, or adapted, and photocopied easily, and on-site. Now if a new field needs to be added or a process is changed it needs to go (I imagine) to some centralised IT development office and fed into a ticketing system where it might get changed in a few weeks or months. There's no room to quickly adapt at a ground-level, so you end up with things like this excel sheet problem.

I'm surprised that MS Access didn't take this kind of niche.

I know it's a pretty shitty program to use - excel is much more user friendly. If MS had made excel, but have all the features of access in it...

It depends which bits of functionality you want, but I think you are close to describing PowerQuery (Get & Transform) which is in Excel, which can ingest multiple data sources into effectively a relational database with hard datatypes and data-cleaning.

In PowerQuery & PowerPivot there are also no practical limits on the number of rows of data you can have (other than the impact on processing time, clearly hundreds of millions of records might start to be a problem). It's not quite access - it doesn't actually persist/store the data, just aggregates it from other sources which is probably what is required here.

Access is a little legacy and would have it's own issues. I think in reality it depends on how you are ingesting data (which is probably hospitals submitting excel templates or similar?). Maybe these could be automatically aggregated and put onto an analytics platform? (e.g. PowerBi). Hospitals could maybe report it on a portal, but I'm not convinced that the leadership would have wanted to change the established process (because of operational focus, change reduction & risk).

For a very long time, it did.

Weaknesses of Access in no particular order: very poor multi-user/multi-device-access features. No real concept of servers. Buggy, had a nasty habit of corrupting its custom file format causing data loss (Excel never does this). Does expect you to understand SQL, data normalisation, etc. VBA just about serviceable for a spreadsheet but not good enough for a 'real' database-backed app.

Probably someone could try and make a better Access. There are surely lots of startups doing that already, along with hosting. All the "no code" tools that are out there, etc. Access had the advantage of coming with Office so lots of people had it already, whereas today's subscription based services that host your data have incremental cost.

IMHO the biggest problem is the 2GB database size limit

LibreOffice Base is something of an attempt at a newer Access, but unfortunately it's far from ready for production use.

Base has very solid data storage foundations (including the ability to connect to real database servers) but the UI is very buggy and is painful to fight with. Note I said 'fight with' rather than 'use.'

Base seems to have a fair bit of potential, however, if someone where to pour some money and/or time into developing it further.

Don't forget the formulas! Excel is essentially a functional reactive programming REPL. No RDBMS I know lets you easily define a table in which cells in a row depend on values from previous cells in a row, and keep this as an invariant.

People working on Excel replacements need to remember about this aspect of spreadsheets too.

Sure. That's why I said it wouldn't bother me if this was in the first few weeks of the pandemic or for one-off ad-hoc analysis. I actually use Excel for that all the time, including for tasks where technically a db is the correct solution. Working is better than perfect and everyone knows (sort of) how to use Excel. To still be using it in production this many months in is not acceptable. Someone should have taken that Excel based workflow and set it up to run through a database, they had the time to do it.

There is also the fact that Excel has some rich editing tools. I have actually seen a system where most editing was done by exporting the relevant items to a spreadsheet, making your changes and reimporting.

Reply on news.ycombinator.com