Hacking around with the ScotRail audio announcements | Hacker News Reader

Hacking around with the ScotRail audio announcements

simonwillison.net

139 pointssimonw30 comments

One more hack: here's a query which lets you enter a sequence of comma separated terms to stick together MP3s of the closest matching audio recordings for each one:

https://scotrail.datasette.io/scotrail?sql=with+phrases+as+%...

In that example I enter "sorry, south, from, dunf, cancelled, due to, gas" and it produces audio for "I am sorry to announce that the Southampton Central service from Dunfermline Queen Margaret has been cancelled due to A fire near the railway involving gas cylinders".

I hadn’t heard of Datasette until you published this blog entry. I have to say, on watching your intro video, it looks really fantastic. Congratulations on creating such an excellent tool. I look forward to playing with it over the coming days and learning more about it.

I turned this into a canned query: https://scotrail.datasette.io/scotrail/assemble_sentence?ter...

And updated the blog entry with a new section about how it works: https://simonwillison.net/2022/Aug/21/scotrail/#assembling-s...

A lot of these systems — those which build phrases from audio fragments — seem like the bastard love child of miserly bean counters and software engineers being too smart.

For variable data — delayed by $N minutes — sure, building that from pieces of audio makes sense.

For the name of a service though — “The service from $start_station to $end_station” — it’s not necessary to go all in on O(n) vs O(n^2) and record individual station names and put the word “to” in between. While a computer scientist will baulk at the inefficiency, a designer will optimise for the end result and record every route pair because (i) it sound much better and (ii) the problem scales quadratically where n is the number of railway stations you have built, a number which is bounded and grows extremely slowly. You could record every route name in a day, guarantee it.

This example is a bit contrived because most announcement only say the name of the destination station but it applies to a lot of other audio announcement systems where “clever programming” and “cheap out on recording studio fees” were optimised over “make it sound good”.

An example of this can be observed in the newer Dutch slow trains (called ‘sprinters’). They seen to have changed the recorded voice (“the train to $destination will arrive at $time”) to some sort of a computer assemblage that sounds like they’re constructing sentences from syllables resulting in very strange and sometimes hilarious emphasis and off sounding timing on certain syllables. What a mess

I'm pretty sure the TFL buses in London have voice announcements built in the same way, as some of the stop names are pronounced really strangely.

I don't agree. over 4000 messages on the list, so clearly, splitting these out individually producing what? 20K+ messages? not what you could record properly in a single day with concentration, breaks etc.

If you look at the examples, they record them with different emphasis so they can appear in different positions in a sentence and sound natural enough that you can hardly notice.

In the last 10 years, I don't think I have heard any of these in the UK that sounded like you describe. We all know the voice is a robot anyway so as long as it is understandable, it's fine.

If trains are involved, someone is sufficiently nerdy to have uploaded it to YouTube:

https://www.youtube.com/watch?v=yxw8p7oo_3Y

https://www.youtube.com/watch?v=4PpUsoIPBsc (much better recording but not Scotland).

I think they're all nicely understandable, and the slight robotic-ness aids comprehension, and makes it easier to ignore the routine announcements (once on the train) but still notice an unusual one.

Also note the usual announcement on the station is something like this:

> The next train at platform [1] will be the [delayed] [16:40] service to Airbles, calling at Hamilton Central, Hamilton West, Blantyre, Newton, Cambuslang, Rutherglen, Dalmarnok, Bridgeton, Argyle Street, Glasgow Central, Balloch, calling at Anderston, Exhibition Centre, Partick, Hyndland, Jordanhill, Scotstounhill, Garscadden, Yoker, Clydebank, Dalmuir, Kilpatrick, Bowling, Dunbarton East, Dunbarton Central, Dalreoch, Renton, Alexandria and Balloch. [We apologize for the delay to this service, which was caused by] [the recorded announcement not being ready.]

Then you get "The train now approaching platform 1 is the ..." and "This is the ..."

On the train itself, you get "This is the ..."

Interestingly they have two clips for the name each station (e.g. [1]), I guess for use in announcements where the destination appears in the middle of a list ("this train will call at Linlithgow, Polmont, Falkirk High, ...") and for where it appears at the end ("the next stop is Linlithgow").

Having used ScotRail for years though, it's always slightly bugged me how the intonations of subsequent stations are unmatched. The announcement for the stations the Glasgow-Edinburgh express trains will stop at is particularly odd.

[1] https://scotrail.datasette.io/scotrail/announcements?_search...

I was surprised ScotRail only has 400 odd excuses for cancelling trains, I assumed based on usual service they'd have at least triple that!

I thought they only had one reason: "operational reasons"!

Add "caused by Network Rail".

Here’s the list of reasons: https://scotrail.datasette.io/scotrail?sql=select+*+from+ann...

Although there are duplicates for “X”, “X earlier today”, “X yesterday”. Also the page is being hugged to death (perhaps due to SQL queries?)

I don't quite understand why this many specific reasons for failure have been recorded. Sure, there's equipment failure, technical problems, etc. but then there's also "...a broken windscreen on the train." which surely falls under one of the earlier two?

Then there's also: * a fault with the on-train signalling system. * a fault with the radio system between the driver and the signaller. * a fault with the signalling system.

plus 2 more copies of all of the above with either "earlier on this train's journey" or "earlier today" appended.

I mean, don't get me wrong, as a public transit nut I love all the information I can get, but over here in The Netherlands itd'be very rare for the automated voice to give any reason more detailed than "a defective train" or "disruption on the tracks" and I can't see why beancounters at ScotRail wouldn't have come to the same conclusion.

Research has indicated that commuters get way less annoyed when given a reason for delays.

Of course the dutch railways noticed that it doesn't have to be a good reason, and hence you'll now often hear "We're driving at a modified speed due to a red signal" announced on the trains with some frequency.

An aside: On https://scotrail.datasette.io/scotrail/random_apology I see heavy use of:

    order by random() limit 1

I understand that in more or less all databases this is quite inefficient: it runs `random()` for all rows, sorts those random values and then chooses the row corresponding to the first one. In tables with millions of values this could be rather expensive.

Is there any generic way in SQL to cheaply choose an arbitrary row from a collection in a way that is not deterministic? [1] I’m thinking about something that has the same characteristics as Ruby’s `Array#sample`.

Postgres has TABLESAMPLE but that only works across an entire table so it’s not flexible enough to be useful in real world situations (ie where the collection to choose a random element from is already filtered by WHERE)

Or are there any databases that know how to recognize this `order by random()` idiom and skip the extra work? Kind of like how `count(*)` usually has optimized semantics to not actually look at the distinct values of each row.

[1] I realize that SELECT without an order clause is technically arbitrary but in practice it’s usually pretty deterministic.

In this case my database is 500KB in size so I'm not thinking about query performance at all.

I've run into horrific performance problems with order by random() in larger databases.

I actually suggested the SRANDMEMER feature that was added to Redis as a result of one of those projects! Details here: https://simonwillison.net/2009/Dec/20/crowdsourcing/

One of the reasons a train might be cancelled:

> "Predicted flooding earlier today"

Not much of a prediction, is it?

If the outbound service was cancelled due to predicted flooding, then the return service will also be cancelled, whether the flooding eventuated or not.

Just guessing, but ScotRail might be extra-careful about flooding after a flood-related derailment killed 3 people at Stonehaven[0]. So if they pre-emptively cancel a service anticipating bad flooding that doesn't transpire, they'll likely want to be open with the public - those inconvenienced might be a bit more sympathetic if the know the whole picture.

[0] - https://en.wikipedia.org/wiki/Stonehaven_derailment

That's what I assumed it meant, just thought it was a funny phrasing—not that I can think of a better way of putting it as succinctly.

well sometimes (a lot of the time, actually) a train going from point a to point b relies on a train getting to point a to begin with, and that cancellation of that trip is likely the cause of the cancellation of the trip from a to b. so this makes sense, to me.

The install nothing web-based screwing around with datasette-lite is impressive. And there's a ton of easy to understand great examples of on-the-fly quick hacks that came out of it. Really informative write-ups.

Im quite impressed by the transcribing:

> It took two hours for a collective of anonymous volunteers to dive through all of the files and transcribe every single one of them.

Seemed kind of ad-hoc using a Google Sheet for collab but still impressive to me. There's interesting parcelling up/stating-requesting/finding/doing work metaproblems about that keep seeming well served in specific domains & niches, but with scant progress at higher-level solutions/tools.

A wartime bomb which has now been made safe

Notes: thank god

Now I need someone to FOI TfL for the Tube announcer voices!

I can't help but wonder if ScotRail meant to publish these announcements, or if the file was posted for internal use only, and then somehow leaked. The hacks that are being done with these announcements are cool though.

a freedom of information request resulted in this being released.

aw this is so cool! Is there a way you (or someone) could swap out the sounds w/ the Sydney trains announcements?? The announcements really sound similar...

Just prepend each announcement with Crikey and append it with mate, and you're good!

niceee

Reply on news.ycombinator.com