Introduction to Bert

towardsml.com

148 points·seshagiric··10 comments

I was hoping this would be more comprehensive, breaking down the transformer architecture as well. Anyone got a resource like this?

Here you go:

http://www.peterbloem.nl/blog/transformers

Wow, I didn't expect such a detailed write up that is still meant for people without a great overview of all the advancements leading up to this.

I'll certainly start digging into this tonight.

Also: https://nlp.seas.harvard.edu/2018/04/03/attention.html

Thank you, this also seems great although more of a practical resource it seems. I'll go through this after I have a base intuition from the other resource.

You know how in movies the "alien" learned the language and customs from popular media and so the story pokes fun at them for how oddly they behave (and how odd our popular media is). From this article, it seems like NLP people are essentially worried about the same thing happening.

Now I have a vision of Dan Akroyd and Steve Martin -esque computers announcing that "we are... two wild and crrrazy guys!"

That was a movie too -- Short Circuit (1986).

"Wouldn't you like to be a Pepper too?"

A timely post, as December is prime 'Bert' month.

If Bert & Ernie are only Sesame Street characters to you, please watch "It's a Wonderful Life" (carefully) this holiday season. I think that's where Sesame Street got the names.

Interesting, but a little Googling tells me that the writers deny that. It's just a coincidence.

Perhaps Jim Henson subconsciously remembered them though.

I went there expecting something about Binary Erlang Term (BERT) [1]. It's unfortunate the acronym clashes, though with so many acronyms it is bound to happen. Interesting article though.

[1] http://bert-rpc.org/

Reply on news.ycombinator.com