How to build a handwriting reader from scratch with deep learning | Hacker News Reader

How to build a handwriting reader from scratch with deep learning

nanonets.com

50 pointsarunpgandhi8 comments

Open articleView on HN

Hey guys, here's the updated URL: https://nanonets.com/blog/handwritten-form-ocr-handwriting-r...

Hello, I am the author of this article

OCR is considered a solved problem in general but not in entirety

A key component of it, Handwriting recognition is still a challenging problem.

Handwriting Text Recognition(HTR) is the task of recognizing handwritten human text

It involves using both Computer Vision and NLP

Every person has a different style of handwriting, thus solving the task of HTR is much more difficult than OCR

In this article I cover the progress of various techniques involved in solving HTR and all the SOTA models

In addition, I have discussed the way to train your own HTR model on your own dataset

Happy to discuss more if you have any queries about handwritten text recognition

Thanks for sharing. I'm curious about a couple of things:

- What's your favourite way to preprocess images? Most real world data isn't already segmented into images of complete words, and that preprocessing step could affect the quality of the final output.

- Do you know of any open source models I could use as a starting point to recognise text chunks with some 2D structure, like math equations that span multiple lines?

Why not use the industry standard acronyms? ICR intelligent character recognition is the term for extracting handwritten characters.

Apologies, I see you clarify in the article.

Sort of off-topic, but an interesting angle on handwriting recignition-- The Sony DPT-RP1 digital paper system has a stylus for handwritten notes and highlighting. They do not have handwriting recognition; however, there is a search feature to find occurrences in your document of handwritten characters _star_ and _asterisk_. Personally, I would like to see these two tokens expanded to a customizable set of handwritten search tokens. The Remarkable tablet's SDK was recently discussed [1] here at HN, and this article is inspiring in this regard.

[1]: https://news.ycombinator.com/item?id=24295443

I don't think it's off topic :) This is a really useful and I suspect under-rated feature. There was a similar feature in an older version of the Livescribe digital pen software. You could search for any text string, without having run the handwriting to text function. This was incredibly useful as you could search for terms not in the dictionary (so for example in my lab book I could search for gene names that handwriting to text would routinely scramble).

Sadly they removed it, and I've never seen anything similar again (with the exception of the DTP-RP1 example you mention).

What a great write up! This looks like an amazingly fun topic to work on.

Reply on news.ycombinator.com