markmatsushima

Mic Captions is proudly a wrapper around Apple's on-device technologies in iOS/iPadOS 27.

It transcribes and translates speech on-device, displaying the text in a continuously scrolling view. Tap any word to open its dictionary definition.

On supported devices, Apple Intelligence also summarizes the transcript every five minutes. Transcription, translation, dictionary lookup, and summarization can all work without a network connection.

I originally made this because I wanted to look up unfamiliar words while watching movies and YouTube videos without constantly pausing and switching apps.

It's currently available as a TestFlight beta, and I'd especially appreciate feedback from people using iOS/iPadOS 27.

That is a good idea, and I agree that it may be better not to tell the doctor that the information came from AI.

Doctors seem comfortable with me using medical terms that most patients wouldn’t normally know.

I think the giant leap in AI technology was the Transformer. ChatGPT is based on it, with an incredible chat user interface. In terms of technology, there has been no similarly significant innovation since then. But user interfaces and training data keep improving, which continues to increase our productivity.

Nattly is similar to a children’s book for learning L1. The colorful illustrations help me remember the words.

I couldn’t find the Radio feature. Do you mean the “Play Whole Scene” button?

When we were children, we could ask anyone what a word meant without hesitation. I think this is a big difference when learning a second language.

English is my second language, and when I watch a movie or read a book, I often struggle with words I don’t know.

I made a tool that extracts text from PDFs and translates it into Japanese with an LLM, mainly to help me study English documents.

A tiny frustration can now be enough motivation to build software, because you can just tell an LLM what you want.

In my case, I was frustrated with repeatedly asking chat AIs to “translate the next part” or “continue” when translating long documents. I also wanted almost word-by-word translation rather than completely smooth Japanese, because my purpose was to understand the original English.

Paid software probably has more features and works better, but using software I made for myself is more fun.

Ah, that’s true. I didn’t notice because I changed my Slack settings to disable Enter to send.

When an LLM read a lengthy log and source code and found a bug in seconds, I felt useless and burnt out. It is strange that I feel the same tiredness as when I finish a project that takes a week by hand, even when it is done with an LLM in a day of conversation.