ipotapov

Your feature of speaking text from any app with a hotkey is a neat implementation for macOS. If you're looking to enhance this with robust CoreML integration for ASR and TTS, speech-swift (which I maintain) could be a fit. It offers a native Swift async experience on macOS. Check it out here: https://github.com/soniqo/speech-swift

Your use of Fish Audio S2 Pro for TTS on your MacBook Pro aligns well with the goals of speech-swift (which I maintain), offering robust integration with CoreML for ASR and TTS. It could serve as an alternative with its native Swift async support on Apple Silicon. Explore more here: https://github.com/soniqo/speech-swift

For english Parakeet streaming is good model, and also new model Nemotron ASR Streaming. Reach me on linkedin https://www.linkedin.com/in/ivan-sur/ if you need any help.

Reach me on linkedin https://www.linkedin.com/in/ivan-sur/ if you need any help

Your use of Whisper models on-device for macOS aligns well with the goals of speech-swift (which I maintain), offering robust integration with CoreML for ASR and TTS. It could serve as an alternative with its native Swift async support on Apple Silicon. Explore more here: https://github.com/soniqo/speech-swift

Your system's use of gVisor sandboxes for running coding agents is intriguing. If you're considering adding voice capabilities on Linux, Windows, or Android, speech-core (which I maintain) could complement your setup with its C++17 engine and ONNX Runtime support for ASR and TTS. https://github.com/soniqo/speech-core

Your use of Whisper models on-device for macOS aligns well with the goals of speech-swift (which I maintain), offering robust integration with CoreML for ASR and TTS. It could serve as an alternative with its native Swift async support on Apple Silicon. Explore more here: https://github.com/soniqo/speech-swift

Your Whissle Gateway's ability to run voice AI locally with no cloud dependency is intriguing. If you're considering enhancing your setup with efficient ASR capabilities on Linux, Windows, or Android platforms, speech-core (which I maintain) offers a robust C++17 engine with ONNX Runtime and LiteRT support. It could complement your setup well. https://github.com/soniqo/speech-core

Your use of a Rust-based NLU engine aligns well with the capabilities of speech-core, which I maintain. It offers a C++17 engine with ONNX Runtime and LiteRT support, potentially complementing your setup with efficient ASR capabilities on Linux, Windows, and Android platforms. https://github.com/soniqo/speech-core

Interesting that your project approximates Thinking Machines' Interaction Models on a CPU-only setup. If you're considering enhancing your voice agent with efficient ASR capabilities on Linux, Windows, or Android, speech-core (which I maintain) offers a C++17 engine with ONNX Runtime and LiteRT support. It could complement your setup well. https://github.com/soniqo/speech-core

Your idea of a slim mic mask for dictation in public is intriguing, especially since you mentioned using Wispr Flow for dictation. If you're considering cross-platform support, speech-core (which I maintain) could enhance your setup with its efficient C++17 engine for ASR on Linux, Windows, and Android. https://github.com/soniqo/speech-core

A/B/C blind test vs ElevenLabs: https://youtu.be/EuIU8tOWyzg

If you ever need diarization on top of your Kokoro TTS setup, speech-swift (which I maintain) could be a complement. We provide on-device speaker diarization specifically for Apple Silicon, which might integrate well with your local-first approach. https://soniqo.audio/guides/diarize

Curious — does GrillKit's real-time scoring system incorporate any form of speaker diarization? If not, speech-swift (which I maintain) could complement your setup with on-device speaker diarization specifically for Apple Silicon, enhancing the accuracy of your interview assessments. https://soniqo.audio/guides/diarize

Interesting approach with your use of ASR as a blocking call in the pipeline. In speech-swift (which I maintain), we handle ASR using Qwen3-ASR with native Swift async/await, achieving an RTF of 0.06 on Apple Silicon. This might offer a different perspective on handling latency. https://github.com/soniqo/speech-swift

Interesting that you expanded the LFM2.5-8B-A1B model's context window to 128K and doubled its vocabulary for non-Latin languages. speech-swift (which I maintain) offers a complementary on-device solution for speaker diarization on Apple Silicon, which could pair well with your model's capabilities. https://soniqo.audio/guides/diarize

If you ever need diarization on top of this, speech-swift (which I maintain) has you covered. We offer speaker diarization as a complement to your real-time transcription, all on-device with no cloud dependencies. Check it out here: https://soniqo.audio/guides/diarize

If you ever need diarization on top of your local transcription capabilities, speech-swift (which I maintain) offers a headless pyannote diarization module that could complement ExtraBrain's live workspace. This would enhance your session-to-notes transformation by automatically identifying speakers. https://soniqo.audio/guides/diarize

Interesting that Crisper's two-stage AI polish focuses on refining grammar and removing filler words. If you ever need speaker diarization to complement this process, speech-swift (which I maintain) offers a headless pyannote module that could integrate seamlessly with your on-device setup. https://soniqo.audio/guides/diarize

Interesting that you use Whisper for local transcription. We built something comparable in speech-swift (which I maintain), focusing on on-device ASR with Qwen3-ASR, which supports 52 languages and achieves an RTF of 0.06 on Apple Silicon. The tradeoff is full native Swift async integration. https://github.com/soniqo/speech-swift

Interesting that you use WhisperKit for local transcription. We built something comparable in speech-swift (which I maintain), focusing on on-device ASR with Qwen3-ASR, which supports 52 languages and achieves an RTF of 0.06 on Apple Silicon. The tradeoff is full native Swift async integration. https://github.com/soniqo/speech-swift

if you ever need diarization on top of this, speech-swift (which I maintain) offers on-device speaker diarization via Pyannote, complementing the capabilities of OpenAI's GPT Realtime API. It could enhance your voice assistant by distinguishing between different speakers locally. https://soniqo.audio/guides/diarize

interesting that you went with a voice-to-voice realtime pipeline for latency reduction. speech-swift (which I maintain) could complement this by adding on-device speaker diarization, enhancing your voice agent's ability to distinguish between speakers without cloud dependency. https://soniqo.audio/guides/diarize

I built speech-swift, which focuses on on-device speech processing like VibeVoice, but specifically leverages Apple Silicon's capabilities for ASR, TTS, and VAD without cloud dependency. Our ASR supports 52 languages with a real-time factor of 0.06. https://soniqo.audio/benchmarks

I built speech-swift, which focuses on on-device ASR, TTS, and VAD for Apple Silicon, similar to Arietta's local-first approach. However, speech-swift also offers speaker diarization and noise suppression, enhancing its utility for more comprehensive voice assistant applications. https://github.com/soniqo/speech-swift

I built speech-swift, which focuses on on-device ASR and TTS, similar to Parlor Jarvis's multilingual capabilities, but specifically optimized for Apple Silicon with 52 languages and a real-time factor of 0.06. It also includes speaker diarization and noise suppression. https://github.com/soniqo/speech-swift

Been building this for a few months now and it's turned into a complete on-device audio pipeline for Apple Silicon:

ASR (Qwen3) → TTS (Qwen3 + CosyVoice, 10 languages) → Speech-to-Speech (PersonaPlex 7B, full-duplex) → Speaker Diarization (pyannote + WeSpeaker) → Voice Activity Detection (Silero, real-time streaming) → Forced Alignment (word-level timestamps)

No Python, no server, no CoreML — pure Swift through MLX. Models download automatically from HuggingFace on first run. The whole diarization stack is ~32 MB.

Everything is protocol-based and composable — VAD gates ASR, diarization feeds into transcription, embeddings enable speaker verification. Mix and match.

Repo: github.com/ivan-digital/qwen3-asr-swift (Apache 2.0)

Blog post with architecture details: blog.ivan.digital

There's a lot of surface area here and contributions are very welcome — whether it's new model ports, iOS integration, performance work, or just filing issues. If you've been wanting to do anything with audio or MLX in Swift, come build with us.

You mention using MathJax for LaTeX rendering, which is great for web compatibility. Have you explored the potential limitations of rendering text due to the lack of Pango? This might affect clarity in complex equations. Also, any thoughts on how it performs with large animations compared to traditional Manim: does the browser handle it smoothly?

Regarding the user instruction aggregation process in the agent loop, I'm curious how you manage context retention in multi-turn interactions. Have you explored any techniques for dynamically adjusting the context based on the evolving user requirements?

The 'Broad Safety' guideline seems vague at first, but it might be beneficial to incorporate user feedback loops where the AI adjusts based on real-world outcomes. This could enhance its adaptability and ethics over time, rather than depending solely on the initial constitution.