Gemma 4 E2B, E4B and 12B unified accept audio - here's a recipe using MLX that can use it for transcription: https://simonwillison.net/2026/Apr/12/mlx-audio/

Only up to 30s though, and the larger 26B A4B and 31B models are text and image only.