Voice Coder – Alexa skill to code by voice | Hacker News Reader

Voice Coder – Alexa skill to code by voice

voicecoder.net

Really cool! Perhaps you could also take a look at what we are building at https://snips.ai (disclaimer: I'm a co-founder) to make a 100% on-device and private-by-design Voice AI which runs even on a RaspberryPi 3

it would be awesome to do this on a raspi, without requiring internet connection!

Great idea. I really like idea of not requiring an internet connection. Self contained projects are more reliable and portable. Also this project is aimed towards students so privacy is a real concern.

The way I built it should be easy to use another voice ai input. The Alexa module of the code just passes the commands to the Core logic. It's all built in node.js.

I built this for Alexa Skills Challenge: Tech for Good. https://devpost.com/software/code-by-voice . More details about how it was built can be found there.

https://mycroft.ai/ is a very popular assistant too.

(No affiliation)

This looks neat!

Is it possible to get started without creating an account on your website? I couldn't figure out how, but I figured there must be a way if it's "100% on-device".

Hi spacehome, you need to create an account in order to use the webconsole to build the assistant (create intents and queries), then once this is done you can use it 100% on-device!

You guys are are also incentivizing for app work done via the console / added to platform right? For a serious dev with experience in app deployment what could I look forward to?

Most of my background is in data security so this is definitely up my alley.

You do not create an account on the website. Right now the only way is to open the Alexa skill and it will give you a 4 digit code for your game. After you open your game on the website you can continue adding rules to your game via Alexa. New game logic will update in the game in real time.

I would like to make a version 100% on device. A few have suggest how.

Thanks, but I was really asking my question about Snips.

This would be so awesome! I'm wondering how Snips.ai functions with no internet connection? I'm an early stage programmer with not much RaspPi experience unfortunately. Is it just a closed circuit for home automation. How does it fetch data for tasks that need external servers to update?

I met you at maker fair! I was impressed by the demo and how well it could recognize my voice even when I was far away or facing hard surfaces.

See https://talonvoice.com for an extensible open source alternative to voice code.

The creator/main developer is a friend of mine and is one of the smartest people I know. The project can be used to control IDA for reverse engineering, play video games, program, and more.

Thanks for the plug! Couple notes: Talon is free, but there are closed source bits, and this Alexa project is "voice coder", which is different than the paid and abandoned voicecode project.

The difference between this and Talon, is Talon can be used for real programming and uses very low latency offline recognition, while voice coder looks like a guided tutorial and the user experience is probably a fairly high latency single command call/response based on my previous Alexa experience. I'm not sure Alexa can recognize a streaming series of commands without large recognition pauses between each command.

Talon seems impressive.

https://youtu.be/ddFI63dgpaI

How is its accuracy for daily use? How about random navigation within a file, for example.

Does it have an active community?

Correct. Voice Coder is a guided tutorial for beginners, not a replacement for developing production code. It does allow for multi step answers, but is a high latency back and forth dialog with a smaller set of commands. The goal is to be extremely human friendly, not productive, to better understand the basics of coding.

If you are interested in state-of-the-art voice coding, I recommend this video: https://www.youtube.com/watch?v=YRyYIIFKsdU

Silvius isn't really state of the art. It's just newish. On their website it even says the project that inspired them has better performance. It references Aenea (which is a convoluted system of using Dragon on Linux in a VM) and calls out to Tavis Rudd's 5-year-old presentation and Dragonfly, none of which are state of the art.

Silvius is unique in being the only project I know of to use Kaldi, which is very cool because it's open-source and doesn't require a GPU, but the recognition accuracy of Silvius isn't great and the voice model is fixed and has no feedback from the grammar.

I have that and several other resources on a Github page:

https://github.com/melling/ErgonomicNotes/blob/master/progra...

Seems like the skill needs a lot of semantic knowledge. This is less coding than a meta game.

The game template supplies the available events/action/value. The Game Creator guides you through the available options supplied by the template. It then tries to use a human speech like interface to receive the input, the game logic. Currently there is only one game template. Each new game template would need to supply it's own possible events/actions/values to suit it's needs.

A game template looks something like this: https://gist.github.com/jamesfdickinson/65b9f5a05f4902468c46...

Is this the template for the example? If so where does the logic for columns of boxes and stars come from?

This game template was design for students with zero experience. So the create new objects command was omitted in the user defined rules. The game itself controls the box placement for this game. The goal of that game was to learn about events and actions.

I plan to add more game templates with the create object command for more advanced users. You can set it's properties, game physic values and so on. The game itself will need to implement and execute that command. The game template just tells the Game Creator what that current game supports. So the magic happens in the game code. Hit view source on the game if want to see it.

The project was built in a 2 weeks; server, Alexa integration, mini-game, and game creator logic. If the project gets any traction, I'd love to keep working on it.

This is more like changing exposed parameters in Unity during gameplay for tweaking.

Mostly yes. The difference is rather than changing values, you are creating event handlers, actions, and their values. You can only trigger actions on events the game already has and execute actions the game supports which is defined in the game template. With that said, there is no reason why it couldn't support all the events and actions from the core game engine it uses. For this release the focus was for beginners and the options were minimized.

I ask it for my code it and Alexa says, oddly and quickly, "0irk.". That code is not accepted.

I tested your code and it works for me. Make sure you type 0IRK and nothing else like spaces or periods. or click this link https://voicecoder.net/game.html?id=0IRK

What browser and device are you using?

I guess the problem was that it said "dot", so I included a period. It should probably not say that. It works now, thanks!

Thank you for the feedback, I'll fix that.

What’s the IP status of recordings sent to Amazon? Might that affect the IP of the code produced?

The project is designed to help students to understand the basic of coding logic using their voice. Think of it more like a coding lesson for beginners. Many at this level use drag and drop coding. Code.org has many drag and drop coding lessons. The difference here is it uses voice. The recordings are not part of the code produced, just plain text. This version cannot make production like code, just add game logic to an already created game template.

Headline should be "Voice Configurator -- Alexa skill to configure by voice."

Me: "Alexa, open Voice Coder"

Alexa: "Sorry, I don't know that"

go to https://www.amazon.com/dp/B07HFWQPKN and enable the skill.

Ah, that's why. It's not available in the UK.

I put it in the catagory kids education. Alexa only supports the kids section in the US right now. I may change that so it can be accessed worldwide.

I have personally been playing with Snips. It has been an exciting project to work with. I do need some help with certain things, but reddit has been a good source of information.

Hi i'm not quite skilled in coding but is the barrier of entry high? Would someone like me with not a lot of knowledge be able to pick this up and start programming?

Nothing in life is impossible. But unless you have some sort of physical disability that prevents you from using a keyboard and monitor, you are much better off learning programming using standard tools that have tons of noob-friendly documentation, rather than something experimental like this.

Well it's simply an alternative to entering code via the keyboard. If you can't code using a keyboard already this won't help in that regard.

I think this approach will need a great more deal of maturity before I start programming in the way demonstrated in the video.

I guess things have moved on from this Perl-Vista nightmare, I was personally looking forward to having all the coders around me screaming at their desktops in a fit of frustration and rage.

https://www.youtube.com/watch?v=MzJ0CytAsec

That’s 11 years old.

It was a joke, relax.

This isn’t Reddit

Reply on news.ycombinator.com