I used the PulseAudio simple API only to find out that it didn't work. Only when I changed to the normal API it started working.