Built a voice bot that can search the web by breaking down queries, extracting and highlighting relevant details. Here it is searching across reddit reviews to extracts pros and cons of a product
Built an AI voicebot that can highlight relevant details on a webpage when answering queries and maintain a running summary with citations across webpages. This can be useful to gather details while comparing products across webpages.
Built an AI voice bot that can guide through PDFs. Here, it is guiding through an insurance policy document by highlighting specific details with simple explanations and updating a running summary with citations
Created a Github bot that can review PRs to catch bugs and security issues. It has been useful as a quick first check for issues before diving into architecture changes. Also, discovered while building this that current LLM eval tools suck.
Open sourcing code for sharing screen with vision language models like gpt-4, Llava and Bakllava.
Aim's to develop open agents that can specialize and assist with coding and using complex tools. You can try the demo here - https://vision-agent.synth.co.in
Will be posting updates here - https://twitter.com/suneel_matham
Iteration 2 of screensharing with gpt-4. Reduced latency, converted the cube into a sphere in blender.
Also, it can summarize and take notes for later use.
Can check this thread for updates - https://twitter.com/suneel_matham/status/1724762783697498265
Yeah, even RL tuning can be applied to improve reasoning paths for specialized applications, something like RLAIF by anthropic but for specialized agents
Yeah, elevenlabs supports input text streaming but in practice it seems to wait for quite a while before it starts streaming. Should look into ways to make TTS much more instant
Created a script to share screen with gpt 4 and asking it to guide through blender. Latency could be better if openai tts api supported a streaming text input