Duck Tales: Voice-mode in Duck.ai. The private, hands-free way to chat with popular AIs (Ep.34)

In this episode, Beah (Chief Product Officer) and Fran (Engineering) discuss voice-mode in Duck.ai, common use cases, and how it’s private.

Disclaimers: (1) The audio, video (above), and transcript (below) are unedited and may contain minor inaccuracies or transcription errors. (2) This website is operated by Substack. This is their privacy policy.

If you have feedback on Duck Tales, or episode ideas, email us at podcast@duckduckgo.com

Beah: Hello, welcome to another episode of Duck Tales, where we go behind the scenes at DuckDuckGo and discuss the stories, technology, and people that help build privacy tools for everyone. In each episode, you’ll hear about our vision, our product updates, engineering, or approach to AI. In this episode, you are going to hear about our voice AI chat feature. And I have with me today Fran, who played a very large role in building and shipping that feature. So Fran, do you want to just briefly introduce yourself?

Fran: Yeah, so hi everyone. I’m Fran. I’m an engineer at DuckDuckGo and I’ve been working on Duck.ai for a long time now and yes, I was heavily involved in building voice chat and yeah, it’s a very exciting feature.

Beah: Awesome. Well, thank you for joining us. And I’m Beah. For anyone who hasn’t already met me on one of these podcasts, I’m on our product team. Before, so I have a bunch of questions for Fran, but before I jump in, I want to make a ask of our listeners, actually. So we’ve been doing these podcasts for a little over six months, I think, now, and have a pretty good audience following on Substack. We would love to hear from you, our listeners, get some feedback about like things that you you like about how we’re approaching this, things maybe you don’t like, and also like what topics you would like us to talk more about so we can plan future episodes with your interests in mind. So if you have any feedback or want to talk to us, send us a note at podcast@duckduckgo.com. and we’ll also put that address in the show notes. So with that said, let’s talk more about voice chat. so Fran, can you just tell me like to get started, what is this feature? How does it work? Maybe you can give us a quick demo.

Fran: Yes, definitely. so voice chat is a hands-free conversation mode inside Duck.ai. so yes, so far our users have been able to talk with any with many chat models via text. And we wanted to provide a privacy focused way for you to just have a conversation with your model. There’s some really cool real-time models out there right now, and we wanted to incorporate this feature on Duck.ai as well. so yeah. With voice mode, you can basically have a call with the model and talk directly to it. It will listen, it will reply out loud. you can also interrupt it mid conversation and like if you want to correct or actually add some extra information, the model will stop and listen to you and then reply back. It can pick up a little bit how are you expressing yourself to also help that play into the response that it will give back to you.

Beah: They have people.

Fran: And so I have here on my phone Duck.ai. And if you want to use it, like we have this very handy voice chat, you can go there and you can just start talking. So can you give me the weather for New York today?

Beah: Yeah, yeah.

Fran: Just new it’s city in New York City, the temperature is around twenty eight degrees Celsius with a heavy thunderstorm. Celsius and the low tonight could drop to twenty-four degrees. There’s a heat advisory in effect as well, so it might feel quite humid. Make sure to stay hydrated and be prepared for possible flooding. You can also check the extended forecast on sites like AccuWeather or the Weather Channel. Yeah, that’s very nice. So yeah, you can ask follow-up questions inside the same session. Right now I’ve muted myself, so you can also mute in case you want the model to not hear you at the moment. And and yeah, it has access to web search as well, so you can also get the news, get weather information or any up to date information.

Beah: Nice. I am not too far from New York City and I can vouch for it being very humid today. cool. Yeah, so t tell me, I know that the demo, you know, I hopefully people can see it well enough f on the screen, but if you couldn’t, there’s like a little ghosty character with a bow tie. tell me a little bit. Who’s that? What’s his deal, his or her deal?

Fran: yes. so that is our voice mascot. we wanted to provide like a visual aid and like represent that the AI is speaking to you so you can just see when the AI is listening and replying back to you. So we created this mascot to be there with you during the session. And we wanted this mascot to also have a little nod to Dax, our technical mascot.

Beah: Yeah.

Fran: so that’s why it has also the green bow tie and it’s white, so it it it’s it’s a nod to Dax. Yeah.

Beah: Yeah, so if you’re not familiar with Dax, that’s the name of the DuckDuckGo duck that you see, the Debonair, yeah. duck with the bow tie. awesome. and by the way, like on the point about the dem demoing this is a little bit challenging because of the audio and the back and forth, but just go like after this podcast or in the middle of this podcast, go try it. it’s you know, if you don’t already have the DuckDuckGo app, you can download the DuckDuckGo app on your phone. You can also try it on desktop.

Fran: Yeah.

Beah: And actually, Fran, I should ask, is it free or do users have to pay to use voice chat?

Fran: It is completely free. so our free tier users can access voice chat directly from either that button or the sidebar as as you could see. And yes, with with the pro and plus subscriptions you do get more time to speak with the model, but it’s available for everyone right now on your phone on or desktop.

Beah: Awesome. cool. So maybe can you tell me a little bit about why we decided to build it at all? Like what instigated this whole project?

Fran: yeah, so like I was saying before, like we there’s some cool voice models out there and we wanted to give like this hands-free way for people to talk with with our with their chat model. and we also wanted to have a private way for them to speak, so we did put in place some technical ways for us to improve like our privacy and anonymizing your voice when we when you’re connecting to the provider to speak with it. and yeah we want to give you this free hands-free way of talking with the model is also better for accessibility like we’ve gotten feedback like some people struggle using the text chat so with this you can just connect and start talking with the model uninterrupted.

Beah: Even the. Yeah. Okay.

Fran: So yeah, it’s been a very, a very cool feature. And we know voice is a very unique thing for the users, so we wanted to provide a way for our users to be able to use that in a private and secure environment.

Beah: Everything. Okay. Yeah, yeah. Can you is there anything else that you wanna add about like how we handled priv like how we made it in fact meaningfully private?

Fran: Yes, definitely. it was challenging because right now there is so many different ways to handle voice models that are out there. Like different providers have different ways that they provide. Like we did this connection directly with the model using real-time web technologies. So you’re actually like having a call with the model. but there are other ways where you could parse the conversation, maybe turn to text, but then a lot of the details and the nuances of your talk can be lost in between. So we found we looked into this balance of having a good user experience while also preserving your privacy. So the way also that we build, like there is this relay in between you and the provider, and we anonymize all the metadata and the information of your call. So the provider doesn’t really know who you are. Although your va your voice is still unique to you. You should be mindful of that. But apart from that, everything else we anonymize during the call. Yeah.

Beah: Hmm. That’s cool. just to like kind of loop back to what you were saying about the why we build it in the first place. I’m personally like I’m not a huge voice input person. I know a lot of people are, but I don’t usually talk to my phone unless I’m actually calling a human. and I found it pretty cool ‘cause like, I don’t know, it is really convenient sometimes just to not I don’t know, they’re like when you’re on the go or something, right? And don’t have your typing fingers available. I’ve I’ve actually found that pretty handy and just been like impressed by how slick it is with the like hand it handling interruption. And I know we had to dial that in a little bit, like at least on my iPhone, it was way too sensitive at first and it thought like all the background noises were interruptions and yeah. But I I think now it’s pretty slick.

Fran: Yeah. No, it’s super cool and we’ve gotten so many positive feedback like from I’ve got people who reach out to me and say like I was stuck in traffic so I wanted to check something real quick so they use voice mode so they could still pay attention around them and like while they were sitting there waiting for traffic to move, they could quickly use and make some lookups. I’ve heard from people that also use it to practice a different language since the model can understand so many different languages. And even me, I’ve tried like speaking in one language and then switching to my native, like speaking English and then switching to my native one, and it can pick up like flawlessly. it’s it’s very nice. Yeah, I I enjoy a lot the the hands-free and versatility of it, and I’ve yeah, I I’ve used for a few things where

Beah: Nee. Yeah.

Fran: Me speaking is easier than me typing down what I want to to ask the model about. Yeah.

Beah: Yeah, I’ve been using it like I guess as like a thought partner. Sometimes if I’m like walking my dog and I’m thinking through something complex, it’s the kind of thing that like if my husband were walking alongside me, I would start just like, you know, spewing thoughts at him as a sounding board. I start spewing it at you know, Duck.ai instead. And I don’t know, it’s been kinda cool.

Fran: Yeah.

Beah: t tell me, Fran, like do you have a either a favorite thing or a favorite use case about about voice chat?

Fran: Yeah, so I I’ve used it a little bit in the past to help me when I need to write a write down like some idea that I have in my mind or like formulate some something that I want to share with a colleague but I really cannot put into words but I can express myself. So I would open voice chat and then like speak for a whole minute saying, I want to say this, but not exactly that word, but so like you’re saying, like this thought process. And then at the end I say, now summarize what I’ve said in two sentences, and it has been like really helpful to help me like consolidate a lot of flying ideas. And with web search that we added after the release, I also find it really nice, like to get, give me a summary of the news in my city today, or do these quick lookups where I can have it on the side speaking to me while also doing multitasking and doing something else. So yeah, it’s it’s been pretty nice for those use cases.

Beah: I mean I could. Nice. Awesome. Well, anything else that you wanna add before we wrap?

Fran: just want to invite everybody to use voice chat. I I’m biased, but I really like this feature. I I find it really cool, really like our ghosty guy. And and yeah, please test it out and share any feedback that you have with us. We are always looking for ways to improve our features and voice chat and Duck.ai in general. So, yes, please hit the share feedback button and we’ll definitely appreciate.

Beah: Does the ghosty guy have a name?

Fran: No, we didn’t give a name. Maybe if if anyone suggests a one that we like, maybe.

Beah: Yeah, I think you should have naming privileges Fran, so think about it.

Fran: Yeah, I’ll think I’ll think about some ideas. Yeah.

Beah: We Yeah, okay. Sounds good. I was gonna suggest Fran, but maybe you don’t want Fran. Yeah. Cool. Okay. Awesome. Well thanks so much for for joining and and congrats on this neat feature that you launched.

Fran: Ghosty Bow tie. Thank you so much.

Beah: All right. Talk to you later, Fran.

Fran: Take it. See you soon.

This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit insideduckduckgo.substack.com