Duck Tales: Building useful, private, optional AI directly into search, with Search Assist (Ep.14)

In this episode, Gabriel (Founder) and Ewa (Product) discuss Search Assist, why we’re so focused on letting users control their experience, and future improvements.

Disclaimers: (1) The audio, video (above), and transcript (below) are unedited and may contain minor inaccuracies or transcription errors. (2) This website is operated by Substack. This is their privacy policy.

Show notes: Learn more about the “More” button in Search Assist here.

Gabriel Hello everybody, welcome to DuckTales. I’m Gabriel, founder of DuckDuckGo. With me is Ewa. You wanna introduce yourself, Ewa?

Ewa Hi everyone, I’m Ewa Sobula. I’m a product person at DuckDuckGo based in Poland.

Gabriel Cool, we are going to talk about search assist today. We’ve done one episode before when we introduced the more button. This is the stuff on the top of search results where our anonymous AI is answering queries for you and you can click more, but we’re gonna go take a step back and just kind of talk about the feature in general. ⁓ As a precursor, I’ve said this a bunch of times. at AI episodes, but ⁓ our guiding principles for AI features are that they’re useful, private, and optional. in this case, and I know people really appreciate that, ⁓ we think Search Assist is extremely useful, and we’ll get into that, but it is also optional. So you can turn it off if you like. It’s really easy to do so. There’s toggles actually within Search Assist itself, but also in the browser and search settings. And of course, it’s private like the rest of our search results. It’s completely anonymous. ⁓ So with that, yeah, let’s just jump from the highest level. What is Search Assist and how does it work?

Ewa So as you mentioned, search assist is our AI generated instant answer to search queries that we show up on top of search results page. Once we are confident that this is gonna be the optimal answer to use a query, meaning for queries where you either ask a question or really are looking for a quick summary. And ⁓ we are now showing it on like roughly quarter of our searches and we are using LLMs to create the answer but what is important is that we actually grounded in the right sources and like verified and checked sources so it’s not like generated literally and just by an LLM but we find the relevant sources to the search query. we analyze them and we synthesize the concise one, two sentences also that we show on top of SERP when it’s really relevant. Or we also show like lower down the page where you might have different intent. ⁓ But still it could be useful if it’s something that you scroll down to.

Gabriel Yeah, absolutely. ⁓ And I want to get into the, you mentioned it being concise and that’s one of the main differences I feel we have with Google and I want to get into a bunch of those. ⁓ before we do that, let’s continue with the basics. I so I actually started working on this feature. Maybe people think it was in reaction to Google or something, but it wasn’t. We actually started working on this as soon as Chat TV came out. ⁓ to really initially focus on Wikipedia and helping, know, giving people better Wikipedia answers, which you mentioned, search exists now appears in about 25 % of searches, Wikipedia appears around about 10%. So, and we had gotten lots of feedback over the years that, hey, it’d be great, you’re showing me the beginning of Wikipedia, but it’d be great if you could just show me the answer from Wikipedia. And we had tried that in different ways and we accomplished that somewhat, but until LMS came along where we could really pull back the paragraphs of Wikipedia and ask the LMP to pull out the answer within that paragraph, which is much better than just showing the paragraph and making you find it. We weren’t really able to unlock search assist. So yeah, we started working on it right when ChatGPD came out and kind of rushed to Wikipedia. And then I know you got involved later. when we started adding lots of other sources. And as you mentioned, we’re trying to use the best sources we can. ⁓ But I’d say more broadly, given that it’s kind of a broader thing than Wikipedia now, what do you see the problem that it’s solving in search results, just kind of for at large? Like you mentioned, sometimes you put it on top, sometimes you put it on the bottom. Obviously, that’s a choice. we’re putting on top because we think it’s solving a search problem,

Ewa So think the key problem is that it’s short that we’re solving with assist is that it shortens the path from when you know what you want to ask and you formulate a query and to actually finding what you’re looking for. And to your point, we’ve already been doing Wikipedia or other modules in the past and we’re still doing them for many of the searches. But Assist allows us to cover more of these informational queries, including the long tail ones, meaning people use different language to ask Search Engine about what they’re looking for. And with Assist, we’ve been able to understand more of these natural language queries or queries that really ever are asked only once to a Search Engine, which is a huge portion of search queries. And, but we still can understand them and can present an answer that is like good enough to answer what you’re really looking for. But also with the more button that you’ve already mentioned also allows you to dig deeper and get more information on demand while still keeping you in the search engine context. In the context that a lot of people are familiar with because we’ve been using it for years, years, some of us ever since they were born. And so it’s kind of like bridges the gap between the value that LLMs bring and how they can enrich the experience of finding information ⁓ without having to move to a totally different user interface, to move to more like conversational chat experiences. It’s still search results that are familiar. It’s the search results that these answers are grounded for. but we’re making use of this technology to present it in a more suitable way for larger volume of different types people ask search engine.

Gabriel Yeah, and so maybe I summarize that way. It really is saving people time. And I think as a primary benefit, I think as a secondary benefit in aligning with our vision of Raise a Standout Trust Online is that we’re trying to ⁓ understand what is the best information in the search results and surface that for you in a concise way. ⁓ So that not only saves you time, but it on average should be giving you better information higher up on the page, ⁓ which is kind of really what you want in a search engine. And just to restate for people who really don’t want AI, you can turn it off. However, this is not to your point earlier, AI making up the answer. This is us grounding the answer on actual search pages that were crawling in real time to look for that answer for you. And then the sources are ⁓ annotated there, which you can click through ⁓ and both check and read more information because we’re only giving you a concise summary. So if you want more, you click through. ⁓ So with that in mind, how you mentioned the more button in the UX before we kind of dive into kind of differences. ⁓ What is the general user reaction been like over time with search assist?

Ewa ⁓ So we’ve been getting a lot of really positive reactions from our users. Assist has been like one of the highest rated parts of search experience historically at DuckDuckGo. And ⁓ I think what people usually appreciate is both that it saves time, it gives this concise answer. The fact that it’s really concise and it’s not like taking over your search experience is just there when you need it but still doesn’t make it hard to get to organics if that’s what you’re looking for. It’s another thing that people have appreciated. ability to drill down to sources, as you’ve mentioned, is also something we’ve heard been ⁓ getting like really good reception. And we are using the feedback we’re getting from people a lot in improving assist. On one hand, that’s because we really don’t track our users. We have very little information about how people interact with our search results. So we really rely on when people make effort to click thumbs up, click thumbs down, leave some additional comment. We use this information both in automated way to improve our answers and also we really do read through them and take lessons and figure out how we can continue improving assist, which is for instance how we’ve gotten the more button.

Gabriel Yeah, I mean, so that seems like a big difference from Google right there for what it’s worth. I mean, I guess I’m not inside Google, so I don’t know, but from reading comments on Hacker News and other places, it does not necessarily seem like they read every piece of feedback, ⁓ but we actually do. And so ⁓ that really is a distinction. ⁓ We mentioned some others too. ⁓ So I mentioned that it was optional. I think we should clarify now that it’s not just you can turn it on or off, which you can. But you can also change the frequency of when search assist appears. We have often and sometimes, as well as on demand. So you can basically make it so it doesn’t show up automatically, but if you want to click on it, you can still click the search assist button on the underneath the search box and it’ll show just for when you want it on demand. And if you really, really don’t want it, including not even seeing that button on the page, you can ⁓ get rid of that. So that’s another one.

Ewa Yes.

Gabriel Another one that you mentioned, so the conciseness, I think, you know, does a couple things. One, it means that the less information there is, the less kind of surface area there is for making stuff up or getting stuff wrong. ⁓ But also, what you had just mentioned, it just takes up less space on the page. So I think some complaints that people have about Google’s A.O. overviews is they really are just taking up. You can’t see the organic links, they’re taking up too much of the page. We’re really keeping it ⁓ tight. And the way we’re doing that is we’re kind of forcing it to be small, but also to your point, we have the more button there as an option, but we’re not starting out with that long explanation. We’re starting out with this ⁓ concise explanation. Is there anything else you want to highlight in terms of kind of differences between us and Google? ⁓ If not, I can probably come up with something else.

Ewa Maybe like building on top of what you just talked about is I ⁓ think what what I can tell about our philosophy of building assist which I think also applies to the whole of search is that We like to give control to our to our users. We trust that they know they can own their experience, which is why we’re giving you search results, which some of it is assist. You get the organics, you get other modules, none of these experiences is like overly dominating the search results page. And you can kind of like choose your own adventure, which I think is... Again, I’m not in Google. I don’t know exactly how decisions are made in there, but from my observation is like different approach to how we give control versus deciding for the user what they should be seeing when they search for a specific topic query.

Gabriel Yeah, that’s a great point. I mean, I would even build on that, which is we’ve also been giving control to publishers. There are some publishers who don’t ⁓ want to be part of AI results. ⁓ We are trying our best to make the click through rate actually pretty decent because we have a short summary. And so really, if you want more, you click through and the links are prominent. Like on Google, sometimes they’re hidden actually behind the click and on DuckDuckGo they’re not. More fundamentally in that if you do want to opt out, we have a kind of way for publishers to just opt out of the AI summaries ⁓ via robots.txt ⁓ and still appear totally fine in search results. Whereas Google has been trying to bundle those things together and basically force you to opt out of Google altogether. ⁓ So that’s another decently big difference in line with control. ⁓ OK, so yeah, what’s What’s next? What are the negative feedback complaints we’re getting? And what are we working on? Where are we going?

Ewa Yes.

Ewa So ⁓ one of the things that we continue working on and we will... is that we want to keep the high bar for answers quality. There are still cases where we get the answer wrong, it’s not incorrect or maybe the sources weren’t right or we misinterpret the query. The percent of these incorrect answers is really low but we, as I said, we want to really take correctness seriously. So we’re working to add even more loops, both with user feedback, observing the queries, especially the fresh ones, like trending topics. This is where information changes really often and we want assist to always give you the fresh and correct answer. So definitely answer quality is something that we will continue working on. ⁓ We also are moving to other languages besides English, as it has been available in English for a while already. That hasn’t been true for other languages, so it’s already available in Italy for some queries. We’re only rolling it down. You’ll start seeing it rather further down the page before we build enough confidence to show it on top of your results, but you can already start seeing it there. And we plan to roll out to ⁓ a few next languages which are gonna be Spanish, Dutch, French, German and we won’t stop there but this is like the current focus which ⁓ I’m personally excited about because yeah it’s...

Gabriel sure a lot of team members are. have, I mean, this is for InterObject.co, we have people, I think, across like 30-something countries now, and a lot of them are not English as their first language. And I think they would prefer to have non-English ⁓ search assist answers. So that’s exciting.

Ewa Absolutely, but also our users. I think we know that people all over the world are using DuckDuckGo search and I personally can’t wait to have them use assist and also get benefits of these short and concise answers. And we’re also looking into improving and expanding or continue improving the user experience of assist. We’ve been already mentioning the more button a few times. We know that there are sometimes different expectations people have when clicking more buttons. Sometimes it’s really you want to have the longer version of the answer, but sometimes it’s more like you want to do more information or additional thing that wasn’t mentioned, the concise answer. Or maybe you want to drill down, more like learn about related things. So we’re now looking into exploring these options and making sure that expand answers are really helpful depending on what you’re looking for.

Gabriel Cool, I know we’re also always trying to speed things up. ⁓ reducing latency, getting answered faster, it’s always on the list. ⁓ Yeah, I guess the only thing else comes to mind is, you know, we’ve been, it might be worth pointing out is, which we mentioned in the past, that this actual ⁓ technology stack is completely independent from being in Microsoft. ⁓ You know, we’ve been kind of, We’re obviously working with the LM partner. We’re not building a foundational model ourselves, but the stack, ⁓ you know, is coming via our own crawling, ⁓ the web pages where we’re generating the answers. And that index is improving every day. And so we’re working on that as well. ⁓ And ultimately we hope to say more about that. ⁓ I think that’s it. I mean, the, ⁓ Anything else you want to add or do we hit everything for the basics?

Ewa we hit everything that maybe one thing which is interesting to me, I don’t know if it’s going to be interesting to others, but ⁓ I think it’s also worth mentioning how we determine, for instance, answer correctness and how we’ve been actually using a lot of manual reviews of queries and answers within the team, which ⁓ I think is not obvious. Some people might assume that if you’re using AI generated also you also delegate to the AI to determine if it’s correct or not or if it includes enough details or not too much details. So the way we’ve been building assist is that we’ve done a lot of that manual judgment ourselves and we rely on these manually reviewed datasets. I’m pretty sure quite a few people in the team ⁓ might complain a lot about how tiresome and ⁓ maybe you it’s It’s not the most exciting work while you do it, but I think it’s extremely useful in really ensuring that these answers, when we say our correctness rate is high, this doesn’t mean that someone or AI tells us we’re doing a good job. It’s really, we’ve reviewed a huge, huge, huge data sets of queries and we retest them over time to ensure that this quality stays high.

Gabriel Yeah ⁓

Gabriel That’s good, that’s a way to end. I guess I have been forgetting to say, if you’re listening to this and you do have follow-up questions, email us. You can reply to the email that you get if you subscribe to us on Substack. Otherwise, we probably need to develop an email address that other people can send to us. So I’ll have to get back on that. But if you subscribe to the Substack, you can just reply to that email and it’ll come to us and we’ll get it for future episodes. So thank you everyone for coming on and thanks everyone for tuning in.

Ewa Thanks, Gabriel. Thanks, everyone.

This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit insideduckduckgo.substack.com