The first time I used ChatGPT, soon after its release, it felt like Star Trek. I asked the computer some questions about things I didn’t know, refined them conversationally, and got clear answers. And then I asked it a question about Swift, something I do know, and it gave me a clear, illustrated answer, with source code snippets and textbook authority, which was flat-out wrong.

In Google’s announcement of Bard, its conversational AI engine, Sundar Pichai proudly showed off a user interaction. In that interaction, the user asked whether piano or guitar were easier to learn. After some vague “some say” type responses, it offered up a very specific answer with textbook-sounding authority: “To get to an intermediate level, it typically takes 3-6 months of regular practice for guitar, and 6-18 months for piano.” That was it. No citations or links whatsoever. Were those numbers from a specific source, or were they synthesized by Bard? Were they based on studies, or blog posts, or music school advertisements? Are these numbers controlled for age, for music style, or literacy? These questions are left entirely up to us to ponder.

Web search, for all its problems, at least conveys clearly that it’s sharing results written by different people, and presents them in their original context. Yes, it’s been gamed with content farms and SEO. Yes, we too often click through the first link and uncritically accept what we find. But at least it puts that in our hands, it makes sources evident and clear.

There’s also the flip side of this, which is that clicking on a search result is often a human connection. Maybe it’s to a blog, a published paper, or someone’s review on Amazon, and maybe there’s a name there. Maybe the author would like to be credited for their work. Maybe they’d like to be able to be contacted or replied to or just given a like or a star. By using their words as training data, you erase all of that. These two problems—the uncited and unwarranted confidence to the searcher and the erasure of the creator—are really the same thing, reflected in two different kinds of harm.

Conversational, natural-language interfaces are worth working on. They could be transformative for computing. And this generation of LLMs have proven to be a massive step toward them. But as currently constructed and promoted, they’re an awful approach to information search.