I’ve seen some talk recently about the virtues of hallucination, implying that it’s a “good” thing because it’s “creative” to make things up or simply what LMs are “trained to do.” I see my own work sometimes come up in these discussions since I’ve written a lot on creativity and open-endedness. So I want to give my two cents: An important aspect of creativity is the ability to know what already exists, which is how you can push towards novelty. If you are confused about what is real and not real, then it’s hard to know what’s novel. So the less-discussed flipside of hallucination is the tendency to present old ideas as novel, fundamentally misunderstanding what it means to be creative. And indeed we see LMs often do just that when pushed to exhibit genuine innovation. Ask e.g. for innovative new recipes, ideas for new genres of music, or inventions to help with some existing problem, and often what you get back is something that either already exists or already has been proposed! (Note I am not saying that’s always what you get back, but neither do you always get hallucinations when asking questions of fact. But it happens too often.) So false creativity and hallucination are two sides of the same coin. If you have one, you would tend to expect to see the other. In this sense, these are not good things, and the argument that hallucination is an asset “because creativity” is a misunderstanding. A firm grasp of reality is an ally of both authority and creativity, and indeed human authorities on subjects are also often creative. But this point is not just to be cynical: it means that if we genuinely “solve” hallucination, creativity will come along for the ride. It also surfaces an interesting hypothesis about which approaches to solving hallucination are most deeply illuminating for progress in the field: if a “solution” only addresses one side of this two-sided coin (e.g. lowers hallucination without increasing creative quality) it is likely ultimately relatively superficial and not the genuinely transformational path that will lead to something spectacular in the field.