For me it's a little more subtle. When I sit down to write, say, an email, I usually have a specific point that I want to make but I usually haven't yet figured out exactly how I want to get there. I'm not even sure it is even possible to conceive of writing without doing it word by word. Perhaps the difference though is that we have the ability to go forward and backwards in the sense that we can usually think faster than we can type or write which allows our thoughts to advance beyond the tokens we have produced. This allows for a kind of real time editing process that incorporates both past and possible future information. But I imagine there us probably a way to enhance LLMs to branch and edit before producing the next word materially (if they aren't already doing that).
To go even more meta, there is an analogy I'm trying to make right now in which I am visualizing a road and thinking about how describing the road relates to the process of writing. In my mind's eye, I can see the full length of the road and all of its contours but I can't actually describe the individual stretches of the road coherently without enumerating them. Something similar happens with writing. I can visualize what I want to say far beyond the next word, but it's true that the actual process of writing goes word to word, much like how the process of token selection is described for an LLM. The question is whether the LLM has an analogous conception of where it is going. Going back to the process above, sometimes I know where I am going and haven't yet figured out how to articulate it yet. It is through the process of writing that I am able to articulate that thought. But the thought preceded my articulation of it. I don't know to what extent LLMs have coherent thoughts that they are articulating or if that even makes sense for the type of intelligence they project. My suspicion is that they don't have additional sensory inputs beyond language that give thoughts the immaterial shape that then is expressed in language. Without that, I am skeptical that they will truly get beyond regurgitating and/or remixing what has already been fed to them textually. That doesn't diminish how amazing they are, but I am somewhat more in the Brooks/Knuth camp that they are impressive and surprising, but there is something that ultimately leaves me a bit cold about them.
I usually have a specific point that I want to make but I usually haven't yet figured out exactly how I want to get there. I'm not even sure it is even possible to conceive of writing without doing it word by word. [...] In my mind's eye, I can see the full length of the road and all of its contours but I can't actually describe the individual stretches of the road coherently without enumerating them.
Not to trivialise the interesting point you’re making, but do you never write with an outline? Write bullets for the big points you want to touch, then go back and flesh out the details?
To go even more meta, there is an analogy I'm trying to make right now in which I am visualizing a road and thinking about how describing the road relates to the process of writing. In my mind's eye, I can see the full length of the road and all of its contours but I can't actually describe the individual stretches of the road coherently without enumerating them. Something similar happens with writing. I can visualize what I want to say far beyond the next word, but it's true that the actual process of writing goes word to word, much like how the process of token selection is described for an LLM. The question is whether the LLM has an analogous conception of where it is going. Going back to the process above, sometimes I know where I am going and haven't yet figured out how to articulate it yet. It is through the process of writing that I am able to articulate that thought. But the thought preceded my articulation of it. I don't know to what extent LLMs have coherent thoughts that they are articulating or if that even makes sense for the type of intelligence they project. My suspicion is that they don't have additional sensory inputs beyond language that give thoughts the immaterial shape that then is expressed in language. Without that, I am skeptical that they will truly get beyond regurgitating and/or remixing what has already been fed to them textually. That doesn't diminish how amazing they are, but I am somewhat more in the Brooks/Knuth camp that they are impressive and surprising, but there is something that ultimately leaves me a bit cold about them.