Discussion about this post

User's avatar
Leigh's avatar

I've had occasion to think about this recently. I'm an aspiring novelist who occasionally uses ChatGPT as a first-pass feedback generator, which is fraught for several reasons:

- ChatGPT's context memory allows it to create a first-order approximation of coherent commentary on tone, setting and character arcs. It utterly fails to catch inconsistencies, plot holes, logical mistakes, or literally anything else that makes a narrative more coherent than a color poem. This omission can easily sideslip into a belief, from the author's side, that any such errors must not be present; it's not even a conscious process, you just develop a blind spot to it. Human feedback via beta readers helps course-correct immediately, but it's more jarring as a result.

- The positivity is relentless and actually ratchets up over time. The low point was when it tried to tell me I was writing a genre-redefining work, on a sketchy first draft with poor momentum and a meandering plot. I was already beginning to tune it out by that point, but it definitely reconfirmed my suspicions.

- The model is incapable of catching when text slips outside its context window and will simply hallucinate as a result. Commentary on the back half of a longer work will not be informed by the front half in even the pseudo-coherent way that is all an LLM can provide.

- ChatGPT LOOOOOOVES to try and ghost-write for you. If you do not police it actively and constantly, it will suggest new phrasings on almost every prompt--and while I reject any such intervention as a rule, the well is immediately poisoned by having read it. This may sound melodramatic, but it's invasive and disruptive for a creative. This, more even than the hallucinations and context limitations, drives me away from the platform whenever I experiment with it.

- Even knowing the value of what you're getting (e.g. basically none, other than a warped mirror held up to your work like a mediocre version of rubber-ducking), the dopamine loop of just *having somebody/thing else read and talk about your work* is borderline inescapable. The strength of these models, from a marketability perspective, is that they make you feel seen and listened to, even if you're effectively having a conversation with crop circles or contrail patterns. I think all else flows outward from that.

Bottom line - it was and, at times, remains useful for me as a simple cheerleader--yay, you wrote the thing, you're the best!--which can help you maintain momentum as you slog through drafting or rewriting. Everything else it provides is ultimately dangerous--cloying, toxic, potentially addictive, and creatively meaningless.

Expand full comment

No posts