I wonder if you have come across one of Roald Dahl's classic short stories in this vein, conveniently online (and probably in GPT-4's training data at some memetic level):
Neal Stephenson plays with some of the same ideas as Vonnegut touches on here in The Diamond Age, although he focuses a little more on the pedagogy of a Universal Storyteller compared to its mechanical execution.
Oh wow, that's fascinating. I'm surprised this is the first time I've seen the Dahl story come up since LLMs came about. Amazing how well he predicted the fallout.
Very cool! Wonder if there are stories where the resulting graph is counterintuitive. What happens if you put in James Joyce's Ulysses or Finnegans Wake? Or the Mahabharata?
Exceptional job. I love that you're focusing on how to use the API in a structured pipeline, rather than manually input free-form prompt engineering. More of this please!
This seems like a primary weakness in the study to me. I would be really interested to see this repeated with better controls in place there. Like, that's probably why Don Quixote seems so chaotic compared to Romeo and Juliet. It's way longer so you have way more data points, there's no way to compare them.
I wonder if you have come across one of Roald Dahl's classic short stories in this vein, conveniently online (and probably in GPT-4's training data at some memetic level):
https://roalddahl.fandom.com/wiki/The_Great_Automatic_Grammatizator_(short_story)
Neal Stephenson plays with some of the same ideas as Vonnegut touches on here in The Diamond Age, although he focuses a little more on the pedagogy of a Universal Storyteller compared to its mechanical execution.
Oh wow, that's fascinating. I'm surprised this is the first time I've seen the Dahl story come up since LLMs came about. Amazing how well he predicted the fallout.
Very cool! Wonder if there are stories where the resulting graph is counterintuitive. What happens if you put in James Joyce's Ulysses or Finnegans Wake? Or the Mahabharata?
Ulysses is in the demo! I was hoping it might line up a little more closely w/ the Odyssey
Exceptional job. I love that you're focusing on how to use the API in a structured pipeline, rather than manually input free-form prompt engineering. More of this please!
Fascinating! Thanks for running this experiment. It really opens the mind about how we can constructively use AI.
How does ChatGPT decide how many data points to give you for each character?
I made the choice here--I split every text into chunks of 10k characters. So I end up with fewer data points for shorter stories.
I'd like to spend more time playing with this parameter.
This seems like a primary weakness in the study to me. I would be really interested to see this repeated with better controls in place there. Like, that's probably why Don Quixote seems so chaotic compared to Romeo and Juliet. It's way longer so you have way more data points, there's no way to compare them.
Exactly. I'd love to e.g. break Don Quixote down into particular chapters/stories too, and annotate each one individually.