LLMs are predictive models. They scraped as much text as possible to create a model that predicts the next word accurately. To generate text, the LLM assembles a sequence of likely next words.
That exact same sort of model can be turned around and asked, how closely did the actual next word match the predicted one? Good test for training the LLM. A better model will make more accurate predictions.
AI checkers are usually doing that test. Does the real text match what the AI predicted? It sounds like a test of the text, but it really isn’t. In this case, yes. Of course an AI trained on Mary Shelly’s Frankenstein can accurately predict the next word of Mary Shelly’s Frankenstein. It has the whole book memorized, if it were accurate to anthropomorphize computer code.
So the “checker” calls it AI generated. These checkers don’t work.
Actually they’re not doing that check as they don’t have access to the models, they’re running their own statistical transformer that asks “how closely does this match our database”?
LLMs are predictive models. They scraped as much text as possible to create a model that predicts the next word accurately. To generate text, the LLM assembles a sequence of likely next words.
That exact same sort of model can be turned around and asked, how closely did the actual next word match the predicted one? Good test for training the LLM. A better model will make more accurate predictions.
AI checkers are usually doing that test. Does the real text match what the AI predicted? It sounds like a test of the text, but it really isn’t. In this case, yes. Of course an AI trained on Mary Shelly’s Frankenstein can accurately predict the next word of Mary Shelly’s Frankenstein. It has the whole book memorized, if it were accurate to anthropomorphize computer code.
So the “checker” calls it AI generated. These checkers don’t work.
Actually they’re not doing that check as they don’t have access to the models, they’re running their own statistical transformer that asks “how closely does this match our database”?