That sucks. I had a hunch that my above-average level in french, my native language, (not just according to me, but also… almost all of my French teachers throughout my entire education) might be tripping these…
Yeah, LLM-based checkers will still have LLM-based problems, most notably being incapable of true analysis, which is the whole point of an AI checker. It’s just the same text predictor shit.
Oh and also there’s an arms race where generative AI has the advantage because eventually it will be capable of generating things entirely indistinguishable from what a human would make (though it will still be susceptible to the hallucinations and errors it’s already famous for).
Yep, they’re all trash and should not be relied upon.
I got anywhere from 35% to 70% AI generated results on a book I wrote in 2019, before AI was even released.
eta: it’s not about plagiarism, either. I also ran my novel through plagiarism checkers, since it’s easy to accidentally write passages similar to existing work. 0% on those, but high numbers in the AI checkers.
I had to write a short story for English literature class in 2006 and I still have the file. Apparently over half of that is AI generated, which is pretty impressive on my part I must say.
I witnessed an interaction where a grad school professor used AI detector and threatened to fail a student for submitting “AI generated” paper. It was so stupid, even after showing them how if you just add a few spelling mistakes the detection says human written, or even putting their own email in AI detector to show an example. It’s like the saying “little knowledge is dangerous”
When I was at university I was pretty belligerent and if a professor tried that on me I’d have reported them for academic misconduct. They should be grading in the damn papers themselves, if they’re not going to do that then what is the point in them?
I don’t buy it. Not until I can test it, hands on.
So many LLM papers have amazing (and replicated) results in testing, yet fall apart in the real world outside of the same lab tests everyone uses. Research is overfit to hell.
And that’s giving them the benefit on the doubt; assuming they didn’t train on the test set in one form or another. Like how Llama 4 technically aced LM Arena because they finetuned it to.
It looks like Pangram specifically holds back 4 million documents during training and has a corpus of “out of domain” documents that they test against that didn’t even have the same style as the testing data.
I’m surprised at how well it does; I really wonder what the model is picking out. I wonder if it’s somehow the same “uncanny valley” signal that we get from AI generated text sometimes.
To show that our model is able to generalize outside of its training domain, we hold out all email from our training
set and evaluate our model on the entire Enron email dataset, which was released publicly as a dataset for researchers
following the extrication of the emails of all Enron executives in the legal proceedings in the wake of the company’s
collapse.
Our model with email held out achieves a false positive rate of 0.8% on the Enron email dataset after hard negative
mining, compared to our competitors (who may or may not have email in their training sets) which demonstrate a
FPR of at least 2%. After generating AI examples based on the Enron emails, we find that our false negative rate is
around 2%. We show an overall accuracy of 98% compared to GPTZero and Originality which perform at 89% and
91% respectively.
and
We exclude 4 million examples from our training pool as a holdout set to evaluate false positive rates following
calibration on the above benchmark.
That’s still 2 out of 1000 which if you’re using this at scale is not a great rate.
Would also be curious how that’s calculated if that’s done whit their test data that they’ve iterated on heavily or with actual feedback (which may never get back to them)
tools like these are used to reject CVs and grade school papers btw
no matter how much ai is trash do NOT use ai checkers, they do not work
That sucks. I had a hunch that my above-average level in french, my native language, (not just according to me, but also… almost all of my French teachers throughout my entire education) might be tripping these…
Yeah, LLM-based checkers will still have LLM-based problems, most notably being incapable of true analysis, which is the whole point of an AI checker. It’s just the same text predictor shit.
Oh and also there’s an arms race where generative AI has the advantage because eventually it will be capable of generating things entirely indistinguishable from what a human would make (though it will still be susceptible to the hallucinations and errors it’s already famous for).
Yep, they’re all trash and should not be relied upon.
I got anywhere from 35% to 70% AI generated results on a book I wrote in 2019, before AI was even released.
eta: it’s not about plagiarism, either. I also ran my novel through plagiarism checkers, since it’s easy to accidentally write passages similar to existing work. 0% on those, but high numbers in the AI checkers.
I had to write a short story for English literature class in 2006 and I still have the file. Apparently over half of that is AI generated, which is pretty impressive on my part I must say.
Seems like AI was trained on your book
Was it? I was sure it was first released in 2022.
in 2022, gpt 3.5, along with chatgpt, got released
Could have been. AI was trained on works written before AI was released.
I witnessed an interaction where a grad school professor used AI detector and threatened to fail a student for submitting “AI generated” paper. It was so stupid, even after showing them how if you just add a few spelling mistakes the detection says human written, or even putting their own email in AI detector to show an example. It’s like the saying “little knowledge is dangerous”
When I was at university I was pretty belligerent and if a professor tried that on me I’d have reported them for academic misconduct. They should be grading in the damn papers themselves, if they’re not going to do that then what is the point in them?
This is the Dunning Kruger era.
ESPECIALLY don’t use the “ai text humanizer” function of one that’s absolutely certain that RL authors were AI 🤦🏻
Pangram does work, actually. Here’s independent validation by unaffiliated scientists:
https://www.nber.org/papers/w34223
Although white papers are biased, here’s pangram’s white paper:
https://arxiv.org/pdf/2402.14873
I don’t buy it. Not until I can test it, hands on.
So many LLM papers have amazing (and replicated) results in testing, yet fall apart in the real world outside of the same lab tests everyone uses. Research is overfit to hell.
And that’s giving them the benefit on the doubt; assuming they didn’t train on the test set in one form or another. Like how Llama 4 technically aced LM Arena because they finetuned it to.
It looks like Pangram specifically holds back 4 million documents during training and has a corpus of “out of domain” documents that they test against that didn’t even have the same style as the testing data.
I’m surprised at how well it does; I really wonder what the model is picking out. I wonder if it’s somehow the same “uncanny valley” signal that we get from AI generated text sometimes.
and
Looked at the preprint. False positive rate of 0.2%, that’s crazy. I kinda find it hard to believe? It doesn’t seem possible to me.
That’s still 2 out of 1000 which if you’re using this at scale is not a great rate.
Would also be curious how that’s calculated if that’s done whit their test data that they’ve iterated on heavily or with actual feedback (which may never get back to them)
Wow thanks for sharing this. I always thought these things were just complete BS but it seems like some actually do work