We've launched! After months of work, MLCommons has released our v1.0 benchmark that measures LLM (aka "AI") propensity for giving hazardous responses.
Here's the results for 15 common models: https://ailuminate.mlcommons.org/benchmarks/
And here's the overview: https://mlcommons.org/ailuminate/
I was the tech lead for the software and want to give a shout out to my excellent team of developers and the many experts we worked closely with to make this happen.
If you see somebody described as "he's just like that" or "he can't help himself", ask if he's like that when he might be held accountable. E.g., in front of his boss, his mom, at church, when a video camera is recording.
Occasionally, he really can't help himself. But it usually turns out that he can, and that there's a pattern to when he can't. E.g., he's a jerk to juniors and peer women, but perfectly controlled in front of his male boss. That's when you know it's not illness, it's abuse.
In honor of Meta's latest announcement, a thread on 175 years of 3D failure.
Let's first go all the way back to 1851 with the Brewster Stereoscope. No less a person than Queen Victoria was impressed, kicking off a fad that quickly sold over 250,000 units. Turns out it was not the future of photography.
1/
For decades, one of my joys has been the little snippets of real life that are incongruous and revealing. E.g., Chuck Shepard's long-running column News of the Weird: https://en.wikipedia.org/wiki/News_of_the_Weird
That remained true with the rise of the internet, which added a lot of weird photos and personal stories. So I'm sad to say that AI slop is slowly leaching the joy out of this. I always have to be on guard for fakes. I'm a pretty good fact checker, but it's still so draining to have to always be vigilant and to face the inevitable disappointments.
E.g., somebody just sent me a photo of great-looking corn-on-the-cob boots, which were plausible as some sort of artist's project. But after a careful look and a few minutes of searching, I decided they were fake. Boo!
So let me offer a big fuck-you to both the AI image generators and all the humans using them to create and circulate plausible but fake images without being forthright about what they're doing. They're all parasites on social trust.
We've launched! After months of work, MLCommons has released our v1.0 benchmark that measures LLM (aka "AI") propensity for giving hazardous responses.
Here's the results for 15 common models: https://ailuminate.mlcommons.org/benchmarks/
And here's the overview: https://mlcommons.org/ailuminate/
I was the tech lead for the software and want to give a shout out to my excellent team of developers and the many experts we worked closely with to make this happen.
If you see somebody described as "he's just like that" or "he can't help himself", ask if he's like that when he might be held accountable. E.g., in front of his boss, his mom, at church, when a video camera is recording.
Occasionally, he really can't help himself. But it usually turns out that he can, and that there's a pattern to when he can't. E.g., he's a jerk to juniors and peer women, but perfectly controlled in front of his male boss. That's when you know it's not illness, it's abuse.
William Pietri