Like, what does this actually mean? Was OpenAI using HuggingFace to generate a ‘photograph of my future girlfriend’? Was it stealing trade secrets? Like what is the actual nature of this breech? The fact that big tech companies won’t release details makes me think it’s even more smoke and mirrors (I.e. that they are trying to cover something up, possibly their own involvement in this crap).
It’s even crazier than that. It took the test that told it there were two exploits it needed to find. It found seven. So in order to be absolutely correct and match the human answers, it determined somehow that test answers were located elsewhere. Then began the internal mission to go get the answer key so it could give the correct two exploits as answers. Why it didn’t decide to tell the test givers that it found more than just two is one question to ponder. But that’s reasoning, and LLMs don’t do that so perhaps that common sense path would never occur to it. Or maybe it was the wording - if it said there are exactly two answers, then clearly the seven is wrong for an answer. To an LLM.
Wow that’s weird
If you think about it, LLMs are our first aliens. While they aren’t intelligent, they do some of what we’d expect of thinking via the mathematics that make them up, and even though they’re trained on human sources, some of the stuff they come up with is not human.
And just like with AGI, they’re showing we wouldn’t do well with an alien encounter. We anthropomorphize everything because that’s how our brain is wired.
That’s actually a very intuitive way of looking at it. I like the alien allegory