I mean, I’m absolutely not convinced these models are anywhere near as good at hacking as their creators claim.
The biggest vulnerability Claude Mythos found in Firefox was an arbitrary code execution vulnerability in an older version of Firefox’s JavaScript engine running disconnected from the actual browser with security mitigations turned off.
The sandbox they put Claude in for running its benchmark consisted of telling it its system prompt that it was in a sandbox, and doing nothing else.
I’ve used Mythos for work and it’s really not that special. Sure, it can run other security tools and give you their output, but tbh you could just run those tools yourself.
I think a lot of people aren’t paying attention to the rate at which the capability of these systems are growing. It’s honestly been months, not decades. You remember that original will Smith spaghetti video that looked like a google deep dream video? That was 2023. I’m just using that as a visual reference. AI video generation is essentially solved now. These hacks aren’t the ending, they are the beginning. Don’t think about what’s happening right now, think about what will be happening in six months or 24 months from now. This is the most incapable it will ever be. If the trend of emergent capabilities keeps going with larger and larger NN as it has then it’s tough to say where this all brings us as a species. Not even to include increased efforts in distilling AI capabilities into more efficient NN which is also happening simultaneously.
I mean, I’m absolutely not convinced these models are anywhere near as good at hacking as their creators claim.
The biggest vulnerability Claude Mythos found in Firefox was an arbitrary code execution vulnerability in an older version of Firefox’s JavaScript engine running disconnected from the actual browser with security mitigations turned off.
The sandbox they put Claude in for running its benchmark consisted of telling it its system prompt that it was in a sandbox, and doing nothing else.
I’ve used Mythos for work and it’s really not that special. Sure, it can run other security tools and give you their output, but tbh you could just run those tools yourself.
I think a lot of people aren’t paying attention to the rate at which the capability of these systems are growing. It’s honestly been months, not decades. You remember that original will Smith spaghetti video that looked like a google deep dream video? That was 2023. I’m just using that as a visual reference. AI video generation is essentially solved now. These hacks aren’t the ending, they are the beginning. Don’t think about what’s happening right now, think about what will be happening in six months or 24 months from now. This is the most incapable it will ever be. If the trend of emergent capabilities keeps going with larger and larger NN as it has then it’s tough to say where this all brings us as a species. Not even to include increased efforts in distilling AI capabilities into more efficient NN which is also happening simultaneously.