Is this the beginning of the end of that slop we call AI?

  • Zephyr@sh.itjust.works
    2·
    4 days ago

    I’m not sure that will be the outcome. You supercharge clandestine groups and nation-states with an increased ability to hack one another’s infrastructure, defense systems, and other important networks and things will get interesting for sure. Unlike nuclear testing there’s no way for one nation to verify another isn’t continuing to develop AI for offensive capabilities so the race will continue regardless of the concerns for humanities wellbeing.

    • AVincentInSpace@pawb.socialBanned from communityEnglish
      2·
      4 days ago

      I mean, I’m absolutely not convinced these models are anywhere near as good at hacking as their creators claim.

      The biggest vulnerability Claude Mythos found in Firefox was an arbitrary code execution vulnerability in an older version of Firefox’s JavaScript engine running disconnected from the actual browser with security mitigations turned off.

      The sandbox they put Claude in for running its benchmark consisted of telling it its system prompt that it was in a sandbox, and doing nothing else.

      • zeroday@lemmy.blahaj.zone
        1·
        3 days ago

        I’ve used Mythos for work and it’s really not that special. Sure, it can run other security tools and give you their output, but tbh you could just run those tools yourself.

      • Zephyr@sh.itjust.works
        1·
        4 days ago

        I think a lot of people aren’t paying attention to the rate at which the capability of these systems are growing. It’s honestly been months, not decades. You remember that original will Smith spaghetti video that looked like a google deep dream video? That was 2023. I’m just using that as a visual reference. AI video generation is essentially solved now. These hacks aren’t the ending, they are the beginning. Don’t think about what’s happening right now, think about what will be happening in six months or 24 months from now. This is the most incapable it will ever be. If the trend of emergent capabilities keeps going with larger and larger NN as it has then it’s tough to say where this all brings us as a species. Not even to include increased efforts in distilling AI capabilities into more efficient NN which is also happening simultaneously.

      • Zephyr@sh.itjust.works
        2·
        4 days ago

        Yeah the thing that essentially halted nuclear development and testing was a means of countries (US and Russia) being able to detect development and testing. Before that point things got a bit wild and essentially followed oppenheimer’s fears about an explosion (pun intended) in nuclear development. With AI, the best anyone could do is detect increased energy and data infrastructure and just assume they’re building the next doomsday AI that will wipe us all out. The only recourse anyone has is to develop a superior AI first.