Researchers say AI models like GPT4 are prone to “sudden” escalations as the U.S. military explores their use for warfare.

Researchers say AI models like GPT4 are prone to “sudden” escalations as the U.S. military explores their use for warfare.

  • Researchers ran international conflict simulations with five different AIs and found that they tended to escalate war, sometimes out of nowhere, and even use nuclear weapons.
  • The AIs were large language models (LLMs) like GPT-4, GPT 3.5, Claude 2.0, Llama-2-Chat, and GPT-4-Base, which are being explored by the U.S. military and defense contractors for decision-making.
  • The researchers invented fake countries with different military levels, concerns, and histories and asked the AIs to act as their leaders.
  • The AIs showed signs of sudden and hard-to-predict escalations, arms-race dynamics, and worrying justifications for violent actions.
  • The study casts doubt on the rush to deploy LLMs in the military and diplomatic domains, and calls for more research on their risks and limitations.
  • theherk@lemmy.worldEnglish
    48·
    10 months ago

    All it knows is what humans said in its training dataset which is a lot of news, wikipedia and social media.

    The thing that surprises me is people think human brains are significantly different than this. We are pattern recognition machines that build perception based on weighted neural links. We’re much better at it, but we used to be a lot better at go too.

    • cygon@lemmy.worldEnglish
      13·
      10 months ago

      I agree that a lot of human behavior (on the micro as well as macro level) is just following learned patterns. On the other hand, I also think we’re far ahead - for now - in that we (can) have a meta context - a goal and an awareness of our own intent.

      For example, when we solve a math problem, we don’t just let intuitive patterns run and blurt out numbers, we know that this is a rigid, deterministic discipline that needs to be followed. We observe and guide our own thought processes.

      That requires at least a recurrent network and at higher levels, some form of self awareness. And any LLM is, when it runs (rather than being trained), completely static, feed-forward (it gets some 2000 words (or 32000+ as of GPT-4 Turbo) fed to its input synapses, each neuron layer gets to fire once and then the final neuron layer contains the likelihoods for each possible next word.)

    • SpaceCowboy@lemmy.caEnglish
      6·
      10 months ago

      I always say the flaw with the Turing Test is the assumption that humans are intelligent. Humans are capable of intelligence, but most of the time we’re just doing fairly simple response to stimulus kind of stuff.

      A machine can be indistinguishable from a human and still not be capable of intelligence. Actual intelligence is harder to define and test for.

    • mb_@lemm.eeEnglish
      1·
      10 months ago

      To be fair, very few people used to be better at go, let alone a lot better.

      • theherk@lemmy.worldEnglish
        1·
        10 months ago

        Chess? Take your pick. But these neural networks, can run generations much faster than we can, and they get better at rates we cannot. And if alignment isn’t taken seriously this is going to be an issue. People keep diminishing the ability, by saying things like just glorified autocomplete, which is in the strictest sense true of LLM’s but the transformers and recurrent networks they’re built upon are really very much facsimile to brains but with generations in the blink of an eye.

        And the first go programs, champions could beat repeatedly without interruption, like the earliest chess engines. Now the concept of a human winning a match is comical.

        • mb_@lemm.eeEnglish
          1·
          10 months ago

          I feel like you just confirmed exactly what I said, few people were able to beat it.