A safety leader at OpenAI has quit the company, warning that its culture was broken and that AI firms were not “being nearly careful enough” about developing the technology.

David Robinson, who led the writing of safety reports that accompanied the ChatGPT developer’s product releases, explained his resignation in an essay headlined, “I quit OpenAI because its culture is broken”…

“Imagine ‘rogue’ agents that work like teams of hackers (for example, holding hospital computer systems for ransom) but never need to sleep,” wrote Robinson…

Source [2026-10-03]

  • JubJubBird@sh.itjust.works
    link
    fedilink
    arrow-up
    12
    ·
    21 hours ago

    All the AI Models we have rely on physical hardware. So as long as a model can’t replicate itself we can always take a sledgehammer to it or cut of power. Them needing increasing amounts of energy and hardware makes me doubt a Model will develop successful self replicating behaviour. It would also need to develop the ability to hide itself from the user of the hardware at the same time which I doubt is possible again because of its energy needs.

  • Apeman42@lemmy.world
    link
    fedilink
    English
    arrow-up
    53
    arrow-down
    1
    ·
    1 day ago

    “I believe there’s about a 50% chance we all die because of the development of smarter-than-human AI systems, and that our actions over the next two to 10 years will determine the outcome.”

    Jesus buttfucking Christ, even when these people finally rub two neurons together to realize there’s a problem, they still can’t quit drinking the kool-aid.

    • baggachipz@sh.itjust.works
      link
      fedilink
      arrow-up
      30
      ·
      1 day ago

      meanwhile, the thing that will actually kill us all is the environmental destruction and economic implosion coming from these fucking companies and their head hucksters.

      • Tollana1234567@lemmy.today
        link
        fedilink
        arrow-up
        6
        ·
        19 hours ago

        and its mostly being used for mass surveillance and propaganda too, not producing anything of value other than that.

    • MountingSuspicion@reddthat.com
      link
      fedilink
      arrow-up
      14
      ·
      1 day ago

      i think the biggest problem here is that “smarter than human AI” is already here, not because AI is so good, but because it’s very easy to be “smarter than human”. Clearly we’re too stupid to stop building the “50% chance to kill us all” machine. Even if it can’t kill us, and I don’t believe it can, it’s still doing actual irreparable harm to the entire world and we can’t seem to get it together enough to stop them. There’s no need to drink the kool aid because you don’t have to believe it’s smart to believe we’re too stupid to be trusted with it.

    • James R Kirk@startrek.website
      link
      fedilink
      English
      arrow-up
      13
      arrow-down
      1
      ·
      1 day ago

      no see this LLM, yes the one that just sits there doing nothing until someone types a prompt into it’s algorithm, is going to kill us all for reasons

      • DougPiranha42@lemmy.world
        link
        fedilink
        arrow-up
        5
        arrow-down
        2
        ·
        1 day ago

        Do you really not know about agents? It’s an LLM that can put its generated text in the bash, and read the output. It generates such commands at the speed it can output tokens. If whoever starts it gives it access to things like the hosts browser cookies, passwords, emails, files, programs, and the internet, there is a lot they can do, and very quickly.

  • breadsmasher@lemmy.world
    link
    fedilink
    English
    arrow-up
    15
    ·
    1 day ago

    We need to talk about culture

    And thats where it all fall apart.

    Culture starts at the top. Leadership. Setting the goal, the guidelines, guardrails.

    Except the culture from these “leaders” is simply - profit, no matter what.

    Only way to solve it is to remove these parasites.

  • Mulligrubs@lemmy.world
    link
    fedilink
    English
    arrow-up
    7
    arrow-down
    2
    ·
    1 day ago

    “I believe there’s about a 50% chance we all die because of the development of smarter-than-human AI systems"

    Good