A shocking story was promoted on the “front page” or main feed of Elon Musk’s X on Thursday:

“Iran Strikes Tel Aviv with Heavy Missiles,” read the headline.

This would certainly be a worrying world news development. Earlier that week, Israel had conducted an airstrike on Iran’s embassy in Syria, killing two generals as well as other officers. Retaliation from Iran seemed like a plausible occurrence.

But, there was one major problem: Iran did not attack Israel. The headline was fake.

Even more concerning, the fake headline was apparently generated by X’s own official AI chatbot, Grok, and then promoted by X’s trending news product, Explore, on the very first day of an updated version of the feature.

  • kadu@lemmy.world
    link
    fedilink
    English
    arrow-up
    109
    arrow-down
    4
    ·
    edit-2
    6 months ago

    I wonder how legislation is going to evolve to handle AI. Brazilian law would punish a newspaper or social media platform claiming that Iran just attacked Israel - this is dangerous information that could affect somebody’s life.

    If it were up to me, if your AI hallucinated some dangerous information and provided it to users, you’re personally responsible. I bet if such a law existed in less than a month all those AI developers would very quickly abandon the “oh no you see it’s impossible to completely avoid hallucinations for you see the math is just too complex tee hee” and would actually fix this.

    • Ottomateeverything@lemmy.world
      link
      fedilink
      English
      arrow-up
      66
      arrow-down
      2
      ·
      6 months ago

      I bet if such a law existed in less than a month all those AI developers would very quickly abandon the “oh no you see it’s impossible to completely avoid hallucinations for you see the math is just too complex tee hee” and would actually fix this.

      Nah, this problem is actually too hard to solve with LLMs. They don’t have any structure or understanding of what they’re saying so there’s no way to write better guardrails… Unless you build some other system that tries to make sense of what the LLM says, but that approaches the difficulty of just building an intelligent agent in the first place.

      So no, if this law came into effect, people would just stop using AI. It’s too cavalier. And imo, they probably should stop for cases like this unless it has direct human oversight of everything coming out of it. Which also, probably just wouldn’t happen.

      • kadu@lemmy.world
        link
        fedilink
        English
        arrow-up
        15
        ·
        6 months ago

        So no, if this law came into effect, people would just stop using AI. And imo, they probably should stop for cases like this unless it has direct human oversight of everything coming out of it.

        Then you and I agree. If AI can be advertised as a source of information but at the same time can’t provide safeguarded information, then there should not be commercial AI. Build tools to help video editing, remove backgrounds from photos, go nuts, but do not position yourself as a source of information.

        Though if fixing AI is at all possible, even if we predict it will only happen after decades of technology improvements, it for sure won’t happen if we are complacent and do not add such legislative restrictions.

      • rottingleaf@lemmy.zip
        link
        fedilink
        English
        arrow-up
        1
        ·
        6 months ago

        Unless you build some other system that tries to make sense of what the LLM says, but that approaches the difficulty of just building an intelligent agent in the first place.

        I actually think an attempt at such an agent would have to include the junk generator. And some logical structure with weights and feedbacks it would form on top of that junk would be something easier for me to call “AI”.

        • atrielienz@lemmy.world
          link
          fedilink
          English
          arrow-up
          2
          ·
          6 months ago

          I actually have been thinking about this some, and all those “jobs” that people are losing to AI? Will probably end up being jobs that add a human component back into AI for the firms that have doubled down on it. Human oversight is going to be necessary and these companies don’t want to admit that. Even for things that the LLM’s are actually reasonably good at. So either companies will not adopt AI and keep their human workers, or they’ll dump them for AI LLM’S, quickly realize they need people in specialities to comb through AI responses, and either hire them back for that, or hire them back for the job they wanted to supplant them with LLM’S for.

          Because reliability and cost are the only things that are going to make one LLM more preferable to another now that the Internet has basically been scraped for useful training data.

          This is algorithms all over again but on a much larger scale. We can’t even keep up with mistakes made by algorithms (see copyright strikes and appeals on YouTube or similar). Humans are supposed to review them. They don’t have enough humans to do that job.

    • rottingleaf@lemmy.zip
      link
      fedilink
      English
      arrow-up
      4
      ·
      6 months ago

      The legislation should work like it would before. It’s not something new, like filesharing in the Internet was.

      Which means - punishment.

      • atrielienz@lemmy.world
        link
        fedilink
        English
        arrow-up
        1
        ·
        6 months ago

        The legislation doesn’t work because part of the problem is what “products” these LLM’s are being attached to. We already had this argument in the early and mid oughts in the US. And nothing was done really about the misinformation proliferated on places like Twitter and Facebook specifically because of what they are. Social media sites are protected by section 230 in the US and are not considered news aggregators. That’s the problem.

        People can’t seem to agree on whether or not they should be. I think if the platform (not the users) is pushing something as a legitimate news source it shouldn’t be protected by 230 for the purposes of news aggregation. But I don’t know that our laws are even attempting to keep up with new tech like LLM’S.

        NY’s for a chatbot that was actively giving out information that was pseudo legal advice. Suggesting that Businesses should do illegal things. They aren’t even taking it down. They aren’t being forced to take it down.

    • rayyy@lemmy.world
      link
      fedilink
      English
      arrow-up
      3
      ·
      6 months ago

      Another of Musk cutting corners to the max and endangering lives but why should he care? He is in control and that is the only thing that matters to him, even if he loses billions of dollars.

    • expr@programming.dev
      link
      fedilink
      English
      arrow-up
      38
      arrow-down
      1
      ·
      6 months ago

      In case you’re not familiar, https://en.m.wikipedia.org/wiki/Grok.

      It’s somewhat common slang in hacker culture, which of course Elon is shitting all over as usual. It’s especially ironic since the meaning of the word roughly means “deep or profound understanding”, which their AI has anything but.

      • umbraroze@lemmy.world
        link
        fedilink
        English
        arrow-up
        13
        ·
        6 months ago

        Yup. Got also added to the Jargon File, which was an influential collection of hacker slang.

        If there’s one thing that Elon is really good at, it’s taking obscure beloved nerd tidbits and then pigeon-shitting all over them.

  • cmnybo@discuss.tchncs.de
    link
    fedilink
    English
    arrow-up
    30
    arrow-down
    2
    ·
    6 months ago

    Oh, what a surprise. Another AI spat out some more bullshit. I can’t wait until companies finally give up on trying to do everything with AI.

    • Cosmic Cleric@lemmy.world
      link
      fedilink
      English
      arrow-up
      15
      arrow-down
      3
      ·
      6 months ago

      I can’t wait until companies finally give up on trying to do everything with AI.

      I don’t think that will ever happen.

      They’re acceptable of AI driving car accidents that causes harm happen. It’s all part of the learning / debugging process to them.

      • rottingleaf@lemmy.zip
        link
        fedilink
        English
        arrow-up
        12
        arrow-down
        1
        ·
        edit-2
        6 months ago

        The issue is that the process won’t ever stop. It won’t ever be debugged sufficiently

        EDIT: Due to the way it works. A bit like static error in control theory, you know that for different applications it may or may not be acceptable. The “I” in PID-regulators and all that. IIRC

        • assassin_aragorn@lemmy.world
          link
          fedilink
          English
          arrow-up
          1
          ·
          6 months ago

          Due to the way it works. A bit like static error in control theory, you know that for different applications it may or may not be acceptable. The “I” in PID-regulators and all that. IIRC

          Oh great, I’m getting horrible flashbacks now to my controls class.

          Another way to look at it is if there’s sufficient lag time between your controlled variable and your observed variable, you will never catch up to your target. You’ll always be chasing your tail with basic feedback control.

        • Cosmic Cleric@lemmy.world
          link
          fedilink
          English
          arrow-up
          5
          arrow-down
          6
          ·
          edit-2
          6 months ago

          It won’t ever be debugged sufficiently

          It will, someday. Probably years and years down the road (pardon the pun), but it will.

          By the way, you reply to me seems very AI-ish. Are you a bot?

          • maynarkh@feddit.nl
            link
            fedilink
            English
            arrow-up
            6
            ·
            6 months ago

            I guess the argument is that this is what “innovation and disruption” looks like. When they finally iron out so that chatbots won’t invent fake headlines, they will pile on a new technology that endangers us in a new way. This is the acceptable margin of error to them.

      • JackGreenEarth@lemm.ee
        link
        fedilink
        English
        arrow-up
        3
        arrow-down
        2
        ·
        6 months ago

        AI isn’t inherently bad. Once AI cars cause less accidents than human drivers (even if they still cause some accidents) it will be moral to use them on roads.

        • anon987@lemmy.world
          link
          fedilink
          English
          arrow-up
          3
          arrow-down
          3
          ·
          6 months ago

          AI cars already cause drastically less accidents. And the accidents they do cause are overwhelmingly minor.

          • Thorny_Insight@lemm.ee
            link
            fedilink
            English
            arrow-up
            1
            ·
            6 months ago

            People hate it when an accident happens and there’s no one to blame. Now it’s still on the driver’s responsibility but that’s not always going to be the case. We’re never reaching zero traffic deaths even with self driving cars that are a hundred times better than the best human driver.

  • Nobody@lemmy.world
    link
    fedilink
    English
    arrow-up
    25
    ·
    6 months ago

    Beware, terminally incompetent interns everywhere. Doing something incredibly damaging to your company over social media on your first day is officially a job that’s been taken by AI.

    • Fedizen@lemmy.world
      link
      fedilink
      English
      arrow-up
      6
      arrow-down
      1
      ·
      6 months ago

      “Somebody I don’t like said something bad about one of the worlds richest oligarchs therefore all criticism of him is invalid”

      The guy has enough money to protect himself from bad criticism and address narratives he doesn’t like; he doesn’t need sad pathetic losers defending him on the internet like he’s a defenseless baby.

      • Ginger666@lemmy.world
        link
        fedilink
        English
        arrow-up
        1
        arrow-down
        4
        ·
        6 months ago

        You people will take a statement, then do mental gymnastics in your mind to make it fit your retarded narrative.

        Holy fuck

        • Cris@lemmy.world
          link
          fedilink
          English
          arrow-up
          3
          ·
          edit-2
          6 months ago

          I mean, he’s responsible for a pretty heinous thing in this instance… It’s not like people are pissing their pants over nothing.

          If your company publishes an explicitly fabricated headline stating there was a missile attack that never happened, generated by ai, and then presented to a ton of people on a major social media site, criticism is pretty deserved… (≖_≖ )