• 1 Post
  • 382 Comments
Joined 11 months ago
cake
Cake day: December 30th, 2025

help-circle





  • Start by not calling them agents

    What should we call them instead? “Agents” is more or less the accepted name for chat bots with access to tools (like writing files and running commands).

    Then use “generate” instead of “say”, “think”, “explain” and by that point you’ll be on a roll.

    “Generate” is too general. We need a way to characterize the text output. “Explanation” is our characterization of the generated text. We humans interpret it as an explanation.

    The idea of letting the LLM “think” before it “responds” is inspired by how humans generally think before we respond. It was quickly discovered after GPT 3.5 that the quality of answers improves if it has a “thinking” phase before it produces an answer.

    The English language is unfortunately not made for describing LLMs in an non-anthropomorphic way - at least not without sounding like a dork.






  • I’m not worried about AI going rogue and destroying society. I’m worried about what human assholes out in the world can do with AI.

    It’s not hard to imagine how AI can have destructive powers. In Europe there was recently news about someone hacking into freezer shelves of grocery stores and turn their temperature to max. One of my local grocery stores got affected had to throw out lots of their food due to this. Not saying it’s due to AI, but I wouldn’t be surprised if some AI was involved.

    This kind of attack on our food supply chain and other parts of our infrastructure could be even more large scale. Not because of some rogue superintelligence, but because some human asshole with AI is able to brute force billions of combinations of preexisting ideas faster than previously possible.

    Putting the models out in the open won’t help a tiny bit. Even if it’s possible to perfectly align open models so they refuse to do these sorts of attacks, anyone could remove these safeguards by modifying its weights or train them on new data.

    The answer is: ban development of frontier models until we better understand how to regulate this technology. But given the incompetency of politicians, I doubt this will happen before it’s too late.