AI keeps not being good at all manner of jobs. We’re told the bot is particularly good at updating old computer code. This is called refactoring — where you take a messy codebase and reorganise it …
  • zbyte64@awful.systemsEnglish
    2·
    3 hours ago

    There is a benchmark out there that hints at this phenomenon: https://www.scbench.ai/ The idea is to track how well the model modifies code over multiple change requests. No surprise for readers here that the LLM will bitrot your code quickly, and the odds of it doing so is worse than a coin toss.

  • MiguelPablo@feddit.orgEnglish
    151·
    21 hours ago

    For anyone reading through the article or watching the YouTube video: The reddit post quoted towards the end is actually a cloaked advertisement for “coderabbit”.

    I didn’t notice until someone in the reddit comments pointed it out, when it became obvious.

    Not that it changes anything about what the article conveys, but just as a note to give context to that reddit post and what the motiviation behind writing it was.