everything it does must be verified in such a way to render the automation aspect useless.
While there’s no formal mathematical proof that P ≠ NP, all signs point it being so. Even if literally everything must be verified as you say, it still very often takes less time to verify a result than to generate it.
If my codebase needs to pass a manual code review which finds an endless slew of LLM hallucinations that need correcting, it adds an immense amount of time to the process to rectify them. And these tools create insanely long winded and bloated code bases so the review itself is much longer from the outset. We’re not talking about rigorous mathematical proofs, it’s much more akin to a smoke test, and even that is a huge burden to bear when using those tools such that I view them as useless from this caveat alone. And it’s basic best practice in the industry to perform these reviews, they are vital to quality results.
Even hobbyist software dev work would feel this impact. If you aren’t testing the functions an LLM writes you will be putting buggy work with major obvious edge cases and vulnerabilities out there. The tests they write for themselves are often incomplete, happy path, or plainly avoiding actual testing by just asserting true.
From seeing these shortcomings in a field where technical proof and automated testing is a possibility, I can only imagine the level of human review necessary on less quantitative work to ensure any sort of reliability or accuracy. They cannot be trusted as accurate or authoritative at any point in the chain. ML can be useful as statistics for fuzzy and approximate data, but that’s really where the usefulness ends
My point is that it doesn’t take less time to verify a result than to generate it. You can generate stuff all day but to verify it takes a lot more effort and tjme
And yet, I’ve gotten help from LLMs with code in two different ways: 1) Where I check everything at the door and manually write out or rewrite in my own words, only using the AI as an assistant for the method at most. 2) Where I don’t know enough in the language, so I go back and forth, trying a solution it gives me and then going back to it when corrections are needed.
2 is definitely more painful and I would probably be extra wary of using it in production business code and insist on review from someone who is literate in the language. BUT, there have been occasions where if I didn’t have 2 as an option, I may have never written the solution at all, or it would have taken me weeks or months of learning to get there. If I could be assured the “personal learning then completion” path would happen, that would lessen the value of 2, but being real, I cannot remotely guarantee it. Whether it’s a matter of time, motivation, etc., even someone who is already literate in some programming like myself, will not necessarily find it easy to learn every new language, library, etc., or always find it to be a motivating pursuit worth doing.
In other words, sometimes we are not talking about space-faring code and we’re talking about whether something with largely trivial consequences for mistakes, and a low friction path to fixing them, will ever get made at all. This is the kind of area where LLMs shine, since they don’t need to be without error, they just need to be “good enough” that it empowered somebody to make something that wouldn’t otherwise have gotten made.
While there’s no formal mathematical proof that P ≠ NP, all signs point it being so. Even if literally everything must be verified as you say, it still very often takes less time to verify a result than to generate it.
If my codebase needs to pass a manual code review which finds an endless slew of LLM hallucinations that need correcting, it adds an immense amount of time to the process to rectify them. And these tools create insanely long winded and bloated code bases so the review itself is much longer from the outset. We’re not talking about rigorous mathematical proofs, it’s much more akin to a smoke test, and even that is a huge burden to bear when using those tools such that I view them as useless from this caveat alone. And it’s basic best practice in the industry to perform these reviews, they are vital to quality results.
Even hobbyist software dev work would feel this impact. If you aren’t testing the functions an LLM writes you will be putting buggy work with major obvious edge cases and vulnerabilities out there. The tests they write for themselves are often incomplete, happy path, or plainly avoiding actual testing by just asserting true.
From seeing these shortcomings in a field where technical proof and automated testing is a possibility, I can only imagine the level of human review necessary on less quantitative work to ensure any sort of reliability or accuracy. They cannot be trusted as accurate or authoritative at any point in the chain. ML can be useful as statistics for fuzzy and approximate data, but that’s really where the usefulness ends
It doesn’t matter whether that specific use case is fruitful or not. I wasn’t making a case for it.
My point is that it doesn’t take less time to verify a result than to generate it. You can generate stuff all day but to verify it takes a lot more effort and tjme
This is true in some use cases and false in others. In the use case you presented, it is true.
And yet, I’ve gotten help from LLMs with code in two different ways: 1) Where I check everything at the door and manually write out or rewrite in my own words, only using the AI as an assistant for the method at most. 2) Where I don’t know enough in the language, so I go back and forth, trying a solution it gives me and then going back to it when corrections are needed.
2 is definitely more painful and I would probably be extra wary of using it in production business code and insist on review from someone who is literate in the language. BUT, there have been occasions where if I didn’t have 2 as an option, I may have never written the solution at all, or it would have taken me weeks or months of learning to get there. If I could be assured the “personal learning then completion” path would happen, that would lessen the value of 2, but being real, I cannot remotely guarantee it. Whether it’s a matter of time, motivation, etc., even someone who is already literate in some programming like myself, will not necessarily find it easy to learn every new language, library, etc., or always find it to be a motivating pursuit worth doing.
In other words, sometimes we are not talking about space-faring code and we’re talking about whether something with largely trivial consequences for mistakes, and a low friction path to fixing them, will ever get made at all. This is the kind of area where LLMs shine, since they don’t need to be without error, they just need to be “good enough” that it empowered somebody to make something that wouldn’t otherwise have gotten made.