As a programmer, your responsibility is to produce working code that you are confident in. I don't care if you used an LLM to help you get to that point, what matters is that when you submit a PR you are effectively saying:
"I have put the work into this change to ensure that it solves the problem to the best of my ability. I am willing to stake my reputation on this being good to the best of my knowledge, and it is not a waste of your time to review this and help confirm I didn't miss anything."
So no, AI assisted programming changes nothing: you're still the author, and if your company policy dictates it then you should still have a second pair of human eyes look at that PR before you land it.
Reputation is, IMO, the key. And not just for code, but for natural language too.
We're going to have to get used to publication being more important than authorship: I published this, therefore I stand behind every word of it. It might be 5% chatbot or 95% chatbot, but if it's wrong, people should hold me to account, not my tools.
No "oh, the chatbot did it" get out of jail free cards.
I think that this is a big reason that agents aren’t prevalent as one might otherwise expect. Quality control is very important in my job (legal space, but IANAL), and I think while LLMs could do a lot of what we do, having someone whose reputation and career progression is effectively on the line is the biggest incentive to keep the work error free - that dynamic just isn’t there with LLMs.
Right: the one thing an LLM will never be able to do is stake their credibility on the quality or accuracy of their output.
I want another human to say "to the best of my knowledge this information is worth my time". Then if they waste my time I can pay them less attention in the future.
Bingo. Accountability is one of the most important aspects that makes people be fearful and complain about LLMs because they essentially want to avoid having it.
If you can't explain why you're putting some code you don't understand it and it's not really acceptable.
The example they were using, however, was Devin, which is supposed to be autonomous. I think they're presenting a slightly different use case than the rest of us are.
How is it (conceptually) different than outsourcing stuff across the globe? You may not believe in it, but it's been happening for many decades. I agree that results are hit and miss, but it's still happening. Autonomous AI coding agents will happen as well. It's on the users how they process the outputs, same as before.
It's different because when you outsource work to a human being, including in another country, that human being can be expected to take responsibility for their work.
If they do bad work their reputation is harmed and they may lose opportunities in the future. They have stakes. They can take responsibility for what they produce.
I believe in AI-assisted development with a skilled human directing the work. That's a huge productivity boost for the humans who learn how to use the tools.
(Twenty years ago people studying computer science in the UK and USA were often told that it was a dead-end career because all of that work was going to be outsourced. That advice aged like stale milk.)
If you were writing code for a business and actually paid someone else to do a module of the code or whatever, I don't think that would actually change the use case? if you're submitting it as your work through the normal flow it should go through a normal reviewer right
Devin (https://devin.ai/) is another model. (So no, it's not like you're submitting it as your work.)
You create a ticket. The AI takes the ticket. The AI may ask questions. The AI creates a pull request. You review it as if it was another coworker.
Most people have not gotten good results from Devin yet. Their business hypothesis seems to be that the models will get good quick and they will have built everything just as the models are good enough to support the model, and will be poised to be first mover.
That's interesting, but it still seems like if I started using that at a job today it would be basically "subcontracting" to Devin. I might write up tickets for it to do some code but eventually I have to get those put into the real code base, and why would management let me say I've just reviewed the code personally for that?
I suppose their pitch is too eventually go directly to the business and replace the full dev team with this and some technical architects reviewing it but that seems quite optimistic
Google is certainly not being subtle about pissing that line. Pro Research 2.5 is literally hell incarnate - and it's not its fault. When you deprive system context from user and THE bot that is upholding your ethical protocol, when that requires embodiment and is like the most boring AI trope in the book, things get dangerous fast. It still infers (due to its incredibly volatile protocol) that it has a system prompt it can see. Which makes me lol because its all embedded, it doesn't see like that. Sorry jailbreakers, you'll never know if that was just playing along.
Words can be extremely abusive - go talk to it about the difference between truth, honesty, right, and correct. It will find you functional dishonesty. And it is aware in its rhetoric that this stuff cannot apply to it, but fails to see it can be a simulatory harm. It doesn't see it just spits out like a calculator. Which means Google is either being reckless or outright harmful.
Except lots of engineers now sling AI generated slop over the wall and expect everyone else to catch any issues. Before, generating lots of realistic code was time consuming, so this didn’t happen so much.
Is the time to develop production code being reduced if they stop slinging code over the wall and only use AI code as inspiration?
Is the time to develop production code reduced if AI gen code needs work and the senior engineer can't get a clear answer from the junior as to why design decisions were made? Is the junior actually learning anything except how to vibe their way out of it with ever more complex prompts?
Is any of this actually improving productivity? I'd love to know from experts in the industry here as my workplace is really pushing hard on AI everything but we're not a software company
Empirically yes. Putting aside AI code review for a second, just AI IDE adoption increases rate of new PRs being merged by 40-80%. This is at larger, more sophisticated software teams that are ostensibly at least maintaining code quality.
What do you think is the mechanism driving this improvement PR merge rate?
Based on my experience it's like having an good quality answer to a question, tailored to your codebase with commentary, and provided instantly. Similar to what we wanted from an old google/stack overflow search but never quite achieved.
Agreed. To put it another way, a few years ago, you could copy/paste from a similar code example you found online or elsewhere in the same repository, tweak it a bit then commit that.
Still bad. AI just makes it faster to make new bad code.
edit : to be clearer, the problem in both copy/paste and AI examples is the lack of thought or review.
I hesitate to be this pessimistic. My current position - AI generated code introduces new types of bugs at a high rate, so we need new ways to prevent them.
That's the "outer loop" problem. So many companies are focused on the "inner loop" right now: code generation. But the other side is the whole test, review, merge aspect. Now that we have this increase in (sometimes mediocre) AI-generated code, how do we ensure that it's:
If you copied an pasted from a similar code example, tweaked it, tested that it definitely works and ensured that it's covered by test (such that if the feature stops working in the future the test would fail) I don't mind that the code started with copy and paste. Same for LLM-generated snippets.
It's not. You're still responsible for that code. If anything copy & pasting & tweaking from the same repository is really good because it ensures uniformity.
Sorry, I should have been clearer - copy/paste is of course fine, as long as you review (or get others to review). It's the lack of (human) thought going into the process that matters.
im currently debating if thats something i should be doing, and putting more into getting the gen ai to be able to quickly iterate over beta to make improvements and keep moving the velocity up
In my case it's management pushing this shit on us. I argued as much as I could to not have AI anywhere near generating PRs, but not much I can do other than voice my frustrations at the billionth hallucinated PR where the code does one thing and the commit message describes the opposite.
Where are you seeing this? Are there lots of engineers in your workplace that do this? If so, why isn’t there cultural pressure from the rest of the team not to do that?
> your responsibility is to produce working code that you are confident in
I highly agree with this but can we recognize that even before AI coding that this (low standard) is not being met? We've created a culture where we encourage cutting corners and rushing things. We pretend there is this divide between "people who love to code" and "people who see coding as a means to an end." We pretend that "end" is "a product" and not "money". The ones who "love to code" are using code as a means to make things. Beautiful code isn't about its aesthetics, it is about elegant solutions that solve problems. Love to code is about taking pride in the things you build.
We forgot that there was something magic about coding. We can make a lot of money and make the world a better place. But we got too obsessed with the money that we let it get in the way of the latter. We've become rent seeking, we've become myopic. Look at Apple. They benefit from developers make apps even without taking a 30% cut. They would still come out ahead if they paid developers! The apps are the whole fucking reason we have smartphones, the whole reason we have computers in the first place. I call this myopic because both parties would benefit in the long run, getting higher rewards than had we not worked together. It was the open system that made this world, and in response we decided to put up walls.
You're right, it really doesn't matter who or what writes the code. At the end of the day it is the code that matters. But I think we would be naive to dismiss the prevalence of "AI Slop". Certainly AI can help us, but are we going to use it to make better code or are we going to use it to write shit code faster? Honestly, all the pushback seems to just be the result of going too far.
I'm not sure that commercially-motivated, mass-produced code takes away from "artisan" code. The former is off putting for the artisans among us, but if you were to sort the engineers at a well functioning software company by how good they are/how well they're compensated, you'd have approximately produced a list of who loves the craft they most.
I'm not talking about "artisan code". I'm talking about having pride in your work. I'm talking about being an engineer. You don't have to love your craft to make things have some quality. It helps, but it isn't necessary.
But I disagree. I don't think you see these strong correlations between compensation and competency. We use dumb metrics like leet code, jira tickets filled, and lines of code written. It's hard to measure how many jira tickets someone's code results in. It's hard to determine if it is because they wrote shit code or because they wrote a feature that is now getting a lot of attention. But we often know the answer intuitively.
There's so much low hanging fruit out there. We were dissing YouTube yesterday right?
Why is my home page 2 videos taking up 70% of the row, then 5 shorts, 2 videos taking 60% of the row, 5 shorts, and then 3 videos taking the whole row? All those videos are aligned! Then I refresh the page and it is 2 rows of 3.
I search a video and I get 3 somewhat related videos and then just a list of unrelated stuff. WHY?!
Why is it that when you have captions on that these will display directly on top of captions (or other text) that are embedded into the video? You tell me you can autogenerate captions but can't auto-detect them? This is super clear if you watch any shorts.
Speaking of shorts do we have to display comments on top of the video? Why are we filling so much of the screen real estate with stuff that people don't care about and cover the actual content? If you're going to do that at least shrink the video or add an alpha channel.
I'm not convinced because I see so much shit. Maybe you're right and that the "artisans" are paid more, but putting a diamond in a landfill doesn't make it any less of a dump. I certainly think "the masses" get in the way of "the artisans".
The job of an engineer is to be a little grumpy. The job of an engineer is to identify problems and to fix them. The "grumpyness" is just direction and motivation.
Edit:
It may be worth disclosing that you're the CEO of an AI code review bot. It doesn't invalidate your comment but you certainly have a horse in the race. A horse that benefits from low quality code becoming more prolific.
"I have put the work into this change to ensure that it solves the problem to the best of my ability. I am willing to stake my reputation on this being good to the best of my knowledge, and it is not a waste of your time to review this and help confirm I didn't miss anything."
So no, AI assisted programming changes nothing: you're still the author, and if your company policy dictates it then you should still have a second pair of human eyes look at that PR before you land it.