The study tracked around 800 developers, comparing their output with and without GitHub’s Copilot coding assistant over three-month periods. Surprisingly, when measuring key metrics like pull request cycle time and throughput, Uplevel found no meaningful improvements for those using Copilot.
I’ve tried it for even some boiler plate code a few times. I’ve had to end up rewriting it every time.
It makes mistakes like Junior engineers, but it doesn’t make them in the same way that junior engineers do, which means that as a senior engineer it takes me significantly more effort to review. It also makes mistakes that humans don’t, which is even weirder to catch in review.
Also my experience. It sometimes tries to be smart and gets everything wrong.
I think code shows clearly, that LLMs don’t actually understand what’s written. Often enough you can clearly see it trying to insert a common pattern even though that doesn’t make sense at this point.
As a junior-to-mid-level developer I find myself having to rewrite the boilerplate code copilot comes up with as often as not, or it will get things slightly wrong that I then have to go back and fix. I’m starting to think that most of the time it would be just as quick for me to just write it all myself.
I basically exclusively use LLMs to explain broad concepts I’m unfamiliar with. a contrived example would be ‘what is a component in angular’ or ‘explain to a c# dev how x,y, and z work in rust’ The answers don’t need to be 100% accurate and they provide a nice general jumping point to get specific information.
The few times I have used AI to help me with coding has mostly been to ask it for examples on how to use a specific feature, then it has been ok for the most part.
I mostly code in PowerShell, HTML and CSS, and Bing Chat helpful when I am stuck on a small issue.
We also recently started testing Copilot Pro 365, the one that can help you make documents or search through company documents and stuff like that.
As a test I asked it to make me a powerpoint presentation about the top ten podcasting microphones to buy.
The result looked great at first glance, but quicly got very generic.
Sure, it did show pictures of some microphones and even spoke about them, but it was just vauge and generic
I got a bridge to sell to anyone who thought AI would help reduce burnout lmao
No really, AI has great uses but I’m in awe anyone thought this was one of them
I do a lot of scripting for cloud infrastructure deployments and linux/windows basic scripting and the bing chat is great for banging out 5 liners in 1 second that would take me an hour even after multiple decades of being an admin.
Anything more complex it is useless for so it is limited but nice to have.
I’m not sure why that’s so surprising actually
I’m mildly surprised at all the bad experiences.
I’ve been using chatgpt for a while now. Tbf, of all the ai code assistant I tried, the only one that isn’t garbage is chatgpt. Annoying part is to provide code snippet and context.
But once you do, it becomes a god
Entire mechanics, algo, template, helper functions, done within a minute.
I legit can save multiple hours of work everyday with it. Obviously, I use this extra bit of time for myself ! Lmao
It’s slightly better Resharper, except when it fucks up and then it’s just an annoying parlour trick.
