Yeah, go get Copilot (VC Code) or Cursor (VS Code fork) and try it out! There’s also Codeium that I tried but did not like that much.
I’ve tried Copilot. I wasn’t enthused. Perhaps time to try again…?
You have to work with it regularly to see the benefit. It’s a skill, just like any other.
7 Likes
I would like to echo this 100%. I do not necessarily disagree with the premise of some of these “bubble” arguments (whether we mean the technology itself or the valuations of AI firms). But… I am a convert. Whether it be chat tools or something else, I use it pretty much daily now.
3 Likes
I cannot agree more with this statement. Copilot is extremely helpful to generate boilerplate codes and to write good comments and docs, especially for someone like me who is not a native English speaker!
2 Likes
I have not been able to persuade Copilot to do function docs properly. It always throws in “implementation goes here” and messes up my code. Teething problems?
I don’t think the bubble conversation is at all about AI’s existence. Even if we continue to ignore other kinds of AI here, code generation and debugging are probably the most justifiable use of genAI now (there’s a good argument that many people didn’t upload their code to be scraped and we can’t feasibly credit the authors as we’d like, but at least we can immediately learn how the code works, unlike most artistic skills), so it’s not unusual that a programming language forum has happy users. The dot-com bubble popped, the internet didn’t die. What died were opportunistic companies that adopted the internet with no understanding of its benefits and more legitimate companies that relied too heavily on them. genAI does have its fair share of such businesses, and ironically the cultural backlash against genAI in some contexts might help stave off rampant misuse and bad investments.
6 Likes
This is rich, considering how your friendly government (wink wink) is by far the biggest offender here.
No doubt. But there are poor guardians of AI and really terrible ones, I am sure. (I have some past and present colleagues at KAUST. I’ve heard some hair-raising stories… )
You can run the big open source LLMs like Llama2 etc yourself on cloud machines that are a few USD per hour, and these models don’t lack too much behind ChatGPT. So if you have a good use case, these models are really not that expensive to run.
1 Like
Is there empirical evidence that copilot/llm’s improves productivity ? In my experience they can help you get a jumpstart but also waste a lot of time, so it’s not so clear to me if a more principled approach (reading the docs, taking the time to do things right, …) doesn’t win on the longer term.
1 Like
Copilot can be sometimes uncannily helpful. At other times I just have to reject the suggestions outright (they are wrong, misleading, or just unhelpful), or accept, but spend significant time editing.
I’ve been using one for about a year and empirically for me it drastically improves productivity
obviously it can’t do everything, but I would say at least 3 times a week there is some task that would have previously taken me 10-30 minutes and GPT can just spit out an answer in seconds
2 Likes
That would still be anecdotal, empirical evidence calls for larger sample sizes and numbers. I didn’t do a lit review, but I did skip over several surveys to find a randomized controlled trial of 95 developers (professional, 20s-30s, mostly Indian or Pakistani) writing an HTTP server in Javascript. The control group could search the internet and use Stack Overflow, and the experimental group additionally accessed GitHub Copilot. Of the 35 who completed the task successfully (negligible difference in success rate), the experimental group was 55.8% faster (95% CI: [21%,89%]).
3 Likes
Of course one has to wonder what the affect is on longterm productivity, because although you get an answer quicker, you might be learning less from the process, and thus your programming knowledge base grows more slowly in the longterm than it would if you figured out the answer on your own. 
1 Like
only if you want to be statistically confident in your estimate of the mean
I don’t need to query any larger sample size to be very confident that LLMs have improved my own productivity
honestly I doubt it. if anything I think it helps me learn important things more quickly and spend less time on junk
3 Likes
That argument would also apply to non-genAI sources of code, like this very forum (genHI, generative human intelligence?). People who are crunched for time for some reason will paste-execute-pray, there’s no getting around that. But every code snippet is an opportunity to narrow down searches for documentation, write educational comments, and (most importantly) sanity-checking the code. I would hope that happens as part of any testing.
A legitimate question is why your programing knowledge base should grow at all if machines can do it.
Thinking purely from industrial automation perspective e.g. when mechanical levers replaced muscles: One can make an argument that puny arms and chicken legs are worse than muscular arms and legs, therefore the industrial automation was bad.
May be, in the near future, we will have office spaces without internet and LLMs, where people will go to code for “training”, much like how people now go to gyms to do bicep curls and lat pull downs to appear like a healthy adult human.
1 Like
Good point. Soon humans will be reduced to having philosophical discussions on Discourse. The machines could of course have the philosophical discussions for us, but they have better things to do with their time.
3 Likes
Several reasons:
- genAI is not deterministic like a compiler and cannot understand what it outputs. You do not want to leave your code at the mercy of stochastic hallucinations.
- genAI is not an independent actor, it is a tool that requires a user to write a prompt. Tasks are often small and simple enough that it’s much easier just to learn the code than learn the prompt with a high success rate.
- Users should not rely on powerful enough machines or internet access to them to get anything done, lesser tools should be and often need to be enough.
2 Likes