A colleague mentioned this article during our Friday AI meetup
Measuring GitHub Copilot’s Impact on Productivity – Communications of the ACM
Three things stood out to me. The first was the ratio of AI suggestions vs accepted suggestions (w/ or w/out alteration) was some 170 to 8. To me, this ratio seems more like pestering than help. Actively ignoring the suggestions must itself be draining. I've not tried to use an AI assistant yet (yea, I need to), so perhaps these unwanted suggestions feel a lot like an IDE's method completion suggestions.
The second was how both student and experienced developer used it similarly to fill in the gaps of their understanding, ie they were both working in a new language. The experienced developer found the AI assistant to be less useful in areas where they already had a comprehensive understanding.
The most significant standout was that the AI assistant improved the perception of productivity and the satisfaction of the developer. These results mirror pair programming in general. In particular, regularly working closely with another is generally more pleasing than always working alone. I assume there is also less of a stigma to not knowing when working with a robot no matter how genial your partner is.
