I just can't do this anymore, Copilot

I’ve been using Github Copilot in my daily coding activity for the last two weeks, and I just can’t anymore.

I read a post on Hacker News this morning talking about the security vulnerabilities of Copilot. It’s really struck a chord with me, not because of the security implications or licensing issues, but because of the implication of the entire piece. Copilot makes bugs.

Talking about bugs in software development is like talking about the air for developers. We all know what it is, ironically with continuous effort to understand it, with sparse agreement on the best remedy, yet it remains utterly vital every moment.

What is a bug? A naive definition: an unintended and unseen side-effect of incomplete logic. As humans, we are wrong much of the time, even when the issue is narrow and specific. The scientific method only works because we try, repeatedly, to get the same result, probing each step along the way, in darkness, until we stumble upon the answer. Confidently proclaiming the answer, while knowing we are probably wrong and someone will soon come along to disprove us. Failure is universal.

Bugs are the only real cost in software development, and they can cost us everything. We should strive to stop making bugs, at whatever cost. We know this. Yet we also have to build. So we build, and work to reduce error continually, over our entire careers.

We find different ways to make this happen. TDD is nice. XP is cool. Refactoring tools become indispensable. We pair, we review, we spell check… all in the name of reducing bugs.

Back to my experience. Two weeks isn’t that long.. It is long enough to know if a tool is helping or hurting you. I feel as if I’ve given it the ol’ college try. But I just can’t anymore. I’ve turned off Copilot.. maybe for good.

It seems as if everyone has an opinion on Copilot. Everyone… it’s everywhere. I can’t escape it. It’s at work and everywhere on Twitter. It seems as if it’s the biggest thing to hit development since the escape key.

My experience: Copilot injects continual subtle bugs into my code, everywhere from variable names, and comments, to complex functions. It seems as if it’s always wrong, but sometimes it’s right, only for me to go back and see how wrong it was, all while questioning myself, that maybe it was right all along… regardless, nothing I write is even close to what Copilot is suggesting. And to me, that’s the worst part.

It takes me out of my flow. Instead of being able to simply talk my way through a problem in my head, and write out the answer, I’m continually interrupted by my copilot, with something that seems reasonable, usually wrong, and never what I really want. It’s like having a jr developer constantly chirping in my ear. “Would this code work?” My productivity has noticeably dropped.

What are revolutionary development tools? Existing libraries, TDD, XP, Refactoring tools, etc… We already have all the tools we need to squash bugs. We simply don’t apply them.

Yet again, here we are, expecting some magical shortcut to writing software where we don’t have to write it, only to realize after much effort and consternation that we still have to use our Brains.

Brains are still valuable, much to our chagrin.

Written by John Nelson, who lives and works in Chattanooga, building things for Batch.sh.