Blog
How AI can enhance the developers productivity
In the past years, we have seen many advances in Natural Language Processing: question answering, text generation, proofreading, etc. So, you should not be surprised about systems to assist developers, right?
That’s exactly what Github Copilot does, a tool from Microsoft that is advertised as “your AI pair programmer”. At Skaylink GmbH we have been using it for a while, and now we’ll share with you the main points of this productivity tool and how you can start using it too.
First, how does it work?
It is powered by GPT-3, a large neural network that was trained with a huge amount of public internet texts (Wikipedia, blogs, StackOverflow, open repositories in Github, etc). There is nothing “magical” about it: its architecture was described in detail in academic papers, its general architecture has been used for many years (including in NLP projects here at Skaylink), and since its debut many competitors have trained similar models. But even if not magical, GPT-3 achieves impressive results in many NLP tasks, and one could dare to say that it “memorized the whole internet”.
The strength of GPT-3 is text generation: you give it an initial text, and the network will return you the most probable text that could follow it. Quite generic, yeah? That’s exactly the idea, and the “text” can be anything:
- You give it “what is the largest river by discharge volume of water in the world” and it answers “amazon river”
- You give it “Angela Merkel was elected because…” and it gives you 3 paragraphs of political analysis
- You give it “int count = 20; for i in …” and it completes a code that makes sense
Microsoft encapsulated that into a system that auto-completes code, and can be installed in many IDEs as a simple plugin. It shows some suggestions automatically while you type the code (which can be even just a comment), and you can accept the suggestion by pressing tab. If you press ctr+enter at any time, it will show 10 suggestions to continue your code. It usually completes from a few words to a few lines of code, but in some situations, it will generate 10 or 20 lines.
What it does well
Generates functional code: its suggestions are mostly functional and make sense in the context of the code.
Followed my code style: it named variables and guessed some string names in the same way as I was doing in my code, as well as correctly using functions and classes that I have just written.
Works for many libraries and APIs: I initially thought that Copilot would shine only for “generic” code, pure logical problems. But no, and it even helped me with some very specific tasks (like AWS LaunchSpecifications JSON descriptions).
Works across files: if I start writing a function in one file and then use this function in another file, copilot will autocomplete correctly.
Understand different languages: both programming languages (python, javascript, etc) and natural languages (English, Spanish, etc). I was surprised when I wrote a comment in Portuguese and it autocompleted my comment in a way that made sense, almost like reading my thoughts, haha.
What it doesn’t
GPT-3 is well known for having problems with “long term memory”: it generates text where the phrases make sense among themselves, but the text as a whole is gibberish. So, you can expect Copilot to generate short pieces of code, but the architecture of the solution is still up to you.
Trash-in, trash-out: if you don’t provide enough context (like just opening a blank new file) or provide a confusing context (like having a lot of unrelated code together), the Copilot’s result will be bad. It may be syntactically correct, but it won’t perform anything meaningful. The developer is in control, and that won’t change soon.
There are also open questions about how copilot behaves regarding copyrights, licenses and privacy matters. For instance:
- It was seen that sometimes it generated valid API keys
- It could have been trained with code that is available on the Internet, but under a license that requires citation or that requires your code to become open source as well
- It may use our own proprietary code to feed the system
Microsoft clarified all those points [at least partially] and there are extensive discussions in the community, but only with time we’ll see how the system works as it becomes more and more popular.
While using it myself, the only moment I saw weird auto-completion was when copilot suggested the comment “This file is part of love-platform” in a Lua file (löve is a game engine for the programming language Lua, and my program had no relationship with it). Also, Copilot recently added a new feature for “allow suggestions matching public code”, with an also optional approval for “Allow GitHub to use my code snippets for product improvements”. Interesting… I don’t completely understand my code will be used by Microsoft, but I trust that they are doing a serious work.
How companies can use it
Copilot is available to individual developers and individual accounts within organizations (but organizations can’t buy bulk licenses yet), and it costs \$10 USD/month or \$100 USD/year. After paying the subscription, you just have to install the plugin in your IDE (VS code, for example) and start coding. The plugin uses only the system’s web API, so your computer doesn’t become slow because of it.
Alternatives
One of the best known alternatives is Tabnine, whose pricing starts at \$ 12 USD/month per developer. The inference can run locally or on the cloud, and you can thwick small configurations like the length of suggestions and if the suggestions should be given for empty lines. Honestly, the code-completion quality was way bellow Copilot, and it requires more “supervision”: if I just go pressing tab (that is, accepting the suggestion) the generated code usually is pure giberish. Disclaimer: I have tested only the free version, and some peers told me that the paid version is considerably better. The paid version also have features that worth mentioning: train a model specific for your team’s code, continuously improve the model from your code, github integration, among others.
There are also competitors that follow a different approaches for AI-assisted coding:
- Captain Stack brings answers from stack overflow (a famous online forum for development questions);
- OpenAI provides a code-competition REST API (they are the ones actually fact behind copilot, not Microsoft, but I won’t go into more details about this);
- Code Clippy provides a solution quite similar in functionality to Copilot, but completely free-and-open-source and using the cloud inference engine of HuggingFace (that is also widely used by the open-source community). I think that 100 dolar per year is an acceptable sum to pay for quality and trustworthiness of Microsoft’s Copilot, so the Code Clippy model makes sense mostly for the companies that explicitly desire to support the open-source community.
Conclusion
I was impressed by Copilot, and it’s usually not easy to impress me.
Besides being fun, it speeds up my coding process: I already know what I want to write, then I start writing it, Copilot completes parts of the code, I fix the wrong parts, and continue writing what I had in mind. The result is faster delivery and more confidence that I didn’t make any silly mistakes.
It also induces me to write a more standardized and comprehensible code, since then the copilot recommendations are better in those cases. This sounds especially advantageous when you are learning something new, since then your code tends to be messier.
In summary, I can easily imagine Copilot and similar tools being widely used in a near future. Do you want to be at the cutting edge of AI-assisted development and productivity tools? We can provide you with the guidance and support needed, contact us and let’s work together.