GitHub Copilot – review of pros and cons in a programmer’s work

Andrzej Chybicki: projekty związane z wykorzystaniem sztucznej inteligencji to znacząca część naszych projektów

GitHub Copilot - review of pros and cons in a programmer's work

GitHub anticipates that in the coming years, 80% of the world’s code will be generated by AI [1]. Although it is difficult to find exact information on the number of sold GitHub Copilot licenses, it is undeniable that we are observing a change in the way organizations and programmers themselves create new IT solutions. At Inero, we are working on many projects simultaneously, so we are eager to test the latest solutions that are designed to optimize teamwork, deliver solutions faster and better.

In this article, we will talk about our experiences with GitHub Copilot.

What is GitHub Copilot?

GitHub Copilot is a tool designed for programmers, utilizing an advanced language model (known as LLM – Large Language Model) tailored for the analysis and understanding of source codes. A set of appropriate plugins and tools, allows for the optimization of the software development process, providing programmers with automatic coding hints and completions, suggestions, syntax analysis, and even generating documentation. Developed by Microsoft, the tool is based on artificial intelligence and was created to enhance the productivity of programmers and improve the quality of the source code.

Further than Closer, Slowly Towards the Goal

The use of GitHub Copilot in most cases allows us to speed up programming work, importantly, not only in creating the code itself but also in more tedious tasks. For example, since we started testing GitHub Copilot in our team, we have visited StackOverflow much less frequently. Copilot accelerates work when our programmers need to generate relatively simple and repetitive code that can be easily assessed as correct at first glance.

• • Copilot greatly speeds up work with regular expressions and certainly optimizes the time for writing repetitive code.
  • Copilot is also an effective tool for generating code documentation, essentially being able to almost entirely do it for us. Of course, the generated content always needs to be verified, but we haven’t encountered significant errors. Usually, we add more details to the result, however, in many cases Copilot is sufficient. Documenting, for example, a single block of code, when it’s quite extensive, can take approximately even a minute. With Copilot, this can be done in a few, up to several seconds, depending on whether we need to elaborate on the content.
  • From the perspective of experienced programmers, the use of this solution can be described as an advanced Intellisense working in Line-By-Line mode. The programmer, starting to create a block of code, further uses Copilot’s suggestions. In such a case, the verification is immediate and if something is wrong, we correct it ourselves right away. When Copilot suggests a larger block of code, e.g., 10-15 lines, then analyzing whether this code does exactly what it should, can take more time than writing it independently. Especially since among these 15 lines, 14 might be correct, but one is erroneous and needs to be identified.
  • From the perspective of a less experienced programmers, the reception of this solution may be different. Writing code takes them more time than it does for an experienced developers, so Copilot seems helpful. However, during the analysis of code from Copilot, the user might not notice at all if something is amiss – the problem might only be detected during a code review conducted by a more experienced person.
  • After some time, we get used to Copilot sending us suggestions, so it’s necessary to learn how to work with this tool. Initially, these suggestions can disrupt the rhythm of normal line-by-line code writing. Instead of continuing to write, we stop to read the proposed suggestions. However, in the plugin’s configuration, it’s possible to disable continuous prompting and leave only key shortcut-based prompting – here the decision depends on the programmer, who can choose the degree of “support” that suits them best.
  • When we work on more complicated functionalities, for example, using a new library, Copilot can guide us in some way. Its suggestions in such cases can be accurate and even if they are not fully correct, they allow for a quicker understanding of how to reference the library, which otherwise would require analyzing the documentation.

Comment from the CEO of Inero Software

Andrzej Chybicki: Measuring the effectiveness of using Copilot from a management perspective is difficult – we don’t have objective indicators telling us whether and to what extent the investment in the license has shortened the time spent on the software projects we are entrusted with. That’s the first point. Secondly, some clients express concerns about the security of code analyzed by AI, especially in projects related to cybersecurity or involving sensitive data processing. Hence, we encounter situations where using Copilot is not advisable.

What advantages and disadvantages have we noticed?

Analyzing the experiences of our team, we can identify the pros and cons of using GitHub Copilot.

Will we continue to use GitHub Copilot at Inero? Definitely yes.

The use of GitHub Copilot in daily work brings many benefits. However, it’s important not to lose vigilance and not to fully rely on its suggestions. It’s crucial to continuously monitor the code prepared with this tool. The development of the technology, which Microsoft is working on, will surely bring many updates and improvements in the coming months.

[1] https://github.blog/2023-02-14-github-copilot-for-business-is-now-available/