Listen to this story
Microsoft has ‘Copilot’, Amazon has ‘Code Whisperer’ and Google has just launched one of their own coding assistants to help improve developer productivity at the end of a two-year research collaboration between Google Core, Google Brain and Google Research.
In a blog post, Google detailed how the researchers have combined machine learning (ML) and semantic engines (SE) to develop Transformer-based hybrid semantic machine learning code completion.
Available to Google developers now
Currently, the coding assistant has only been made available to Google’s internal developers. So far, there is no indication from Google that such facilities could be made available to non-Googlers but the possibility remains. The researchers then compared this semantic ML code completion of over 10,000 Google developers to a control group. The results were deemed impressive. The team saw a 6% reduction in coding iteration time and a 7% reduction in context switches when they were exposed to single-line ML completion, claims Google.
(Image source: Google)
Hybrid semantic machine learning code completion
In the blog post, Google also delves into great detail about how the hybrid procedure is expected to work.
- The code was represented with sub-word tokens and a SentencePiece vocabulary and the researchers used encoder-decoder transformer models running on TPUs to make completion predictions. “Sequences are generated with a beam search or tree exploration on the decoder”, explains Google.
- When the coder is typing in the IDE, the code completions are requested from the ML along with the SE in the backend. Google reports that while the semantic engines predict a single token, the machine learning models predict multiple tokens. Although, the researchers consider the first token only.
Following these steps, the researchers consider the top three ML suggestions that are also contained in the SE suggestions and boost their rank to the top. Finally, the re-ranked results come up as suggestions for the coder in the IDE.
- As a third step, the researchers use semantic engines to conduct fast semantic correctness checks within a given latency budget. Then, they use cached abstract syntax trees to enable a ‘full’ structural understanding, explains Google.
GitHub Copilot developer productivity results
Copilot was one of the major highlights in tech in 2021. Quite recently, Microsoft announced the general availability of GitHub Copilot to all developers for USD 10/month or USD 100/year. GitHub Copilot draws context from comments and code to suggest individual lines and whole functions instantly. It is powered by Codex from OpenAI.
Previously, Github’s CEO, Thomas Dohmke praised Copilot claiming that in files where Copilot is enabled, nearly 40% of the code is written by GitHub Copilot in popular coding languages. Dohmke believes that this value is expected to increase in the future.
In July 2022, GitHub released a report on how Copilot helped increase developer productivity. It analysed survey data collected from more than 2,000 developers in the US to research the contribution Copilot had made to enhance these developers’ productivity.
GitHub claims that the survey focused on three primary queries—if the developers felt that Copilot enhanced their productivity, if that feeling could be measured in any objective usage measurements and if any specific measurement reflected developers’ feeling of increased productivity better than others.
(Image source: GitHub)
GitHub observed that the acceptance rate of completions was much higher for those who had reported the biggest productivity gains. It was found that the developers didn’t assign much value to reworking the suggestion, as long as GitHub Copilot provided them with a suitable starting point. For more details, refer to this paper by GitHub.
“GitHub Copilot offers developers the parts but leaves it up to them to assemble and design the finished product”, adds Albert Ziegler, staff ML engineer at GitHub.
The recent launch of Code Whisperer has created much anticipation around the impact it is expected to have on universal developer productivity.
Everyone wants to release a coding assistant now
Of late, code generation is one of the key focus areas for big techs along with other areas such as large language models and text-to-image generation. There have been a number of quick releases in coding assistant tools from leading global tech companies.
Around the announcement of Copilot’s general availability, Amazon launched its ‘Code Whisperer’. At the time, Vasi Philomin, Amazon’s VP for AI services emphasised that the Code Whisperer was not an imitation of GitHub’s ‘Copilot’ and had been in Amazon’s pipeline for years.
Following suit, CRM software giant ‘Salesforce’ also revealed its CodeGen that claimed to possess the capability to turn English prompts into executable code. Salesforce claimed that users would be able to solve simple coding problems with little to no coding experience with CodeGen. Likewise, Google-owned AI research lab, ‘DeepMind’, also developed Alphacode with the capability to write computer programs.