ChatGPT Can Fix Bugs in Computer Code

ChatGPT, the AI chatbot developed by tech company OpenAI, can find and fix bugs in computer code as well as standard machine learning approaches – and does even better when engaged in conversation. A number of tools exist that use artificial intelligence to check programming code to ensure there are no mistakes.

ChatGPT chatbot came out and we thought it seems very good at fixing software bugs, but its main advantage over other methods and AI models is its unique ability to converse with humans allowing it to improve the answer correctness.

Researchers from Johannes Gutenberg University Mainz and University College London compared OpenAI’s ChatGPT to “standard automated programme repair techniques” as well as two deep-learning approaches to programme repairs: Codex, OpenAI’s GPT-3-based model that underpins GitHub’s Copilot paired programming and auto code-completion service, and CoCoNut from the University of Waterloo, Canada.

“We find that ChatGPT’s bug fixing performance is competitive to the common deep learning approaches CoCoNut and Codex, and significantly better than the results reported for the standard programme repair approaches,” the researchers write in a new arXiv paper, which New Scientist first spotted.

Although the ability of ChatGPT to solve coding problems is not novel, the researchers emphasise that its unique ability to converse with humans gives it a potential advantage over other approaches and models.

Microsoft Corp on Wednesday rolled out a premium Teams messaging offering powered by ChatGPT to simplify meetings using the AI chatbot that has taken Silicon Valley by storm. OpenAI-owned ChatGPT will generate automatic meeting notes, recommend tasks and help create meeting templates for Teams users. At the same time, Google reportedly has employees testing its own AI-powered chatbot rivals to ChatGPT right now, and it wouldn’t surprise me if it was doing something similar with its currently internal tools for turning text into imagesor even video.

ChatGPT is built on the transformer architecture, which was developed by Google, according to Meta’s AI chief Yann LeCunn this week. Codex, Microsoft Research’s CodeBERT, and Google’s predecessor BERT are all based on Google’s transformer method.

In examples for debugging code, OpenAI highlights ChatGPT’s dialogue capability, where it can ask for clarifications and receive hints from a person to arrive at a better answer. Companies that create bug-fixing software — and software engineers themselves — are taking note. However, an obvious barrier to tech companies adopting ChatGPT on a platform like Sentry in its current form is that it's a public database.

However, the implications for developers are unclear. While ChatGPT’s ability to discuss can help it arrive at a more correct answer, the researchers note that the quality of its suggestions is unknown. That’s why they wanted to assess ChatGPT’s bug-fixing capabilities.