The results showed that out of the 512 questions, 259 (52%) of ChatGPT's answers were incorrect and only 248 (48%) were correct. Moreover, a whopping 77% of the answers were verbose.

Surprisingly, the study revealed that ChatGPT's performance in providing accurate responses to programming questions was subpar, with the model delivering correct answers only 52% of the time.

Disturbingly, programmers in the study didn’t always catch the mistakes being produced by the AI chatbot. “However, they also overlooked the misinformation in the ChatGPT answers 39% of the ...