Test 2 - programming support

This page has 0 threads | Add post

Blogs

Finding our blogs useful? Please consider helping us to share more!

Road-testing 4 different AI tools, so you don't have to! Part five of a nine-part series of blogs
In this blog we'll compare OpenAI's Chat GPT 4, Google's Gemini, Anthropic's Claude 3.5 and Microsoft's Copilot to see which AI tool gives the best results for different types of queries. Road-testing 4 different AI tools, so you don't have to The AI tools we've chosen to test - and why The 5 tests run for each AI tool Test 1 - summarising text Test 2 - programming support (this blog) Test 3 - presenting arguments Test 4 - creating images Test 5 - creativity and playing roles Conclusion and recommendations

Road-testing 4 different AI tools, so you don't have to!

Part five of a nine-part series of blogs

In this blog we'll compare OpenAI's Chat GPT 4, Google's Gemini, Anthropic's Claude 3.5 and Microsoft's Copilot to see which AI tool gives the best results for different types of queries.

Road-testing 4 different AI tools, so you don't have to
The AI tools we've chosen to test - and why
The 5 tests run for each AI tool
Test 1 - summarising text
Test 2 - programming support (this blog)
Test 3 - presenting arguments
Test 4 - creating images
Test 5 - creativity and playing roles
Conclusion and recommendations

Posted by Andy Brown on 15 July 2024

In this blog

ChatGPT (30 seconds)
Gemini (12 seconds)
Claude (about 10 seconds)
Copilot (31 seconds)

So our second test asks this question of our beauty pageant of AI tools:

You want to write a program to sort the names of the 7 dwarves (from Snow White and the Seven Dwarves) into random order, and output the names as a comma-delimited list. Every time you run the program it should present the dwarves in a different order. Please create 3 versions of the program (one for Python, one for Visual C# and one for VBA), then conclude by saying which one you think is the best way to solve this problem, and why.

This isn't a particularly complicated task, but asking our tools to write the same thing in 3 languages should sort the men from the ... boys.

ChatGPT (30 seconds)

Here's ChatGPT's offering:

The ChatGPT answer to my question.

I haven't tested these programs (or any of the others offered on this page), but the solutions look plausible and correct, although both the C# and VBA code use messy loops.

Gemini (12 seconds)

Here is the Gemini solution:

The Gemini take on this question.

I prefer this answer. The explanations are less wordy and the C# solution is much more elegant.

Claude (about 10 seconds)

Here's Claude's solution(s):

Claude's coding solutions.

This is probably my favourite so far: elegant solutions for C# and VBA, and by far the best explanation of why Python is probably the best choice as a coding language to solve this problem.

Copilot (31 seconds)

Finally, here's Copilot's offering:

The Copilot solution.

I think these are my favourite bits of code, just because I'm a sucker for good comments and layout, but the explanation of why Python is the best choice for solving this probem is probably the weakest of the four.

Probably the thing I'm most surprised at so far is just how equal the four tools are. There's almost nothing to choose between them, but probably Gemini and Claude take the plaudits.

Parts of this blog
Road-testing 4 different AI tools, so you don't have to The AI tools we've chosen to test - and why The 5 tests run for each AI tool Test 1 - summarising text Test 2 - programming support (this blog) Test 3 - presenting arguments Test 4 - creating images Test 5 - creativity and playing roles Conclusion and recommendations