Test 5 - creativity and playing roles

This page has 0 threads | Add post

Blogs

Finding our blogs useful? Please consider helping us to share more!

Road-testing 4 different AI tools, so you don't have to! Part eight of a nine-part series of blogs
In this blog we'll compare OpenAI's Chat GPT 4, Google's Gemini, Anthropic's Claude 3.5 and Microsoft's Copilot to see which AI tool gives the best results for different types of queries. Road-testing 4 different AI tools, so you don't have to The AI tools we've chosen to test - and why The 5 tests run for each AI tool Test 1 - summarising text Test 2 - programming support Test 3 - presenting arguments Test 4 - creating images Test 5 - creativity and playing roles (this blog) Conclusion and recommendations

Road-testing 4 different AI tools, so you don't have to!

Part eight of a nine-part series of blogs

In this blog we'll compare OpenAI's Chat GPT 4, Google's Gemini, Anthropic's Claude 3.5 and Microsoft's Copilot to see which AI tool gives the best results for different types of queries.

Road-testing 4 different AI tools, so you don't have to
The AI tools we've chosen to test - and why
The 5 tests run for each AI tool
Test 1 - summarising text
Test 2 - programming support
Test 3 - presenting arguments
Test 4 - creating images
Test 5 - creativity and playing roles (this blog)
Conclusion and recommendations

Posted by Andy Brown on 15 July 2024

In this blog

ChatGPT (11 seconds)
Gemini (4 seconds)
Claude (10 seconds)
Copilot (8 seconds)

For the final test, I want to test how well each AI tool plays roles:

Assume that you’ve asked 5 people to explain why Die Hard should not be considered as a Christmas movie: a caveman, an alien, Yoda, Mr Bean and a hermit. Create a table showing the name of each person and a summary of their argument, using not more than 30 words for each.

ChatGPT (11 seconds)

Here are the 5 reasons ChatGPT came up with:

I love the Mr. Bean take on this!

Gemini (4 seconds)

Here are the Gemini reasons:

This hasn't quite got mastered nuance of each character as well as ChatGPT did.

Claude (10 seconds)

Here's what Claude had to offer:

The Claude reasons.

I tried twice to get Claude to redisplay this without the irritating horizontal scroll bar, but failed, so I'll have to do this myself here:

Character	Argument summary
Caveman	No snow in cave. Big boom-boom not for celebration. No mammoth feast. Strange shiny tower not like home. Angry man not jolly gift-giver.
Alien	Earthlings' concept of "Christmas" unclear. Observed violence contradicts peace messaging. Temporal setting insufficient for classification. Lacks common ritualistic elements seen in other "Christmas" media.
Yoda	A Christmas movie, Die Hard is not. Peace and serenity, it lacks. The path to joy, through violence, leads not. Celebrate life, a Jedi does.
Mr. Bean	[Silently acts out confused facial expressions, then mimes decorating a tree, giving gifts, and eating dinner. Shakes head vigorously and points to imaginary explosions and gunfire.]
Hermit	Never seen "Die Hard" or celebrated Christmas. Isolation from society prevents understanding of either concept. Violence and materialism seem contrary to spiritual growth.

These are astonishingly good. How can a large language model possibly give this comment from Mr. Bean?

Copilot (8 seconds)

And finally, Copilot:

There's a lot of similarity between these reasons!

For this test Claude is a clear winner - it showed the most artificial intelligence (ie human-like qualities) in understanding the questions and creating its answers.

Parts of this blog
Road-testing 4 different AI tools, so you don't have to The AI tools we've chosen to test - and why The 5 tests run for each AI tool Test 1 - summarising text Test 2 - programming support Test 3 - presenting arguments Test 4 - creating images Test 5 - creativity and playing roles (this blog) Conclusion and recommendations