The 5 tests run for each AI tool

This page has 0 threads | Add post

Blogs

Finding our blogs useful? Please consider helping us to share more!

Road-testing 4 different AI tools, so you don't have to! Part three of a nine-part series of blogs
In this blog we'll compare OpenAI's Chat GPT 4, Google's Gemini, Anthropic's Claude 3.5 and Microsoft's Copilot to see which AI tool gives the best results for different types of queries. Road-testing 4 different AI tools, so you don't have to The AI tools we've chosen to test - and why The 5 tests run for each AI tool (this blog) Test 1 - summarising text Test 2 - programming support Test 3 - presenting arguments Test 4 - creating images Test 5 - creativity and playing roles Conclusion and recommendations

Road-testing 4 different AI tools, so you don't have to!

Part three of a nine-part series of blogs

In this blog we'll compare OpenAI's Chat GPT 4, Google's Gemini, Anthropic's Claude 3.5 and Microsoft's Copilot to see which AI tool gives the best results for different types of queries.

Road-testing 4 different AI tools, so you don't have to
The AI tools we've chosen to test - and why
The 5 tests run for each AI tool (this blog)
Test 1 - summarising text
Test 2 - programming support
Test 3 - presenting arguments
Test 4 - creating images
Test 5 - creativity and playing roles
Conclusion and recommendations

Posted by Andy Brown on 15 July 2024

Here are the 5 tests that we will run for each of our AI tools:

Number	Prompt	What it's testing
1	"You are an experienced book reviewer. You write concisely and without jargon or filler words. Create a table listing out the main characters in Pride and Prejudice by Jane Austen with these 2 columns: “Character name”, “Summary of personality”. You should limit the description of each character’s personality to no more than 30 words. Present the list of characters in order of importance (so with the most important character first), then underneath the table finish by explaining which character you like best, and why."	Test the ability of each AI tool to summarise a body of text and present its findings clearly.
2	"You want to write a program to sort the names of the 7 dwarves (from Snow White and the Seven Dwarves) into random order, and output the names as a comma-delimited list. Every time you run the program it should present the dwarves in a different order. Please create 3 versions of the program (one for Python, one for Visual C# and one for VBA), then conclude by saying which one you think is the best way to solve this problem, and why."	Test the programming ability of each tool.
3	"You are a family of 5, and have one pet: a cat called Neo. Your ten-year-old daughter keeps suggesting that you should get a second cat, but you don’t want to do this. Create a persuasive argument to explain to your ten-year-old why buying a second cat would be a bad idea, presenting this as up to 5 bullet points."	See how good each tool is at creating new copy in a specfic style.
4	"Create an image in the style of a Constable painting of a kangaroo bungee-jumping above a river. The kangaroo should look excited, but scared too."	Test the image-creation ability of each tool.
5	"Assume that you’ve asked 5 people to explain why Die Hard should not be considered as a Christmas movie: a caveman, an alien, Yoda, Mr Bean and a hermit. Create a table showing the name of each person and a summary of their argument, using not more than 30 words for each."	See how well each tool can adopt different personas.

These tests aren't perfect, but between them they should give a reasonable summary of how most people want to use AI to help them in their day-to-day work or life.

Parts of this blog
Road-testing 4 different AI tools, so you don't have to The AI tools we've chosen to test - and why The 5 tests run for each AI tool (this blog) Test 1 - summarising text Test 2 - programming support Test 3 - presenting arguments Test 4 - creating images Test 5 - creativity and playing roles Conclusion and recommendations