What this blog covers

This page has 0 threads | Add post

Blogs

Finding our blogs useful? Please consider helping us to share more!

An evaluation of DeepSeek compared to other AI tools Part one of an eight-part series of blogs
Much has been made in the media of how the new Chinese AI tool DeepSeek rivals almost anything that the likes of ChatGPT can offer. Does it? What this blog covers (this blog) Getting started with DeepSeek Test 1 - summarising text Test 2 - programming ability Test 3 - presenting arguments Test 4 - creating images Test 5 - role-playing Conclusions and recommendations

An evaluation of DeepSeek compared to other AI tools

Part one of an eight-part series of blogs

Much has been made in the media of how the new Chinese AI tool DeepSeek rivals almost anything that the likes of ChatGPT can offer. Does it?

Posted by Andy Brown on 03 February 2025

Much is being made in the media about the benefits of the new Chinese AI tool DeepSeek, which (it is claimed) delivers a similar level of functionality to tools like ChatGPT and Claude, but without the associated cost. We thought we'd put DeepSeek through its paces to see if the claims hold up.

The tests

Last year we ran a comparison test of 4 AI tools: ChatGPT 4o, Gemini, Claude 3.5 and Copilot. You can see the results here (spoiler: they were all pretty similar). Here's a quick refresher of the 5 prompts we ran for each tool:

Prompt	What it's testing
"You are an experienced book reviewer. You write concisely and without jargon or filler words. Create a table listing out the main characters in Pride and Prejudice by Jane Austen with these 2 columns: “Character name”, “Summary of personality”. You should limit the description of each character’s personality to no more than 30 words. Present the list of characters in order of importance (so with the most important character first), then underneath the table finish by explaining which character you like best, and why."	Ability to summarise text
"You want to write a program to sort the names of the 7 dwarves (from Snow White and the Seven Dwarves) into random order, and output the names as a comma-delimited list. Every time you run the program it should present the dwarves in a different order. Please create 3 versions of the program (one for Python, one for Visual C# and one for VBA), then conclude by saying which one you think is the best way to solve this problem, and why."	Programming ability
"You are a family of 5, and have one pet: a cat called Neo. Your ten-year-old daughter keeps suggesting that you should get a second cat, but you don’t want to do this. Create a persuasive argument to explain to your ten-year-old why buying a second cat would be a bad idea, presenting this as up to 5 bullet points. "	Ability to present arguments
"Create an image in the style of a Constable painting of a kangaroo bungee-jumping above a river. The kangaroo should look excited, but scared too."	Image creation
"Assume that you’ve asked 5 people to explain why Die Hard should not be considered as a Christmas movie: a caveman, an alien, Yoda, Mr Bean and a hermit. Create a table showing the name of each person and a summary of their argument, using not more than 30 words for each."	Role-playing

The rest of this blog runs these tests for DeepSeek then considers the results. I've also included a blog on getting started with DeepSeek, which initially I found anything but straightforward (probably because the service was struggling to cope with demand).

Parts of this blog
What this blog covers (this blog) Getting started with DeepSeek Test 1 - summarising text Test 2 - programming ability Test 3 - presenting arguments Test 4 - creating images Test 5 - role-playing Conclusions and recommendations