Constraints on the use of AI

This page has 0 threads | Add post

Blogs

Finding our blogs useful? Please consider helping us to share more!

What you can - and can't - hope to understand about how AI works Part four of a four-part series of blogs
I'm sick of reading blogs pretending to explain how Large Language Models work, when the truth is that few people on earth can ever hope to understand how they are constructed. This blog is my attempt to explain as much as 99.99% of us can ever hope to understand about how AI works, and why the other bits will always remain opaque to most people. What you can - and can't - hope to understand about how AI works Tokenisaton and embedding Large language models Constraints on the use of AI (this blog)

What you can - and can't - hope to understand about how AI works

Part four of a four-part series of blogs

I'm sick of reading blogs pretending to explain how Large Language Models work, when the truth is that few people on earth can ever hope to understand how they are constructed. This blog is my attempt to explain as much as 99.99% of us can ever hope to understand about how AI works, and why the other bits will always remain opaque to most people.

What you can - and can't - hope to understand about how AI works
Tokenisaton and embedding
Large language models
Constraints on the use of AI (this blog)

Posted by Andy Brown on 16 July 2024

In this blog

The Context Window (Token Limits)
How Good is the Underlying LLM?
Speed of Processing
Data integrity
Accuracy (the Danger of Hallucination)

The main benefit of understanding (insofar as is possible) how AI tools work is to help you appreciate the constraints intrinsic in their user. There as follows:

Constraint	Notes
The context window	How much can your AI tool remember of what you've been asking it (what is the token limit)?
How good is the model?	Does the large language model underlying your tool have enough (and sufficiently sensible) parameters, for example?
Speed of processing	The larger and more complex a LLM is, the longer it will take for it to generate your answer.
Data integrity	Is there intrinsic bias in the data on which the LLM was trained (or are there privacy or copyright concerns)?
Accuracy	Is your AI tool prone to hallucination?

Each of these possible constraints is discussed under a separate heading below.

The Context Window (Token Limits)

AI tools are (like all websites) stateless - they have no memory. This means that when you submit a request the entire previous conversation goes with it:

AI tools get the whole of the current conversation sent to them, subject to the maximum size of the "context window".

So for the example above the number of tokens has risen to 100, because when you submit your question you will also submit the whole of the conversation leading up to it:

The Tokenizer (see earlier in this blog) shows that we have submitted 100 tokens.

Thsi matters, because each AI tool has maximum token counts. I asked ChatGPT 4 to show these:

Trying to pin down the AI tool to provide accurate information!

Here's what ChatGPT came up with:

You should never trust an AI tool, but these figures are pretty much what I expected to see.

How Good is the Underlying LLM?

An answer is only good as its underlying model. Again, I asked ChatGPT 4 how many parameters each of the main LLMs use:

The prompt I used and the answer given (I'll spare you the footnotes this time). The number of parameters is a pretty good proxy for the size and complexity of a model.

However, a model as only as good as its underlying data, training and generation algorithm allow.

Our experience is that the tools all give surprisingly similar results, although for some tasks one tool will outperform the others and for other tasks another will. You can see a more detailed comparison of ChatGPT, Claude, Copilot and Gemini here.

Speed of Processing

When you submit a query, you're basically asking your AI tool to do some complex matrix multiplication, which can take time. Based on our experience, ChatGPT (especially) and Copilot are signficantly slower than Claude and Gemini at producing the answers to queries.

Data integrity

A large language model can only ever be as good as the data on which it's trained.

To appreciate this, imagine comparing the AI tools based on large language models trained on two different sources of data: the collected lyrics of Taylor Swift and the tweets and social media messages of Donald Trump. Now imagine how different the generated answers would be!

Here are two of the biggest problems you can encounter with a Large Language Model trained on less-than-perfect data:

Problem	Notes
Bias	To give one example, the English-speaking world's webpages are heavily skewed towards news and views of countries in the West, so events and opinions in (for example) sub-Saharan Africa will be under-represented in any answer generated.
Copyright infringement	The more specific you make a query, the more likely it is that the answer may infringe the privacy or copyright of one particular supplier of data.

Accuracy (the Danger of Hallucination)

AI tools are often compared to personal assistants, but they have one unendearing characteristic: a refusal ever to say "I don't know". Here's an example of ChatGPT tripping itself up with its willingness to give an answer, however inaccurate:

There's only one problem with this clue: it doesn't work!

Feeling I was perhaps missing something, I asked ChatGPT to justify this clue, and it got increasingly panicky:

This is as close as you'll ever see to an AI tool saying "Sorry".

You should never assume that an AI tool's answer is correct, and certainly never submit or use it without checking it thoroughly first (or at least giving a warning, as I've done for the two tables of constraints shown above).

Parts of this blog
What you can - and can't - hope to understand about how AI works Tokenisaton and embedding Large language models Constraints on the use of AI (this blog)

Some other pages relevant to the above blogs include:

Classes in Artificial Intelligence

This blog has 0 threads

Add a new post

BLOGS BY TOPIC

BLOGS BY AUTHOR

The Context Window (Token Limits)

How Good is the Underlying LLM?

Speed of Processing

Data integrity

Accuracy (the Danger of Hallucination)

Head office

London

Manchester