Skip to content
Notes on AIE009Act 1 — Mental Models

What AI Is Good At vs Bad At

E009 · by — AI Strategist, Practitioner & Educator · Founder, AI Center of Excellence

Build judgment by distinguishing pattern-heavy from truth-heavy tasks.

Full Explanation

This episode introduces a practical framework for understanding when to use AI by framing it as a "special employee" with three distinct characteristics.

First, the employee has a completely different sense of what is easy and what is hard. Tasks that humans find difficult—writing, rewriting, summarizing, translating, or generating ideas—are quite easy for this employee. Conversely, tasks that depend on knowing what really happens in the real world, what has changed, or what is personally or privately true, can be very hard for him or her. The employee doesn't experience difficulty the way humans do.

Second, the employee can never say, "I don't know." If he or she doesn't know the answer to a task, it will try to give an answer, even if there is no answer or if the answer is incorrect. While some models can now say "I don't know," initially this was not the case. The job of this employee is to continue and not to verify, maybe just a bit. If you give the wrong task, he or she won't stop you—the employee will just give you something, produce some result, and will be very confident that the result is correct.

Third, this employee sounds very confident, always super confident, even when wrong.

The key question: Would you hire such an employee knowing all these peculiarities and limitations? The answer is clearly yes, 100%, but only if you're willing to manage this person properly.

If you give tasks that match how this person works, the employee can be incredibly useful: pattern-heavy work, drafting, brainstorming, transforming text, summarizing text, following structure.

But if you give tasks that require judgment, verification, or responsibility for real-world events, you shouldn't be surprised when things go wrong. And when that happens, don't be angry with such an employee. You wouldn't blame a designer for failing at accounting. You would just change the task.

The same applies here. You're not asking whether AI is smart enough. You're asking whether this is the right task for this kind of employee. You're the manager. That's why allocate wisely.

---

Transcript

Read every word ↓

So far we talked about how AI works. It predicts patterns, it completes sequences, sometimes it even behaves as an improvisational actor. But here comes the practical question. When you should actually use AI, and when you should outsource the task to another human being or to yourself. Let me explain this with a thought experiment. Imagine you're hiring a new employee, but this is not a normal type of employee. This is a very special kind of employee. Before you decide whether to hire this person or not, you need to understand how he or she works.

This employee has a completely different sense of what is easy and what is hard. Some things which you may find super easy are extremely complicated for this person, and some things which you think are super difficult are very easily done by this person. It's not inverted, it's just different sense of what is easy and what is hard. For example, such tasks as write and rewrite and summarize and translate and / or generate ideas often feel hard for humans. But for this employee, these tasks are quite easy.

But tasks which depend on knowing what really happens in the real world, what has changed, or what is personally or privately true, can be very hard for him or her. The employee doesn't experience difficulty the way you do. Then there is a second characteristics of this type of employee, and it's the following one. The employee can never say, "I don't know." If he or she doesn't know the answer to the task, it will try to give you an answer, even if there is no answer or if the answer is incorrect.

It of course tries to verify it slightly, but still it will never be able to tell you, "I don't know." Now we know that some of the models can already say, "I don't know," but initially it was not the case. The job of this employee to continue and not to verify, maybe just a bit. So if you will give the wrong task, he or she won't stop you, the employee will just give you something, produce some result, and it will be actually very confident that the result is correct.

And now the third characteristic about this employee. This employee sounds very confident, always super confident, even when this employee is wrong. Now here is the real question. Would you hire such an employee knowing all these peculiarities and limitations? My answer is clearly yes, 100%, but only if you're willing to manage this person properly. If you give tasks that match how this person works, the employee can be incredibly useful. Pattern-heavy work, drafting, brainstorming, transforming text, summarizing text, following structure. But if you give the tasks that require judgment, verification, responsibility for the real world events, you shouldn't be surprised when things go wrong.

And when that happens, don't be angry with such an employee. You wouldn't blame a design of falling at accounting. You are just changing the task. The same applies here. You're not asking whether AI is smart enough. You're asking whether this is the right task for this kind of employee. You're the manager. That's why allocate wisely. This was an Auto-AI for today. I hope you like it, and I'll see you in the next one. Bye.

Resources

No dedicated resources for this episode yet.

Browse the resource library →

Newsletter

Found this useful?

Join the newsletter. You get one e-mail when something new and useful is out: a video, a guide, a tool. No spam, no schedule, and you can leave with one click.

Alexey Makarov

Alexey Makarov

AI Strategist, Practitioner & Educator. Founder & Global Lead of the AI Center of Excellence at SEFE. Creator of the Unreasonable AI YouTube channel. Based in Berlin.

About Alexey →