My AI diary: AI agents are impressive. But can I actually trust them?

Category: AI

By Thomas | Financial Enthusiast

In 30 seconds: AI agents promise to automate entire workflows, not just answer questions. But the more responsibility we give them, the more important reliability, transparency and human control become. And that’s where things get interesting.

---

My AI diary: November 9, 2026 — The uncomfortable side of AI automation.

I’ve been thinking about something lately.

We keep talking about AI agents as if they’re the next big productivity revolution. Give an AI a task, let it figure out the steps, and come back when everything is finished.

Sounds fantastic.

Especially for someone like me who works in IT and enjoys automating things.

But there’s one question I can’t get out of my head.

What happens when the AI confidently does the wrong thing?

And worse, what happens when nobody notices?

From answering questions to making decisions

Traditional chatbots are relatively straightforward.

You ask something. They respond. You decide what to do with the information.

AI agents change that relationship.

Instead of simply suggesting an action, an agent might execute it.

Reading emails, updating databases, modifying code, scheduling meetings or interacting with external services.

Suddenly, we’re not just dealing with generated text anymore.

We’re dealing with software that can influence real systems.

That’s a completely different level of responsibility.

And as someone who has spent plenty of time troubleshooting IT systems, I know how quickly a small mistake can turn into a much bigger problem.

The problem with almost perfect

Imagine an AI agent that completes 95% of its tasks correctly.

That sounds pretty good, doesn’t it?

Now imagine giving that agent 100 independent tasks.

Statistically, you’d expect around five failures if that success rate remained constant.

And if a workflow requires ten consecutive steps, each with a 95% success probability, the chance of every step succeeding drops to roughly 60%.

Damned. That’s not particularly reassuring.

Of course, real-world reliability is more complicated. Errors aren’t always independent, and agents can sometimes detect and correct their own mistakes.

But the underlying problem remains.

Small failure rates become significant when automation scales.

A chatbot giving me a slightly inaccurate answer is annoying.

An autonomous agent deleting the wrong database records is something else entirely.

Why my trading bots taught me to be careful

This reminds me of my experiments with automated trading.

Building a strategy that looks good in a backtest is relatively easy.

Making it behave reliably under unpredictable market conditions?

That’s another story.

You need logging, safeguards, position limits and a way to stop everything when something unexpected happens.

I think AI agents need exactly the same mindset.

Not because they’re trading systems, but because autonomous software needs boundaries.

An agent should know what it’s allowed to do, what requires confirmation and when it should stop.

And we should be able to understand what happened afterward.

Otherwise, we’re simply replacing manual work with automated uncertainty.

The missing ingredient: trust

I don’t think the biggest challenge for AI agents will be intelligence.

Models are already capable of handling increasingly complicated tasks.

The real challenge is making their behavior predictable enough for people and businesses to trust them.

That means better monitoring, clear permissions, reliable testing and transparent decision histories.

It also means accepting that not every process should be fully autonomous.

Sometimes having a human approve the final action isn’t inefficient.

It’s sensible engineering.

Especially when money, personal information or critical infrastructure is involved.

Where I think this is heading

I’m still excited about AI agents.

Actually, probably more than ever.

The idea of automating repetitive IT tasks, analyzing information and connecting different applications is incredibly attractive.

But I suspect the winners won’t necessarily be the companies building the most autonomous agents.

They might be the ones building the most dependable ones.

The systems that know their limits.

The ones that ask before doing something irreversible.

And the ones that make failures visible instead of hiding them behind a convincing response.

Because in the end, automation isn’t valuable simply because a machine can do something.

It’s valuable when we can rely on the result.

What do you think? Would you trust an AI agent to manage important tasks completely on its own, or would you always want a human involved?

---

Disclaimer: This article reflects personal opinions and is provided for informational purposes only. It does not constitute investment, financial or technical advice.