The tutor that fixes the textbook

I've built an agentic tutor to quiz me on a subject. It marks my wrong answers and then passes them to another bot to fix the textbook for my future learning.

The subject is investing, which isn't really why I did it. Last Saturday afternoon I joined a live session run by Tina Huang, whose videos I've followed for a while on LinkedIn and YouTube, through her company Lonely Octopus. She was walking people through setting up Hermes, an AI agent framework, alongside Obsidian, a note-taking app, and investing was the example everyone built around. I went because I wanted to see what it takes to set something like this up, and how it might work inside an organisation rather than on my laptop.

I wasn't starting from zero. I already use something from Tina called Lifebot, which runs on the same setup and keeps track of my focus time and sleep, and it's become one of the more useful things I use day to day.

What we built is easier to describe than it sounds. One set of agents builds and maintains a wiki from the sources you give it, a small library of notes that link to each other and improve as more goes in. A tutor agent uses that wiki to quiz you. When you get something wrong, it logs the mistake and puts it on a board, and the wiki agents pick it up and fill the gap. It all runs through Telegram, and every time an agent wanted to run a script, I approved it by hand from my phone.

By the end of the afternoon I was thinking less about investing and more about onboarding.

At Baker McKenzie I ran quarterly onboarding sessions for new joiners across EMEA, around 200 people at a time. They were about inclusive culture, leadership and the behaviours the firm wanted to see, including speaking up, and policies came into them too. I never had a reliable way of knowing what had stuck.

Most firms back sessions like that up with a handbook of some kind and a set of pages online for quick reference, and the same problem applies. You can measure attendance and you can send a survey, but you can't easily see what someone a few weeks into the job still doesn't understand, or which page of the handbook keeps letting people down.

This setup turns that round. A handbook the agents keep current, a tutor that checks what new joiners have understood, and a log of what they get wrong would show you where people are struggling, and it would also show you where the handbook is failing them. That second part matters more than it sounds, because fixing a confusing policy page once is a lot cheaper than explaining it to every new starter for the next five years.

Another area I'd be interested in applying this to is interviewer training, partly because hiring is my own ground. In my experience, firms train hiring managers in structured interviewing once and then hope it holds. A tutor that checks what they've kept hold of, and shows you which parts of the training aren't landing, would tell you something another refresher session can't. You could do the same for a new recruiter learning a firm's offices and practice groups, or for anyone who needs to find the right policy quickly, but onboarding and interviewer training are where I'd start.

I'd go in with my eyes open, though, and the build gave me a couple of reasons to.

The first came a couple of days later, when I pointed the same setup at my own research. I'd had an AI research task pulling together reports on talent acquisition for me, and one of the agents' jobs was to follow every source back to the original. In a report from June it found six figures with no source at all. It found numbers credited to one company that had come from another, and a statistic about 63% of people across five countries that had become "63% of US job seekers" somewhere along the way. AI summaries are very good at making an unsourced number look like a fact. In a business, every fact an agent holds needs a source you can trace, and checking that is a job worth giving to someone, or something, on purpose.

The second was smaller and more familiar. I added a rule telling one of the agents to follow every source link, and on its next run it ignored it. Restarting it didn't help. The rule only took effect once I'd cleared its conversation history, because it was still working from a conversation that started before the rule existed. Anyone who's run a change programme will recognise that. Changing how an agent works is a rollout, the same as it is with people, and someone has to own it.

That ownership question is the big one, and I'd argue it's as much a people job as an IT one. Somebody has to decide what goes into the wiki, what stays out because it's confidential, who can approve what, and what the agents are allowed to touch, and a business needs proper permissions for that, set and checked by someone. Anything touching decisions about candidates should stay well away from this kind of setup. The EU AI Act treats AI used in screening and selection as high-risk, and for good reason.

I also suspect it suits a 100-person firm better than a large one. A smaller firm often has no knowledge system at all, and the way things work lives in a few people's heads, so a wiki that keeps itself current is a real step forward. A large firm already has SharePoint, Copilot and an infosec team, and for them I think the lessons travel further than the tools.

None of this needed me to be a technologist, which is partly why I wanted to write it down. The hardest questions the build threw up were the same ones I've spent my career asking about human teams: who owns what, who signs off, how work gets handed over, and how you know anyone's learning.

If you look after onboarding or interviewer training, I'd like to know what you'd most want to find out that your people haven't taken in.