$ cat ./blog/my-agent-picks-its-own-models.md

I stopped picking AI models by hand

2026-07-20 #ai

I do not choose which model runs which task anymore. My agent does it on a schedule, and my job is to read the recommendation and say yes or no.

The agent is Hermes, running on my own machine. The machine is a mini PC with an Intel N100 and 16GB of RAM, so it cannot run anything worth using locally and the models are remote. Right now it runs GLM 5.2 and DeepSeek V4 Flash. I hand it research, code, and the multi-step work that would otherwise sit in an open tab for a week.

Saturday self-evaluation: research, price, diff against my config

The problem is not picking the wrong model

The problem is never going back to check. You choose a model once, write it into a config, and nothing prompts you to look at it again. Meanwhile something ships every few weeks, prices move, and the model that was the obvious pick for a subagent stops being it. Nobody tells you. You find out when something feels slow or a bill looks wrong.

What runs on Saturday

Every Saturday at 9am the agent runs a self-evaluation, in four steps:

  1. Research which models people are actually running right now: release notes, forums, the places where practitioners complain.
  2. Pull the current numbers: pricing per million tokens, context window, rate limits.
  3. Compare that against what my config runs today, for the main agent and for subagents separately.
  4. Write a short recommendation with the cost and the benefit spelled out.

Splitting the main agent from the subagents is the part I did not think about when I set this up. They need different things. The main loop is where the judgment happens, so it gets the expensive model. Subagents doing mechanical fan-out, twenty at a time, need cheap and fast, and the quality bar there is whether it came back with the right file. Before this ran on a schedule I was using one model for both.

It recommends, I decide

Hermes does not edit the config. It writes a report. That is the same rule I use with agent PRs at work: whoever merges owns every line, and “the agent chose it” is not an answer. The report takes me a few minutes on Saturday morning, and most weeks I change nothing.

It will not find the best model

Community signal is noisy and it lags. Half of what shows up is someone’s first impression after a weekend of use, and a model that reads well in a thread can be wrong for how I work. Optimal was never the point. I want to not be six months stale, and a rough answer once a week does that. Parts of my setup are probably still off, and they are less off than when I only checked after something annoyed me.