A personal AI assistant as a butler
An AI agent like Hermes or OpenClaw can do real work out in the world, like writing code or filing bug reports. So it's tempting to use it for exactly that.
I see it differently.
To me, a personal assistant is an agent system that shouldn't do any of that outside work itself. Instead it orchestrates the other systems and gives you one interface to all of them.
A butler is a good way to think about it.
A butler doesn't clean the rooms, set the table, or cook the food himself.
He manages the people who do, signs off on their work, and gives the client one easy interface. The client doesn't have to talk to every worker or organize their work.
So a personal AI assistant is just a proxy.
Some people like to work in text (chat, or Obsidian), some by voice, some through boards. The assistant's job is to be right there and make it easy for you to work with all the systems around you.
With that in mind, you really want to keep the agent local and as close to the body as you can. To cut latency and reduce privacy risks.
And you have to watch that the agent doesn't do anyone else's work, just hands it off and takes back the result.
Of course, by then all the rest of the work has to be set up so the agent can assign it and accept it.