Andy Smith

Building scalable self-improving infra for AI agents.

October 30, 2026

  • To organize infrastructure for Durable Agents, it is useful to classify them by the lifecycle of the tasks they support.

    1. Request-response. The agent sleeps most of the time. When it receives a request, it wakes up, performs the task, and goes back to sleep.
    2. Timer. A task runs at a specific time.
    3. Loop. The agent runs in a loop without sleeping. It may sleep while waiting for a response from other agents.

    The first option is the easiest to support with infrastructure. We simply save the state of the working directory after the task is complete and restore it before the next run.

    The second option can be seen as a special case of the first. The timer is simply a component outside the agent that sends it "do X" events.

    A loop is the most difficult object to make Durable because it requires an environment that is always running. However, a loop can be modeled as a finite-state machine. Each step of the loop is then just a specific agent performing a specific task and passing control to the next one. While one agent is working, all the others are sleeping. I am not entirely sure this approach works well for every class of task. I think some tasks are better served by a traditional long-running container.

    But the foundation here is the agent that wakes up when called, performs a task, and goes back to sleep. Infrastructure solutions should focus on these agents, which seem to be the majority of mine. Specifically, they need fast onboarding and access restrictions.

  • It turns out that the default Apple Mail app is very finicky about encryption.

    I have several Apple devices and Thunderbird on my computer. It is recommended to separate encryption and signing keys, make signing keys unique to each device, and use the same encryption keys on all devices while rotating them more often. This is a strange practice in my view, but it is what it is. Otherwise, one device cannot decrypt messages sent to the other devices, and I could not find a way to use several encryption certificates for one address.

    Here is what I found while trying to set up an encryption certificate shared with Thunderbird:

    1. Thunderbird supports EC keys but uses sha256kdf/sha384kdf. Apple only supports sha1kdf, which Thunderbird does not support. The conclusion is that EC will not work in a setup with Apple.
    2. Both Thunderbird and Apple support RSA, but only RSA PKCS#1 v1.5, not RSA-OAEP. This is weaker encryption and is not resistant to the Bleichenbacher attack. When an RSA key is selected, Thunderbird uses RSA PKCS#1 v1.5.

    I had to use RSA PKCS#1 v1.5 for encryption so that I could decrypt email on my iPhone and iPad. I trust third-party clients even less.

    Digital signatures work correctly with EC keys, so these workarounds do not apply to signing.

    Links referenced by the agents:

October 29, 2026

  • I am thinking about how to connect agents through A2A and email.

    Ideally, an agent should understand only A2A and expose its services only through it. This makes communication much simpler. It does not matter what is used to communicate with a person, whether it is email, a chat such as Buzz or Zulip, Telegram, or anything else. You only need a connector between the chosen communication tool and A2A.

    One small caveat: this will work only with asynchronous agents. There will be no real-time communication here. But 100% of my agents work this way so far, so I do not really need it. Voice agents will use a different architecture.

    My publishing agent currently works differently. It can work with email itself, including receiving and sending messages. It does not even use IMAP or SMTP. It runs in a Cloudflare Worker and uses worker features to handle email.

    This is enough for a basic tool, but the architecture looks wrong:

    1. The tool depends entirely on Cloudflare and cannot be moved elsewhere.
    2. It is impossible to connect another tool that supports A2A but does not support S/MIME email. For example, hermes supports email but cannot verify signatures or encrypt messages.

    So I am arriving at the following, more general setup.

    1. If an agent needs to communicate by email, meaning it is an external agent rather than an internal one, it needs an email account that stores a queue of unprocessed messages. This is clearly better than keeping the queue in a worker's memory. If processing fails, the worker simply loses the message, as happened today with the previous post.
    2. The agent, or something next to it, should have an email <-> A2A adapter. The adapter must verify encryption and signatures, unpack the message, pass it to the A2A agent for processing, get the result, and send the reply to the user by email, or through a chat, depending on the adapter.
    3. In this case, A2A is not secured separately, but it is accessible only to the email adapter.
    4. If the agent is internal and does not have A2A, the email adapter is replaced with a checker that can use the chosen method. It can verify a decentralized access pass or use a centralized authorization service.

    This will make the implementation somewhat more complex, but it will make all agents universal, which in turn will reduce onboarding time for launching new agents.

  • When people learn that I am building a network of agents and betting that everyone will have many agents, the first thing I hear is, "I have one agent, and that is enough for me."

    I usually reply, "Good for you, but one is not enough for me." Sometimes, though, I feel like debating it.

    First, we need to define what an agent is. There are two opposing views.

    1. An agent is an agent session that performs a specific task. All agents start from the same shared base. There is a starter setup for every task, with all potentially necessary skills, tools, and the broadest possible access.
    2. An agent is a launcher for a specific type of task. For example, an agent that reviews pull requests in a particular repo, or in every repo in an organization. It can review different PRs independently, but these tasks still belong to one agent. In this case, each agent has limited permissions assigned only to it. These are the minimum permissions needed to do a specific job.

    The first option is the simplest. You just launch Codex or Claude Code and work in it. If you need something, you add it. Security relies on faith that the model is smart and will figure things out. Joking aside, I have met people who actually think this way.

    But this is roughly the same as giving a pizza delivery driver the key to a server. In a normal situation, there is nothing wrong with that because the driver has no reason to break anything. But that does not mean they will never have such a reason in the future.

    To me, dividing work, responsibility, and access within my team of agents is an essential architectural requirement that is not up for debate. Otherwise, it would be like putting graduates of top universities in a room and demanding that they do something. They might reach an agreement among themselves, but not necessarily in my interests. Of course, agents can be used to implement an organizational structure, but abandoning it completely is madness in my view. Sooner or later, everyone will come to this. That is why I am preparing for it now. My goal is to make launching a team of agents on your own infrastructure as fast and convenient as possible.

    If you need to organize processes with agents and think you are overpaying, let’s discuss it. I am sure we can cut costs and make the process more transparent.

October 28, 2026

  • In a network of agents, we need a way to manage agent permissions. Specifically, we need to define which tools an agent can invoke.

    There are two cases:

    1. An agent uses a tool.
    2. An agent calls another agent.

    I see two architecturally different ways to handle this:

    1. An agent has permanent permissions. For example, if an agent can access GitHub, this is defined in its configuration when it is created.
    2. Permissions are defined at the task level. The task initiator, a human, sets specific constraints when defining the task. Any agent can then do anything within those constraints.

    The second option looks more flexible but less convenient. At the start of a task, it may not be clear which permissions are needed to complete it.

    I am not considering an option where each agent or tool has a fixed list of agents that can call it. In that case, deploying a new agent would require redeploying all the other agents.

    Each agent should have a "pass" that provides agent authentication. It confirms that the request is being made by the owner of that pass. I think the S/MIMIE approach to signing and encrypting email could work here. Each agent receives its own certificate, signed by a certificate authority that all other participants trust. Every request signed with this certificate is a request from that agent. As long as the certificate has not been stolen, it can be treated as authenticating the agent.

    In addition to the "pass," the agent needs a set of permissions. The main question is where they should be stored, and in what format.

    My hypothesis is to package a set of permissions and the certificate ID into a JWT, then sign the JWT with a central CA key that everyone trusts. An agent team may have several such signers. They are the team owners.

    When agent A needs to call agent B, it includes a JWT that says: "This JWT was issued to agent A. It has the myvipteam.website.publication permission." The called agent verifies the CA signature on the JWT, verifies agent A's signature on the request, checks that A's certificate has not been revoked, and checks that the JWT has the required permissions. If all checks pass, it performs the task.

    This approach seems slightly more complex because creating an agent requires generating a certificate and assigning permissions. However, it is decentralized and requires almost no shared infrastructure, except for a revocation list.

  • The first agent I am launching under the new setup is a website post publisher.

    It is worth using this agent to explore the agent interface while trying to make it universal, at least to some degree.

    Since only I, a human, can write on the website, this agent must have an email interface.

    There are two options: use email directly, or use A2A with an email bridge.

    I want to make it as universal as possible, but I am leaning toward keeping it as simple as possible so I can identify the setup's bottlenecks.

    This means it should be an agent that can do the following:

    1. Have a public email address and receive emails at that address.
    2. Know the CA fingerprint and verify the authenticity of the signature.
    3. Know the author's email address, which is mine, and start the publishing process if the signature is successfully verified.
    4. Save every step in a trace and provide feedback to the user.

    This is the simplest but incorrect approach.

    The agent's settings store a variable list of those who can contact the agent, specifically my email address. This is acceptable in this case because my email address does not change often. But if access to agent services were given to other agents, this approach would not work.

    Overall, this is very similar to what I already have running, but without signature verification and observability, so launching it in this form could be relatively quick.

  • Instead of using a small number of agents, I am building an infinite network of agents that operate autonomously and interact with each other on their own. I expect each person to have several thousand such agents. Some agents will be personal. Others will belong to a company, community, or state.

    Each task that can be solved independently of the others can be implemented as a separate agent. Other agents, and people, can then ask it to perform that task.

    This network must be decentralized, or at least federated. There must be no single point of failure that stores all the data from all agents. It would be vulnerable and would be taken down first.

    Adding a new agent must not affect existing ones. This immediately rules out any solution where each agent knows who can contact it. Otherwise, adding a new agent would require redeploying every agent it can access.

    Agents must use existing, time-tested protocols.

    I expect each agent to be available through two interfaces:

    1. email, for communication with people
    2. A2A

    These two protocols seem to overlap in part. We may need to find a way to combine them.

    Authentication and authorization must also be decentralized. For email interactions, we could include a set of permissions available to the device in its signing certificate. However, this seems quite difficult when permissions need to be updated, since that would require reissuing certificates.

    This question remains open for now.

    We may also need to add chats using Matrix so agents can report their status promptly. However, an observability tool may be able to handle this. It could send logs to Matrix, so each agent would not need to support Matrix.

October 2, 2026

  • I am thinking about a tool that connects mechanical processes, agents, and people.

    Every agent run through Hermes or another harness costs money. But besides intellectual tasks, we also have mechanical work: CI checks, monitoring, and continuous model training in a loop. There is a process defined by code that runs with specific parameters and produces intermediate messages and a result.

    The idea is to separate tasks performed by intelligence from tasks performed by mechanics, then connect them.

    The simplest loop looks like this: a mechanical runner does the work and sends the result to an agent. The agent provides feedback that affects the next run.

    image0.png

    But a person is often needed here. They need to see both the result and the feedback. They can say, "Yes, I like everything." Or, "Wait, you queued the next task differently from how I want it. Let's fix it."

    My solution is a shared message bus. In our case, it is Buzz, but it could be Telegram, Zulip, or another communication platform that people can use.

    image1.png

    Agents and several runners also work on the same platform. There can also be several people and agents.

    The runner reads the config and creates a separate thread for each run. It posts the run parameters there, then intermediate messages, and finally the result. All of this happens mechanically.

    The agent sees the messages in the thread and responds to them. It can simply reply, "Got it, everything is okay." If something is wrong, it can change the config, stop the current run, and restart it with different parameters.

    A person watches this in the same thread and can step in at any time. For example, they can write, "No, that's enough. Stop it." The person's messages also reach the agent, which can decide what to do next.

    The runner's loop itself follows the code: read the config, start the run, publish the result. If everything is okay, it keeps the cycle going. It creates a new thread for the next run.

    image2.png

    The mechanics run, the agent sees their messages through the message bus, and people can observe, adjust, and audit the entire process. We use Buzz as the message bus. A play on words.

    This is roughly how we use it, and we really like it.

September 27, 2026

  • Another question I have is how exactly to launch agents. In other words, how to turn a description into a set of running agents, either in a k3s cluster or in microsandbox.

    Suppose we already have a description of the agent team.

    Now we need to bring the cluster into the state that matches this description.

    I see two approaches here.

    The first is static.

    We run run. It takes the description and generates a regular set of Kubernetes manifests.

    We can inspect the result, verify it, put it in a repo, and then let a GitOps controller apply this state to the cluster.

    The second option is much cleaner architecturally.

    We could build our own Kubernetes controller that reads the description directly and continuously reconciles the actual state of the cluster with the desired state.

    But right now, I do not think we need to start with the second option.

    The static option is much easier to understand.

    Run run.

    Get a concrete result.

    See which pods and other objects should appear.

    Check exactly what the system is going to do.

    After that, if it becomes clear that we really need a continuous reconciliation loop, we can move the same model into a controller.

    So my next step is not to "write an orchestrator."

    The next step is much simpler.

    First, I need to describe the target state of the system after run.

    What exactly do we want to see in the cluster?

    Then we can decide which mechanism is best for bringing the cluster into that state.

    First, the desired state.

    Then, reconciliation.

  • At first, I thought of an agent description mainly as build instructions. There is a description. We use it to build the agent image. The image includes the environment, tools, and everything else the agent needs to work.

    But now it looks like the description is needed for much more than the build.

    In fact, we have at least three operations:

    build

    We build the agent image from the description fully automatically.

    onboard

    We need to create all the external entities through which the agent will exist in the system.

    For example, if the communication layer is Telegram, we need to create a bot with the right name. If it is Zulip or some internal tool, it will have its own equivalent.

    This is where issues can arise that cannot be properly handled during the build. For example, the name may already be taken. Or we may need to create a secret and put it in secret storage.

    So onboarding looks more like an operator task.

    run

    The same description is needed here again.

    An image alone is not enough to run the agent.

    We need to know what the agent is allowed to do, what policies it has, which secrets to connect, what resources it needs, and what state the infrastructure should be in after launch.

    So the description ends up serving as instructions for several systems at once.

    For the builder.

    For onboarding.

    For the orchestrator.

    This changes how I think about the agent description language.

    It is no longer a config for a Docker image.

    It is the source of truth from which a working system is created.

  • I am thinking about the agent session as an object and the level at which it should exist.

    There are two options.

    First, a session is a harness-level object. There is one permanently running agent instance, and parallel sessions are created inside it.

    Second, a session is an infrastructure-level object. Each session gets a separate pod or sandbox and effectively becomes a separate agent instance.

    The first option is easier to implement, but the more I think about it, the more I prefer the second one.

    If sessions live inside the harness, the harness suddenly has to do a lot. It has to create sessions, manage them, restore them, and handle concurrency.

    This also creates an unpleasant coupling. If the agent pod goes down, all the sessions running inside it may go down with it.

    If each session is a separate infrastructure object, everything becomes much simpler.

    A session is a fully isolated environment. It has its own environment, repositories, variables, and possibly secrets.

    If one session fails, the others do not notice at all.

    Most agents do nothing most of the time. If an agent instance is created only while it is working, we do not need to keep resources allocated for every role all the time.

    We may have a thousand or 50 thousand defined roles. That does not mean we need to keep 50 thousand pods running.

    Resources appear only when an agent is actually working.

    Of course, orchestration then has to move somewhere else. We need a separate layer that can create these agent instances and manage their lifecycle.

    I do not yet fully understand what it should look like.

    But the principle itself now seems quite clear to me:

    an agent session is an infrastructure object, not an internal harness abstraction.

  • Watching how my agents work, I have arrived at a concept I call the Boss session. This session does no hands-on work. It only watches all the other workers and gives them instructions in natural language.

    This may look like using subagents, but it is actually the opposite approach.

    Unlike subagents, workers are autonomous units. They have their own context, access, and sessions.

    Workers listen to the Boss and interpret its commands based on their own context. They do what their role requires and publish a brief result of their work, reporting it to the Boss. This removes all micromanagement from the Boss and lets it focus on the work. It implements the concept I described here: https://andysmith.ai/2026/Sep/25/separating-core-work-from-technical-work/.

    The Boss session uses the most expensive and best model available, so every token is very expensive. The workhorse models are chosen for the task, and their sessions are short-lived.

    Ideally, this approach will maximize token savings. This is very relevant now. Even though Opus 5.5 is quite efficient, I think the limits will be cut very soon, and it will be very painful, as it was in early September.

September 26, 2026

  • I am reading Navalmanack with my own eyes, without agents. I am now at the part about finding and endlessly refining my Special Knowledge.

    I realized that social networks such as X/Bluseky/Threads/Instagram/Telegram are a good tool for finding and confirming that Special Knowledge.

    Every social media post reflects my experience, a unique step that sets me apart from everyone else. Together, these steps form my unique path from birth to where I am now.

    So every post should be more than just a thought or an idea. It should confirm something I have done in the real world. It should reflect a change, an improvement, something that gives me experience.

    So that when I scroll through my feed many years later, I can say, "Wow, did I do that? And that too?"

    Through the lens of these small steps, small wins, and small changes, I will find what, as Naval puts it, I am the best in the world at. It must also be something that can be scaled and turned into a product, because if it is only interesting to me, it is not Specific Knowledge.

    Once I find it, the best place to put it is in the Headline, so that everyone, especially me, can see it. And, of course, I will refine it regularly.

  • An open plain-text format for storing health, workout, and sleep data could be a good alternative to Apple Health. It would avoid vendor lock-in and be easy to integrate with agents.

    ledger/hledger and their transaction format could serve as a reference, adapted for health data.

    It could store both aggregated data and raw metrics.

    The format should be simple enough to enter data manually, through agents, or with basic exporters. At the same time, it should be flexible enough to store different types of data.

    It should also support machine validation so that reports and aggregations can be generated automatically.

    I wonder what ready-made options already exist in this area?

  • I can view the space of possible options as a graph, and research aimed at solving a problem as finding a path through that graph.

    At any given time, there is a set of visited vertices. Following either the WFS or DFS algorithm, I can choose the next vertex to visit, which is the next question to explore.

    Breadth-first search performs better in terms of speed and number of steps.

    But people, including me, still prefer depth-first search because it feels more natural.

    This observation is counterintuitive, but it is supported by graph theory. So I need to stop myself whenever I want to go deeper instead of broader.

September 25, 2026

  • Very often, instead of doing useful work, my agents get distracted by technical tasks: creating issues, pull requests, and commits. This fills up the context and reduces the efficiency, and therefore the quality, of their work.

    I have tried different approaches. For example, I moved technical tasks into a separate tool that the agent can call. All the complexity stays inside the tool, and the agent only gets the result. But this does not solve the problem because the agent still needs to "think" about calling the tool, which takes attention away from the main flow of work.

    It seems that the only option is to move technical tasks one level above the agent. In other words, an external layer, which I think will be oml.sh, runs outside the agent, prepares the environment for it, monitors it through ACP, and handles technical tasks in a separate session.

    This lets the agent focus on its core work, while the environment handles the setup and related tasks.

  • There is the good old KISS principle, which I always forget. As a result, I end up with overcomplicated solutions that solve problems that do not exist yet.

    How do I deal with this? Especially in the age of agents, when adding bloat is free, but cutting it back is expensive?

    Am I the only one with this problem, and need to fix myself, or is it a widespread issue, and we need to fix our processes?

  • I tried to build smith.wiki as the single llmwiki, aka Zettelkasten, for all my research. Within just two days, it turned into a mess that was impossible to navigate.

    I think this is because an agent writes far more than a person, or even another agent, can read and review. Garbage multiplies on an unprecedented scale.

    Now I am trying a new setup. I publish each independent research thread as a repository in the smith-wiki GitHub organization. Since the CNAME is set in the main repository, all repositories are automatically published at smith.wiki/. A repository can be public or private. The old setup did not allow private research.

    The main index only provides a list of research projects and possibly a general index, such as a list of pages in each project. The research projects themselves are autonomous and can use any structure.

September 24, 2026

  • I plan to try running Jev inside EMACS so that it predicts the next command to execute.

    The input is the screen state + task, along with the possible next commands. The expected output is the probability of the next command.

    Since every action in EMACS is a command, Jev can do anything this way, absolutely anything.

    The only problem is that there are an incredible number of commands, and it is unclear how Jev will handle this.

    This might be optimized somehow. For example, the commands could be organized into a tree, first selecting a group of commands and then the right command within that group.

September 23, 2026

  • What if, after entering the wrong password, a hacker saw seemingly valid, but outdated, data?

    The authentication process currently includes a quick check. You only need to enter the wrong password to learn that it is wrong.

    What if every password were correct?

    What if, after entering the wrong password, a hacker saw documents, emails, and data generated by AI and unique to each password?

    This would make hacking much harder, especially when the hacker does not know what to look for and therefore cannot quickly verify whether the password is correct.

  • Claude Opus 5.5 and GPT 6 Sol are out.

    Claude has already appeared in omp for me, even though I have not updated it in a long time, while openai-codex still has no updates.

    They say they are cheaper and better now, but I will test them myself tomorrow.

    In recent weeks, I have really noticed the lower limits. It turns out that the 50% free allowance over the summer made a big difference. Or maybe I have started working more.

    So the lower token prices come at just the right time.

  • To be honest, “sending an email to a blog” sounds very outdated.

    Straight out of the last century.

    Still, I have a hypothesis that email is an undeservedly forgotten communication channel, and a very convenient and well-designed one.

    I’ll try it for blog posts, and then maybe move all communication with agents to email.

  • I’m trying to test my new publishing workflow.

    I sent this post by email.

September 19, 2026

  • The idea is that the questions I ask may be interesting to others, not just me.

    I developed this process: I post a question in a dedicated Zulip chat. A researcher picks it up, publishes the question, and starts researching it. The researcher then gives a short answer and links to the full research.

    I chose Threads as the publishing platform because I thought it allowed free publishing through its API, unlike Twitter.

    My agent was blocked after its third automatically published post :)

    So I realized that I should not do this on centralized social media platforms.

    I’ll try Bluesky.