Skip to content

feat(chats): add the coordinator agent - #13158

Open
mudler-agent wants to merge 42 commits into
masterfrom
feat/coordinator-agent
Open

mudler-agent wants to merge 42 commits into
masterfrom
feat/coordinator-agent

Conversation

@mudler-agent

Copy link
Copy Markdown
Collaborator

What this adds

A coordinator is an agent that keeps working on its own for a long time. It wakes on a schedule and when someone answers, starts helper agents for big tasks, writes down what it did, and contacts a named person or team only when it needs them.

The person who creates it writes its behaviour: system prompt, standing rules and topics. LocalAI decides no policy. It enforces only what the creator configured: who it may contact, how much it may send, a pause switch and an audit log.

How it works

  • Creation. A coordinator is an agent plus a coordinator record (new package and tables). There is a create and edit page, an API and a card with state and usage. The agent struct is unchanged.
  • One long-lived chat. Wake-ups, contact replies and helper reports all arrive as messages in that chat. Its state lives in a page folder it reads at the start of each wake-up. Writing its own page is not a proposal.
  • Wake. A schedule (minimum 5 minutes), a contact's reply through a connector, or a helper finishing. A scheduled wake runs from the existing reaper leader tick, so only one frontend does it and there is no new lock key.
  • Contacts. The creator lists people and teams. A person is either a LocalAI user (inbox item) or an external identity reached through a connector. A reply from a listed contact in the listed chat goes back into the coordinator's chat. Inbox contacts cannot reply there; they can talk to the coordinator in its chat.
  • Guardrails, enforced by the server.
    • Send caps per minute and per day, and an audit log (90 days by default, queryable, records denied attempts).
    • A run budget and a cluster-wide cap on running coordinators, checked before a wake starts.
    • Pause: any admin can press it. It blocks all outbound, stops running turns and helpers, refuses new helpers, and holds what arrives (the owner's messages too) until resume.
    • Helper limits are settings (8 helpers, depth 2 by default; ceilings 16 and 3, set by the operator). A helper has the coordinator's rights at any depth and never gets the contact tool.
  • Recovery. Helper reports are stored rows with a lease. A leader-elected sweeper delivers them, also after a restart or a dead frontend, and the coordinator is told about stopped, failed and interrupted helpers.
  • Admin and assistant. An admin section lists coordinators, pauses and resumes them, and shows the audit. Four assistant tools: list, pause, resume, audit.

Decisions worth a second look

  • Coordinator chats always run on the frontend, never on a chat worker, so the run budget, helper limits and contact tool always apply.
  • In a cluster, chat workers and add-on owner processes no longer take the reaper lock, so the leader tick always lands on a frontend. They also no longer reap stale turns. Frontends do that.
  • There is no queue for coordinator sends: a send that would queue returns "try later".
  • Creation has no assistant tool: it is a user action on a feature route.

Known limits

  • The answer to a contact's reply goes out through the normal messaging path, so the send caps and audit do not cover it. Only the run budget limits it.
  • A user contact receives inbox messages without consent. Only the caps limit them.
  • Tools the creator grants besides contact (add-on actions, commands, web tools) are not limited by the contact list or caps.
  • A wake that falls due while the chat is busy is not retried until the next period.
  • The concurrency limit and the helper cap are read just before a turn starts, so two coordinators starting at once can exceed them by one.
  • A crash right after a report is sent, before it is marked done, can deliver that report twice.
  • Loops armed by helpers are not paused by Pause. I did not check whether they can restart a helper.
  • Upgrade all frontends together: an older frontend would not hold a paused coordinator or count its runs.
  • No real X or social connector is part of this. No real cluster was run; leader election, leases and the two-frontend behaviour are tested with a fake bus and shared stores.
  • New translations are AI-written and unreviewed.

How it was tested

  • Journeys on a two-frontend fake cluster with a scripted model and a fake messaging bot: scheduled wake, contacts, helpers, reply routing, pause and resume across frontends, caps and audit, budget, a dying frontend. Each journey was made to fail by deliberately breaking the rule it covers.
  • Go tests for every touched package, go test -race ./core/services/studio/..., golangci-lint with no issues, and the Postgres store suites run against a throwaway Postgres 16 (7 stores, all passing).
  • UI: unit tests (1358 passed), i18n and lint gates at baseline, and Playwright over all specs except the ignored ones: 3183 passed, 49 failed. 47 failures need a missing headless browser and also fail on master. The two others are flakes that fail intermittently on master too.
  • The UI coverage ratchet passes by hand (89% against a 40% baseline).
  • Not run: the strict Go coverage check, the full core/http Go suite (it times out here and on master), and any real connector or provider.

🤖 Generated with Claude Code

https://claude.ai/code/session_01DjrQyipS6MmvKffdvVGjCg

mudler added 30 commits November 8, 2026 21:50
Assisted-by: Claude:claude-sonnet-5-5

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Assisted-by: Claude:claude-sonnet-5-5

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Assisted-by: Claude:claude-sonnet-5-5

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Assisted-by: Claude:claude-sonnet-5-5

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
The label chats:triggersPage.runLimit was added in English only, which
raised the missing-key count of the chats namespace above its baseline in
the eight other locales, so the i18n gate failed.

Assisted-by: Claude:claude-sonnet-5-5

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
A paused coordinator holds every message to its chat, the owner's too,
as a waiting message. The wake sources (schedule, reply, helper) take
one run of the daily budget with an atomic counter, and are held when
the budget is spent or the cluster already runs the most coordinator
chats allowed. The owner's own message and a message that joins a
running turn are not counted.

Loop runs, resumes of interrupted turns and the waiting-message
continuation honour the pause. A chat worker gets the wake source and
reports a held start.

Assisted-by: Claude:claude-sonnet-5-5

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Pause sets the flag in the shared store first, so every frontend holds
messages and denies sends at once, then stops the chat turn, its helpers,
its goal and its active loops over the usual stop path. Resume clears the
flag and starts the held messages as one turn. The owner or an
administrator may press the switch. The audit read is limited to the
owner's own rows unless the caller is an administrator.

Assisted-by: Claude:claude-sonnet-5-5

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Assisted-by: Claude:claude-sonnet-5-5

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Assisted-by: Claude:claude-sonnet-5-5

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Assisted-by: Claude:claude-sonnet-5-5

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Assisted-by: Claude:claude-sonnet-5-5

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
…iling

Assisted-by: Claude:claude-sonnet-5-5

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Assisted-by: Claude:claude-sonnet-5-5

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Assisted-by: Claude:claude-sonnet-5-5

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Assisted-by: Claude:claude-sonnet-5-5

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Assisted-by: Claude:claude-sonnet-5-5

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Assisted-by: Claude:claude-sonnet-5-5

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
…posal

Assisted-by: Claude:claude-sonnet-5-5

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Assisted-by: Claude:claude-sonnet-5-5

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Create, read, replace, pause, resume and audit coordinators, with the
administrator's list, pause, resume and audit routes. The agent list
marks coordinators.

Assisted-by: Claude:claude-sonnet-5-5

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Four admin tools: list_coordinators, pause_coordinator,
resume_coordinator and list_coordinator_audit, in both clients, with a
skill recipe. Creating a coordinator is a user act on a feature route,
so it has no tool.

Assisted-by: Claude:claude-sonnet-5-5

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
A separate page for a coordinator: name and model, instructions with a note that the rules are the creator's, a contacts editor in a side sheet, wake schedule, caps and helper limits, tools and skills, a state card with pause and resume, and the audit log. The agent list shows a Coordinator mark from the list flag and sends Edit to the coordinator page. Texts are in all nine locales, including the administrator section added next.

Assisted-by: Claude:claude-sonnet-5-5

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Chats of all users lists every coordinator with its state, next wake and usage today. Pause asks first, because it stops everything and holds what arrives. The audit log loads on demand with filters and paging. A 404 leaves the section out and a failed read is shown in the section.

Assisted-by: Claude:claude-sonnet-5-5

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
A coordinator is its own inbox source with the robot icon, a message from one is titled with its name, and the coordinator tool group has a label in every locale.

Assisted-by: Claude:claude-sonnet-5-5

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Assisted-by: Claude:claude-sonnet-5-5

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Six journeys over a two-frontend cluster with a scripted model and a fake messaging bot: schedule, contacts, helpers, own page, reply routing, pause and resume, a frontend that dies, caps and the run budget.

Assisted-by: Claude:claude-sonnet-5-5

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
mudler added 12 commits November 9, 2026 00:12
Assisted-by: Claude:claude-sonnet-5-5

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
A chat worker has no coordinator host and no connectors, so a coordinator turn started there skipped the run budget charge, the helper limits and the contact tool. SendBackground and the resume of an interrupted turn now run a coordinator chat on the frontend. The start request no longer carries the wake source or a held flag.

Assisted-by: Claude:claude-sonnet-5-5

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
… spec

The audit rows of one spec leaked into the next on a shared database, so five specs failed on PostgreSQL while SQLite passed.

Assisted-by: Claude:claude-sonnet-5-5

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
A listed contact who wrote to the bot from a group got its reply routed into the coordinator chat, and the answer of the turn went back to that group. A contact is now the sender in the listed chat.

Assisted-by: Claude:claude-sonnet-5-5

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
A turn that was still running when the pause arrived could start a helper that the pause no longer stopped.

Assisted-by: Claude:claude-sonnet-5-5

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Assisted-by: Claude:claude-sonnet-5-5

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Assisted-by: Claude:claude-sonnet-5-5

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Assisted-by: Claude:claude-sonnet-5-5

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
# Conflicts:
#	core/http/react-ui/public/locales/id/chats.json
#	core/http/react-ui/public/locales/ko/chats.json

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
A connector poll that was in flight when leadership moved to another
frontend could return an update, and the old leader handed it over
although the new leader reads it again from the stored offset. The
message was then answered twice.

Check the lead before each update is handed over.

Assisted-by: Claude:claude-sonnet-5-5 go

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
A turn whose claim was reaped can still be unwinding in this process.
Its end releases the chat by key, so a resume that started in that
window was ended by the old turn and never ran.

Resume now reports that nothing started while this process still holds
a run for the chat. The caller asks again, as it does for any turn that
is still running.

Assisted-by: Claude:claude-sonnet-5-5 go

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
A turn gave up its admission under the shared agent's grant before it
released its claim. An owner's revocation answers once the admission is
released, so for a moment it answered while the chat still read as
running. Give up the admission after the claim is released.

The spec for this waited for the claim, which exists before the turn
reads the agent. A revocation in that gap lets the turn run without the
agent and end on its own, so the turn ended as done and not stopped.
Wait until the turn is blocked in its tool instead.

Assisted-by: Claude:claude-sonnet-5-5 go

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
@mudler
mudler force-pushed the feat/coordinator-agent branch from 9f0b607 to ff63c56 Compare November 9, 2026 09:23

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants