Skip to content
PatMartinPublic

About

The APIs and contracts for OPAL.

Resources

Stars

0 stars

Watchers

0 watching

Forks

Repository files navigation

opal-api

Validate OpenAPI contracts

OpenAPI contracts and validation tooling for the OPAL suite: speech recognition, visual perception, reasoning, event routing, and speech synthesis.

This repository defines the HTTP interfaces that OPAL services and their clients agree to implement. It contains five OpenAPI 3.1 service specifications, a shared contract kernel, documentation, and automated checks. It is intended for service implementers, client authors, and contributors evolving the interfaces together.

The current service contracts and kernel are version 2.0.0. These files describe required and optional service behavior; passing the checks here verifies the contracts, not a running implementation. Service servers, the opal command-line application, and a browser application are not implemented or generated by this repository's scripts.

Contents

Capabilities

  • Reusable contract definitions: services reference shared headers, errors, schemas, and route definitions rather than maintaining separate copies.
  • Consistent service management: health, identity, configuration snapshots, settings, profiles, personas, and process lifecycle routes appear across the suite.
  • Traceable requests: four correlation headers connect HTTP exchanges and derived events across services.
  • Explicit concurrency and lifecycle semantics: configuration changes use strong ETags; asynchronous admissions, terminal results, cancellation, and streaming behavior are documented separately.
  • Defined action execution boundary: Think uses Event's registered actions and execution records, with a declared principal that can discover and invoke actions but cannot administer them.
  • Validation for authors and consumers: structural linting, suite-specific lint rules, standalone bundles, schema fixtures, and contract regression tests run without service processes or model downloads.

Services and architecture

The service contracts

All files below declare OpenAPI 3.1.0 and contract version 2.0.0.

Contract Responsibility Main resources and operations
Event Event publication, routing, and registered action execution Event ingress/history, topics, rules, shell/HTTP/Python actions, execution records, statistics
Listen Wakewords, speech-to-text (STT), intent recognition, and slot extraction Recognition start/stop/status/stream, text/audio diagnostics, intents and phrases, slots and aliases, regression cases, failed utterances, performance
Look Visual observation: video-only or inference RTSP streams Optional engine discovery (YOLO, RF-DETR, RT-DETR, D-FINE adapters), engine-owned models, single-image prediction, engine-local hardware, ephemeral stream sessions with SSE metadata, MJPEG viewing, snapshots, and metrics
Say Text-to-speech (TTS) over host-installed engines Engine discovery/configuration, voice registration and validation, completed audio generation, progressive PCM streaming
Think Bounded reasoning work using configured models and Event actions Asynchronous tasks, progress/output streams, cancellation, optional conversation sessions, model discovery
Look-YOLO Standalone YOLO detection engine adapter Engine discovery/capabilities, models, hardware, prediction, and the shared configuration operations
Shared kernel Common contract building blocks Parameters, response headers, error responses, schemas, reusable Path Items, and suite extensions

The shared kernel has no public routes of its own (paths: {}). Service documents mount reusable Path Items—OpenAPI definitions of operations at a path—and reference common components through external $ref links.

openapi/shared/opal_kernel.yaml
            |
            | external $ref references
            v
openapi/opal_{event,listen,look,say,think}.yaml
openapi/engines/opal_look_yolo.yaml
            |
            +--> Redocly and Spectral validation
            +--> Python contract assertions
            |
            +--> npm run bundle --> dist/opal_*.yaml
                                    standalone consumer documents

How services cooperate

Listen can publish live microphone intent matches. Event routes matching events to registered actions. An HTTP action can submit a task to Think; Think can in turn discover and invoke Event actions through REST.

Listen -- live intent events --> Event -- registered HTTP action --> Think
                                     ^                                  |
                                     +-- action discovery/execution ----+

Event owns the action definitions and execution records. Its think principal has actions:read, actions:execute, and executions:read permissions, with configuration write scopes explicitly denied. Action definitions can include a description, keywords, suggested prompts, and a JSON Schema describing EventInput.data.

Several distinctions matter when building integrations:

  • Admission is not completion. Event publication and action execution use 202 Accepted. Think task creation also returns 202, with a durable task ID and Location. Consumers inspect the appropriate task or execution record for the result.
  • Events are best effort. The contract does not promise delivery, ordering, automatic retries, or exactly-once side effects. Listen's external publication is disabled by default; diagnostic text/audio requests never publish actionable events. Look has no event-bus surface in the MVP Plus contract.
  • Streams have different guarantees. Look stream Server-Sent Events (SSE) support bounded replay with Last-Event-ID; Think task SSE has no guaranteed replay, and the final task snapshot remains authoritative. Say streams raw signed 16-bit little-endian mono PCM audio.
  • Capabilities can be optional. Look's inference engines are optional discoverable adapters — video-only streams work with zero installed engines. Say engines and voice resources are installed on the host; its API registers and uses them rather than downloading models. Say returns audio bytes; playback is outside its API.

Repository layout

openapi/
  opal_event.yaml                  Event contract
  opal_listen.yaml                 Listen contract
  opal_look.yaml                   Look contract
  opal_say.yaml                    Say contract
  opal_think.yaml                  Think contract
  shared/opal_kernel.yaml          Shared components and reusable Path Items
tests/
  spec_util.py                    Resolves references and discovers module specs recursively
  bundle_specs.py                 Bundles every discovered module spec into dist/
  test_config_standard.py         Suite-wide configuration-interface conformance
  test_lint.py                    Mutates specs to exercise lint rules
  test_event_150.py                Event action schemas, fixtures, and semantics
  test_think_event.py              Think/Event integration boundary
  test_look_cli.py                 Look CLI metadata and route parity
  test_listen_150.py               Listen settings concurrency
  test_say_150.py                  Say native JSON settings
  test_rest_security.py            Suite authentication declarations
  fixtures/event_150/              Valid, legacy, and invalid action payloads
docs/
  COOKBOOK.md                     CLI reference and cross-service recipes
  ECOSYSTEM_COOKBOOK.md           Multi-service recipe book
  create_cli.txt                  Documentation-generation prompts
.github/workflows/openapi-validate.yml
                                  CI validation workflow
package.json / package-lock.json   npm scripts and pinned tool dependencies
requirements.txt                  Python test dependencies
redocly.yaml                      Structural validation configuration
.spectral*.yaml                   Generic, suite, Look, and Think lint rules

dist/ is generated by bundling and ignored by Git. node_modules/, .venv/, and Python caches are also ignored. Edit the source contracts in openapi/, then regenerate bundles.

Requirements

Tool Requirement
Node.js Prefer 22.12.0 or newer in the Node 22 line, matching CI's Node 22 configuration. The pinned Redocly CLI accepts >=22.12.0 or >=20.19.0 <21.0.0.
npm 10 or newer, as required by the pinned Redocly CLI
Python Python 3 with pip and venv; Python 3.11+ is recommended. The project does not pin a minimum Python version, and CI requests 3.x.
Git Needed to clone the repository and submit changes

package.json pins @redocly/cli to 2.66.1 and @stoplight/spectral-cli to 6.17.0; npm ci installs the dependency tree in package-lock.json. Python dependencies are PyYAML>=6.0 and jsonschema>=4.21, declared in requirements.txt.

No camera, microphone, model provider, broker, credentials, or running OPAL services are needed to validate this repository.

Installation and quickstart

Clone the repository and install the Node tooling:

git clone https://github.com/PatMartin/opal-api.git
cd opal-api
npm ci
python -m venv .venv

Activate the Python environment using the command for your shell.

Bash or zsh on Linux/macOS:

source .venv/bin/activate

PowerShell on Windows:

.\.venv\Scripts\Activate.ps1

Git Bash on Windows:

source .venv/Scripts/activate

Then install the Python dependencies and run the complete check:

python -m pip install -r requirements.txt
npm run check

The successful end-to-end result is an exit status of 0, passing contract assertions, and five bundled YAML documents under dist/. Lint warnings can still appear on a successful run.

There is no server start command for this repository. Its executable workflow is validation and bundling; the service/start API routes describe behavior for service implementations.

Working with the contracts

Validate and build consumer artifacts

For a quick structural check followed by bundle generation:

npm run validate
npm run bundle

The bundle script writes:

dist/opal_event.yaml
dist/opal_listen.yaml
dist/opal_look.yaml
dist/opal_say.yaml
dist/opal_think.yaml

Each bundle incorporates external kernel references into a single document suitable for OpenAPI viewers and downstream tooling. Internal references can remain: the script uses Redocly bundle without its --dereferenced option. These files are build artifacts, not separately maintained source specifications.

Inspect a resolved operation

The test helper resolves local and shared-base references, copies path-level CLI mappings onto the corresponding operations, and substitutes modules.<module-key> with each module's declared profile compartment. From the repository root, run:

python -c "import sys; sys.path.insert(0, 'tests'); import spec_util; d = spec_util.load('openapi/opal_think.yaml'); print(d['info']['title'], d['info']['version']); print(d['paths']['/api/v1/task/']['post']['x-opal-cli'])"

Expected output:

OPAL Think API 2.0.0
{'group': 'task', 'command': 'add', 'aliases': ['create', 'ask', 'chat']}

tests/spec_util.py is a repository-specific test utility. Use the generated bundles when a consumer needs a portable standalone OpenAPI document.

Understand CLI metadata

The kernel's x-opal-cli-format extension defines the mapping:

opal <service> [parent] [group] <command>

For example, Think's task creation operation declares:

x-opal-cli:
  group: task
  command: add
  aliases:
  - create
  - ask
  - chat

Descriptors have a required command and optional parent, group, and aliases. Kernel Path Items carry per-operation descriptors directly, making the kernel the single source of truth for every shared-operation binding — config/config setting, info health, service, and the profile lifecycle. For profile --module FILE, the kernel binds modules.<module-key> and each service substitutes its declared x-opal-profile-namespace.compartment (e.g. modules.opal-say on opal-say). Module-declared bindings on $ref mounts may instead use a path-level per-method map; the test helper hoists that map onto operations for assertions.

Look and Think currently annotate all their operations. Event, Listen, and Say have partial annotation coverage. The general Spectral rules validate descriptor shape where annotations exist, while the Look parity test explicitly checks operation coverage and command uniqueness.

The cookbook describes the intended unified Typer CLI, a Python command-line interface, and its workflows. This repository supplies metadata and documentation; installing its dependencies does not install an opal executable.

Shared API conventions

The kernel describes OPAL Suite Standard v1.2. This standard version is distinct from the files' info.version values, the OpenAPI format version, and event metadata versions. The private npm package's 1.0.0 version identifies the tooling package rather than the API contract.

Routes and common management

Routes use /api/v1, singular resource names, and trailing slashes on collection roots, such as /api/v1/task/. All five services expose these common families, except where noted:

Family Contract surface
Health GET /api/v1/health/ — unified report (info health): name, version, api_version, status, live, ready, details; GET /api/v1/health/live, GET /api/v1/health/ready — HTTP-only machine probes
Configuration POST /api/v1/config/save, POST /api/v1/config/load, POST /api/v1/config/validate — save/load/validate a portable configuration snapshot (managed store or client file)
Settings GET /api/v1/setting/; GET/PUT/DELETE /api/v1/setting/{key}
Profiles Collection and item CRUD (create, read, update, delete), plus activation
Personas Collection and item CRUD, plus service-local binding validation — mounted by every module including Look and Look-YOLO
Service lifecycle POST /api/v1/service/start, POST /api/v1/service/stop, GET /api/v1/service/log

Say and Look also expose engine listing and item read/update operations. Every published module contract — the five services and the nested Look-YOLO engine adapter — mounts the same seven configuration operations defined once in the kernel: config save, config load, config validate, and config setting list|show|set|remove. Persona operations are mounted from the kernel's persona_collection, persona_item, and persona_validate Path Items with kernel-owned persona CLI bindings.

Correlation, concurrency, and errors

  • Correlation: X-Opal-Session-Id, X-Opal-Transaction-Id, X-Opal-Request-Id, and X-Opal-Causation-Id carry context. Request and transaction response IDs are required; session and causation IDs are conditional. A trace session ID does not itself create a service-managed session.
  • Configuration revisions: strong ETags identify portable configuration revisions. Configuration mutations require an exact strong If-Match validator: missing is 428, stale is 412, and malformed is specified as 422. Runtime admissions and data operations have their own concurrency rules.
  • HTTP outcomes: resource creation uses 201 with Location; asynchronous admission uses 202; empty deletion uses 204. Consult the operation for its exact response headers and statuses.
  • Errors: JSON error responses use the OpalError envelope. Health probes are a deliberate exception: their 503 response uses a status/ready health body.
  • Collections: bounded configuration catalogs return full arrays, subject to deployment caps. Histories use bounded arrays and limit, with operation-specific ordering. The contracts do not expose cursor pagination.
  • Authentication: non-loopback deployments require authentication; loopback deployments may explicitly disable it. 401 means missing or invalid credentials and 403 means insufficient permission. Correlation IDs never grant access.

The event envelope extension describes two ZeroMQ frames: a topic and a JSON envelope. Optional correlation metadata lives under data.metadata, and consumers must ignore unknown metadata fields.

Profiles and personas

A profile is a portable document of named presets: a shared section plus one compartment per module (modules.<module-id>), stored locally by each service. At most one profile is locally active at a time; its shared values and the service's own compartment fill in settings a request leaves out. Compartment schemas are cataloged in openapi/shared/opal_kernel.yaml and keyed by canonical module identity; foreign compartments are preserved verbatim for transfer. A persona is an identity that can carry service-specific bindings, such as a Say voice. The kernel defines per-field resolution as:

explicit request field > persona binding > selected profile's module override > selected profile's shared value > service default

The ownership contract names Think as the authoritative persona registry. Services can keep local records for standalone use or fetch and revalidate owner records when an owner URL is configured. Each service's persona validation reports its own binding resolvability. The concrete request schemas determine which bindings and selectors a service accepts. The v2→v3 flat-to-namespaced profile migration is recorded in docs/PROFILE_MIGRATION.md.

Configuration

Repository tooling

No application environment variables or secret configuration are required for the checks. The validation tools read these repository files:

File Role
redocly.yaml OpenAPI structural checks using Redocly's recommended rules
.spectral.yaml Generic OpenAPI linting
.spectral.opal.yaml Suite conventions, including correlation headers, error shapes, creation headers, and CLI descriptor format
.spectral.opal.look.yaml Look-specific concurrency, health, and event-contract checks
.spectral.opal.think.yaml Think-specific concurrency and health checks

Trailing-slash lint rules are disabled because the suite requires collection trailing slashes. Spectral's oas3-schema rule is disabled for the external Path Item reference issue documented in the rulesets; Redocly still performs structural validation. Redocly's security-defined rule is also disabled, while the Python security suite checks the explicit authentication declarations.

Service configuration described by the APIs

Actual service deployment configuration belongs to the implementations. At the contract level:

  • Settings use native JSON values. config setting set validates and immediately persists one override; config setting remove deletes the override and restores the registered default. Setting records report value (effective), configured_value, default, source (default|persisted|dotenv|environment), and restart provenance.
  • Configuration save/load replaces the former backup/restore and export/import surface. config save writes one managed recovery snapshot (or returns it for --out); config load atomically replaces the full portable configuration from the managed snapshot or a supplied file snapshot, requiring If-Match; config validate checks live configuration or a candidate snapshot without side effects. Snapshots contain persistent desired configuration only: secrets, history, runtime IDs, and generated artifacts are excluded.
  • Credential references such as token_env and Think provider credential_ref identify configured secrets; secret values must not be returned.
  • The shared service-start options default append to false, truncating logs/opal_<service>.log; log retrieval defaults to 10 trailing lines.

Check each spec's info.description, request schemas, and extensions for service-specific settings and limits. Server URLs in the specs are local examples, not a deployment plan for running all five services on one machine.

Configuration migration (v1.7.1 → v2.0.0)

Version 2.0.0 replaces the backup/restore and export/import families with a single configuration interface that every runtime module implements identically — including nested engine contracts under openapi/engines/. The old route families and CLI groups are removed, not aliased.

Retired (v1.7.1) Replacement (v2.0.0)
backup create / POST /backup/create config save / POST /api/v1/config/save
backup export --out F / GET /backup/export config save --out F / POST /api/v1/config/save (destination: response)
backup restore <file> / POST /backup/restore/{filename} config load --in F / POST /api/v1/config/load (snapshot body)
backup import --file F / POST /api/v1/config/import config load --in F / POST /api/v1/config/load
backup validate / GET/POST /backup/validate config validate [--in F] / POST /api/v1/config/validate
backup list/download/remove none — one managed snapshot per module; no archive management
config setting update / PATCH /api/v1/setting/ config setting set --name K --value V per key, or config load for a full snapshot
config <resource> ... <resource> ... — resource groups sit directly beneath the module command

Snapshots are portable desired-configuration envelopes (format_version, module, created_at, settings, resources). They do not restore the operational history that whole-database backups previously included; that recovery-scope change is intentional.

Validation and tests

All commands run from the repository root. The Node CLIs are installed locally by npm ci; global installations are unnecessary.

Command Purpose
npm run validate Redocly structural linting and reference resolution for every published module spec (top-level services, configuration-only contracts, and openapi/engines/)
npm run lint Generic Spectral OpenAPI rules
npm run lint:opal Suite Spectral rules on service specs
npm run lint:kernel Suite Spectral rules directly on the kernel
npm run lint:look Look-specific Spectral rules
npm run lint:think Think-specific Spectral rules
npm run bundle Build a standalone document in dist/ for every discovered module spec
npm run test:config Configuration-standard conformance on every discovered module spec, plus a negative-discovery proof
npm run test:lint Mutate temporary spec copies and check that selected lint rules report the expected findings
npm run test:event150 Event action metadata, JSON Schema fixtures, references, input validation semantics, permissions, and settings
npm run test:think-event Declared Think/Event scopes, integration routes, admission semantics, and execution states
npm run test:look-cli Look operation CLI mappings and selected operational routes, parameters, and error declarations
npm run test:listen150 Listen native JSON settings and concurrency preconditions
npm run test:say150 Say native JSON settings
npm run test:rest-security Bearer authentication declarations, suite authorization semantics, and 401/403 response coverage
npm run check Run all 15 stages above, stopping at the first failing command

The 150 script/file names are historical; those suites load the current contracts and assert version 2.0.0. Tests inspect schemas and declarations rather than sending requests to running services. A passing suite does not establish runtime conformance or prove every behavioral requirement.

The GitHub Actions workflow runs the same set of checks on Ubuntu with Node 22 and Python 3.x. Unlike the local && chain, CI runs validation stages with continue-on-error and aggregates their outcomes. It runs for pull requests, manual dispatches, and pushes to main, integration/**, and proposal/**.

Spectral warnings are non-blocking under the current commands; error findings fail the check. Read warnings when changing related areas rather than treating a successful exit as a warning-free result.

Troubleshooting

Symptom What to check
Redocly reports an unsupported engine Check node --version and npm --version against the requirements above
Python cannot import yaml or jsonschema Activate the intended environment and run python -m pip install -r requirements.txt
Lint regression tests cannot find Spectral Run npm ci; the harness prefers node_modules/.bin/spectral or spectral.cmd
A new kernel reference does not resolve Check the reference path and fragment, then run npm run validate and npm run bundle
npm run check stops before a later test Fix the failing stage, or use that later stage's individual npm command to inspect it

Development and contributing

  1. Read the relevant contract and nearby tests. Start with the service's info.description, its operation schemas, and the shared kernel. Check both structured definitions and prose when changing behavior.
  2. Choose the right source location. Keep service-specific schemas in the service spec. Add reusable components to the kernel and reference them with paths such as ./shared/opal_kernel.yaml#/components/schemas/OpalError. Shared edits can affect every service.
  3. Preserve interface conventions. Review correlation parameters and response headers, authorization errors, error envelopes, item-not-found responses, configuration preconditions, creation headers, and CLI metadata where applicable.
  4. Extend meaningful coverage. Reuse the matching Python test file. Event action payload fixtures belong in tests/fixtures/event_150/; new lint guarantees should have a mutation case in tests/test_lint.py. Use spec_util.load when assertions need the resolved contract.
  5. Review compatibility and documentation. Changes to required fields, routes, response types, operation IDs, or CLI bindings can affect clients. Update affected descriptions, examples, version assertions, and cookbook material when revising the published contract.
  6. Run npm run check and review the diff. Generated dist/ files stay ignored. Submit source changes and relevant documentation/tests in a pull request.

The repository has no configured formatter or type-checker command. Follow nearby YAML and Python style and use the validation commands above.

Existing branches and CI support proposal/<topic> and integration/<topic> naming. The history contains both plain descriptive commits and prefixed subjects; no mandatory commit-message format is configured. Pull requests should explain the concrete contract change, compatibility implications, and validation results.

The security test also scans openapi/proposals/*.yaml, but most validation scripts target only openapi/*.yaml. If a change introduces a new contract location, update the relevant script and test inputs so its intended coverage is explicit.

Documentation and support

License

package.json declares ISC. All specifications omit the info.license field; the repository does not include a standalone license file.

About

The APIs and contracts for OPAL.

Resources

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages