Repository navigation
Conversation
1fb7af1 to
17765c4
Compare
Discover trusted package manifests at runtime and reuse ConsoleModule for menus, routes and overview cards. Share host React and shell providers without adding plugin management UI. Fixes modelsphere#113
A Plugin CR is the registration; the Helm release is the runtime. Console watches modelsphere.dev/v1alpha1 Plugin objects, reads the installed release's plugin.yaml and frontend/ files from Helm's Secret storage, and serves /plugins/index.json, /plugins/<cr-name>/<asset> and /api/<plugin>/* through the release's ReverseProxy. Disabling or deleting the CR removes the frontend and routes without a Console restart. The CLI no longer edits Console Helm values. install applies only the host capabilities the package declares in plugin.yaml requires, and registration prints a CR for a separate apply.
The strict v1 decode used PluginMeta, which had no requires field, so every package that declared host capabilities failed inspection. Requirements now live on PluginMeta and the package manifest aliases that type.
console.yaml is mounted from a ConfigMap, and nothing in the pod template changed when it did, so a values edit that only alters plugins or backends stayed invisible until pods were restarted by hand. Hash the rendered ConfigMap into the pod template.
helm upgrade does not install crds/ on an existing release, so the readyz failure seen on CPU16 needs an explicit kubectl apply. Record the same for clearing the old values plugins list, which otherwise keeps a duplicate index entry beside the CR one.
Plugins now register only through modelsphere.dev/v1alpha1 Plugin CRs; Console reads each installed release's plugin.yaml and frontend/ files. - drop Console Helm values plugins, the config type, static/dev proxies, and the legacy modelsphere.io/v1alpha1 manifest conversion - build the plugin index and assets solely from the CR registry - take menus from SDK registration; descriptor keeps only file locations - remove the SDK Vite dev preset that targeted the deleted proxy
Plugins load after the shell's first render and add their bundles then. The sidebar computed its labels before that, kept the fallback literal, and never re-rendered: i18next lands bundles on the resource store, which emits "added" on itself, not on the instance. Subscribe to both languageChanged and the store's added, and route the SDK's subscribeLocale through the same function. Also serve the SPA shell with no-cache and content-hashed assets as immutable, so a tab opened before an upgrade revalidates instead of running the old bundle.
Plugins no longer ask the host for named capabilities with boolean flags. A provider — the platform or another plugin — declares what it publishes in `provides`; a consumer names what it binds to in `consumes`, optionally pinning the provider and product version with `from`. The CLI resolves each binding before Helm and writes the published values under `consumed.<group>/<name>.<key>`. - api: replace PluginRequirements with PluginCapability and PluginCapabilityBinding; regenerate deepcopy and the Plugin CRD - pluginpackage: validate capability ids, schema versions, value paths, provider kind and SemVer constraints - plugincli: resolve platform built-ins (image pull secrets, inference namespaces, Swiss gateway) plus registered plugins; a plugin declaration shadows the matching built-in; fail preflight on missing, ambiguous, out-of-range, cross-namespace or unpublished values - docs: describe the capability model in plugins.md, plugin-cli.md and plugin-platform-config.md
Console serves the bundle from the installed Chart's frontend/ directory, so a package without the declared entry can never load. validate, package and install now reject it before Helm runs. install also passes --atomic, so a timed-out upgrade rolls back and a failed first install is uninstalled instead of leaving a pending release that blocks the next attempt.
workflow_dispatch only: stamps the chart's image reference, lints it, packages it and uploads console-<version>.tgz. With push on it also builds the image and pushes it; any registry but GHCR authenticates with the REGISTRY_USERNAME and REGISTRY_PASSWORD secrets, and chart_repository optionally pushes the chart.
Discovery only needs the name, namespace and status labels Helm puts on release Secrets, but kubectl returned every Secret body: a release record carries the packaged chart and values, so the cluster-wide list here is megabytes. Over a slow API path the response is cut off mid-stream and install fails before Helm runs with a stream ID INTERNAL_ERROR. Read the list through client-go's metadata client instead, which asks the API server for PartialObjectMetadata and pages at 500 records. The same discovery now transfers 61 KB instead of about 8 MB and finishes in seconds.
Move the ReverseProxy types next to User, IAMRole and IAMRoleBinding in internal/iam, faithful to the CRD schema, and drop the separate api/iam package whose SchemeBuilder and generated deepcopy nothing used. Plugin registration parses the Helm-rendered manifest, so it only needs decoding, not scheme registration.
17765c4 to
c8d1d98
Compare
Codex Review Summary: ✅ CompletedThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: c8d1d98517
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| prefix := strings.TrimSuffix(route.Spec.Matcher.Path, "*") | ||
| if !strings.HasPrefix(prefix, "/api/") || !strings.HasSuffix(prefix, "/") || !route.Spec.Directives.AuthProxy { | ||
| return nil, fmt.Errorf("plugin ReverseProxy must authenticate an /api/ prefix") |
There was a problem hiding this comment.
Reject plugin routes that overlap Console-owned APIs
A registered chart can supply /api/iam/* or even /api/* and pass this check. Server.Handler runs pluginBackends before the built-in mux, so such a route captures requests such as GET /api/iam/users and authorizes them as backends/<plugin> instead of reaching the native handler. Existing config.validateBackend explicitly reserves these paths for static backends, but registrations bypass that protection. Reject Console-owned prefixes and broader prefixes covering them before publishing routes, so installing a plugin cannot take over session or IAM APIs.
Useful? React with 👍 / 👎.
| if bindsPlatformCapability(bindings, capabilitySwissGateway) { | ||
| provider, err := swissCapabilityProvider(console) | ||
| if err != nil { | ||
| return nil, err |
There was a problem hiding this comment.
Resolve plugin providers before validating the Swiss fallback
When externalSwiss.url is empty—the chart default—and an installed Plugin CR provides swiss.modelsphere.dev/gateway, a consumer with an unpinned binding still fails here. swissCapabilityProvider returns an error before runner.Registrations is queried, so preferPluginProviders never lets the real plugin shadow the built-in as documented. This prevents installing consumers against a plugin-provided gateway unless the unrelated built-in URL is also configured. Discover matching plugin providers first and validate the built-in fallback only when it is needed.
Useful? React with 👍 / 👎.
| const data = Object.fromEntries( | ||
| dataExtensions.map((extension, index) => [extension.id, queries[index]?.data]), | ||
| ); |
There was a problem hiding this comment.
Namespace extension results by their owning plugin
When two plugins both declare a data extension named owner for inference-service, both registrations are accepted because duplicate checks in createRegistration are plugin-local. This map nevertheless keeps only the last response because its key omits extension.plugin, and both plugins' presentation and visibility callbacks receive the overwritten data. A valid pair of plugins can therefore silently display another plugin's ownership or status results. Construct plugin-local data maps for callbacks, or introduce plugin-qualified keys consistently in the SDK contract.
Useful? React with 👍 / 👎.
| asset.onload = () => finish(); | ||
| asset.onerror = () => finish(new Error(`Cannot load ${asset instanceof HTMLScriptElement ? asset.src : asset.href}`)); |
There was a problem hiding this comment.
Reject execution failures before committing plugin registrations
If an entry registers its definition, component and extensions and then throws synchronously during the remaining initialization, a classic script reports that exception through the window's error event and still emits its element's load event. These listeners therefore resolve successfully: runtime.finish() and commit() retain the partial module and extensions, while loadPlugins reports no failure. Observe execution errors for the active entry as well as resource-loading errors before committing, so this scenario follows the intended per-plugin rollback path.
Useful? React with 👍 / 👎.
pluginBackends matches before the native mux, so a registered chart could claim /api/* or /api/iam/* and answer Console's own API as a backend. Reject the bare /api prefix and the prefixes Console serves itself, sharing the reserved list with configured backends.
A consumer of swiss.modelsphere.dev/gateway failed when externalSwiss.url was unset, even when an installed plugin provided the capability, because the built-in was validated before registrations were read. Discover plugins first and validate the built-in only when a binding still needs it.
Two plugins that both named a result "owner" shared one map key, so one overwrote the other for both visibility and presentation callbacks. Build the map from the owning plugin's extensions only.
A classic script that throws still fires load, so a partial registration committed and the failure went unreported. Watch window's error event for the active entry and reject before commit.
The guide was tied to one environment: a named cluster, a VPN step, personal paths, an internal Console URL, private branches and an internal registry. Rewrite it in English alongside the other developer docs, keep every CLI behavior, and use plugin.yaml 'my-plugin' as the running example so the same identity appears in all three plugin documents.
The chart_repository help named one team's registry. Keep the example generic so the workflow reads the same for every fork and every cluster.
…ixture The fixture carried an internal registry address. A documentation-range host exercises the same host:port split.
Verdict: Request ChangesThe overall direction of (Note on the 4 follow-up fixes since 1. Security & Public-Repo Blockers
2. High-Severity Correctness & Runtime BugsBackend, Helm & Upgrade Path
Plugin CLI & Packaging
Frontend Shell, Extensions & Plugin SDK
3. Medium Issues & Cleanups
Verified Clean
|
| # With push on the same run also builds and pushes the image, and pushes the chart | ||
| # when chart_repository names a target. | ||
| on: | ||
| workflow_dispatch: |
There was a problem hiding this comment.
I'd like to drop it, workflow_dispatch does not help. the lib's own publish will added to publish workflow, we can do it later after merged
zoboth
left a comment
There was a problem hiding this comment.
Blocker
A plugin's ReverseProxy route can silently hijack an existing configured backend's prefix (or another plugin's), because the only prefix checks are against Console's own reserved list.
internal/pluginregistry/release.go(readRoutes) only rejects a plugin route when the prefix is the literal/apior matchesconfig.ReservedAPIPrefix(/api/iam,/api/me,/api/router,/api/capabilities). It never checks the prefix againsts.cfg.ConfiguredBackends()(the statically configured backends mounted bymountBackendsininternal/server/proxy.go) or against routes otherPluginentries already publish.- In
internal/server/server.go, the handler chain iss.pluginBackends(pluginPaths(mux)), andpluginBackends(internal/server/plugin_registry.go) walks plugin routes and proxies a match before falling through tonext.ServeHTTP(themux, which is wheremountBackendsregistered the static backends).
So if a chart registers a ReverseProxy whose prefix equals (or is a parent of) an existing configured backend's prefix — e.g. an admin's backends: [{name: deploy, prefix: /api/deploy}] — installing that Plugin CR makes pluginBackends intercept and proxy all /api/deploy/* traffic to the plugin's own (CR-declared) upstream instead of the intended backend, with no startup validation or conflict error anywhere in the registration path. The same gap lets two plugins silently shadow each other if they pick the same prefix. Please cross-check a candidate route's prefix against ConfiguredBackends() and already-registered entries (same way /api and the reserved Console prefixes are rejected today) before publishing it.
Non-blocking note
internal/pluginregistry/registry.goreconcile: whenspec.enabledflips tofalse, the entry is removed from the in-memory registry immediately (assets/backends stop being served), but the earlyreturn nilskipsr.report(...), sostatus.phasestaysInstalledandstatus.frontendStatus.distributedstaystrueeven though Console no longer serves that release. Minor status/observability drift rather than a functional break.
Everything else I checked reads cleanly: ValidPluginAsset is sound against path traversal/absolute paths/escapes (backed by fs.ValidPath plus the extra char blocklist), asset bytes are served from an in-memory map keyed by validated relative paths (no filesystem traversal even if a key were unexpected), the Helm-release/CR identity/scope/type checks in release.go are strict, backend proxy requests still go through s.authorize, and the mandatory crds/ pre-apply step for helm upgrade is documented with the exact symptom and command in docs/development/plugin-cli.md.
A package that ships the Plugin CR registers itself, so the old fixed message told the user to do work that was already done. Read the registrations after the upgrade and report what is actually there; a Chart without the CR still gets the follow-up step.
Registration belongs to the release: the recommended package ships the Plugin CR, so install registers and uninstall removes it. Document the template, the version and enabled values, and keep the separate registration command for Charts that do not ship the CR.
The CLI had no way to report which build it was; --version now prints the same version constant the Console binary and the Chart appVersion share.
What type of PR is this?
What this PR does and why
Plugins register through
modelsphere.dev/v1alpha1Plugin CRs instead of Console Helm values. Console watches CRs through an informer, reads each installed release'splugin.yamlandfrontend/files, and serves/plugins/index.json,/plugins/<group--name>/<asset>and/api/<plugin>/*from that registry.The branch also carries the frontend plugin SDK and loader work from #114, rebased onto current
main. That PR is closed in favour of this one. Its last commit, YAML navigation folders, is not carried over: it extends the static registration path this PR removes.The old static path is removed outright:
helm/consoleno longer haspluginsvalues; the renderedconsole.yamlno longer carries them, and the server no longer reads a static plugin list.modelsphere.io/v1alpha1manifest conversion are deleted.id/group/version/entry/styles/backend; menus, components and internal routes come from the plugin's own SDK registration.Documentation added:
docs/development/plugin-architecture.md— the package, registration contract, loader, serving, proxy, capabilities and failure isolation.docs/development/plugins.md— build a plugin end to end, including the SDK bundle, ReverseProxy and IAM.docs/console-design.mdlink the new guides;modelsphere plugin installhelp text no longer claims it registers the runtime.Without this, the two registration paths would keep diverging and the static one would still be able to serve assets no release owns.
Which issue(s) this PR fixes
Fixes #113
How it was tested
go build ./...,go vet ./...go test ./internal/...tsc -b --noEmitandtsc -p tsconfig.swiss.jsoninweb/vitest runinweb/(726 tests)tsc --noEmitandvitest runinweb/packages/plugin-sdk(13 files, 185 tests)helm lint --strict helm/consolemainwith no conflicts.Cluster-level install/upgrade/uninstall remains to be walked through on CPU16.
Checks
go test ./...andgofmt -l .is emptynpm run typecheck && npm test && npm run buildinweb/(ifweb/changed)helm lint --strict helm/console(if the chart changed)Notes for reviewers
Start with
internal/server/plugins.go(descriptor and asset serving),web/src/shell/global-sdk/runtime.tsx(SDK-only menus), theninternal/plugincli(CR selection) anddocs/development/plugin-cli.md.Release note