Skip to content

feat(web): deploy video models with the LLM form plus their own fields - #129

Draft
zhanghaohit wants to merge 15 commits into
mainfrom
feat/video-deploy
Draft

zhanghaohit wants to merge 15 commits into
mainfrom
feat/video-deploy

Conversation

@zhanghaohit

@zhanghaohit zhanghaohit commented Nov 11, 2026 •

Copy link
Copy Markdown
Contributor

What type of PR is this?

  • feature

What this PR does and why

Catalog variants can now carry engine: video (first user: MiniMax H3, chart minimax-h3; swissd accepts it in a parallel change). A video variant deploys with the same form as an LLM, plus the video chart's own fields (pools, Turbo, shared volume, database). The form leaves out what only an LLM chart takes: replicas and scaler bounds, route tuning, ServiceMonitor, the engine image, the compile cache.

Where Video LLM
Console deploy / upgrade sheet (inferences) DeployForm with video: adds VideoStorage (Basic) and VideoPools (Resources), no Routing section DeployForm, unchanged
swiss standalone Deploy / Upgrade pages DeploySettings with video: adds the VideoSettings cards, no Routing options card DeploySettings, unchanged
Form state and plan request Form.video + planRequest(..., { video: true }) Form + planRequest, unchanged
Edit SLO (console sheet, swiss dialog and list button) VideoSLOCard: one section per pool SLOCard, unchanged
Cluster resources JobSLO panel (per-pool bounds and queue depth) LLMSLORequirement panel, unchanged
Access card / Check tab / Endpoint submit, poll, download examples + readiness check (/readyz) chat example + serving and health checks
Playground video releases are not offered unchanged

The type comes from variant.engine on deploy and plan.engine on upgrade and detail pages. List rows use Deployment.engine (modelsphere/swiss#59), falling back to the chart name for an older swissd: swissd only allows LLM engines on the sglang / vllm charts.

Autoscaling follows the PD pattern. The form's existing SLO switch turns on the pools' scalers and SLOs. The chart then gives each pool a JobSLORequirement with its default minimum and queue depth and no maximum, so nothing is scaled. Scaling is set after the deploy in the SLO card.

Chart values the deploy form writes

Matched against modelsphere/helm-charts#105. Every key is in web/src/modules/swiss/lib/videoChart.ts.

Key Written Form default
pools.<p>.enabled (fl2va, ref2va) always true
pools.<p>.replicaCount when the pool is on (the chart ignores it while scaler.enabled) 1
pools.<p>.sglang.args.tpSize / .ulyssesDegree when the pool is on 4 / 1
pools.<p>.sglang.gpuCount when the pool is on: tp × ulysses (the chart takes it over model.gpus and refuses a render where they differ) 4
pools.ref2va.turbo.enabled when ref2va is on true
scaler.enabled + sloRequirement.enabled always, together, from the SLO switch the site's default, as for an LLM
sharedStorage.storageClass when set (empty = cluster default, must be RWX); locked on upgrade. When the variant declares it as an input, the input field asks for it instead and the site profile can answer it empty
sharedStorage.size when set; on upgrade, empty keeps it and it can only grow 500Gi
database.url when typed (password field, never read back; on upgrade, empty keeps the release's) empty
modelRoute.enabled always the site's route.enabled
modelRoute.nginx.route only when a custom route is typed (empty = the service ID, which the chart and swissd both fall back to) empty
sglang.tolerations always; replaces the chart's list. The chart has no top-level tolerations, so this is the one scheduling key that differs from an LLM deploy the chart's nvidia.com/gpu toleration
priorityClassName / schedulerName / affinity as for an LLM deploy (GPU products go to swissd, which writes the affinity). The chart applies the top-level keys to the GPU pods

Upgrade. An upgrade lays the form over the release's form layer, so a key the form leaves out keeps its old value, as for an LLM. A switch the form layer does not name (a release deployed through the API, or by an older form) is seeded from the last plan layer that sets it rather than a hard-coded default, for LLM and video alike, and an emptied custom route is sent as "" so it falls back to the service ID. Earlier preview builds wrote sglang.priorityClassName / .schedulerName / .gpuProduct; the form reads them back and an upgrade clears them, so the top-level keys apply. database.url, sharedStorage.storageClass and sharedStorage.size are never cleared: empty there means keep. An existing PVC's class cannot change and its size cannot shrink, and an empty size renders an invalid PVC. On upgrade the form locks the storage class and says the size can only grow.

Not written: any pools.<p>.sloRequirement.* or pools.<p>.scaler.* (the chart's defaults apply), and server.config.publicBaseUrl (swissd derives modelRoute.url from the site's route.gateway, and the chart uses it as the default).

swissd API used

Call Use
GET /api/releases/{ns}/{release}/objects JobSLORequirement objects: one per pool, spec.serviceId, minimumDeployment, maximumDeployment, queue
GET/PUT/DELETE /api/releases/{ns}/{release}/slo?serviceId=<id> read, save (highPriority, minimumDeployment, maximumDeployment only when set, queue.maxDepth), reset one pool's JobSLO. Without serviceId, LLM calls are unchanged
POST /api/releases/{ns}/{release}/probe the readiness check; swissd probes <url>/readyz for a video release
GET /api/deployments engine on each row (swiss#59)

Shared pieces were exported rather than copied: Row, Suggest, PathInput, Tolerations, Products and an extracted InputFields from swiss DeploySettings; Row and Select from SLOCard; Suggest, Tolerations, PresetField and an extracted InputFields from DeployForm. The LLM forms and SLO card render as before.

Which issue(s) this PR fixes

Refs modelsphere/helm-charts#105, modelsphere/swiss#59.

How it was tested

  • New tests:
    • swiss/routes/Deploy.test.tsx and inferences/components/VideoFields.test.tsx: a video variant shows the LLM form's shared rows plus the video fields and none of the LLM-only ones; the storage class field gives way to a declared input; the LLM form is unchanged.
    • swiss/lib/video.test.ts: planRequest with video writes the shared keys and the video keys and nothing LLM-only; gpuCount = tp × ulysses; empty keeps the DB URL and volume; the legacy sglang.* scheduling keys are cleared; formFromPlan round-trips a video release (and reads the legacy keys); GPU totals; list-row detection.
    • swiss/components/VideoSLOCard.test.tsx covers JobSLO discovery from objects, ?serviceId= paths, edit bodies (no max when empty), validation, and a render per pool without TTFT/OTPS.
    • swiss/components/VideoEndpoint.test.tsx covers the job API examples and the readiness check vs the LLM checks.
    • playground/targets.test.ts covers that video releases are left out.
  • swiss/components/VideoSLOCard.test.tsx and inferences/lib.test.ts also cover Edit SLO following a video release's live JobSLOs, and the scaler step reading only the pools with a maximum (an unscaled pool's "no decision" is not a failure).
  • npm run typecheck, npm test (651 tests; shell/preferences.test.ts fails to load on this Node with or without this change), npm run build, npm run build:swiss.
  • On a test cluster with swiss#59 and helm-charts#105 as prereleases: deployed MiniMax H3 through swissd, the probe answered on /readyz, a T2V and a Ref2VA task each completed and downloaded through the derived public URL, and per-pool SLO edits, saves and resets from the console reached the JobSLOs and the decision server.

Checks

  • npm run typecheck && npm test && npm run build in web/
  • Docs updated (docs/content/{en-US,zh-CN}/deploy.md, "Video models")
  • No internal hostnames, IPs, registries or credentials

Notes for reviewers

  • A JobSLO save merges, like the LLM one, so emptying a stored maximum does not clear it. The card says so, and Reset goes back to the CRD defaults: no maximum (not scaled) and normal priority, keeping the minimum and the queue.
  • database.url lands in the plan's form layer, which is stored in a ConfigMap beside the release. Keeping it in a Secret would need a swissd change.

Release note

Video models (catalog engine "video", such as MiniMax H3) deploy with the LLM form plus their own fields: pools, Turbo, shared storage and database. Each pool's SLO (replica bounds, queue depth, priority) is edited after the deploy. Video releases show the job API and a readiness check instead of chat.

A catalog variant with engine "video" (first one: MiniMax H3 on the
minimax-h3 chart) now opens a video deploy form instead of the LLM one,
in both the console's deploy sheet and swiss's standalone pages. The LLM
form and its requests are unchanged.

- The video form writes the chart's pool, Turbo, storage, database and
  autoscaling values. Every key it writes lives in swiss/lib/videoChart.ts.
- Upgrades of a video release use the same form, seeded from its plan.
  The database URL is never read back into the form; empty keeps it.
- Video releases show the job API (submit, poll, download) instead of the
  chat example and checks, have no SLO action or panel, and are left out
  of the Playground.
Per-pool autoscaling goes to the pool's JobSLORequirement
(pools.<p>.sloRequirement.{minReplicas,maxReplicas,queue.maxDepth}), the
LLMScaler bounds keep the chart's, and a custom route is written as
modelRoute.nginx.route, which the chart and swissd both read before
serviceId. An empty route field now writes nothing. The deploy docs say how
server.config.publicBaseUrl takes __ROUTE__ for a site-wide setting.
Autoscaling follows the PD pattern. The deploy form has one Autoscale / SLO
switch (scaler.enabled + sloRequirement.enabled) and no per-pool fields.
The chart gives each pool a JobSLORequirement with no maximum, which is not
scaled until the SLO card sets one.

- The SLO card for a video release has one section per pool: min and max
  (empty max = autoscaling off), queue max depth, priority, and the queue
  query read-only. Each pool's service id comes from the release's
  JobSLORequirement objects and is sent to swissd's /slo as ?serviceId=.
- The release page's Check is a readiness probe of the route's /readyz again,
  next to the job API examples.
- publicBaseUrl guidance is gone from the form: swissd derives it from the
  site's route.gateway. The docs say so.
…on upgrade

- pools.<p>.sglang.gpuCount = tp × ulysses for every enabled pool. swissd
  sends model.gpus as the variant's requires.gpus, and the chart refuses a
  render where the pod's GPU count is not tp × ulysses. Without this,
  changing the parallelism in the form broke the deploy.
- An upgrade lays the form over the release's form layer, so a field the form
  leaves out keeps its old value. Emptying the route, storage class or size,
  priority class, scheduler or GPU products now sends "" (or []) when the
  release had set it. database.url is not cleared: empty there means keep.
An existing PVC's storage class cannot change and its size cannot shrink,
and the chart renders an empty size as an invalid PVC. So an empty storage
class or size on upgrade now keeps the release's value, like database.url,
instead of sending a clear. The upgrade form locks the class and says the
size can only grow. Route, GPU products, priority class and scheduler still
clear when emptied.
A video variant used a form of its own, which repeated the LLM form's rows
and drifted from them (no model path, scheduling written under other keys).
It now shares the LLM form and its plan request: the form adds the video
chart's pools, Turbo, shared volume and database, and leaves out what only
an LLM chart takes (replicas and scaler bounds, route tuning, ServiceMonitor,
the engine image, the compile cache). The SLO switch turns on the pools'
scalers and JobSLOs. Scheduling goes to the top-level keys an LLM deploy
writes, which the chart applies to the GPU pods; tolerations stay under
sglang, the only place the chart reads them. An upgrade clears the sglang.*
scheduling keys earlier forms wrote, and the storage class is asked by the
variant's input when it declares one.
@zhanghaohit zhanghaohit changed the title feat(web): deploy video models with a form of their own feat(web): deploy video models with the LLM form plus their own fields Nov 12, 2026
Edit SLO was greyed out for a video release whose form layer never named
the SLO switch, though the chart's default had created its JobSLOs: for a
video release the button now follows the JobSLOs that exist. The scaler step
showed a pool with no maximum as failing, since the decision server publishes
nothing for an unscaled pool and its LLMScaler reports that as unavailable:
the step now reads only the pools that are scaled, and is off when none is.
…et keeps

The SLO write answers with the fields it took, so the card lost the chart's
queue query until the next read: it reads the whole SLO back now. Reset goes
to the CRD defaults, which is normal priority rather than the chart's high;
the button and the docs say so.
… zero in a video pool

The deploy sheet rebuilt its whole form whenever the variant's values changed,
so picking another version of an LLM variant threw away what was typed. The
video fields are now seeded into the current form instead, once per video
variant's values, and taken back off (GPU tolerations included) on leaving a
video variant, on both deploy pages. An enabled pool with 0 replicas, TP or
Ulysses is refused before compose.
…d clear an emptied route

An upgrade read each switch from the release's form layer alone, so a release
whose route, scaler, SLO or ServiceMonitor came from the site or the defaults
opened with that switch off, and an upgrade made unchanged turned it off. The
form layer still decides when it names a switch; otherwise the last layer that
sets it does. An emptied custom route is sent as an empty route, so the
overlay drops the old one and the route falls back to the service ID. A video
release's shared volume size reads back empty when its form layer has none,
which keeps the volume rather than writing the default over a larger one.
…O from its pools

The console detail page already left unscaled pools out of the scaler step and
offered Edit SLO from the live JobSLOs; the standalone swiss page did not. Both
now share the selection of the scaled pools' scalers.
The PromQL the decision server reads a pool's backlog with is the chart's,
and the card is for the people who run the service: it shows the bounds,
the queue depth and the priority, and no longer the query.
The storage size and database fields read "kept", which did not say what
was kept. An upgrade now reads the release's shared volume size and class
from every plan layer and shows them: "currently 500Gi; empty keeps it",
and the locked class shows the class itself. The database field says it keeps
the current setting, and the deploy hint says an empty URL gives the service
a database of its own.
Conflicts were the video form meeting main's image slots, kyaml input keys,
per-role resources and cluster taints. Resolved to main's behaviour, with the
video path kept:

- Images: the video form shows the Images section too, from the chart's own
  slots only; the LLM engine's `image` fallback row is not offered for it.
- Upgrade seeding goes through main's formFromPlan(plan, gpuSpecs, variant);
  the video branch reads back inputs, images and form affinity the same way.
- The release's form layer is no longer laid under a new request (main's
  "the new layer is the form alone"); the video test now asserts that.
- Inputs stay in the shared InputFields component, with main's note for an
  answer recorded on the release.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant