Skip to content

config: Add versioned operator configuration through a CRD - #203

Merged
alicefr merged 15 commits into
bootc-dev:mainfrom
HarshwardhanPatil07:issue-92-configmap-config
Nov 12, 2026
Merged

alicefr merged 15 commits into
bootc-dev:mainfrom
HarshwardhanPatil07:issue-92-configmap-config

Conversation

@HarshwardhanPatil07

@HarshwardhanPatil07 HarshwardhanPatil07 commented Sep 29, 2026 •

Copy link
Copy Markdown
Member

Administrators currently need to edit operator workload arguments to change settings and preserve those edits during upgrades. Add an optional, administrator-owned, cluster-scoped BootcOperatorConfig using the versioned node.bootc.dev/v1alpha1 API. The resource may have any valid Kubernetes name, but the controller and daemon require at most one instance at startup.

The API exposes controller insecure-registry policy and image-tag resolution period, plus the daemon fallback status-poll period. Kubernetes validates types and positive whole-second periods. Both processes make one bounded, uncached list request at startup: zero instances use existing defaults, one supplies settings, and multiple instances or API errors stop startup with a contextual error. Explicit CLI flags take precedence, including false and fractional durations. Changes require a manual restart of the affected workload. Both service accounts receive only list permission for this resource.

The bink workflow now installs the CRD, waits for it, creates or updates its local-registry configuration, and only then deploys workloads. Install and Helm manifests include the CRD and permissions without creating or overwriting administrator configuration. Releases publish an optional, editable operator-config.yaml separately from install.yaml. Documentation covers precedence, restarts, revert and upgrade behavior, and Helm CRD upgrade ordering. Live updates, operator status reporting, and feature gates from #181 remain deferred.

Closes #92.

@alicefr

alicefr commented Sep 29, 2026

Copy link
Copy Markdown
Collaborator

I personally prefer a new CRD over a configmap. Mostly for 2 reasons, first it is versioned, secondly eventually we could use the crd status for reporting the operator status. The second point right now isn't used so the motivation isn't that strong.

I'd love also @cheesesashimi's opinion

@HarshwardhanPatil07 HarshwardhanPatil07 changed the title config: Support operator configuration through an optional ConfigMap config: Add versioned operator configuration through a CRD Sep 30, 2026
@HarshwardhanPatil07
HarshwardhanPatil07 force-pushed the issue-92-configmap-config branch 5 times, most recently from 16f3e66 to 19361d9 Compare September 30, 2026 11:24
@HarshwardhanPatil07

Copy link
Copy Markdown
Member Author

updated PTAL

Comment thread internal/config/config.go Resolved
Comment thread Makefile Resolved
Comment thread test/e2e/config_test.go Resolved
Comment thread test/e2e/config_test.go Resolved
@alicefr

alicefr commented Oct 2, 2026

Copy link
Copy Markdown
Collaborator

We should also probably generate a cr as part of the release with some good defaults. This is at least how kubevirt is doing it

@HarshwardhanPatil07
HarshwardhanPatil07 force-pushed the issue-92-configmap-config branch 6 times, most recently from bfbeb93 to e2740b9 Compare October 6, 2026 07:48
@HarshwardhanPatil07

Copy link
Copy Markdown
Member Author

PTAL

Comment thread config/rbac/daemon_role.yaml Resolved
Comment thread Makefile Resolved
@alicefr

alicefr commented Oct 6, 2026

Copy link
Copy Markdown
Collaborator

@HarshwardhanPatil07 it looks great many thanks for doing this. Few last comment and this requires a rebase

@HarshwardhanPatil07
HarshwardhanPatil07 force-pushed the issue-92-configmap-config branch from e2740b9 to 7a4ebda Compare October 9, 2026 10:15
HarshwardhanPatil07 added a commit to HarshwardhanPatil07/bootc-operator that referenced this pull request Oct 12, 2026
Parallel E2E tests create daemon pods on other managed nodes. Restrict the membership lookup to its own node so unrelated pods do not cause the exactly-one assertion to time out.

Refs: bootc-dev#226
Refs: bootc-dev#203

Assisted-by: AI
Signed-off-by: HarshwardhanPatil07 <harshpat@redhat.com>
alicefr pushed a commit that referenced this pull request Oct 12, 2026
Parallel E2E tests create daemon pods on other managed nodes. Restrict the membership lookup to its own node so unrelated pods do not cause the exactly-one assertion to time out.

Refs: #226
Refs: #203

Assisted-by: AI
Signed-off-by: HarshwardhanPatil07 <harshpat@redhat.com>
@HarshwardhanPatil07

Copy link
Copy Markdown
Member Author

rebased

@alicefr

alicefr commented Nov 11, 2026

Copy link
Copy Markdown
Collaborator

@HarshwardhanPatil07 unfortunately, the rebase isn't enough. You need to take care to the new option added by #273

@HarshwardhanPatil07

HarshwardhanPatil07 commented Nov 11, 2026 •

Copy link
Copy Markdown
Member Author

added flag --pull-secret-namespaces changes and some cache and permission restriction added by other pr

Comment thread test/e2e/pullsecret_namespace_test.go Outdated
// so it must stay serial. No node is provisioned: reconciling a digest-based pool
// proves that the controller can read a Secret through its namespace-scoped cache.
func TestOperatorConfigPullSecretNamespaces(t *testing.T) {
e2eutil.Providers(t, "bink", "eks")

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

let's avoid EKS until we have properly tested

Comment thread test/e2e/upgrade_test.go
return append(args, legacyArgs...)
}

func TestReleasedControllerArgs(t *testing.T) {

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can we add a description for this test?

Comment thread test/e2e/upgrade_test.go
return values
}

func TestEffectivePullSecretNamespaces(t *testing.T) {

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can we add a description for this test?

@alicefr

alicefr commented Nov 11, 2026 •

Copy link
Copy Markdown
Collaborator

@HarshwardhanPatil07 I also think we should change the priority of the flags. A flag defined in bootcoperatorconfig should take the precedence over one defined as a direct flag for the daemon or the controller. I apologize for the late request. But I think it will make the transition smoother when we will enable the dynamic change of the option in the bootcoperatorconfig. Otherwise, the static flag defined in the daemonset and in the deployment will always overwrite the dynamic configuration . Basically, the priority should be CR field → CLI flag → built-in default

Give administrators a typed cluster-scoped API for controller and daemon settings without editing workloads. Keep naming flexible; startup selection enforces that only one instance is used.

Refs: bootc-dev#92

Assisted-by: AI
Signed-off-by: HarshwardhanPatil07 <harshpat@redhat.com>
Verify that the API preserves omitted fields for CLI fallback and rejects
invalid period, type, and field values before startup consumes the resource.

Refs: bootc-dev#92

Assisted-by: AI
Signed-off-by: HarshwardhanPatil07 <harshpat@redhat.com>
Make administrator configuration authoritative while retaining CLI values
and built-in defaults for omitted fields. Preserve explicit false and empty
namespace lists, and retain CLI duration precision when the CR field is absent.

Refs: bootc-dev#92

Assisted-by: AI
Signed-off-by: HarshwardhanPatil07 <harshpat@redhat.com>
Read the administrator resource before informer caches exist. Treat missing CRDs, permission failures, API errors, and multiple instances as actionable startup errors while an absent instance keeps defaults.

Refs: bootc-dev#92

Assisted-by: AI
Signed-off-by: HarshwardhanPatil07 <harshpat@redhat.com>
Exercise absent, custom-named, updated, conflicting, and missing-CRD resources through envtest so the loader behavior is covered beyond mocked HTTP responses.

Refs: bootc-dev#92

Assisted-by: AI
Signed-off-by: HarshwardhanPatil07 <harshpat@redhat.com>
Load registry policy, tag resolution, and pull-secret namespaces before
constructing the manager. CR fields override CLI settings; resolve the same
namespace allowlist for the Secret cache and reconciler. Grant only the
configuration list permission and retain namespaced Secret access.

Refs: bootc-dev#92

Assisted-by: AI
Signed-off-by: HarshwardhanPatil07 <harshpat@redhat.com>
Read the administrator resource before starting each daemon. Its polling
period takes precedence over CLI settings when present; omitted fields keep
the existing CLI value or built-in default. Grant list permission for the read.

Refs: bootc-dev#92

Assisted-by: AI
Signed-off-by: HarshwardhanPatil07 <harshpat@redhat.com>
Bink now installs the operator through Helm. Apply the local registry
configuration after the chart creates the CRD, then restart the controller
so it reads the settings and refreshes the development image.

Refs: bootc-dev#92

Assisted-by: AI
Signed-off-by: HarshwardhanPatil07 <harshpat@redhat.com>
Preserve administrator configuration across the release upgrade test and
keep Helm-managed Secret grants separate from CR settings. Translate supported
CR fields into winning CLI arguments for older releases.

Exercise CR-only namespace configuration with namespaced RBAC on bink and
restore the original settings afterward. A digest-based pool proves allowed
Secret reads and denied references without provisioning a worker node.

Refs: bootc-dev#92

Assisted-by: AI
Signed-off-by: HarshwardhanPatil07 <harshpat@redhat.com>
Document CR-first precedence, omitted-field fallback, manual restarts, and
administrator ownership so users can configure and revert settings safely.

Refs: bootc-dev#92

Assisted-by: AI
Signed-off-by: HarshwardhanPatil07 <harshpat@redhat.com>
Support installing CRDs, applying configuration, then starting the operator.
Keep the CRD subchart lifecycle from bootc-dev#234 and remove the old generated directory
to prevent duplicate CRDs.

Retain explicit Helm namespace values as CLI fallbacks with namespaced RBAC
from bootc-dev#273. CR fields take precedence without creating permission grants;
document how administrators provide matching access.

Refs: bootc-dev#92
Refs: bootc-dev#234

Assisted-by: AI
Signed-off-by: HarshwardhanPatil07 <harshpat@redhat.com>
Provide an editable example without putting an administrator-owned instance into install.yaml, so routine installs and upgrades cannot overwrite configuration.

Refs: bootc-dev#92

Assisted-by: AI
Signed-off-by: HarshwardhanPatil07 <harshpat@redhat.com>
Keep the downloadable example aligned with built-in defaults and ensure
install.yaml ships the CRD without creating a configuration instance. Run
the checks on every PR.

Refs: bootc-dev#92

Assisted-by: AI
Signed-off-by: HarshwardhanPatil07 <harshpat@redhat.com>
Make the versioned API discoverable alongside each release. Validate both assets before publishing and direct administrators to check for an existing instance before applying the example.

Refs: bootc-dev#92

Assisted-by: AI
Signed-off-by: HarshwardhanPatil07 <harshpat@redhat.com>
CI selects its Go toolchain from go.mod. Require the patched release to resolve GO-2026-6859 without suppressing vulnerability checks.

The container builder remains on Go 1.27.3 and needs a separate update when the patched Hummingbird image is available.

Reference: https://pkg.go.dev/vuln/GO-2026-6859

Assisted-by: AI
Signed-off-by: HarshwardhanPatil07 <harshpat@redhat.com>

@alicefr alicefr left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It looks great, many thanks for this work! LGTM!

@alicefr
alicefr merged commit 8c33182 into bootc-dev:main Nov 12, 2026
16 of 17 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Support a configmap-backed config dir for configuring the bootc-operator

2 participants