Skip to content

e2e: Wait for controller leadership after upgrade test - #306

Open
alicefr wants to merge 2 commits into
bootc-dev:mainfrom
alicefr:fix-flake-controller-ready
Open

alicefr wants to merge 2 commits into
bootc-dev:mainfrom
alicefr:fix-flake-controller-ready

Conversation

@alicefr

@alicefr alicefr commented Nov 12, 2026

Copy link
Copy Markdown
Collaborator

TestOperatorUpgrade restores the operator in cleanup. The controller can land on the test's worker node, and removing that node reschedules it while parallel tests are already running. Pool creation then races the new pod's image pull and leader election, flaking TestControllerMembership.

Wait for the single running controller to hold the lease after the node is removed, so parallel tests start with a working controller.

TestOperatorUpgrade restores the operator in cleanup. The controller can
land on the test's worker node, and removing that node reschedules it
while parallel tests are already running. Pool creation then races the
new pod's image pull and leader election, flaking TestControllerMembership.

Wait for the single running controller to hold the lease after the node
is removed, so parallel tests start with a working controller.

Assisted-by: AI
Signed-off-by: Alice Frosi <afrosi@redhat.com>
TestPullSecretAuth failed in CI (run 43199018221, e2e 1.37): the
authenticated-registry stage took about 4 minutes, and the first
reboot landed back on the previous deployment because the staged
deployment was silently lost at shutdown when the /boot automount
had idle-expired (bootc-dev/bootc#2622). The daemon correctly
re-staged and issued a second reboot, which converges, but the
extra cycle pushed the total to about 8.7 minutes, just past the
8 minute WaitForNodeIdle budget.

Raise the four authenticated-pull waits to 10 minutes so one extra
stage+reboot cycle fits. This is a workaround for the flake, not a
fix: the root cause is tracked in bootc-dev/bootc#2622 (via
ostreedev/ostree#3689) and the operator-side impact in bootc-dev#256.

Assisted-by: AI
Signed-off-by: Alice Frosi <afrosi@redhat.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant