You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Flaky test: TestPartitionCompactor_DeleteLocalSyncFiles (arm64) — data race in thanos extprom.TxGaugeVec via concurrent MetaFetcher.Fetch #7707
TestPartitionCompactor_DeleteLocalSyncFiles failed on the test (arm64) CI job with a data race detected in vendored Thanos code. Observed on PR #7706 (a pkg/ring/kv/dynamodb-only change, unrelated to the compactor). test (amd64) passed on the same commit.
Two concurrent MetaFetcher.Fetch() calls race on the shared TxGaugeVec: one resets the tx gauge (ResetTx) while the other is still reading/incrementing it (WithLabelValues). Syncer.SyncMetas uses singleflight per group, but concurrent per-user compactions in the test share fetcher metrics across different singleflight groups.
Context
This test family has a flaky history on arm64: #7565 (fixed by #7567), #7608 (fixed by #7619). Those fixes addressed timing/determinism of TestCompactor_DeleteLocalSyncFiles; this failure is the partitioning variant and a different failure mode (race detector rather than poll timeout).
To Reproduce
Flaky — observed on arm64 with -race. go test -race -count=N ./pkg/compactor/ -run TestPartitionCompactor_DeleteLocalSyncFiles on arm64 may reproduce.
Expected behavior
No data race; test passes reliably on all architectures.
Additional context
The race is in vendored Thanos code (extprom/tx_gauge.go), so the fix likely belongs upstream (guard ResetTx/WithLabelValues with the existing mutex consistently) or in how the cortex compactor/test shares FetcherMetrics across concurrent user compactions.
Activity
extprom: make TxGaugeVec.ResetTx and Submit goroutine-safe