[namest504]

Main Issue: #27045

#27350 makes the broker /metrics read the Prometheus Java client 1.x default registry, and the parity test in this PR needs it. It is on master now and this branch has master merged in. This PR does not depend on the other pulsar-broker port PRs; they can merge in any order.

Motivation

Part of the pulsar-broker item in #27045: BrokerOperabilityMetrics and DimensionStats, the summary wrapper behind pulsar_topic_load_times that also feeds the admin broker stats.

Modifications

• DimensionStats wraps a 1.x Summary with the same 6 quantiles and the same 60 second window. The admin broker stats entry (brk_topic_load_time_*) is byte for byte what it was. In 1.x Summary.clear() can race with observe() and collect(), so reset() takes a write lock and recording and getSnapshot() share a read lock. A registry scrape reads the summary without the lock, so a scrape that overlaps reset() can see the family without its data point once. An instance that lost the registration to an earlier one is not registered again when that one closes.
• BrokerOperabilityMetrics: the 3 static counters are 1.x counters. The two *_skipped_total names are declared without _total because the 1.x writer adds it.
• PulsarStats.close() now calls BrokerOperabilityMetrics.close(). Nothing called it before, so the first broker in a JVM kept pulsar_topic_load_times registered forever and later brokers in the same JVM, for example in tests, never got their own. With this, a broker started after another one closed registers its own summary. Brokers running at the same time in one JVM behave as before: the first one wins and the others log a warning.
• pulsar-broker adds prometheus-metrics-core (already shipped).

Metric and exposition format changes (for the 5.1 release notes)

Names, labels and values are unchanged. The text format differs the same way as in #27350: counter # TYPE lines carry _total, _created series are gone, summary and histogram counts are integers, labels are sorted.

Verifying this change

• Make sure that the change passes the CI checks.

This change added tests and can be verified as follows:

• PrometheusMetricsTest.testBrokerOperabilityMetricsAreScraped: after a topic load, the broker /metrics shows the 6 quantiles, count and sum of pulsar_topic_load_times, the 14 topic_load_failed_total series and the 2 skipped counters, all with cluster.
• DimensionStatsTest.shouldRecordAndReadWhileResetRuns: recording and reading while another thread resets the summary never fail. It fails without the lock.
• DimensionStatsTest.shouldReadUnusedStatsAsEmpty and BrokerOperabilityMetricsTest.testTopicLoadMeanIsNaNWhenNothingWasRecordedSinceTheLastReset: an unused or just reset summary reads count 0 and NaN for the mean and the quantiles, also through brk_topic_load_time_mean_ms.
• The parity test does not assume this broker owns the registered pulsar_topic_load_times summary: when another broker in the same JVM registered it first, only the shape is checked. That is the pre-existing first broker wins behaviour.
• BrokerServiceTest, AdminApiMultiBrokersTest, OpenTelemetryBrokerOperabilityStatsTest and DimensionStatsTest pass. In PrometheusMetricsTest, which starts 37 brokers one after another, the "is already registered" warning for pulsar_topic_load_times no longer appears.
• Personal CI on this branch after merging master: https://github.com/namest504/pulsar/actions/runs/43231431083

Does this pull request potentially affect one of the following parts:

If the box was checked, please highlight the changes

• Dependencies (add or upgrade a dependency)
• The public API
• The schema
• The default values of configurations
• The threading model
• The binary protocol
• The REST endpoints
• The admin CLI options
• The metrics
• Anything that affects deployment

Dependencies: pulsar-broker adds io.prometheus:prometheus-metrics-core 1.8.0 (already shipped). Metrics: format changes listed above.

Documentation

• doc
• doc-required
• doc-not-needed
• doc-complete

[lhotari]

Thanks for porting these metrics and for the lifecycle fix in PulsarStats.close(); the names, labels and quantiles line up with master and the new tests are a good guard for the registry hand-over. Two small points on behavior parity: the empty-summary mean now reads 0 where master produced NaN (so the "byte for byte" statement is not quite exact), and Summary.observe() can race with clear() in 1.x in a way simpleclient did not. Neither blocks the port; the rest are minor test and lifecycle notes. I only looked at the last two commits of this branch, since the earlier ones belong to #27350.

[lhotari]

LGTM. All four threads are addressed in 7a188bf and 6fca2fb. The empty-summary mean is NaN again and pinned through the exported metric, reset() is now serialized against recording and reading, and the unneeded guard is gone. I ran DimensionStatsTest and BrokerOperabilityMetricsTest on this head and they pass.