Skip to content

chore: type checking with pyright - #142

Open
tromai wants to merge 3 commits into
juju:mainfrom
tromai:add-pyright
Open

tromai wants to merge 3 commits into
juju:mainfrom
tromai:add-pyright

Conversation

@tromai

@tromai tromai commented Sep 2, 2026 •

Copy link
Copy Markdown
Contributor

This PR adds type checking with pyright.

The section below discusses the errors raised by pyright, and my proposed fixes.

ops.testing.SIMULATE_CAN_CONNECT not found

In tests/__init__.py

import ops.testing

ops.testing.SIMULATE_CAN_CONNECT = True

Solution: Remove ops.testing.SIMULATE_CAN_CONNECT = True following canonical/operator#873.

urllib.request not recognized

In both src/configchangesocket.py and src/controlsocket.py.

import urllib

...python
def __init__(self, socket_path: str, opener: urllib.request.OpenerDirector | None = None):
    super().__init__(socket_path, opener=opener)

This is a trivial fix.

- import urllib
+ import urllib.request

body[...] = value type mismatch

In controlsocket.py

body = { 
    "grpc_endpoint": grpc_endpoint,
    "http_endpoint": http_endpoint,
    "ca_cert": ca_cert,
}

...
if stack_traces is not None:
    body["stack_traces"] = stack_traces   # stack_traces is of type bool.

body is inferred as dict[str, str | None] when it is first initialized. However, bool/float values are later inserted to it.

Solution: annotate the type of body.

- body = { 
+ body: dict[str, str | bool | float | None] = {
    "grpc_endpoint": grpc_endpoint,
    "http_endpoint": http_endpoint,
    "ca_cert": ca_cert,
}

_current_open_telemetry_config returns type mismatch

In src/charm.py

def _current_open_telemetry_config(self) -> tuple[bool, float, str, bool]:
    sample_ratio = float(self.config["workload-tracing-sample-ratio"])
    self._validate_open_telemetry_sample_ratio(sample_ratio)
    return (
        self.config["workload-tracing-stack-traces"],
        sample_ratio,
        self.config["workload-tracing-tail-sampling-threshold"],
        self.config["workload-tracing-insecure-skip-verify"],
    )

Values of self.config (of type ConfigData) has a Union type bool | int | float | str. Therefore, pyright infers the returned object as:

tuple[ 
    bool | int | float | str,
    float,
    bool | int | float | str,
    bool | int | float | str
]

which doesn't match tuple[bool, float, str, bool] in the function signature.

Solution: use cast(T, ...)

def _current_open_telemetry_config(self) -> tuple[bool, float, str, bool]:
    sample_ratio = float(self.config["workload-tracing-sample-ratio"])
    self._validate_open_telemetry_sample_ratio(sample_ratio)
    return (
        cast(bool, self.config["workload-tracing-stack-traces"]),
        sample_ratio,
        cast(str, self.config["workload-tracing-tail-sampling-threshold"]),
        cast(bool, self.config["workload-tracing-insecure-skip-verify"]),
    )

hdrs=None type mismatch in urllib.error.HTTPError

In tests/test_sockets.py:

error=urllib.error.HTTPError(
    url="http://localhost/loki-endpoint",
    code=500,
    msg="",
    hdrs=None,
    fp=io.BytesIO(rb'{"error":"internal error"}'),
),

urllib.error.HTTPError expects hdrs: email.message.Message, not None. See here.

Solution: initialize an empty email.message.Message object.

test_sockets.py — MockOpener not assignable to OpenerDirector

A MockOpener object is passed into ControlSocketClient, for example:

def test_connection_error(self):
    mock_opener = MockOpener(self)
    control_socket = ControlSocketClient("fake_socket_path", opener=mock_opener)

The opener parameter of ControlSocketClient has type urllib.request.OpenerDirector | None = None. Therefore, pyright flags a non-compatible type.

This is an expected type error, but not a serious one. Since this is for testing purposes, and we control the behaviour of MockOpener, I proposed ignoring this error with # type: ignore.

JujuControllerCharm.api_port should return int

If this function doesn't raise, it returns urllib.parse.SplitResult.port. This .port attribute is an int.

JujuControllerCharm.ca_cert return type annotation

Previously:

    def ca_cert(self) -> str | None:
        """Return the controller's CA certificate."""
        return self._controller_runtime_config("ca-cert")

In theory, self._controller_runtime_config("ca-cert") can return None, or a value that is not of type str. I updated this function to strictly handle mis-match type.

As a drive-by, I added JujuControllerCharm._controller_runtime_config return type since runtime_conf.get(key) returns None if key doesn't exist.

with open(runtime_conf_path) as runtime_conf_file:
    runtime_conf = yaml.safe_load(runtime_conf_file) or {}
    return runtime_conf.get(key)

Follow up

  • Create an issue to track: turning on ruff ANN rules and typeCheckingMode = "strict".

@tromai
tromai marked this pull request as ready for review September 10, 2026 05:16
@tromai

tromai commented Sep 10, 2026

Copy link
Copy Markdown
Contributor Author

@SimonRichardson This PR is ready for review. However, it is based on top of #141's branch (I performed a couple of merges). I suggest reviewing and merging #141 first before this one.

@tonyandrewmeyer tonyandrewmeyer left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for this, and for the detailed description. Mostly small suggestions. The one I'd like an answer on before merging is the website handler, since catching the exception there means the hook no longer fails and gets retried.

Comment thread src/charm.py Outdated
Comment on lines 302 to 307
try:
port = self.api_port()
except AgentConfException:
logger.error("machine does not appear to be a controller")
self.unit.status = BlockedStatus("machine does not appear to be a controller")
return

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a behaviour change: before, api_port() raising meant the hook errored and Juju would retry it (so the relation would get the port once runtime.conf had one), but now the hook succeeds and the relation never gets port, since relation-joined won't run again. The status set here is also replaced by _on_collect_status, which already reports this failure.

Is swallowing the error what we want here? If so, I think it needs a test, and the status line can go.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changing its original behaviour is not what I wanted.

I made a mistake here. I missed that self.api_port() raises AgentConfException from other reasons besides a None port.

Therefore:

        if port is None:
            logger.error("machine does not appear to be a controller")
            self.unit.status = BlockedStatus("machine does not appear to be a controller")
            return

is not identical to

        try:
            port = self.api_port()
        except AgentConfException:

Although that original if port is None doesn't do anything because api_port() always returns an int (not a zero) 🤔

I have reverted this change.

Comment thread pyproject.toml
[tool.pyright]
pythonVersion = "3.10"
reportUnnecessaryTypeIgnoreComment = "error"
include = ["src", "tests"]

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Running pyright on its own (or in an editor) can't resolve the charms.* imports (7 errors for me), since it's only run_tests that puts lib on the path. What about adding this?

Suggested change
include = ["src", "tests"]
include = ["src", "tests"]
extraPaths = ["lib"]

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh thanks for the catch. I added it.

Comment thread src/charm.py Outdated

parsed_url = urllib.parse.urlsplit("//" + api_addresses[0])
if not parsed_url.port:
if parsed_url.port is None:

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the old check was actually doing the right thing here, since port 0 isn't a usable API port either (with is None we'd end up with a *:0 scrape target). Is there a case where we'd want to accept it? Feel free to push back.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No push back from me. This change introduced a behaviour change, where a zero port will be passed through without an exception. I have reverted it.

Comment thread src/charm.py Outdated
},
"tls_config": {
"ca_file": self.ca_cert(),
"ca_file": ca_cert if ca_cert is not None else "",

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Previously this would have been sent as null when there's no ca-cert. Is "" handled the same way on the Prometheus side? If we're not sure, I'd rather keep ca_cert() returning str (or raising), since a controller should always have one.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

From my small exploration, I don't have an evidence that they are handled differently on the Prometheus side yet.

But I wouldn't want to introduce an empty string here. I will revert it.

I agree that ca_cert reaching here should always be a string. Just to double check:

  • self.ca_cert() should raise AgenConfException when CA certificate cannot be found.
  • Here we will catch the exception and put the charm into BlockedStatus
        try:
            ca_cert = self.ca_cert()
        except AgentConfException as e:
            self.unit.status = BlockedStatus(
                f"can't read CA certificates from runtime.conf: {e}"
            )
            logger.error("cannot read CA certificates from agent runtime.conf: %s", e)
            return None

        return [
            {
                ...
                "tls_config": {
                    "ca_file": ca_cert,
                    "server_name": "juju-apiserver",
                },
            }
        ]

And I should probably add this try catch to _on_collect_status too.

Comment thread run_tests Resolved
Comment thread tests/test_sockets.py Resolved
Comment thread src/charm.py Resolved
Comment thread src/charm.py Resolved
Comment thread pyproject.toml
[project.optional-dependencies]
dev = [
"coverage",
"pyright",

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

CI installs with uv pip install ".[dev]", which doesn't use uv.lock, so we'll pick up each new pyright as it's released, and a new check could turn CI red without any change here. Do we want a pin (or uv sync) so that new checks arrive deliberately? (ruff is the same already, so feel free to leave this for a separate PR.)

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This problem will be addressed in a subsequent PR (#146) - although there are a couple of issues from that PR I need to fix too. Thanks for raising this!

I prefer the CI respects uv.lock, and doesn't run on new version everytime.

Comment thread pyproject.toml
] No newline at end of file
]

[tool.pyright]

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is the plan to annotate the rest in a follow-up? pyright's standard mode doesn't report missing annotations at all (it infers return types, and treats unannotated parameters as Unknown), so most of the handlers that take an event aren't really being checked yet. ruff check --select ANN gives 107 hits across src/ for me. Turning on the ruff ANN rules and/or typeCheckingMode = "strict" once those are fixed would stop new unannotated code creeping in. Propbably not for this PR, but it would be good to have an issue for it.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes. I enabled strict mode before and found that would make the PR a bit bigger. I wanted this PR to be minimal first, so that it was easier for me to stack other PRs on top, and doesn't affect incoming PRs from other maintainers too much (at the end of the cycle).

I will create an issue to track it as a follow up.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants