Skip to content

Too many Redis calls happen randomly #471

Open

Description

@SlavaSkvortsov

From time to time we notice that ARQ starts to produce a lot of calls to Redis. It consumes a lot of connections and puts a lot of load on Redis
Here how it looks like during the incident:
image

And most of the time it looks like this:
image

It feels like it starts to attempt to execute jobs over and over nonstop. I think this behaviour could happen when you have queue_read_limit lower than max jobs on all servers, but that's not the case for us.

We're using:

python==3.11.3

arq==0.26.0
redis==4.6.0

Unfortunately, I can't reproduce the issue locally, but it happens a couple times a month on prod

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions