Repository navigation
Allow partial cache keys #627
Description
Description:
When restoring cache, the current cache management system only uses the primary cache key.
GitHub Actions cache allows specifying multiple prefixes, and restoring older caches, to speed up setup even when there has been a dependency change. Cf corresponding documentation
I'd recommend setting this as an opt-in behaviour, toggleable with a new input, because :
- That way, this would not be a breaking change
- Never rebuilding from scratch means stale data could stay forever in cache, which can be dangerous (e.g. a former dependency still present in a virtualenv)
Here's an example of what the configuration could look like (for poetry) :
- uses: actions/setup-python@v4
id: setup-python
with:
python-version: "3.10"
cache: "poetry"
allow-partial-restores: trueHere's a configuration roughly equivalent to what I'm proposing here (the cache key is slightly different, but the idea is the same)
- uses: actions/setup-python@v4
id: setup-python
with:
python-version: "3.10"
- uses: actions/cache@v3
with:
path: /home/runner/.cache/pypoetry/virtualenvs
key: poetry-${{ steps.setup-python.outputs.python-version }}-${{ hashFiles('poetry.lock') }}
restore-keys: |
poetry-${{ steps.setup-python.outputs.python-version }}-${{ hashFiles('poetry.lock') }}
poetry-${{ steps.setup-python.outputs.python-version }}-We could also expose cache key management to the users through an input parameter, but that seems much more complex and brittle.
Justification:
This change can represent a significant performance improvement on lockfile change, easily several minutes, depending on the project.
Are you willing to submit a PR?
Not at the time, as I have a working workaround (presented above).
Activity
dmitry-shibanov commentedon Mar 15, 2023
Hello @toadjaune. Thank you for your report. We'll investigate the feature request.
dmitry-shibanov commentedon Apr 5, 2023
Hello @toadjaune. The purpose of caching is to cover the majority of use cases. If you need more customization we advise to use actions/cache
For now I'm going to close the issue.
tragiclifestories commentedon Apr 21, 2023
@dmitry-shibanov can this be reopened? Partial cache keys are not some obscure use case but an important optimisation widely used in the CI ecosystem. I would guess that 90+% of jobs that use setup-python would benefit from it.
It's built into setup-ruby, coming in setup-node, and indeed is actually implemented for some package managers in this code base, just not all of them (eg poetry).
iamstarkov commentedon Nov 29, 2023
@dmitry-shibanov can I ask you to reevaluate your stance and incorporate partial keys how to speed up CI builds and be consistent with the rest of setup-{tool} actions?