Repository navigation
Keep select_related column aliases within identifier length limits - #2395
Open
sinnercode228 wants to merge 2 commits into
Open
sinnercode228 wants to merge 2 commits into
sinnercode228 wants to merge 2 commits into
Conversation
select_related() aliased each column of a joined table as "<table name>.<column>", and the executor took the field name from the part after the first ".". PostgreSQL truncates identifiers to 63 bytes, so for a long table name the alias came back cut short, the field name no longer matched, and the field was silently dropped. The related instance was then partial and save() raised IncompleteInstanceError. Use the index of the joined table as the prefix instead. The executor only needs the prefix to be unique per joined table, and the same table still gets the same alias when it is selected twice. Fixes tortoise#2018
Merging this PR will not alter performance1 warning
Comparing |
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
select_related()aliases every column of a joined table as"<table name>.<column>", andBaseExecutor.execute_selecttakes the field name from the part after the first.. PostgreSQL truncates identifiers to 63 bytes. With a long table name the alias comes back cut short, the field name no longer matches any field, and that field is silently dropped. The related instance ends up partial, andsave()raisesIncompleteInstanceError.With the models from the issue on PostgreSQL 16:
The alias is 64 bytes. It comes back as
...not_too_long_user_ag, so the loaded instance is{'_partial': True, 'user_id': 1, 'id': 1}, withoutnot_too_long_user_age.This PR uses the index of the joined table in
_joined_tablesas the alias prefix ("0.not_too_long_user_age"), in both_join_select_relatedand_resolve_only, through one helper. The executor only needs the prefix to be unique per joined table, and it is. When the same table is selected again (e.g.select_related("left", "left__left")), it gets the same index and the same aliases, as it did with the table name. The executor is unchanged.The prefix no longer depends on the table name, so the alias is only too long if the column name alone is close to the limit.
Not addressed here: table aliases of deep nested joins (
table__field__field...) are truncated as well. That does no harm unless two of them share the first 63 bytes, which needs a far deeper chain than the column case.Motivation and Context
Fixes #2018. It only shows up on PostgreSQL (SQLite has no identifier limit), which is why it could not be reproduced on SQLite in the issue. The alias format predates #1923, so the cause is the 63-byte limit rather than that PR.
How Has This Been Tested?
test_select_related_with_long_table_nameintests/test_relations.py, with a self-referencing modelModelWithATableNameLongEnoughToHitIdentifierLimitsadded totests/testmodels.py("<table>__parent.title"is 64 bytes). It checks the related instance's fields afterselect_related("parent")and that it can be saved. Ondevelopit fails on PostgreSQL with both asyncpg and psycopg (AttributeError: ... object has no attribute 'title') and passes on SQLite. With the change it passes on all three.make check.Checklist: