Repository navigation
Conversation
PyramidParser passes the request charset to core.parse_json and aiohttp's Request.json() decodes with the request charset. When the Content-Type header names a codec Python does not know (e.g. 'application/json; charset=not-a-charset'), bytes.decode() raises LookupError, which was not handled and surfaced as a 500. parse_json now converts LookupError into JSONDecodeError (as it already does for UnicodeDecodeError), and AIOHTTPParser.load_json routes LookupError to _handle_invalid_json_error, so both frameworks respond with the documented 400 'Invalid JSON body.' error.
Collaborator
|
Blanket catching LookupError like this is not a valid solution. Please read our AI Policy. Sending us LLM outputs without any investment of real effort is actively harmful to the project maintainers and therefore harmful to the project. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
When a JSON request's
Content-Typenames a codec Python doesn't know (for exampleapplication/json; charset=not-a-charset),PyramidParserandAIOHTTPParserlet aLookupErrorescape, and the server returns a 500. This PR makes them return the documented 400{"json": ["Invalid JSON body."]}instead, the same way undecodable bodies are already handled since #427.Fixes #1120
Changes
webargs.core.parse_json: ifbytes.decode(encoding)raisesLookupError(unknown codec), raisejson.JSONDecodeErrorinstead.UnicodeDecodeErroris already converted the same way.Parser.load_jsonthen sends it to_handle_invalid_json_error. This fixesPyramidParser, which passesreq.charsetasencoding.AIOHTTPParser.load_json: catch theLookupErrorthat aiohttp'sRequest.text()raises for an unknown charset and pass it to_handle_invalid_json_error. The annotation of_handle_invalid_json_erroris widened to match.tests/test_core.py::test_parse_json_with_unknown_encoding_raises_json_decode_errortest_parse_json_with_unknown_charsetintests/test_pyramidparser.pyandtests/test_aiohttpparser.pydevand pass with this change.Flask, Django, Bottle and Falcon ignore the charset parameter and decode as UTF-8, so they are unaffected. This PR does not change their behaviour.
Testing
pytest: 506 passed, 4 skipped (devbaseline: 503 passed, 4 skipped)ruff check/ruff format --check: cleanmypy src/webargs: no new errors (the 2 existingunreachableerrors intornadoparser.pyare also ondev)