Skip to content

Should responses with content-type application/json have UTF-8 encoding by default? #714

Closed

Description

@drwl

Currently, responses without a specified charset encoding in the header are stored internally with binary/ascii 8bit encoding. According to the spec, the specified charset should be redundant. Should the default for json responses by utf-8 instead?

From RFC8259:

JSON text exchanged between systems that are not part of a closed
ecosystem MUST be encoded using UTF-8 [RFC3629].

Previous specifications of JSON have not required the use of UTF-8
when transmitting JSON text. However, the vast majority of JSON-
based software implementations have chosen to use the UTF-8 encoding,
to the extent that it is the only encoding that achieves
interoperability.

Also in RFC8259, at the end of Section 11 (or just above of Section 12):

Note: No "charset" parameter is defined for this registration.
Adding one really has no effect on compliant recipients.

Activity

  1. tarcieri commented on Jun 13, 2022

    @tarcieri
    Member

    That seems like a reasonable default, yes

  2. drwl commented on Jun 14, 2022

    @drwl
    ContributorAuthor

    That seems like a reasonable default, yes

    I can try to put up a PR. Where do you suggest this code change take place?

    My initial thought is to change HTTP::Response initializer:

            encoding   = opts[:encoding] || charset
    
            if encoding.nil?
              if mime_type == 'application/json'
                encoding = Encoding::UTF_8
              else
                encoding = Encoding::BINARY
              end
            end

    thoughts?

  3. tarcieri commented on Jun 14, 2022

    @tarcieri
    Member

    Sounds good to me /cc @ixti

  4. ixti commented on Jun 16, 2022

    @ixti
    Member

    @tarcieri I like the idea. Reviewed the PR.

  5. drwl commented on Jun 16, 2022

    @drwl
    ContributorAuthor

    @ixti @tarcieri thanks for quick turn around. Not to rush, but do yall have any idea when I can expect a release with this change?

    I came across this related to work and so I'm figuring out if I should make changes to support this internally or if I can wait and update the gem (when it's out).

  6. ixti commented on Jun 17, 2022

    @ixti
    Member

    Released as 5.1.0

  7. ixti commented on Oct 11, 2022

    @ixti
    Member
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions