Repository navigation
Assign implicit Enum values like the server - #616
Open
Indian-boult wants to merge 2 commits into
Open
Indian-boult wants to merge 2 commits into
Indian-boult wants to merge 2 commits into
Conversation
CreateColumnByType() read the elements of an Enum type name as strict
name and number pairs. The parser opens a new element at each = and at
each comma, so a name without a number is followed directly by the
next name. With an even element count the client built a different
type and gave no error. Enum8('a', 'b') became Enum8('a' = 0), and
Enum8('a' = 5, 'b', 'c' = 10, 'd'), which the server rejects, became
Enum8('' = 0, 'a' = 5). With an odd element count it threw
ValidationError, so Enum8('a', 'b', 'c') and Enum8('a' = 5, 'b')
failed although the server accepts them.
The factory now reads each quoted name and the number after it, if the
name has one. A name without a number gets the number of the first
name plus its position. If the first name has no number, the count
starts at 1. This is the rule of the server, so Enum8('a', 'b') is
Enum8('a' = 1, 'b' = 2) and Enum8('a' = -3, 'b', 'c') is
Enum8('a' = -3, 'b' = -2, 'c' = -1).
The factory also rejects three forms that the server rejects. These are
a list where some names after the first have a number and some do not,
a name without quotes, and a number out of the range of Enum8 or
Enum16. The range check runs after the numbering, so
Enum8('a' = 127, 'b') is an error.
TypesCase.ErrorEnumContent expected an error for Enum8('ONE') and
Enum8('ONE'=1,'TWO'). The server accepts both, so they move to the new
list of valid type names.
Fixes ClickHouse#604
CreateColumnFromAst calls itself once per nesting level, and without optimization each call reserves stack for the locals of every case. The new Enum code made each level 528 bytes larger with clang -O0, and the Windows Bazel fastbuild ran out of its 1 MB stack in TypeParser.NestingDepthLimitBoundsColumnFactory, which nests 512 arrays.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #604.
CreateColumnByType()incolumns/factory.cppreads the elements of an Enum type name as strict name and number pairs. The parser intypes/type_parser.cppopens a new element at each=and at each,, so a name without a number is followed directly by the next name.With an even element count the client builds a different type than the server and gives no error.
Enum8('a', 'b')becomesEnum8('a' = 0).Append("b")on that column throwsstd::out_of_range, and an INSERT into a table with the same DDL fails withEnum conversion changes value. With an odd element count the factory throwsValidationError, soEnum8('a', 'b', 'c')andEnum8('a' = 5, 'b')fail although the server accepts them.The factory now walks the flat list. Each quoted name can have one number after it. A name without a number gets the number of the first name plus its position. If the first name has no number, the count starts at 1. This is the rule of the server.
The factory also rejects three forms that the server rejects. These are a list where some names after the first have a number and some do not, a name without quotes, and a number out of the range of
Enum8orEnum16. The range check runs after the numbering, soEnum8('a' = 127, 'b')is an error, as on the server.Enum8('a', 'b')Enum8('a' = 1, 'b' = 2)Enum8('a' = 0)Enum8('a', 'b', 'c')Enum8('a' = 1, 'b' = 2, 'c' = 3)ValidationErrorEnum8('a' = 5, 'b', 'c')Enum8('a' = 5, 'b' = 6, 'c' = 7)Enum8('b' = 0, 'a' = 5)Enum8('a' = -3, 'b')Enum8('a' = -3, 'b' = -2)ValidationErrorEnum8('a', 'b' = 5)Enum8('a' = 1, 'b' = 5)ValidationErrorEnum16('a' = 127, 'b')Enum16('a' = 127, 'b' = 128)ValidationErrorEnum8('x=y', 'b')Enum8('x=y' = 1, 'b' = 2)Enum8('x=y' = 0)Array(Enum8('a', 'b'))Array(Enum8('a' = 1, 'b' = 2))Array(Enum8('a' = 0))Enum8('a' = 5, 'b', 'c' = 10, 'd')Enum8('' = 0, 'a' = 5)Enum8('a' = 126, 'b', 'c')Enum8('b' = 0, 'a' = 126)Enum8(aa, 1)Enum8('' = 1)Enum8('a' = 128)Enum8('a' = 128)I got the server column from ClickHouse 26.10 with
SELECT toTypeName(defaultValueOfTypeName(type_name)). ClickHouse 25.8 gives the same results.Only type names that the application writes are affected. Type names from the server always have explicit numbers. The parser and the
TypeAstshape do not change.Each of these tests fails on master and passes with this change.
Enum/CreateColumnByEnumTypeWithImplicitValuesinut/CreateColumnByType_ut.cpphas 23 type names and the type name that the server gives for each one. It covers the count from 1 and from the first number, negative numbers, the range limits, names with=and\', names that differ only in case, and Enum types inNullable,Array,MapandTuple.CreateColumnByType.EnumImplicitValuesAppendappends labels to a column built fromEnum8('a' = -3, 'b', 'c')and checks their numbers.TypesCase.ErrorEnumContentexpected an error forEnum8('ONE')andEnum8('ONE'=1,'TWO'). The server accepts both, so they move to the valid list. The test now has the mixed lists, a name without quotes and out-of-range numbers.ClientCase.Enum_ImplicitValuescreates a table withEnum8('a' = -3, 'b', 'c'), inserts rows with a column built from the same type name and reads the labels and numbers back.Some forms stay as they are. The flat list does not keep the difference between
=and,. ThusEnum8('a', 1)is still read asEnum8('a' = 1), andEnum8('a' = 'b')is now read likeEnum8('a', 'b'). The server rejects both. A trailing comma such asEnum8('a', 'b',)is still an error, although the server accepts it. The client still does not support the bareEnumtype without a width.This PR does not change duplicate detection. That is #607 and #609. With implicit numbers,
Enum8('a', 'b' = 1)gives both names the number 1, and #609 rejects it. The doubled quote in'it''s'is #605 and #608. Both PRs apply on top of this one without conflicts, and the full test suite passes with all three.Note
Medium Risk
Changes core type-to-column parsing for Enums and can alter numeric assignments for hand-written DDL, affecting inserts and appends; scope is limited to client-side type construction with added server-aligned validation.
Overview
CreateColumnByTypeno longer treats Enum type strings as fixed name/value pairs. Enum construction moves intoCreateEnumColumn, which walks the parser’s flat AST: each label may optionally be followed by a number, then applies the same implicit numbering rules as ClickHouse (base from the first explicit value or start at 1, sequential offsets for unnamed labels).Validation is tightened to reject mixed explicit/implicit numbering after the first entry, unquoted names, and values outside Enum8/Enum16 range—cases the server also rejects.
Tests add broad coverage for implicit Enum definitions (including nested types), append/read behavior, invalid definitions in
TypesCase.ErrorEnumContent, and a live client round-trip insert/select forEnum8('a' = -3, 'b', 'c').Reviewed by Cursor Bugbot for commit 2011a88. Bugbot is set up for automated code reviews on this repo. Configure here.