Repository navigation
Description
Description
ColumnArray::AppendAsColumn() (clickhouse/columns/array.cpp:49) relies on data_->Append() throwing when the supplied column type does not match the array's element type:
void ColumnArray::AppendAsColumn(ColumnRef array) {
// appending data may throw (i.e. due to ype check failure), so do it first to avoid partly modified state.
data_->Append(array);
AddOffset(array->Size());
}But most Column::Append(ColumnRef) implementations silently no-op on a type mismatch instead of throwing:
ColumnVector<T>::Append—clickhouse/columns/numeric.cpp:72-76:if (auto col = column->As<ColumnVector<T>>()) { ... }, noelse.ColumnString::Append—clickhouse/columns/string.cpp:248-260: same shape.ColumnFixedString::Append—clickhouse/columns/string.cpp:73-79: same shape, and additionally silent whenstring_size_differs.
So passing a wrong-typed column to AppendAsColumn appends nothing to data_ while AddOffset(array->Size()) still advances the offsets by array->Size(). The ColumnArray is left internally inconsistent — exactly the state its own constructor rejects with ValidationError("Mismatch between data and offsets: ...") — and SaveBody() then writes offsets promising N elements followed by zero element bytes.
On the wire this desynchronizes the native-protocol block: the server reads the following column's bytes as this array's element data. The user gets no client-side error, only a confusing (and misleading) server-side exception, and the connection is left unusable for the next operation.
This is the C++ analogue of ClickHouse/clickhouse-java#3041, where SerializerUtils.serializeArrayData silently writes zero bytes for a non-null, non-array/non-List value.
Note the same failure class was reported for ColumnNullable in PR #376 (closed, unmerged): nulls_ grows even when the nested Append() silently fails. ColumnArray has the identical problem with offsets_.
ClickHouse server version
26.7.2.59 (native protocol, port 9000), verified against a live server.
Repo at commit 737145d.
Reproduction
Added to ut/column_array_ut.cpp (needs #include <clickhouse/client.h> for the second test):
TEST(ArrayDesync, WrongTypedAppendAsColumn) {
// Array(String), but we append a UInt64 column as one row's elements
auto arr = std::make_shared<ColumnArray>(std::make_shared<ColumnString>());
auto wrong = std::make_shared<ColumnUInt64>();
wrong->Append(1); wrong->Append(2); wrong->Append(3);
EXPECT_NO_THROW(arr->AppendAsColumn(wrong)); // currently passes -- the defect
std::cerr << "arr->Size()=" << arr->Size()
<< " claimed elems row0=" << arr->GetSize(0)
<< " actual data size=" << arr->GetAsColumn(0)->Size() << std::endl;
Buffer buf;
BufferOutput out(&buf);
arr->SaveBody(&out);
out.Flush();
std::cerr << "SaveBody bytes = " << buf.size() << std::endl;
}
TEST(ArrayDesync, EndToEndInsert) {
clickhouse::Client client(clickhouse::ClientOptions().SetHost("localhost").SetPort(9000));
client.Execute("DROP TABLE IF EXISTS test_arr_desync");
client.Execute("CREATE TABLE test_arr_desync (id UInt32, val Array(String), tail String) ENGINE = Memory");
clickhouse::Block b;
auto id = std::make_shared<ColumnUInt32>(); id->Append(1);
auto val = std::make_shared<ColumnArray>(std::make_shared<ColumnString>());
auto bad = std::make_shared<ColumnUInt64>(); bad->Append(7); bad->Append(8);
val->AppendAsColumn(bad); // wrong element type, silently dropped
auto tail = std::make_shared<ColumnString>(); tail->Append("TAILVALUE");
b.AppendColumn("id", id);
b.AppendColumn("val", val);
b.AppendColumn("tail", tail);
try { client.Insert("test_arr_desync", b); std::cerr << "INSERT SUCCEEDED" << std::endl; }
catch (const std::exception& e) { std::cerr << "INSERT threw: " << e.what() << std::endl; }
try {
client.Select("SELECT id, val, tail FROM test_arr_desync", [](const clickhouse::Block&) {});
} catch (const std::exception& e) { std::cerr << "SELECT threw: " << e.what() << std::endl; }
}Actual output
[ RUN ] ArrayDesync.WrongTypedAppendAsColumn
arr->Size()=1 claimed elems row0=3 actual data size=0
SaveBody bytes = 8
[ OK ] ArrayDesync.WrongTypedAppendAsColumn
[ RUN ] ArrayDesync.EndToEndInsert
INSERT threw: DB::Exception: Unknown data type family: TAILVALUE
SELECT threw: cannot execute query while executing another operation
[ OK ] ArrayDesync.EndToEndInsert
Reading that: SaveBody emitted only the 8-byte offset (3) and zero element bytes. End-to-end, the server consumed the tail column's string payload as the type name of the next column — hence Unknown data type family: TAILVALUE. The connection was then left mid-operation, so the follow-up SELECT also failed.
Expected
AppendAsColumn (or the underlying Append) should reject a column whose type does not match the array's element type with a clear client-side ValidationError / std::runtime_error naming both types, leaving the ColumnArray unmodified. Appending a correctly-typed column must keep working, and AppendAsColumn of an empty correctly-typed column must still add a zero-length row.
Suggested fix
Two possible layers, not mutually exclusive:
-
Narrow — in
ColumnArray::AppendAsColumn(clickhouse/columns/array.cpp:49), validate before mutating:void ColumnArray::AppendAsColumn(ColumnRef array) { if (!data_->Type()->IsEqual(array->Type())) throw ValidationError("Cannot append column of type " + array->Type()->GetName() + " to Array of " + data_->Type()->GetName()); data_->Append(array); AddOffset(array->Size()); }
This covers the array path only, but it is where the offsets/data desync is introduced.
-
General — make the
Append(ColumnRef)implementations throw on mismatch instead of silently returning (numeric.cpp:72,string.cpp:73,string.cpp:248, and any sibling with the sameif (auto col = column->As<...>())-with-no-elseshape). That would also fix theColumnNullablevariant from PR [Bug fix] Append() in Nullable columns causes nulls_ array to be incorrectly appended to if nested column Append() fails #376 and make the existing comment inAppendAsColumntrue. It is a behavior change for callers currently relying on the silent no-op, so it may warrant its own discussion.
Regression tests: a wrong-typed AppendAsColumn throws and leaves Size()/GetOffset() unchanged, plus contrast cases that a correctly-typed column and an empty correctly-typed column still behave as today.
Link
Relayed from ClickHouse/clickhouse-java#3041
Activity
polyglotAI-bot commentedon Nov 18, 2026
Recommendation: Close — already fixed
Commit 0dac477 ("Validate ColumnArray element types before appending", merged to
masteron 2026-09-16 in 09613d4) fixes this. Not released yet: the latest release v2.6.2 still shows the defect, so the fix reaches users with the next release.master: Fixed on main —master58453c2 throwsValidationErrorand sends no block; v2.6.2 reproduces the report (ClickHouse 26.10.4.207, default 2.x build options,WITH_OPENSSL=ON)ColumnNullable), Array(Nested(...)) columns silently decode as Array(Nothing) and desynchronize the native block stream #571 (a differentArraydesync, on the read path).Analysis: On
master,ColumnArray::AppendAsColumnchecks the element type before it changes the column (clickhouse/columns/array.cpp:96-101, helperCanAppendTypeatarray.cpp:31-54) and throwsValidationError. The rows, offsets and data stay unchanged, soSaveBodyandClient::Insertcannot send a desynchronized block. The check keeps the conversions that worked before: equal types,Tuplewith other field names,LowCardinality(T)fromT, andBoolfromUInt8; the same commit adds unit tests for them and for the empty-column row (ut/column_array_ut.cpp). In v2.6.2,AppendAsColumncallsdata_->Append()with no check (array.cpp:49-53at v2.6.2), and the run below shows the reported desync. The general layer of the report is still onmaster:ColumnVector<T>::Append(numeric.cpp:71-75),ColumnFixedString::Append(string.cpp:73-79) andColumnString::Append(string.cpp:252-265) ignore a column of another type with no error. To make them throw is a separate behavior change; if it is wanted, it needs its own issue.Reproduction: complete program and output
Output on
master(58453c2), ClickHouse 26.10.4.207:Output on v2.6.2, ClickHouse 26.10.4.207:
Automated triage by Polyglot AI. Classification added on 2026-11-19. A maintainer makes the final decision.