Repository navigation
Description
Symptom
Two CH_* API options change declarations in public headers: CH_USE_ABSEIL_FOR_BIGNUM and CH_NON_OPTIONAL_CURRENT_ENDPOINT. The build gives them to users only as PUBLIC compile definitions of the CMake target. cmake --install installs the headers and the libraries, but no installed file records the option values of the library.
A program that compiles against the installed headers without CMake (plain compiler, Homebrew, distro package) does not get the definitions. In #if, an undefined macro is 0. Thus the program always sees the declarations for CH_USE_ABSEIL_FOR_BIGNUM=0 and CH_NON_OPTIONAL_CURRENT_ENDPOINT=0, whatever values the library was built with.
- Default build (
CH_USE_ABSEIL_FOR_BIGNUM=ON): every API that hasInt128/UInt128in its parameters does not link (ColumnInt128,ColumnUInt128,ColumnDecimal::Append(const Int128&),Bignum::Int128ToString, ...). - Build with
CH_USE_3X_API=ON(the build thatREADME.mdrecommends): a call toClient::GetCurrentEndpoint()compiles and links with no warning, and then crashes with a segmentation fault. The return type is not part of the mangled name, so the 2.x declaration (const std::optional<Endpoint>&) links to the 3.x definition (Endpointby value).
Reproduction
master at 58453c2, clang 18.1, Linux x86-64, ClickHouse server 26.10.4.207 on localhost:9000.
Default build:
cmake -G Ninja -S . -B build -DCMAKE_BUILD_TYPE=Release -DCMAKE_INSTALL_PREFIX=/tmp/inst
cmake --build build && cmake --install buildint128.cpp:
#include <clickhouse/client.h>
#include <iostream>
using namespace clickhouse;
int main() {
Client client(ClientOptions().SetHost("localhost"));
client.Execute("DROP TABLE IF EXISTS t_i128");
client.Execute("CREATE TABLE t_i128 (x Int128) ENGINE = Memory");
auto col = std::make_shared<ColumnInt128>();
col->Append(Bignum::MakeInt128(1, 2)); // 2^64 + 2
col->Append(Bignum::MakeInt128(-1, 0xFFFFFFFFFFFFFFFFULL)); // -1
Block b;
b.AppendColumn("x", col);
client.Insert("t_i128", b);
client.Select("SELECT x FROM t_i128", [](const Block& blk) {
for (size_t i = 0; i < blk.GetRowCount(); ++i)
std::cout << "read: " << Bignum::Int128ToString(blk[0]->As<ColumnInt128>()->At(i)) << "\n";
});
}clang++ -std=c++17 -I/tmp/inst/include int128.cpp \
-L/tmp/inst/lib -lclickhouse-cpp-lib -lcityhash -llz4 -lzstdstatic -labsl_int128undefined reference to `clickhouse::ColumnVector<clickhouse::Int128>::Append(clickhouse::Int128 const&)'
undefined reference to `clickhouse::ColumnVector<clickhouse::Int128>::At(unsigned long) const'
undefined reference to `clickhouse::Bignum::Int128ToString[abi:cxx11](clickhouse::Int128)'
undefined reference to `clickhouse::ColumnVector<clickhouse::Int128>::ColumnVector()'
The library defines these functions for absl::int128; the program asks for the non-Abseil struct clickhouse::Int128.
Build with the 3.x API:
cmake -G Ninja -S . -B build-3x -DCMAKE_BUILD_TYPE=Release -DCH_USE_3X_API=ON -DCMAKE_INSTALL_PREFIX=/tmp/inst-3x
cmake --build build-3x && cmake --install build-3xendpoint.cpp:
#include <clickhouse/client.h>
#include <iostream>
using namespace clickhouse;
int main() {
Client client(ClientOptions().SetHost("localhost").SetPort(9000));
const auto& ep = client.GetCurrentEndpoint();
std::cout << "has_value=" << ep.has_value() << std::endl;
if (ep) std::cout << "endpoint=" << ep->host << ":" << ep->port << std::endl;
}clang++ -std=c++17 -I/tmp/inst-3x/include endpoint.cpp \
-L/tmp/inst-3x/lib -lclickhouse-cpp-lib -lcityhash -llz4 -lzstdstatic
./a.out # Segmentation fault (core dumped), exit code 139Expected vs actual
Expected: the installed headers give the same declarations that the installed library was built with, with no extra flags. At a minimum, a mismatch must fail at build time and not at run time.
| Library build | Program | Program flags | Result |
|---|---|---|---|
| default | int128.cpp |
none | link errors (above) |
| default | int128.cpp |
-DCH_USE_ABSEIL_FOR_BIGNUM=1 + Abseil headers on the include path |
links, prints 18446744073709551618 and -1 (the server returns the same values for SELECT x FROM t_i128) |
| default | endpoint.cpp |
none | prints has_value=1, endpoint=localhost:9000 |
CH_USE_3X_API=ON |
endpoint.cpp |
none | compiles, links, segmentation fault |
CH_USE_3X_API=ON |
same call written for 3.x (Endpoint ep = client.GetCurrentEndpoint();) |
-DCH_NON_OPTIONAL_CURRENT_ENDPOINT=1 |
prints endpoint=localhost:9000 |
nm -C on libclickhouse-cpp-lib.a of the 3.x build shows only one symbol, clickhouse::Client::GetCurrentEndpoint() const, for the two declarations.
Root cause
clickhouse/CMakeLists.txt:133-148: the options reach users only throughTARGET_COMPILE_DEFINITIONS (clickhouse-cpp-lib PUBLIC CH_...=0|1). Only a CMake consumer of the build tree (add_subdirectory,FetchContent) gets them.clickhouse/CMakeLists.txt:184-271: the install step installs the libraries and the headers. It installs no generated header, no CMake package config and no pkg-config file.- The headers test the macros with
#if:clickhouse/types/bignum.h:7,:36,:174,:236andclickhouse/client.h:343. CH_MAP_BOOL_TO_UINT8is not affected: onlyclickhouse/types/type_parser.cpp:45reads it.
Suggested fix
Record the configured values in an installed header:
- Add a template, for example
clickhouse/build_options.h.in, that definesCH_USE_ABSEIL_FOR_BIGNUMandCH_NON_OPTIONAL_CURRENT_ENDPOINT(and, for completeness,CH_MAP_BOOL_TO_UINT8) as0or1. Generate it withCONFIGURE_FILEinto the build tree. - Include it from
types/bignum.handclient.h(or from one header that both include), and install it toinclude/clickhouse/. - Remove the PUBLIC compile definitions, or make sure that they cannot disagree with the header (for example, the header gives
#errorwhen a macro is already defined with a different value).
Cases that must keep their current behavior:
- In-tree CMake consumers (
add_subdirectory,FetchContent) get the same values as now. - The Bazel build (
BUILD.bazelsetsCH_NON_OPTIONAL_CURRENT_ENDPOINT=1) must also supply the header, for example with agenruleor a static header for Bazel.
Related
- Generate CMake config files when installing the project #105 and CMake packaging improvements #418 (CMake package config files): a CMake package config gives the definitions to
find_packageusers only. Programs that do not use CMake (the case above, and the Homebrew formula test) still do not get them. - abseil library isn't installed resulting in compilation error #284 (Abseil headers are not installed): a separate gap. With
-DCH_USE_ABSEIL_FOR_BIGNUM=1, the installedtypes/bignum.hincludesabsl/numeric/int128.h, which the install does not supply. v2.6.2does not have these options, so packages built from it are not affected. The first release that has them is affected.
Provenance
Found by automated analysis (Polyglot AI) during the triage of #597. Verified by a build and install of master in a container and by the programs above, compiled against the install. Not from code reading only.
Classification: impact: Stability · area: Environment (install layout, CH_* options)
Activity