Skip to content

mssql: expression typing (type precedence and built-in functions) - #4

Open
EliSauder wants to merge 21 commits into
mainfrom
mssql/expression-typing
Open

EliSauder wants to merge 21 commits into
mainfrom
mssql/expression-typing

Conversation

@EliSauder

@EliSauder EliSauder commented Nov 13, 2026 •

Copy link
Copy Markdown
Owner

How SQL Server combines types and types its built-in functions.

  • Precedence: set-operation columns from every branch (dialect set_operations rules, int/bit as decimals, argument-free families), mixed arithmetic operators, CASE/COALESCE/GREATEST/LEAST/CHOOSE/IIF, placeholders among alternatives.
  • Functions: common and further built-in seeds, datepart keywords, AT TIME ZONE, ISNULL, ROUND, DATEADD/DATETRUNC over strings, LEFT/RIGHT/TRIM, REPLICATE/CHARINDEX, xml/geography/geometry/hierarchyid methods.

Commits

  • e7b7f555c fix(mssql): pass the datepart of DATEADD and its relatives as a keyword
  • 52b528bcf feat(mssql): type AT TIME ZONE
  • 7aaa35cd8 feat(mssql): seed common built-in functions
  • d6fb19e91 fix(mssql): type mixed arithmetic by data type precedence
  • 94962e827 fix(mssql): type ISNULL as nullable only when both arguments are
  • f6e56e3db fix(mssql): type set operation columns from every branch
  • f424edbbf fix(mssql): count int and bit as decimals when combining types
  • 64e9b6a85 fix(mssql): type CASE, COALESCE, GREATEST, LEAST and CHOOSE by precedence
  • 25b675338 fix(mssql): seed ROUND with its function argument
  • f66c55192 fix(mssql): type DATEADD and DATETRUNC over a string date
  • a0842fc65 feat(mssql): type the methods of xml, geography, geometry and hierarchyid
  • f04eabadb fix(core): type a placeholder among alternatives as the others combine
  • 34bc380b0 feat(mssql): seed more built-in functions
  • 13138f46f feat(mssql): type LEFT, RIGHT and TRIM(chars FROM string)
  • 95835ee18 fix(mssql): take REPLICATE's and CHARINDEX's integer arguments as int
  • 5ba6c853b fix(core): type a set operation column of an argument-free type as its family

Test changes for independence

Independence. This is one of five PRs split from #1, each off main with no dependency on the others: it builds, passes the base suite and keeps the core context at or above main's, alone. Its goldens are generated on this branch, and they hold unchanged with all five merged, whose code then equals the original branch. Parallel PRs add lines next to each other in convert.go, functions.jsonl and the dialect settings, so whichever merges later may need a rebase that keeps both sides.

Verification. Behavior was checked against SQL Server 2025 in #1 (goldeneye or sp_describe_*). go test ./internal/endtoend -run TestReplay/base/ and the touched packages' tests pass; generated Go goldens pass go vet.

Review follow-up: engine-supplied type combination

  • f669ca0cc: replace the earlier named-strategy approach with an optional generic core.TypeCombiner. The engine owns its implementation, configuration schema and validation; shared analysis/seeding has no MSSQL imports or strategy names. The compiler constructs runtime behavior after either catalog seeding or cache restoration, and nested analyzers inherit it.
  • 91aab3b0e: update the extension/cache documentation. The earlier seven-engine comparison remains; no new coercion policies are enabled for other engines.

Backward compatibility: engines without a combiner retain their existing first-branch/first-typed-candidate inference. A combiner can decline individual contexts with the same fallback. Existing call sites can still call Prepare without options. Existing goldens remain unchanged; only two new nested MSSQL cases were appended.

Verification: the complete TestReplay/base/ corpus passes. Selected experimental-core replays have identical outcomes for all 71 recorded test nodes and identical diffs for five pre-existing MySQL/SQLite failures, compared against 002e9f690. Touched package tests and targeted MSSQL end-to-end tests pass under -race; vet, build and whitespace checks pass. A warm catalog exercised with a different query matches a cold catalog and returns the expected decimal(12,2). Invalid embedded configuration and injected failing/declining policies have focused tests because those inputs cannot be supplied through the CLI. No live database validation was performed for this refactor.

The separate JSONL/converter-metadata discussions have been resolved; this PR does not introduce that metadata system.

ISNULL policy correction and leakage audit

41a769124 removes FirstArgType entirely. The shared AST preserves source function identity, and a qualified FunctionName in the generic request lets the MSSQL policy choose ISNULL's first typed candidate. Shared first-typed selection remains only as the compatibility fallback. AST traversal and printing retain ISNULL rather than silently printing COALESCE.

Audit scope: reviewed this PR's shared analyzer, AST, catalog/seed and compiler changes, including actual inference/control flow rather than just engine-name searches. No additional MSSQL-specific policy leakage was found. Precedence, decimal/length combination, configuration validation and ISNULL selection are engine-owned. Shared set-operation nullability, family-type resolution, parameter inference and no-policy fallback are generic mechanisms; the compiler's MSSQL factory reference is engine assembly, not shared policy dispatch.

Verification for this correction: the complete base replay corpus passes. New end-to-end cases pin the pre-change types, nullability and inferred parameters for differing string families, literal NULLs and placeholders. Race-enabled touched-package and targeted end-to-end checks, vet, build and whitespace checks pass. CLI AST inspection confirms source identity for all eight ISNULL calls with no behavioral flag. Fresh/restored catalogs both retain varchar(10) for ISNULL while COALESCE widens to nvarchar(40). Direct printer coverage is needed because the fmt CLI does not support MSSQL. No live database validation was needed for the policy-ownership refactor.

EliSauder and others added 16 commits November 12, 2026 18:34
DATEADD(day, ...), DATEDIFF, DATEPART, DATENAME, DATETRUNC and
DATE_BUCKET take a datepart keyword first, which teesql reports as a
column reference, so the analyzer failed with unknown column "day".
The converter now passes it as the string it names.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
teesql parses x AT TIME ZONE tz, but the converter dropped it, leaving
the column untyped. It is now the call of a seeded pseudo-function, so
the result is the nullable datetimeoffset SQL Server describes, with
datetime's precision of 3, and a time zone placeholder is nvarchar(4000).

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Window, string, math, date and JSON functions such as ROW_NUMBER, LAG,
ISNULL, REPLACE, ROUND, EOMONTH and JSON_VALUE were unknown to the
analyzer and came out untyped. Seed them with the result types and
nullability SQL Server reports for them, so queries using them get real
column types.

DATEADD, DATEDIFF, DATEPART and DATENAME are now typed, so the
analyze_datepart case is regenerated, its placeholder moved to WHERE
where SQL Server can describe the query to check it.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Only same-type arithmetic was seeded, so an expression mixing types such
as decimal * int fell back to a cast and came out as bigint. SQL Server
gives such an expression the operand type with the higher data type
precedence, and + on strings concatenates, giving a Unicode result when
either side is Unicode and a variable-length one when either side is.
Seed those operator pairs.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
ISNULL was seeded as a function returning its first argument's type,
never NULL, which is wrong when both arguments can be NULL. It is now
converted to the COALESCE node it behaves as: the first argument's type,
NULL only when every argument is.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
A UNION, INTERSECT or EXCEPT column was typed from the first branch, as
it is named, but SQL Server types it from every branch: the type of
highest precedence, varying when either branch varies, with the length,
fractional precision or decimal precision and scale that hold both, and
reads it from no one table. It is nullable in a UNION when either branch
is, in an INTERSECT when both are, and in an EXCEPT when the first is.

A dialect may now give these rules as data, in set_operations; SQL
Server does. Without them a column is typed from the first branch, as
before.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
A set operation combining an int or bit with a decimal kept the decimal's
precision, which is what SQL Server reports for a NULL constant of those
types, but not for a column: an int column counts as decimal(10, 0) and
a bit as decimal(1, 0), so int UNION ALL decimal(5,2) is decimal(12,2).

Addresses review comment r4557025343 on #1.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…ence

These yield one of several values, and were typed as the first typed one,
so CASE WHEN on_sale THEN 0 ELSE price END was an int and generated an
int32 the decimal price could not scan into. SQL Server types them as it
does a set operation's columns: the candidates' types combined by data
type precedence. They now are, by the dialect's set_operations rules;
a dialect without them types them as before.

GREATEST and LEAST convert to the min/max node and CHOOSE to the CASE
over its index it is defined as, whose placeholder now takes the type of
what it is compared with. ISNULL keeps its first argument's type, as
SQL Server gives it.

Addresses review comment r4557025398 on #1.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
ROUND(value, length, function) truncates rather than rounds when its
third argument is not zero, but only the two-argument form was seeded,
so a three-argument call matched no overload by its arguments and took
the first, bigint. Each overload now has a three-argument form returning
what its two-argument form does.

Addresses review comment r4557025437 on #1.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
DATEADD and DATETRUNC return the type of the date they are given, which
a string literal is not: SQL Server reads one as datetime for DATEADD
and datetime2(7) for DATETRUNC. The seed returned the argument's own
type, varchar. String overloads now return what the server does, while
a typed date keeps its type.

Addresses review comment r4557025498 on #1.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…hyid

A method called on a value, as doc.exist('/a') or node.GetLevel() is,
was taken for a function of that name, so its result went untyped. A
method is now seeded as ".name" taking the value first, and a static
one, as geography::Point(...), as "geography::point"; xml's value(path,
'type') returns the type it names. Each may return NULL.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
A placeholder passed to COALESCE, GREATEST, LEAST or IIF, or returned by
a CASE branch, went untyped. It now takes the type the other
alternatives combine to; one in COALESCE may be NULL, since that is what
passing it there is for.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
GROUPING, GROUPING_ID, APPROX_COUNT_DISTINCT, CHECKSUM_AGG and
PERCENTILE_CONT; the metadata functions, as DB_NAME, OBJECT_ID and
IDENT_CURRENT; the ERROR_* functions, XACT_STATE and SESSION_CONTEXT;
STRING_ESCAPE, SOUNDEX, DIFFERENCE, FORMATMESSAGE, COMPRESS and
DECOMPRESS; the bit functions; DATE_BUCKET; the REGEXP_* functions;
VECTOR_DISTANCE; and LTRIM and RTRIM given the characters to trim.
Each was untyped. Their types are those SQL Server reports.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
LEFT and RIGHT, which the parser gives nodes of their own, were not
converted, so their results went untyped, and TRIM(chars FROM string)
took the type of the characters. LEFT and RIGHT are now called, taking
their length as the int SQL Server does, and TRIM given the characters
returns the string's type.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
A placeholder for the count REPLICATE repeats a string, or the position
CHARINDEX starts at, was a bigint, where SQL Server takes an int.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…s family

A column of UNION, INTERSECT or EXCEPT whose combined type has no
arguments, as a string literal's varchar or decimal arithmetic's
decimal, was looked up as a declared type, which takes the dialect's
defaults: varchar(1) and decimal(18,0). It is now its family.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

@EliSauder EliSauder left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some things to consider.

Comment thread internal/core/analyzer/expr.go Resolved
Comment thread internal/core/analyzer/setop.go Resolved
Comment thread internal/engine/mssql/convert.go Resolved
Comment thread internal/engine/mssql/convert.go Resolved
Comment thread internal/engine/mssql/convert.go Resolved
Comment thread internal/engine/mssql/convert.go Resolved
EliSauder and others added 2 commits November 12, 2026 23:18
Keep SQL Server precedence and modifier rules in its typepolicy package, validate the serialized selector, and preserve the existing fallback for all other dialects. Cover invalid embedded seed metadata separately because it cannot be supplied through the CLI.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Compare all seven seeded engines' common-type semantics and explain why SQL Server settings must not be copied to them. Clarify converter dispatch and the metadata needed to replace its recognition maps.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

@EliSauder EliSauder left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not quite there

Comment thread internal/core/analyzer/setop.go Resolved
Comment thread internal/core/seed/seed.go Resolved
EliSauder and others added 2 commits November 12, 2026 23:56
Remove SQL Server dispatch and typed policy settings from shared analysis and seeding. Bind the optional engine factory after catalog creation or restore, inherit it in nested queries, and retain existing fallback inference when no combiner handles a context.

Cover nested SQL Server combination end to end and inaccessible policy-contract/configuration failures with focused tests. Preserve single-candidate column provenance and all existing engine goldens.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Replace named-strategy documentation with the optional generic interface, engine-owned metadata validation and runtime construction after both catalog paths. Document backward-compatible inference for engines without a combiner.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

@EliSauder EliSauder left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Something to think about

Comment thread internal/core/analyzer/expr.go Resolved
Comment thread internal/core/typecombiner.go Resolved
Replace the shared FirstArgType instruction with source function identity passed through the generic combination request. Keep first-typed inference in shared code only as the compatibility fallback. Preserve ISNULL identity in AST traversal and printing.

Cover first-argument selection versus COALESCE widening, literal NULLs and placeholders with pre-change end-to-end goldens. Exercise formatting directly because the fmt CLI does not support MSSQL.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant