[Bug] Table-model UDFs fail on TAG/ATTRIBUTE predicates, SHOW/COUNT DEVICES and UPDATE since 2.0.11 #19178

Search before asking

• I searched in the issues and found nothing similar.

Version

The official apache/iotdb:2.0.11-standalone Docker image (2.0.11, build 7c0de06, linux/arm64), and master at d8800f1 built from source. The same steps work on apache/iotdb:2.0.10-standalone (build 3c26382).

Describe the bug and provide the minimal reproduce step

A user-defined table scalar function fails when it is used on a TAG or ATTRIBUTE column in WHERE, in SHOW DEVICES / COUNT DEVICES ... WHERE, or in UPDATE ... SET. The same function works in the SELECT list, and the built-in upper() works in all of these places.

The function only returns its argument:

package lazy;

import org.apache.iotdb.udf.api.customizer.analysis.ScalarFunctionAnalysis;
import org.apache.iotdb.udf.api.customizer.parameter.FunctionArguments;
import org.apache.iotdb.udf.api.relational.ScalarFunction;
import org.apache.iotdb.udf.api.relational.access.Record;
import org.apache.iotdb.udf.api.type.Type;

public class StrF implements ScalarFunction {
  @Override
  public ScalarFunctionAnalysis analyze(FunctionArguments arguments) {
    return new ScalarFunctionAnalysis.Builder().outputDataType(Type.STRING).build();
  }

  @Override
  public Object evaluate(Record input) {
    return input.isNull(0) ? null : input.getBinary(0);
  }
}

CREATE DATABASE db;
CREATE TABLE db.t (device_id STRING TAG, attr1 STRING ATTRIBUTE, v INT64 FIELD);
INSERT INTO db.t(time, device_id, attr1, v) VALUES (1,'d1','a',1),(2,'d1','a',2),(1,'d2','b',100);
CREATE FUNCTION str_f AS 'lazy.StrF' USING URI 'file:/tmp/lazyudf.jar';

SELECT time, device_id, str_f(device_id) FROM db.t ORDER BY device_id, time;
SELECT count(*) FROM db.t WHERE str_f(device_id) = 'd1';
SELECT count(*) FROM db.t WHERE str_f(attr1) = 'a';
SHOW DEVICES FROM db.t WHERE str_f(device_id) = 'd1';
COUNT DEVICES FROM db.t WHERE str_f(device_id) = 'd1';
UPDATE db.t SET attr1 = str_f(device_id) WHERE device_id = 'd2';

What did you expect to see?

What 2.0.10 returns: the first SELECT returns the three rows, both count(*) queries return 2, SHOW DEVICES returns d1 | a, COUNT DEVICES returns 1, and the UPDATE succeeds and sets attr1 of d2 to d2.

What did you see instead?

On 2.0.11 and master the first SELECT still works, but the rest fail:

SELECT count(*) ... WHERE str_f(device_id) = 'd1'   301: outerQueryDeadlineMs must be positive for UDF execution
SELECT count(*) ... WHERE str_f(attr1) = 'a'        301: outerQueryDeadlineMs must be positive for UDF execution
SHOW DEVICES ... WHERE str_f(device_id) = 'd1'      301: outerQueryDeadlineMs must be positive for UDF execution
COUNT DEVICES ... WHERE str_f(device_id) = 'd1'     301: IoTDBLocalFactory must not be null for UDF execution
UPDATE db.t SET attr1 = str_f(device_id) ...        305: null

For the UPDATE, the DataNode log shows IoTDBLocalFactory must not be null for UDF execution, thrown from SchemaRegionMemoryImpl#constructDevicePredicateUpdater under SchemaRegionStateMachine#write. On 2.0.11, the same statements with upper() in place of str_f all succeed.

Anything else?

The messages come from TableOperatorGenerator.IoTDBLocalFactory#createIoTDBLocal, added with IoTDBLocal in #18035. UserDefineScalarFunctionTransformer calls it in its constructor, so every user-defined scalar function needs a factory, a fragment instance id, a query id and a positive deadline, even when it never uses IoTDBLocal.

• For schema fragments, FragmentInstanceManager#execSchemaQueryFragmentInstance creates the FragmentInstanceContext without a deadline, so it stays at -1. The data path passes DriverTask.computeDeadlineTimeInMs(instance.getTimeOut()).
• COUNT DEVICES builds its transformer context in DataNodeTableOperatorGenerator#visitTableDeviceQueryCount without a factory.
• UPDATE builds its transformers inside SchemaRegionStateMachine#write, where there is no query.

The UPDATE case also came up in a review comment on #18889 (#18889 (comment)), and I ran into the rest while testing #19107.

I can see two directions: give schema fragments the same deadline and factory as data fragments, and create IoTDBLocal only when a function actually uses it, with a clear error where there is no query, as in UPDATE. I'm happy to send a PR once it's clear which one is preferred.

Are you willing to submit a PR?

• I'm willing to submit a PR!