Skip to content

batch insert fails the whole batch with "ValueError: Unexpected type" when a value is a Decimal #2263

Open

Description

@steven-asearcy

ran into this in a singer target. the singer sdk parses json numbers as decimal.Decimal, and postgres numeric columns come back as Decimal through psycopg too, so probably not just me.

data.insert gives a clear error:

WeaviateInvalidInputError: Invalid input provided: Cannot serialize value of type <class 'decimal.Decimal'> to Weaviate..

the grpc paths don't. insert_many raises ValueError: Unexpected type from inside protobuf, and the batch context managers just log it. every object in that batch ends up in failed_objects, including the good ones:

with col.batch.fixed_size(batch_size=100) as batch:
    for i in range(99):
        batch.add_object({"price": float(i)})
    batch.add_object({"price": Decimal("9.99")})

len(col.batch.failed_objects)  # 100
len(col)                       # 0

the message doesn't say which property or type it was, so it's hard to track down.

cause is _serialize_primitive in collections/batch/grpc_batch.py. it passes anything it doesn't handle straight to Struct.update. numpy.float32 hits the same thing.

weaviate-client 4.24.1, weaviate 1.40.5, python 3.13

i'll put up a PR that raises the same WeaviateInvalidInputError as data.insert. one bad value would still fail the whole batch after that, which felt like a bigger change so i left it alone. can take a look at that too if you want.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions