Repository navigation
feat(fluentbit): add the type_converter filter - #2132
Open
NopeCharon wants to merge 1 commit into
Open
NopeCharon wants to merge 1 commit into
NopeCharon wants to merge 1 commit into
Conversation
cvas22
reviewed
Nov 12, 2026
cvas22
left a comment
Contributor
There was a problem hiding this comment.
Checked the classic and YAML output against the Fluent Bit type_converter docs: one Int_Key/Uint_Key/Float_Key/Str_Key line per conversion in classic, and the repeated keys turn into a list in YAML, which is what the YAML loader expects. The CEL rule for at least one list makes sense too. Looks good to me.
Collaborator
|
Can you rebase to resolve the conflicts? |
Fluent Bit's type_converter filter converts the value of a key to int, uint, float or string and appends it under a new key. It's commonly used to make a numeric field matchable by grep or modify, or to turn a numeric string into a number before it reaches a metrics or search backend. FilterItem had no field for it, so ClusterFilter and Filter users had to fall back to customPlugin. Add a typeConverter filter with one list for each source type Fluent Bit supports (intKeys, uintKeys, floatKeys, strKeys). Each entry is a TypeConversion with fromKey, toKey and type, rendered as "<Src>_Key FROM TO TYPE" in the classic format and as a list in the YAML format. Fluent Bit splits each entry into exactly three tokens and won't start when the filter has no rules, so the CRD rejects: - keys that are empty or contain whitespace - types other than int, uint, float and string - empty lists, and a typeConverter with no conversions (CEL rule) The deepcopy code, CRDs, setup.yaml and API docs are regenerated, and the filter is listed in docs/plugins/fluentbit/index.md. Fixes fluent#2131 Signed-off-by: NopeCharon <115794254+NopeCharon@users.noreply.github.com>
Contributor
Author
|
Rebased onto master and resolved the conflicts (kept both sysinfo and typeConverter). |
NopeCharon
force-pushed
the
feat/2131-fluentbit-type-converter-filter
branch
from
November 12, 2026 13:42
b72e70f to
0ffe540
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this PR does / why we need it:
Adds a
typeConverterfilter to ClusterFilter and Filter, for Fluent Bit'stype_converterplugin. Until now the only way to use it wascustomPlugin.There is one list for each source type Fluent Bit supports:
intKeys,uintKeys,floatKeysandstrKeys. Each entry has afromKey, atoKeyand the targettype:This renders as:
In the YAML config format, a key with several conversions becomes a list.
I used structured entries rather than raw
"FROM TO TYPE"strings so the CRD can check them. Fluent Bit wants exactly three tokens per entry and won't start without at least one rule ([filter:type_converter] no rules). So the API server rejects:int,uint,floatandstringtypeConverterwith no conversions (a CEL rule)The deepcopy code, CRDs,
setup.yamland API docs are regenerated, and the filter is listed indocs/plugins/fluentbit/index.md.Which issue(s) this PR fixes:
Fixes #2131
Does this PR introduced a user-facing change?
Additional documentation, usage docs, etc.:
Testing
TestFilter_TypeConverter_Paramschecks the rendered parameters, including several entries for one key and an empty filter.TestClusterFilterList_Load_TypeConverterchecks the classic and YAML output of a ClusterFilter.make test,make lint,make verifyandmake generate manifests docs-updatepass locally, with no diff left over.fluent/fluent-bit:5.2.0with adummyinput, and the converted fields showed up with the right types, e.g."code_int":404,"ratio_float":0.5,"n_str":"42".typeConverter, an empty list, an unsupportedtype, an empty key and a key with a space are all rejected with the messages above.