Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

remove json column v4 format writers #17625

Open
wants to merge 3 commits into
base: master
Choose a base branch
from

Conversation

clintropolis
Copy link
Member

Description

Removes the ability to write JSON columns in the older version '4' format, leaving only the latest current version, '5', as the only option. (Internally it isn't really v5, rather v0 of the 'auto' format, but it is branded as v5 for the sake of continuity).

I have left the structures to control which version is written added in #14955, in the event we need to introduce any newer versions in the future, we just only allow v5 right now.

Release note

JSON columns can no longer be written in the older V4 format, the current latest version, V5, is the only version which can be written to new segments. This means both the formatVersion of a JSON column spec on JSON based ingestion specs, e.g.

          {
            "type": "json",
            "name": "someColumn",
            "formatVersion": 5
          }

and druid.indexing.formats.nestedColumnFormatVersion in runtime.properties only allow 5.

All older version of JSON columns can still be read and queried just fine, this change just prevents writing new segments in the older format.

@github-actions github-actions bot added Area - Batch Ingestion Area - MSQ For multi stage queries - https://github.com/apache/druid/issues/12262 labels Jan 14, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Area - Batch Ingestion Area - MSQ For multi stage queries - https://github.com/apache/druid/issues/12262 Area - Segment Format and Ser/De
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant