Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Feature Request] Improve indexing performance by using a single Lucene field #17022

Open
bugmakerrrrrr opened this issue Jan 14, 2025 · 0 comments
Labels
enhancement Enhancement or improvement to existing feature or request Indexing:Performance untriaged

Comments

@bugmakerrrrrr
Copy link
Contributor

Is your feature request related to a problem? Please describe

Currently, OS usually create multiply Lucene fields for a single OS field during indexing, for example, when parsing number field, OS creates three Lucene fields.

When Lucene processes one document, it needs two passes for each fields. The first pass is to verify that doc schema matches the index schema, and the second pass is to index each field. Obviously the more fields there are, the more passes need to be processed.

Describe the solution you'd like

For each OS field, only create a single Lucene field to reduce the number of passes to be processed. Take a Long field as a example, we can create a Lucene field as follows:

final FieldType type = new FieldType();
type.setDimensions(1, Long.BYTES);
type.setDocValuesType(DocValuesType.SORTED_NUMERIC);
type.setStored(true);

new CustomField(name, value, type)

Related component

Indexing:Performance

Describe alternatives you've considered

No response

Additional context

No response

@bugmakerrrrrr bugmakerrrrrr added enhancement Enhancement or improvement to existing feature or request untriaged labels Jan 14, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement Enhancement or improvement to existing feature or request Indexing:Performance untriaged
Projects
None yet
Development

No branches or pull requests

1 participant