Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUG] Parsing exception while storing negative epoch millis with certain length. #16979

Open
kbharathy opened this issue Jan 8, 2025 · 1 comment
Labels
bug Something isn't working Indexing Indexing, Bulk Indexing and anything related to indexing untriaged

Comments

@kbharathy
Copy link

Describe the bug

While storing negative epoch millis with certain length, getting parsing execption

These negative values(-561600000, -5616000000) are parsed incorrectly with default format "strict_date_optional_time||epoch_millis".

But, when format changed to "epoch_millis||strict_date_optional_time", then these values are parsed correctly.

other closer ranges(-56160000 or -56160000000), are parsed correctly with default format.

Related component

Indexing

To Reproduce

  1. Create index with date field and format as default.
curl -X PUT "http://localhost:9200/testindex" -H 'Content-Type: application/json' -d'
{
  "mappings" : {
    "properties" :  {
      "event_timestamp" : {
        "type" : "date"
      }
    }
  }
}'
  1. Insert a document with date value as "-561600000"( With 9 digits.)
curl -X POST "localhost:9200/testindex/_doc/?pretty" -H 'Content-Type: application/json' -d'{"event_timestamp": "-561600000"}'
  1. Response contains error message as,
{
  "error": {
    "root_cause": [
      {
        "type": "mapper_parsing_exception",
        "reason": "failed to parse field [event_timestamp] of type [date] in document with id 'XkCoRpQBBkWe45k1QzUB'. Preview of field's value: '-561600000'"
      }
    ],
    "type": "mapper_parsing_exception",
    "reason": "failed to parse field [event_timestamp] of type [date] in document with id 'XkCoRpQBBkWe45k1QzUB'. Preview of field's value: '-561600000'",
    "caused_by": {
      "type": "arithmetic_exception",
      "reason": "long overflow"
    }
  },
  "status": 400
}
  1. Insert a document with date value as "-5616000000"( With 10 digits.)
curl -X POST "localhost:9200/testindex/_doc/?pretty" -H 'Content-Type: application/json' -d'{"event_timestamp": "-5616000000"}' 
  1. Response contains error message as,
{
  "error": {
    "root_cause": [
      {
        "type": "mapper_parsing_exception",
        "reason": "failed to parse field [event_timestamp] of type [date] in document with id 'X0CoRpQBBkWe45k1sjXW'. Preview of field's value: '-5616000000'"
      }
    ],
    "type": "mapper_parsing_exception",
    "reason": "failed to parse field [event_timestamp] of type [date] in document with id 'X0CoRpQBBkWe45k1sjXW'. Preview of field's value: '-5616000000'",
    "caused_by": {
      "type": "date_time_exception",
      "reason": "Invalid value for Year (valid values -999999999 - 999999999): -5616000000"
    }
  },
  "status": 400
}
  1. With format as "", then parsed correctly
curl -X PUT "http://localhost:9200/testindex" -H 'Content-Type: application/json' -d'
{
  "mappings" : {
    "properties" :  {
      "event_timestamp" : {
        "type" : "date",
        "format" : "epoch_millis||strict_date_optional_time"
      }
    }
  }
}'

curl -X POST "localhost:9200/testindex/_doc/?pretty" -H 'Content-Type: application/json' -d'{"event_timestamp": "-561600000"}'

Expected behavior

With default format, negative epoch values should be parsed correctly.

Additional Details

Plugins
Please list all plugins currently enabled.

Screenshots
If applicable, add screenshots to help explain your problem.

Host/Environment (please complete the following information):

  • OS: [e.g. iOS]
  • Version: OpenSearch 1.3 onwards

Additional context
Add any other context about the problem here.

OpenSearch and ElasticSearch code base of handling negative values is different. In OpenSearch, -561600000 is parsed as a Year but in Elasticsearch parsed correctly

ES code: https://github.com/elastic/elasticsearch/pull/73034/files#diff-27cb6b4759805385f2b88731b2071c644258c9842be25536d2c7b31314f1348d
OpensSearch code: https://github.com/opensearch-project/OpenSearch/blob/main/server/src/main/java/org/opensearch/common/time/DateFormatters.java#L93

@kbharathy kbharathy added bug Something isn't working untriaged labels Jan 8, 2025
@github-actions github-actions bot added the Indexing Indexing, Bulk Indexing and anything related to indexing label Jan 8, 2025
@kbharathy
Copy link
Author

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working Indexing Indexing, Bulk Indexing and anything related to indexing untriaged
Projects
None yet
Development

No branches or pull requests

1 participant