You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Describe the bug, including details regarding any error messages, version, and platform.
The code[1] would cleanup the min-max stats in Parquet. For ByteArray, we may "Merge" multiple stats when reading from file. Things would be tricky in the code below when min = ""
Code in [2] is empty, so PlainDecode will not be called, and has_min_max_ is true. But ByteArray keeps default constructor, which leaves ptr == nullptr [3]
When call TypedStatistics::Merge, this will call Cleanup [1], and finally, the min-max statistics would leave unchanged.
So, when min = "" being merged, the min-max will keep the old statistics.
Describe the bug, including details regarding any error messages, version, and platform.
The code[1] would cleanup the min-max stats in Parquet. For ByteArray, we may "Merge" multiple stats when reading from file. Things would be tricky in the code below when
min = ""
PlainDecode
will not be called, andhas_min_max_
istrue
. ButByteArray
keeps default constructor, which leavesptr == nullptr
[3]TypedStatistics::Merge
, this will call Cleanup [1], and finally, the min-max statistics would leave unchanged.So, when
min = ""
being merged, the min-max will keep the old statistics.[1]
arrow/cpp/src/parquet/statistics.cc
Line 408 in ea47172
[2]
arrow/cpp/src/parquet/statistics.cc
Line 609 in ea47172
[3]
arrow/cpp/src/parquet/types.h
Line 587 in ea47172
Component(s)
C++, Parquet
The text was updated successfully, but these errors were encountered: