LJpeg: support square MCU layout (Sony LJpeg, -32%) #698

LebedevRI · 2024-03-26T02:05:46Z

This is essentially rebased #483, and partially reverts #484,
now that i've figured out how we can deduce the MCU size:
e9e3f1b + 63f7170 !

This, hopefully, partially solves the roadblock towards different predictor support.

Comparing /home/lebedevri/rawspeed/build-Clang18-release/src/utilities/rsbench/rsbench-old to /home/lebedevri/rawspeed/build-Clang18-release/src/utilities/rsbench/rsbench
Benchmark                                                                                                                               Time             CPU      Time Old      Time New       CPU Old       CPU New
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
./Adobe DNG Converter/Canon EOS 5D Mark III/5G4A9394-compressed-lossless.DNG/threads:32/process_time/real_time_pvalue                 0.0081          0.0066      U Test, Repetitions: 27 vs 27
./Adobe DNG Converter/Canon EOS 5D Mark III/5G4A9394-compressed-lossless.DNG/threads:32/process_time/real_time_mean                  -0.0160         -0.0162            11            11           364           359
./Adobe DNG Converter/Canon EOS 5D Mark III/5G4A9394-compressed-lossless.DNG/threads:32/process_time/real_time_median                -0.0181         -0.0156            11            11           361           356
./Adobe DNG Converter/Canon EOS 5D Mark III/5G4A9394-compressed-lossless.DNG/threads:32/process_time/real_time_stddev                +0.1098         +0.1135             0             0             8             9
./Adobe DNG Converter/Canon EOS 5D Mark III/5G4A9394-compressed-lossless.DNG/threads:32/process_time/real_time_cv                    +0.1279         +0.1318             0             0             0             0
./Adobe DNG Converter/Canon EOS 5D Mark III/5G4A9395-compressed-lossless.DNG/threads:32/process_time/real_time_pvalue                 0.3683          0.4781      U Test, Repetitions: 27 vs 27
./Adobe DNG Converter/Canon EOS 5D Mark III/5G4A9395-compressed-lossless.DNG/threads:32/process_time/real_time_mean                  +0.0089         +0.0082            15            15           464           468
./Adobe DNG Converter/Canon EOS 5D Mark III/5G4A9395-compressed-lossless.DNG/threads:32/process_time/real_time_median                +0.0145         +0.0152            15            15           461           468
./Adobe DNG Converter/Canon EOS 5D Mark III/5G4A9395-compressed-lossless.DNG/threads:32/process_time/real_time_stddev                +0.2341         +0.2787             0             0            11            14
./Adobe DNG Converter/Canon EOS 5D Mark III/5G4A9395-compressed-lossless.DNG/threads:32/process_time/real_time_cv                    +0.2232         +0.2683             0             0             0             0
./Adobe DNG Converter/Canon EOS 5D Mark IV/B13A0729.dng/threads:32/process_time/real_time_pvalue                                      0.2913          0.2326      U Test, Repetitions: 27 vs 27
./Adobe DNG Converter/Canon EOS 5D Mark IV/B13A0729.dng/threads:32/process_time/real_time_mean                                       -0.0098         -0.0102            30            29           939           930
./Adobe DNG Converter/Canon EOS 5D Mark IV/B13A0729.dng/threads:32/process_time/real_time_median                                     -0.0193         -0.0181            30            29           936           920
./Adobe DNG Converter/Canon EOS 5D Mark IV/B13A0729.dng/threads:32/process_time/real_time_stddev                                     -0.1583         -0.1647             1             1            28            23
./Adobe DNG Converter/Canon EOS 5D Mark IV/B13A0729.dng/threads:32/process_time/real_time_cv                                         -0.1500         -0.1561             0             0             0             0
./Fujifilm/X100S/fujifilm-x100s-daylight-DSCF9505.dng/threads:32/process_time/real_time_pvalue                                        0.0001          0.0000      U Test, Repetitions: 27 vs 27
./Fujifilm/X100S/fujifilm-x100s-daylight-DSCF9505.dng/threads:32/process_time/real_time_mean                                         -0.0312         -0.0321             4             4           129           125
./Fujifilm/X100S/fujifilm-x100s-daylight-DSCF9505.dng/threads:32/process_time/real_time_median                                       -0.0479         -0.0480             4             4           128           122
./Fujifilm/X100S/fujifilm-x100s-daylight-DSCF9505.dng/threads:32/process_time/real_time_stddev                                       +0.8491         +0.7719             0             0             4             8
./Fujifilm/X100S/fujifilm-x100s-daylight-DSCF9505.dng/threads:32/process_time/real_time_cv                                           +0.9087         +0.8306             0             0             0             0
./Samsung/Galaxy S21 Ultra/20230712_115041.dng/threads:32/process_time/real_time_pvalue                                               0.0000          0.0000      U Test, Repetitions: 27 vs 27
./Samsung/Galaxy S21 Ultra/20230712_115041.dng/threads:32/process_time/real_time_mean                                                +0.0128         +0.0128           149           151           149           151
./Samsung/Galaxy S21 Ultra/20230712_115041.dng/threads:32/process_time/real_time_median                                              +0.0128         +0.0128           149           151           149           151
./Samsung/Galaxy S21 Ultra/20230712_115041.dng/threads:32/process_time/real_time_stddev                                              -0.0387         -0.0437             0             0             0             0
./Samsung/Galaxy S21 Ultra/20230712_115041.dng/threads:32/process_time/real_time_cv                                                  -0.0509         -0.0558             0             0             0             0
./Samsung/Galaxy S23 Ultra/20231214_130645.dng/threads:32/process_time/real_time_pvalue                                               0.0066          0.7687      U Test, Repetitions: 27 vs 27
./Samsung/Galaxy S23 Ultra/20231214_130645.dng/threads:32/process_time/real_time_mean                                                +0.0055         -0.0000           580           584          7001          7001
./Samsung/Galaxy S23 Ultra/20231214_130645.dng/threads:32/process_time/real_time_median                                              +0.0076         -0.0107           579           583          7224          7147
./Samsung/Galaxy S23 Ultra/20231214_130645.dng/threads:32/process_time/real_time_stddev                                              +0.5071         -0.0321             4             6          1181          1143
./Samsung/Galaxy S23 Ultra/20231214_130645.dng/threads:32/process_time/real_time_cv                                                  +0.4988         -0.0320             0             0             0             0
./Sony/ILCE-7RM5/7RM5-S35-LosslessCompressedMedium.ARW/threads:32/process_time/real_time_pvalue                                       0.0000          0.0000      U Test, Repetitions: 27 vs 27
./Sony/ILCE-7RM5/7RM5-S35-LosslessCompressedMedium.ARW/threads:32/process_time/real_time_mean                                        -0.3269         -0.3256            29            19           910           614
./Sony/ILCE-7RM5/7RM5-S35-LosslessCompressedMedium.ARW/threads:32/process_time/real_time_median                                      -0.3205         -0.3224            29            19           908           615
./Sony/ILCE-7RM5/7RM5-S35-LosslessCompressedMedium.ARW/threads:32/process_time/real_time_stddev                                      -0.0947         -0.0594             1             1            21            20
./Sony/ILCE-7RM5/7RM5-S35-LosslessCompressedMedium.ARW/threads:32/process_time/real_time_cv                                          +0.3449         +0.3946             0             0             0             0
OVERALL_GEOMEAN                                                                                                                      -0.0593         -0.0594             0             0             1             1

…frame size

…tInterval/`

codecov · 2024-03-26T03:14:37Z

Codecov Report

Attention: Patch coverage is 60.00000% with 50 lines in your changes are missing coverage. Please review.

Project coverage is 60.84%. Comparing base (8297c38) to head (7d97477).

Files	Patch %	Lines
...rc/librawspeed/decompressors/LJpegDecompressor.cpp	62.13%	39 Missing ⚠️
src/librawspeed/decompressors/LJpegDecoder.cpp	60.00%	4 Missing and 2 partials ⚠️
...zz/librawspeed/decompressors/LJpegDecompressor.cpp	0.00%	5 Missing ⚠️

Additional details and impacted files

@@             Coverage Diff             @@
##           develop     #698      +/-   ##
===========================================
- Coverage    60.87%   60.84%   -0.03%     
===========================================
  Files          272      272              
  Lines        16464    16490      +26     
  Branches      2084     2090       +6     
===========================================
+ Hits         10022    10033      +11     
- Misses        6314     6328      +14     
- Partials       128      129       +1

Flag	Coverage Δ
benchmarks	`11.78% <0.80%> (-0.02%)`	⬇️
integration	`44.95% <68.18%> (-0.01%)`	⬇️
linux	`57.05% <63.02%> (-0.06%)`	⬇️
macOS	`25.03% <0.00%> (-0.04%)`	⬇️
rpu_u	`44.95% <68.18%> (-0.01%)`	⬇️
unittests	`21.46% <0.80%> (-0.04%)`	⬇️
windows	`∅ <ø> (∅)`

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

kmilos · 2024-03-26T09:51:58Z

This, hopefully, partially solves the roadblock towards different predictor support.

Yay! 🚀

LebedevRI added 25 commits March 26, 2024 02:41

LJpegDecoder: check that frame/tile areas match, not dimensions

3ee4910

LJpegDecoder: Maximal output tile size should be a multiple of LJpeg …

63f7170

…frame size

LJpegDecoder: compute MCU layout from tile size and ljpeg frame size

97c82e3

LJpegDecoder: extract jpegFrameDim helper variable

01470cc

LJpegDecompressor: extract cps variable from frame

bce1bee

LJpegDecoder: plumb MCUSize into LJpegDecompressor

00ef6d6

LJpegDecompressor: rewrite checks in ctor for MCU size

debd42d

LJpegDecompressor: we do not expect to support row-partial MCU's

dfdbb30

LJpegDecompressor: template decodeN() on MCUSize

087d5c6

LJpegDecompressor: template decodeRowN() on MCUSize

c64abc5

LJpegDecompressor: `s/numRowsPerRestartInterval/numLJpegRowsPerRestar…

f6214c3

…tInterval/`

LJpegDecompressor: s/rowOfRestartInterval/ljpegRowOfRestartInterval/

627ae6b

LJpegDecompressor: s/row/ljpegRow/

08270f7

LJpegDecompressor: correctly calculate numRestartIntervals

a5392fc

LJpegDecompressor: correctly calculate output row

16599fb

LJpegDecompressor: s/fullBlocks/numFullMCUs/

a87d59c

LJpegDecompressor::decodeRowN(): iterate over mcu index, not col

43dea6c

LJpegDecompressor: s/block/mcu/

acac68f

LJpegDecompressor: take outStripe, not outRow

af901e7

LJpegDecompressor: actually produce a stripe

ad03fff

LJpegDecompressor: rewrite predictor reloading

b6da886

LJpegDecompressor: extract outTile in the non-edge-pixel code

dfff2ae

LJpegDecompressor: drop incorrect invariant

9f89219

LJpegDecompressor: support {2,2} MCU's

e2ff718

ArwDecoder: LJpeg decompressor supports {2,2} MCU's, drop workaround

7d97477

LebedevRI force-pushed the ljpeg-mcu branch from b2498f6 to 7d97477 Compare March 26, 2024 02:41

LJpegDecompressor: don't inline decodeN() into decode()

53dbe15

LebedevRI changed the title ~~LJpeg: support square MCU layout (as used by Sony LJpeg)~~ LJpeg: support square MCU layout (Sony LJpeg, -32%) Mar 26, 2024

LebedevRI merged commit 1ef967a into darktable-org:develop Mar 26, 2024
1 check passed

LebedevRI deleted the ljpeg-mcu branch March 26, 2024 03:38

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

LJpeg: support square MCU layout (Sony LJpeg, -32%) #698

LJpeg: support square MCU layout (Sony LJpeg, -32%) #698

LebedevRI commented Mar 26, 2024 •

edited

Loading

codecov bot commented Mar 26, 2024 •

edited

Loading

kmilos commented Mar 26, 2024

LJpeg: support square MCU layout (Sony LJpeg, -32%) #698

LJpeg: support square MCU layout (Sony LJpeg, -32%) #698

Conversation

LebedevRI commented Mar 26, 2024 • edited Loading

codecov bot commented Mar 26, 2024 • edited Loading

Codecov Report

kmilos commented Mar 26, 2024

LebedevRI commented Mar 26, 2024 •

edited

Loading

codecov bot commented Mar 26, 2024 •

edited

Loading