Skip to content

Commit

Permalink
perf(tm2/pkg/amino): reduce RAM heavy-handedness by *bytes.Buffer poo…
Browse files Browse the repository at this point in the history
…led reuse

This change comes from an analysis of a bunch of RAM and CPU profiles
and noticing that realm storage needs to invoke amino.MustMarshalAny
but that in the profile for TestStdlibs, it was consuming 1.28GB.
```shell
ROUTINE ======================== github.com/gnolang/gno/tm2/pkg/amino.MustMarshalAny in /Users/emmanuelodeke/go/src/github.com/gnolang/gno/tm2/pkg/amino/amino.go
         0     1.28GB (flat, cum)  0.61% of Total
         .          .     80:func MustMarshalAny(o interface{}) []byte {
         .     1.28GB     81:	return gcdc.MustMarshalAny(o)
         .          .     82:}
         .          .     83:
         .          .     84:func MarshalAnySized(o interface{}) ([]byte, error) {
         .          .     85:	return gcdc.MarshalAnySized(o)
         .          .     86:}
```

but after this change, we see more than 560MB shaved off

```shell
ROUTINE ======================== github.com/gnolang/gno/tm2/pkg/amino.MustMarshalAny in /Users/emmanuelodeke/go/src/github.com/gnolang/gno/tm2/pkg/amino/amino.go
         0   560.95MB (flat, cum)  0.26% of Total
         .          .     80:func MustMarshalAny(o interface{}) []byte {
         .   560.95MB     81:	return gcdc.MustMarshalAny(o)
         .          .     82:}
         .          .     83:
         .          .     84:func MarshalAnySized(o interface{}) ([]byte, error) {
         .          .     85:	return gcdc.MarshalAnySized(o)
         .          .     86:}
```

and even more after the change on ensuring that tm2/pkg/amino benchmarks
could run we have quite good improvements, running out of RAM is much
worse than a couple of microseconds so we can tolerate an increase in
some CPU time benchmarks.

```shell
name                                   old time/op    new time/op    delta
Binary/EmptyStruct:encode-8              3.86µs ± 5%    3.92µs ± 5%     ~     (p=0.548 n=5+5)
Binary/EmptyStruct:decode-8              3.79µs ± 5%    3.79µs ± 6%     ~     (p=0.690 n=5+5)
Binary/PrimitivesStruct:encode-8         35.5µs ± 2%    36.5µs ± 5%     ~     (p=0.151 n=5+5)
Binary/PrimitivesStruct:decode-8         35.0µs ± 2%    38.6µs ±11%  +10.17%  (p=0.016 n=5+5)
Binary/ShortArraysStruct:encode-8        5.91µs ± 6%    6.36µs ± 8%   +7.61%  (p=0.032 n=5+5)
Binary/ShortArraysStruct:decode-8        6.07µs ±21%    6.39µs ± 8%     ~     (p=0.151 n=5+5)
Binary/ArraysStruct:encode-8             95.1µs ± 8%   100.6µs ± 7%     ~     (p=0.222 n=5+5)
Binary/ArraysStruct:decode-8             91.3µs ± 5%    98.5µs ±12%     ~     (p=0.222 n=5+5)
Binary/ArraysArraysStruct:encode-8        131µs ± 3%     132µs ± 6%     ~     (p=0.841 n=5+5)
Binary/ArraysArraysStruct:decode-8        136µs ± 9%     134µs ± 3%     ~     (p=0.548 n=5+5)
Binary/SlicesStruct:encode-8             85.4µs ± 1%    92.3µs ± 9%   +8.15%  (p=0.008 n=5+5)
Binary/SlicesStruct:decode-8             87.1µs ± 8%    94.8µs ± 7%     ~     (p=0.056 n=5+5)
Binary/SlicesSlicesStruct:encode-8        506µs ± 2%     545µs ± 9%     ~     (p=0.151 n=5+5)
Binary/SlicesSlicesStruct:decode-8        506µs ± 3%     523µs ± 3%     ~     (p=0.095 n=5+5)
Binary/PointersStruct:encode-8           56.8µs ± 4%    65.5µs ±20%  +15.43%  (p=0.016 n=5+5)
Binary/PointersStruct:decode-8           57.5µs ± 3%    55.9µs ± 3%     ~     (p=0.095 n=5+5)
Binary/PointerSlicesStruct:encode-8       162µs ± 4%     172µs ±21%     ~     (p=0.841 n=5+5)
Binary/PointerSlicesStruct:decode-8       163µs ± 5%     185µs ±13%     ~     (p=0.095 n=5+5)
Binary/ComplexSt:encode-8                 314µs ± 3%     354µs ±11%  +12.90%  (p=0.008 n=5+5)
Binary/ComplexSt:decode-8                 319µs ± 2%     338µs ± 4%   +5.87%  (p=0.008 n=5+5)
Binary/EmbeddedSt1:encode-8              39.8µs ± 7%    39.3µs ± 8%     ~     (p=1.000 n=5+5)
Binary/EmbeddedSt1:decode-8              37.0µs ± 4%    37.8µs ± 6%     ~     (p=0.690 n=5+5)
Binary/EmbeddedSt2:encode-8               316µs ± 7%     307µs ± 3%     ~     (p=0.222 n=5+5)
Binary/EmbeddedSt2:decode-8               316µs ± 3%     306µs ± 2%     ~     (p=0.095 n=5+5)
Binary/EmbeddedSt3:encode-8               217µs ± 7%     201µs ± 1%   -7.26%  (p=0.008 n=5+5)
Binary/EmbeddedSt3:decode-8               222µs ±10%     204µs ± 2%   -8.50%  (p=0.032 n=5+5)
Binary/EmbeddedSt4:encode-8               332µs ± 4%     325µs ± 3%     ~     (p=0.421 n=5+5)
Binary/EmbeddedSt4:decode-8               332µs ± 4%     324µs ± 5%     ~     (p=0.095 n=5+5)
Binary/EmbeddedSt5:encode-8               218µs ± 2%     212µs ± 3%     ~     (p=0.056 n=5+5)
Binary/EmbeddedSt5:decode-8               224µs ± 8%     209µs ± 1%   -6.85%  (p=0.008 n=5+5)
Binary/AminoMarshalerStruct1:encode-8    9.03µs ± 6%    8.97µs ±12%     ~     (p=0.841 n=5+5)
Binary/AminoMarshalerStruct1:decode-8    8.91µs ± 5%    8.81µs ± 4%     ~     (p=0.841 n=5+5)
Binary/AminoMarshalerStruct2:encode-8    13.2µs ±10%    12.2µs ± 2%   -7.26%  (p=0.008 n=5+5)
Binary/AminoMarshalerStruct2:decode-8    13.2µs ± 6%    12.5µs ± 5%     ~     (p=0.095 n=5+5)
Binary/AminoMarshalerStruct3:encode-8    7.17µs ± 3%    7.50µs ± 8%     ~     (p=0.548 n=5+5)
Binary/AminoMarshalerStruct3:decode-8    7.12µs ± 4%    7.84µs ±10%  +10.12%  (p=0.016 n=5+5)
Binary/AminoMarshalerInt4:encode-8       6.60µs ± 5%    6.96µs ±11%     ~     (p=0.421 n=5+5)
Binary/AminoMarshalerInt4:decode-8       6.79µs ±12%    7.04µs ±15%     ~     (p=0.690 n=5+5)
Binary/AminoMarshalerInt5:encode-8       6.64µs ± 4%    6.92µs ± 5%   +4.09%  (p=0.032 n=5+5)
Binary/AminoMarshalerInt5:decode-8       6.55µs ± 3%    7.76µs ±10%  +18.44%  (p=0.008 n=5+5)
Binary/AminoMarshalerStruct6:encode-8    11.7µs ± 5%    13.2µs ±10%  +13.09%  (p=0.008 n=5+5)
Binary/AminoMarshalerStruct6:decode-8    11.4µs ± 3%    11.6µs ± 2%     ~     (p=0.222 n=5+5)
Binary/AminoMarshalerStruct7:encode-8    9.86µs ± 1%   10.10µs ±19%     ~     (p=0.310 n=5+5)
Binary/AminoMarshalerStruct7:decode-8    9.55µs ± 3%    9.75µs ±10%     ~     (p=0.690 n=5+5)

name                                   old alloc/op   new alloc/op   delta
Binary/EmptyStruct:encode-8              1.50kB ± 0%    1.41kB ± 0%   -6.32%  (p=0.008 n=5+5)
Binary/EmptyStruct:decode-8              1.50kB ± 0%    1.41kB ± 0%   -6.32%  (p=0.008 n=5+5)
Binary/PrimitivesStruct:encode-8         10.4kB ± 0%     9.6kB ± 0%   -7.82%  (p=0.008 n=5+5)
Binary/PrimitivesStruct:decode-8         10.4kB ± 0%     9.6kB ± 0%   -7.82%  (p=0.000 n=4+5)
Binary/ShortArraysStruct:encode-8        2.11kB ± 0%    1.92kB ± 0%   -9.04%  (p=0.008 n=5+5)
Binary/ShortArraysStruct:decode-8        2.11kB ± 0%    1.92kB ± 0%   -9.04%  (p=0.008 n=5+5)
Binary/ArraysStruct:encode-8             25.9kB ± 0%    22.0kB ± 0%  -15.04%  (p=0.008 n=5+5)
Binary/ArraysStruct:decode-8             25.9kB ± 0%    22.0kB ± 0%  -15.04%  (p=0.008 n=5+5)
Binary/ArraysArraysStruct:encode-8       37.7kB ± 0%    25.3kB ± 0%  -33.07%  (p=0.008 n=5+5)
Binary/ArraysArraysStruct:decode-8       37.7kB ± 0%    25.3kB ± 0%  -33.07%  (p=0.008 n=5+5)
Binary/SlicesStruct:encode-8             28.2kB ± 0%    25.1kB ± 0%  -10.96%  (p=0.008 n=5+5)
Binary/SlicesStruct:decode-8             28.2kB ± 0%    25.1kB ± 0%  -10.97%  (p=0.008 n=5+5)
Binary/SlicesSlicesStruct:encode-8        183kB ± 0%     147kB ± 0%  -19.92%  (p=0.008 n=5+5)
Binary/SlicesSlicesStruct:decode-8        183kB ± 0%     147kB ± 0%  -19.92%  (p=0.008 n=5+5)
Binary/PointersStruct:encode-8           14.4kB ± 0%    13.6kB ± 0%   -5.64%  (p=0.008 n=5+5)
Binary/PointersStruct:decode-8           14.4kB ± 0%    13.6kB ± 0%   -5.64%  (p=0.008 n=5+5)
Binary/PointerSlicesStruct:encode-8      43.9kB ± 0%    40.2kB ± 0%   -8.49%  (p=0.008 n=5+5)
Binary/PointerSlicesStruct:decode-8      43.9kB ± 0%    40.2kB ± 0%   -8.49%  (p=0.008 n=5+5)
Binary/ComplexSt:encode-8                95.3kB ± 0%    78.2kB ± 0%  -17.97%  (p=0.008 n=5+5)
Binary/ComplexSt:decode-8                95.3kB ± 0%    78.2kB ± 0%  -17.97%  (p=0.008 n=5+5)
Binary/EmbeddedSt1:encode-8              11.3kB ± 0%    10.2kB ± 0%   -9.62%  (p=0.000 n=5+4)
Binary/EmbeddedSt1:decode-8              11.3kB ± 0%    10.2kB ± 0%   -9.61%  (p=0.000 n=5+4)
Binary/EmbeddedSt2:encode-8              95.5kB ± 0%    78.3kB ± 0%  -17.96%  (p=0.008 n=5+5)
Binary/EmbeddedSt2:decode-8              95.5kB ± 0%    78.4kB ± 0%  -17.94%  (p=0.008 n=5+5)
Binary/EmbeddedSt3:encode-8              68.3kB ± 0%    56.6kB ± 0%  -17.22%  (p=0.008 n=5+5)
Binary/EmbeddedSt3:decode-8              68.3kB ± 0%    56.6kB ± 0%  -17.21%  (p=0.008 n=5+5)
Binary/EmbeddedSt4:encode-8              97.2kB ± 0%    82.3kB ± 0%  -15.32%  (p=0.008 n=5+5)
Binary/EmbeddedSt4:decode-8              97.2kB ± 0%    82.3kB ± 0%  -15.31%  (p=0.008 n=5+5)
Binary/EmbeddedSt5:encode-8              65.9kB ± 0%    55.3kB ± 0%  -16.19%  (p=0.008 n=5+5)
Binary/EmbeddedSt5:decode-8              66.0kB ± 0%    55.3kB ± 0%  -16.18%  (p=0.008 n=5+5)
Binary/AminoMarshalerStruct1:encode-8    2.87kB ± 0%    2.66kB ± 0%   -7.23%  (p=0.008 n=5+5)
Binary/AminoMarshalerStruct1:decode-8    2.87kB ± 0%    2.66kB ± 0%   -7.23%  (p=0.008 n=5+5)
Binary/AminoMarshalerStruct2:encode-8    4.58kB ± 0%    3.62kB ± 0%  -20.95%  (p=0.008 n=5+5)
Binary/AminoMarshalerStruct2:decode-8    4.58kB ± 0%    3.62kB ± 0%  -20.95%  (p=0.008 n=5+5)
Binary/AminoMarshalerStruct3:encode-8    2.42kB ± 0%    2.31kB ± 0%   -4.62%  (p=0.008 n=5+5)
Binary/AminoMarshalerStruct3:decode-8    2.42kB ± 0%    2.31kB ± 0%   -4.62%  (p=0.008 n=5+5)
Binary/AminoMarshalerInt4:encode-8       2.38kB ± 0%    2.15kB ± 0%   -9.38%  (p=0.008 n=5+5)
Binary/AminoMarshalerInt4:decode-8       2.38kB ± 0%    2.15kB ± 0%   -9.38%  (p=0.008 n=5+5)
Binary/AminoMarshalerInt5:encode-8       2.36kB ± 0%    2.27kB ± 0%   -4.07%  (p=0.008 n=5+5)
Binary/AminoMarshalerInt5:decode-8       2.36kB ± 0%    2.27kB ± 0%   -4.07%  (p=0.008 n=5+5)
Binary/AminoMarshalerStruct6:encode-8    3.51kB ± 0%    3.19kB ± 0%   -9.05%  (p=0.008 n=5+5)
Binary/AminoMarshalerStruct6:decode-8    3.51kB ± 0%    3.19kB ± 0%   -9.05%  (p=0.008 n=5+5)
Binary/AminoMarshalerStruct7:encode-8    2.89kB ± 0%    2.67kB ± 0%   -7.72%  (p=0.008 n=5+5)
Binary/AminoMarshalerStruct7:decode-8    2.89kB ± 0%    2.67kB ± 0%   -7.72%  (p=0.008 n=5+5)

name                                   old allocs/op  new allocs/op  delta
Binary/EmptyStruct:encode-8                38.0 ± 0%      36.0 ± 0%   -5.26%  (p=0.008 n=5+5)
Binary/EmptyStruct:decode-8                38.0 ± 0%      36.0 ± 0%   -5.26%  (p=0.008 n=5+5)
Binary/PrimitivesStruct:encode-8            439 ± 0%       429 ± 0%   -2.28%  (p=0.008 n=5+5)
Binary/PrimitivesStruct:decode-8            439 ± 0%       429 ± 0%   -2.28%  (p=0.008 n=5+5)
Binary/ShortArraysStruct:encode-8          56.0 ± 0%      52.0 ± 0%   -7.14%  (p=0.008 n=5+5)
Binary/ShortArraysStruct:decode-8          56.0 ± 0%      52.0 ± 0%   -7.14%  (p=0.008 n=5+5)
Binary/ArraysStruct:encode-8                977 ± 0%       919 ± 0%   -5.94%  (p=0.008 n=5+5)
Binary/ArraysStruct:decode-8                977 ± 0%       919 ± 0%   -5.94%  (p=0.008 n=5+5)
Binary/ArraysArraysStruct:encode-8        1.28k ± 0%     1.08k ± 0%  -15.05%  (p=0.008 n=5+5)
Binary/ArraysArraysStruct:decode-8        1.28k ± 0%     1.08k ± 0%  -15.05%  (p=0.008 n=5+5)
Binary/SlicesStruct:encode-8              1.01k ± 0%     0.97k ± 0%   -3.77%  (p=0.008 n=5+5)
Binary/SlicesStruct:decode-8              1.01k ± 0%     0.97k ± 0%   -3.77%  (p=0.008 n=5+5)
Binary/SlicesSlicesStruct:encode-8        6.33k ± 0%     5.95k ± 0%   -5.90%  (p=0.008 n=5+5)
Binary/SlicesSlicesStruct:decode-8        6.33k ± 0%     5.95k ± 0%   -5.90%  (p=0.008 n=5+5)
Binary/PointersStruct:encode-8              637 ± 0%       627 ± 0%   -1.57%  (p=0.008 n=5+5)
Binary/PointersStruct:decode-8              637 ± 0%       627 ± 0%   -1.57%  (p=0.008 n=5+5)
Binary/PointerSlicesStruct:encode-8       1.62k ± 0%     1.56k ± 0%   -3.28%  (p=0.008 n=5+5)
Binary/PointerSlicesStruct:decode-8       1.62k ± 0%     1.56k ± 0%   -3.28%  (p=0.008 n=5+5)
Binary/ComplexSt:encode-8                 3.37k ± 0%     3.22k ± 0%   -4.62%  (p=0.008 n=5+5)
Binary/ComplexSt:decode-8                 3.37k ± 0%     3.22k ± 0%   -4.62%  (p=0.008 n=5+5)
Binary/EmbeddedSt1:encode-8                 453 ± 0%       440 ± 0%   -2.87%  (p=0.008 n=5+5)
Binary/EmbeddedSt1:decode-8                 453 ± 0%       440 ± 0%   -2.87%  (p=0.008 n=5+5)
Binary/EmbeddedSt2:encode-8               3.37k ± 0%     3.22k ± 0%   -4.62%  (p=0.008 n=5+5)
Binary/EmbeddedSt2:decode-8               3.37k ± 0%     3.22k ± 0%   -4.62%  (p=0.008 n=5+5)
Binary/EmbeddedSt3:encode-8               2.32k ± 0%     2.20k ± 0%   -5.38%  (p=0.008 n=5+5)
Binary/EmbeddedSt3:decode-8               2.32k ± 0%     2.20k ± 0%   -5.38%  (p=0.008 n=5+5)
Binary/EmbeddedSt4:encode-8               3.67k ± 0%     3.54k ± 0%   -3.73%  (p=0.008 n=5+5)
Binary/EmbeddedSt4:decode-8               3.67k ± 0%     3.54k ± 0%   -3.73%  (p=0.008 n=5+5)
Binary/EmbeddedSt5:encode-8               2.32k ± 0%     2.20k ± 0%   -5.00%  (p=0.008 n=5+5)
Binary/EmbeddedSt5:decode-8               2.32k ± 0%     2.20k ± 0%   -5.00%  (p=0.008 n=5+5)
Binary/AminoMarshalerStruct1:encode-8      97.0 ± 0%      94.0 ± 0%   -3.09%  (p=0.008 n=5+5)
Binary/AminoMarshalerStruct1:decode-8      97.0 ± 0%      94.0 ± 0%   -3.09%  (p=0.008 n=5+5)
Binary/AminoMarshalerStruct2:encode-8       149 ± 0%       133 ± 0%  -10.74%  (p=0.008 n=5+5)
Binary/AminoMarshalerStruct2:decode-8       149 ± 0%       133 ± 0%  -10.74%  (p=0.008 n=5+5)
Binary/AminoMarshalerStruct3:encode-8      77.0 ± 0%      76.0 ± 0%   -1.30%  (p=0.008 n=5+5)
Binary/AminoMarshalerStruct3:decode-8      77.0 ± 0%      76.0 ± 0%   -1.30%  (p=0.008 n=5+5)
Binary/AminoMarshalerInt4:encode-8         71.0 ± 0%      68.0 ± 0%   -4.23%  (p=0.008 n=5+5)
Binary/AminoMarshalerInt4:decode-8         71.0 ± 0%      68.0 ± 0%   -4.23%  (p=0.008 n=5+5)
Binary/AminoMarshalerInt5:encode-8         74.0 ± 0%      73.0 ± 0%   -1.35%  (p=0.008 n=5+5)
Binary/AminoMarshalerInt5:decode-8         74.0 ± 0%      73.0 ± 0%   -1.35%  (p=0.008 n=5+5)
Binary/AminoMarshalerStruct6:encode-8       122 ± 0%       117 ± 0%   -4.10%  (p=0.008 n=5+5)
Binary/AminoMarshalerStruct6:decode-8       122 ± 0%       117 ± 0%   -4.10%  (p=0.008 n=5+5)
Binary/AminoMarshalerStruct7:encode-8       101 ± 0%        98 ± 0%   -2.97%  (p=0.008 n=5+5)
Binary/AminoMarshalerStruct7:decode-8       101 ± 0%        98 ± 0%   -2.97%  (p=0.008 n=5+5)
```

Fixes gnolang#3488
  • Loading branch information
odeke-em committed Jan 12, 2025
1 parent 1b89166 commit ef6cc1e
Show file tree
Hide file tree
Showing 5 changed files with 108 additions and 24 deletions.
70 changes: 54 additions & 16 deletions tm2/pkg/amino/amino.go
Original file line number Diff line number Diff line change
Expand Up @@ -219,7 +219,11 @@ func (cdc *Codec) MarshalSized(o interface{}) ([]byte, error) {
cdc.doAutoseal()

// Write the bytes here.
buf := new(bytes.Buffer)
buf := poolBytesBuffer.Get().(*bytes.Buffer)
defer func() {
buf.Reset()
poolBytesBuffer.Put(buf)
}()

// Write the bz without length-prefixing.
bz, err := cdc.Marshal(o)
Expand All @@ -239,7 +243,7 @@ func (cdc *Codec) MarshalSized(o interface{}) ([]byte, error) {
return nil, err
}

return buf.Bytes(), nil
return copyBytes(buf.Bytes()), nil
}

// MarshalSizedWriter writes the bytes as would be returned from
Expand Down Expand Up @@ -271,7 +275,11 @@ func (cdc *Codec) MarshalAnySized(o interface{}) ([]byte, error) {
cdc.doAutoseal()

// Write the bytes here.
buf := new(bytes.Buffer)
buf := poolBytesBuffer.Get().(*bytes.Buffer)
defer func() {
buf.Reset()
poolBytesBuffer.Put(buf)
}()

// Write the bz without length-prefixing.
bz, err := cdc.MarshalAny(o)
Expand All @@ -291,7 +299,7 @@ func (cdc *Codec) MarshalAnySized(o interface{}) ([]byte, error) {
return nil, err
}

return buf.Bytes(), nil
return copyBytes(buf.Bytes()), nil
}

func (cdc *Codec) MustMarshalAnySized(o interface{}) []byte {
Expand Down Expand Up @@ -357,7 +365,12 @@ func (cdc *Codec) MarshalReflect(o interface{}) ([]byte, error) {

// Encode Amino:binary bytes.
var bz []byte
buf := new(bytes.Buffer)
buf := poolBytesBuffer.Get().(*bytes.Buffer)
defer func() {
buf.Reset()
poolBytesBuffer.Put(buf)
}()

rt := rv.Type()
info, err := cdc.getTypeInfoWLock(rt)
if err != nil {
Expand All @@ -377,7 +390,7 @@ func (cdc *Codec) MarshalReflect(o interface{}) ([]byte, error) {
if err = cdc.writeFieldIfNotEmpty(buf, 1, info, FieldOptions{}, FieldOptions{}, rv, writeEmpty); err != nil {
return nil, err
}
bz = buf.Bytes()
bz = copyBytes(buf.Bytes())
} else {
// The passed in BinFieldNum is only relevant for when the type is to
// be encoded unpacked (elements are Typ3_ByteLength). In that case,
Expand All @@ -387,7 +400,7 @@ func (cdc *Codec) MarshalReflect(o interface{}) ([]byte, error) {
if err != nil {
return nil, err
}
bz = buf.Bytes()
bz = copyBytes(buf.Bytes())
}
// If bz is empty, prefer nil.
if len(bz) == 0 {
Expand Down Expand Up @@ -443,16 +456,26 @@ func (cdc *Codec) MarshalAny(o interface{}) ([]byte, error) {
}

// Encode as interface.
buf := new(bytes.Buffer)
buf := poolBytesBuffer.Get().(*bytes.Buffer)
defer func() {
buf.Reset()
poolBytesBuffer.Put(buf)
}()
err = cdc.encodeReflectBinaryInterface(buf, iinfo, reflect.ValueOf(&ivar).Elem(), FieldOptions{}, true)
if err != nil {
return nil, err
}
bz := buf.Bytes()
bz := copyBytes(buf.Bytes())

return bz, nil
}

func copyBytes(bz []byte) []byte {
cp := make([]byte, len(bz))
copy(cp, bz)
return cp
}

// Panics if error.
func (cdc *Codec) MustMarshalAny(o interface{}) []byte {
bz, err := cdc.MarshalAny(o)
Expand Down Expand Up @@ -764,15 +787,20 @@ func (cdc *Codec) JSONMarshal(o interface{}) ([]byte, error) {
return []byte("null"), nil
}
rt := rv.Type()
w := new(bytes.Buffer)
w := poolBytesBuffer.Get().(*bytes.Buffer)
defer func() {
w.Reset()
poolBytesBuffer.Put(w)
}()
info, err := cdc.getTypeInfoWLock(rt)
if err != nil {
return nil, err
}
if err = cdc.encodeReflectJSON(w, info, rv, FieldOptions{}); err != nil {
return nil, err
}
return w.Bytes(), nil

return copyBytes(w.Bytes()), nil
}

func (cdc *Codec) MarshalJSONAny(o interface{}) ([]byte, error) {
Expand Down Expand Up @@ -802,12 +830,17 @@ func (cdc *Codec) MarshalJSONAny(o interface{}) ([]byte, error) {
}

// Encode as interface.
buf := new(bytes.Buffer)
buf := poolBytesBuffer.Get().(*bytes.Buffer)
defer func() {
buf.Reset()
poolBytesBuffer.Put(buf)
}()

err = cdc.encodeReflectJSONInterface(buf, iinfo, reflect.ValueOf(&ivar).Elem(), FieldOptions{})
if err != nil {
return nil, err
}
bz := buf.Bytes()
bz := copyBytes(buf.Bytes())

return bz, nil
}
Expand Down Expand Up @@ -863,12 +896,17 @@ func (cdc *Codec) MarshalJSONIndent(o interface{}, prefix, indent string) ([]byt
if err != nil {
return nil, err
}
var out bytes.Buffer
err = json.Indent(&out, bz, prefix, indent)
out := poolBytesBuffer.Get().(*bytes.Buffer)
defer func() {
out.Reset()
poolBytesBuffer.Put(out)
}()

err = json.Indent(out, bz, prefix, indent)
if err != nil {
return nil, err
}
return out.Bytes(), nil
return copyBytes(out.Bytes()), nil
}

// ----------------------------------------
Expand Down
41 changes: 36 additions & 5 deletions tm2/pkg/amino/binary_encode.go
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@ import (
"fmt"
"io"
"reflect"
"sync"
)

const beOptionByte = 0x01
Expand Down Expand Up @@ -209,6 +210,10 @@ func (cdc *Codec) encodeReflectBinary(w io.Writer, info *TypeInfo, rv reflect.Va
return err
}

var poolBytesBuffer = &sync.Pool{
New: func() any { return new(bytes.Buffer) },
}

func (cdc *Codec) encodeReflectBinaryInterface(w io.Writer, iinfo *TypeInfo, rv reflect.Value,
fopts FieldOptions, bare bool,
) (err error) {
Expand Down Expand Up @@ -250,7 +255,12 @@ func (cdc *Codec) encodeReflectBinaryInterface(w io.Writer, iinfo *TypeInfo, rv

// For Proto3 compatibility, encode interfaces as google.protobuf.Any
// Write field #1, TypeURL
buf := bytes.NewBuffer(nil)
buf := poolBytesBuffer.Get().(*bytes.Buffer)
defer func() {
buf.Reset()
poolBytesBuffer.Put(buf)
}()

{
fnum := uint32(1)
err = encodeFieldNumberAndTyp3(buf, fnum, Typ3ByteLength)
Expand All @@ -269,7 +279,12 @@ func (cdc *Codec) encodeReflectBinaryInterface(w io.Writer, iinfo *TypeInfo, rv
{
// google.protobuf.Any values must be a struct, or an unpacked list which
// is indistinguishable from a struct.
buf2 := bytes.NewBuffer(nil)
buf2 := poolBytesBuffer.Get().(*bytes.Buffer)
defer func() {
buf2.Reset()
poolBytesBuffer.Put(buf2)
}()

if !cinfo.IsStructOrUnpacked(fopts) {
writeEmpty := false
// Encode with an implicit struct, with a single field with number 1.
Expand Down Expand Up @@ -356,7 +371,11 @@ func (cdc *Codec) encodeReflectBinaryList(w io.Writer, info *TypeInfo, rv reflec

// Proto3 byte-length prefixing incurs alloc cost on the encoder.
// Here we incur it for unpacked form for ease of dev.
buf := bytes.NewBuffer(nil)
buf := poolBytesBuffer.Get().(*bytes.Buffer)
defer func() {
buf.Reset()
poolBytesBuffer.Put(buf)
}()

// If elem is not already a ByteLength type, write in packed form.
// This is a Proto wart due to Proto backwards compatibility issues.
Expand Down Expand Up @@ -431,20 +450,28 @@ func (cdc *Codec) encodeReflectBinaryList(w io.Writer, info *TypeInfo, rv reflec
// form) are represented as lists of implicit structs.
if writeImplicit {
// Write field key for Value field of implicit struct.
buf2 := new(bytes.Buffer)
buf2 := poolBytesBuffer.Get().(*bytes.Buffer)
buf2Done := func() {
buf2.Reset()
poolBytesBuffer.Put(buf2)
}

err = encodeFieldNumberAndTyp3(buf2, 1, Typ3ByteLength)
if err != nil {
buf2Done()
return
}
// Write field value of implicit struct to buf2.
efopts := fopts
efopts.BinFieldNum = 0 // dontcare
err = cdc.encodeReflectBinary(buf2, einfo, derv, efopts, false, 0)
if err != nil {
buf2Done()
return
}
// Write implicit struct to buf.
err = EncodeByteSlice(buf, buf2.Bytes())
buf2Done()
if err != nil {
return
}
Expand Down Expand Up @@ -497,7 +524,11 @@ func (cdc *Codec) encodeReflectBinaryStruct(w io.Writer, info *TypeInfo, rv refl

// Proto3 incurs a cost in writing non-root structs.
// Here we incur it for root structs as well for ease of dev.
buf := bytes.NewBuffer(nil)
buf := poolBytesBuffer.Get().(*bytes.Buffer)
defer func() {
buf.Reset()
poolBytesBuffer.Put(buf)
}()

for _, field := range info.Fields {
// Get type info for field.
Expand Down
7 changes: 6 additions & 1 deletion tm2/pkg/amino/codec.go
Original file line number Diff line number Diff line change
Expand Up @@ -113,7 +113,12 @@ func (info *TypeInfo) String() string {
// before it's fully populated.
return "<new TypeInfo>"
}
buf := new(bytes.Buffer)
buf := poolBytesBuffer.Get().(*bytes.Buffer)
defer func() {
buf.Reset()
poolBytesBuffer.Put(buf)
}()

buf.Write([]byte("TypeInfo{"))
buf.Write([]byte(fmt.Sprintf("Type:%v,", info.Type)))
if info.ConcreteInfo.Registered {
Expand Down
7 changes: 6 additions & 1 deletion tm2/pkg/amino/json_encode.go
Original file line number Diff line number Diff line change
Expand Up @@ -156,7 +156,12 @@ func (cdc *Codec) encodeReflectJSONInterface(w io.Writer, iinfo *TypeInfo, rv re
}

// Write Value to buffer
buf := new(bytes.Buffer)
buf := poolBytesBuffer.Get().(*bytes.Buffer)
defer func() {
buf.Reset()
poolBytesBuffer.Put(buf)
}()

cdc.encodeReflectJSON(buf, cinfo, crv, fopts)
value := buf.Bytes()
if len(value) == 0 {
Expand Down
7 changes: 6 additions & 1 deletion tm2/pkg/amino/wellknown.go
Original file line number Diff line number Diff line change
Expand Up @@ -342,7 +342,12 @@ func encodeReflectBinaryWellKnown(w io.Writer, info *TypeInfo, rv reflect.Value,
}
// Maybe recurse with length-prefixing.
if !bare {
buf := bytes.NewBuffer(nil)
buf := poolBytesBuffer.Get().(*bytes.Buffer)
defer func() {
buf.Reset()
poolBytesBuffer.Put(buf)
}()

ok, err = encodeReflectBinaryWellKnown(buf, info, rv, fopts, true)
if err != nil {
return false, err
Expand Down

0 comments on commit ef6cc1e

Please sign in to comment.