tm2/pkg/amino, gnovm/pkg/golang: amino.*marshal functions are used in saving realms and could be optimized for reuse of *bytes.Buffer instead of allocations and discarding

[odeke-em]

The gnovm needs to be as lightweight as possible on RAM usage and also fast. While analyzing profiles to figure out what's slow and RAM heavy I noticed that:

• github.com/gnolang/gno/tm2/pkg/amino.EncodeString
• github.com/gnolang/gno/gnovm/pkg/gnolang.ObjectID.MarshalAmino
• github.com/gnolang/gno/gnovm/pkg/gnolang.ValueHash.MarshalAmino
• github.com/gnolang/gno/tm2/pkg/amino.(*Codec).encodeReflectBinaryStruct
• github.com/gnolang/gno/tm2/pkg/amino.typeToTyp3
• github.com/gnolang/gno/tm2/pkg/amino.(*Codec).encodeReflectBinaryInterface
• github.com/gnolang/gno/tm2/pkg/amino.EncodeUvarint

have routes that involves creating bytes.Buffer and in all the cases I could find, those paths create buffers and then discard them.

TL;DR:

With my suggestion, TestStdlib's RAM profile massively reduces RAM by more than 570MB

Before

ROUTINE ======================== github.com/gnolang/gno/tm2/pkg/amino.MustMarshalAny in /Users/emmanuelodeke/go/src/github.com/gnolang/gno/tm2/pkg/amino/amino.go
         0     1.28GB (flat, cum)  0.61% of Total
         .          .     80:func MustMarshalAny(o interface{}) []byte {
         .     1.28GB     81:	return gcdc.MustMarshalAny(o)
         .          .     82:}
         .          .     83:
         .          .     84:func MarshalAnySized(o interface{}) ([]byte, error) {
         .          .     85:	return gcdc.MarshalAnySized(o)
         .          .     86:}

After

ROUTINE ======================== github.com/gnolang/gno/tm2/pkg/amino.MustMarshalAny in /Users/emmanuelodeke/go/src/github.com/gnolang/gno/tm2/pkg/amino/amino.go
         0   560.95MB (flat, cum)  0.26% of Total
         .          .     80:func MustMarshalAny(o interface{}) []byte {
         .   560.95MB     81:	return gcdc.MustMarshalAny(o)
         .          .     82:}
         .          .     83:
         .          .     84:func MarshalAnySized(o interface{}) ([]byte, error) {
         .          .     85:	return gcdc.MarshalAnySized(o)
         .          .     86:}

Focuses

After

   focus=MarshalAny
Showing nodes accounting for 1303.02MB, 0.6% of 217023.96MB total
Dropped 13 nodes (cum <= 1085.12MB)
----------------------------------------------------------+-------------
      flat  flat%   sum%        cum   cum%   calls calls% + context 	 	 
----------------------------------------------------------+-------------
                                          539.49MB   100% |   bytes.(*Buffer).grow
  539.49MB  0.25%  0.25%   539.49MB  0.25%                | bytes.growSlice
----------------------------------------------------------+-------------
                                          706.50MB   100% |   bytes.(*Buffer).Write
  167.01MB 0.077%  0.33%   706.50MB  0.33%                | bytes.(*Buffer).grow
                                          539.49MB 76.36% |   bytes.growSlice
----------------------------------------------------------+-------------
                                              93MB 58.68% |   github.com/gnolang/gno/tm2/pkg/amino.(*Codec).encodeReflectBinaryInterface (inline)
                                           56.50MB 35.65% |   github.com/gnolang/gno/tm2/pkg/amino.(*Codec).encodeReflectBinaryStruct (inline)
                                               9MB  5.68% |   github.com/gnolang/gno/tm2/pkg/amino.(*Codec).encodeReflectBinaryList (inline)
  158.51MB 0.073%   0.4%   158.51MB 0.073%                | bytes.NewBuffer

Before

   focus=MarshalAny
Showing nodes accounting for 560.45MB, 0.26% of 215641.47MB total
Dropped 4 nodes (cum <= 1078.21MB)
----------------------------------------------------------+-------------
      flat  flat%   sum%        cum   cum%   calls calls% + context 	 	 
----------------------------------------------------------+-------------
                                              43MB 53.75% |   github.com/gnolang/gno/tm2/pkg/amino.(*Codec).writeFieldIfNotEmpty
                                           29.50MB 36.88% |   github.com/gnolang/gno/tm2/pkg/amino.(*Codec).encodeReflectBinaryInterface
                                            7.50MB  9.38% |   github.com/gnolang/gno/tm2/pkg/amino.(*Codec).encodeReflectBinaryList
      80MB 0.037% 0.037%       80MB 0.037%                | github.com/gnolang/gno/tm2/pkg/amino.encodeFieldNumberAndTyp3
----------------------------------------------------------+-------------
                                              70MB   100% |   github.com/gnolang/gno/tm2/pkg/amino.EncodeByteSlice
      70MB 0.032%  0.07%       70MB 0.032%                | github.com/gnolang/gno/tm2/pkg/amino.EncodeUvarint
----------------------------------------------------------+-------------
                                          560.95MB   100% |   github.com/gnolang/gno/tm2/pkg/amino.(*Codec).MustMarshalAny
   69.27MB 0.032%   0.1%   560.95MB  0.26%                | github.com/gnolang/gno/tm2/pkg/amino.(*Codec).MarshalAny
                                          471.67MB 84.09% |   github.com/gnolang/gno/tm2/pkg/amino.(*Codec).encodeReflectBinaryInterface
                                              20MB  3.57% |   reflect.Value.Interface (inline)

Remedy

When I tweaked the benchmarks to get them to run inside amino and applied this diff we get wins in every dimension!

diff --git a/tm2/pkg/amino/amino.go b/tm2/pkg/amino/amino.go
index 262f5d9a..bcd52e97 100644
--- a/tm2/pkg/amino/amino.go
+++ b/tm2/pkg/amino/amino.go
@@ -219,7 +219,11 @@ func (cdc *Codec) MarshalSized(o interface{}) ([]byte, error) {
 	cdc.doAutoseal()
 
 	// Write the bytes here.
-	buf := new(bytes.Buffer)
+	buf := poolBytesBuffer.Get().(*bytes.Buffer)
+	defer func() {
+		buf.Reset()
+		poolBytesBuffer.Put(buf)
+	}()
 
 	// Write the bz without length-prefixing.
 	bz, err := cdc.Marshal(o)
@@ -239,7 +243,7 @@ func (cdc *Codec) MarshalSized(o interface{}) ([]byte, error) {
 		return nil, err
 	}
 
-	return buf.Bytes(), nil
+	return copyBytes(buf.Bytes()), nil
 }
 
 // MarshalSizedWriter writes the bytes as would be returned from
@@ -271,7 +275,11 @@ func (cdc *Codec) MarshalAnySized(o interface{}) ([]byte, error) {
 	cdc.doAutoseal()
 
 	// Write the bytes here.
-	buf := new(bytes.Buffer)
+	buf := poolBytesBuffer.Get().(*bytes.Buffer)
+	defer func() {
+		buf.Reset()
+		poolBytesBuffer.Put(buf)
+	}()
 
 	// Write the bz without length-prefixing.
 	bz, err := cdc.MarshalAny(o)
@@ -291,7 +299,7 @@ func (cdc *Codec) MarshalAnySized(o interface{}) ([]byte, error) {
 		return nil, err
 	}
 
-	return buf.Bytes(), nil
+	return copyBytes(buf.Bytes()), nil
 }
 
 func (cdc *Codec) MustMarshalAnySized(o interface{}) []byte {
@@ -357,7 +365,12 @@ func (cdc *Codec) MarshalReflect(o interface{}) ([]byte, error) {
 
 	// Encode Amino:binary bytes.
 	var bz []byte
-	buf := new(bytes.Buffer)
+	buf := poolBytesBuffer.Get().(*bytes.Buffer)
+	defer func() {
+		buf.Reset()
+		poolBytesBuffer.Put(buf)
+	}()
+
 	rt := rv.Type()
 	info, err := cdc.getTypeInfoWLock(rt)
 	if err != nil {
@@ -377,7 +390,7 @@ func (cdc *Codec) MarshalReflect(o interface{}) ([]byte, error) {
 		if err = cdc.writeFieldIfNotEmpty(buf, 1, info, FieldOptions{}, FieldOptions{}, rv, writeEmpty); err != nil {
 			return nil, err
 		}
-		bz = buf.Bytes()
+		bz = copyBytes(buf.Bytes())
 	} else {
 		// The passed in BinFieldNum is only relevant for when the type is to
 		// be encoded unpacked (elements are Typ3_ByteLength).  In that case,
@@ -387,7 +400,7 @@ func (cdc *Codec) MarshalReflect(o interface{}) ([]byte, error) {
 		if err != nil {
 			return nil, err
 		}
-		bz = buf.Bytes()
+		bz = copyBytes(buf.Bytes())
 	}
 	// If bz is empty, prefer nil.
 	if len(bz) == 0 {
@@ -443,16 +456,26 @@ func (cdc *Codec) MarshalAny(o interface{}) ([]byte, error) {
 	}
 
 	// Encode as interface.
-	buf := new(bytes.Buffer)
+	buf := poolBytesBuffer.Get().(*bytes.Buffer)
+	defer func() {
+		buf.Reset()
+		poolBytesBuffer.Put(buf)
+	}()
 	err = cdc.encodeReflectBinaryInterface(buf, iinfo, reflect.ValueOf(&ivar).Elem(), FieldOptions{}, true)
 	if err != nil {
 		return nil, err
 	}
-	bz := buf.Bytes()
+	bz := copyBytes(buf.Bytes())
 
 	return bz, nil
 }
 
+func copyBytes(bz []byte) []byte {
+	cp := make([]byte, len(bz))
+	copy(cp, bz)
+	return cp
+}
+
 // Panics if error.
 func (cdc *Codec) MustMarshalAny(o interface{}) []byte {
 	bz, err := cdc.MarshalAny(o)
@@ -764,7 +787,11 @@ func (cdc *Codec) JSONMarshal(o interface{}) ([]byte, error) {
 		return []byte("null"), nil
 	}
 	rt := rv.Type()
-	w := new(bytes.Buffer)
+	w := poolBytesBuffer.Get().(*bytes.Buffer)
+	defer func() {
+		w.Reset()
+		poolBytesBuffer.Put(w)
+	}()
 	info, err := cdc.getTypeInfoWLock(rt)
 	if err != nil {
 		return nil, err
@@ -772,7 +799,8 @@ func (cdc *Codec) JSONMarshal(o interface{}) ([]byte, error) {
 	if err = cdc.encodeReflectJSON(w, info, rv, FieldOptions{}); err != nil {
 		return nil, err
 	}
-	return w.Bytes(), nil
+
+	return copyBytes(w.Bytes()), nil
 }
 
 func (cdc *Codec) MarshalJSONAny(o interface{}) ([]byte, error) {
@@ -802,12 +830,17 @@ func (cdc *Codec) MarshalJSONAny(o interface{}) ([]byte, error) {
 	}
 
 	// Encode as interface.
-	buf := new(bytes.Buffer)
+	buf := poolBytesBuffer.Get().(*bytes.Buffer)
+	defer func() {
+		buf.Reset()
+		poolBytesBuffer.Put(buf)
+	}()
+
 	err = cdc.encodeReflectJSONInterface(buf, iinfo, reflect.ValueOf(&ivar).Elem(), FieldOptions{})
 	if err != nil {
 		return nil, err
 	}
-	bz := buf.Bytes()
+	bz := copyBytes(buf.Bytes())
 
 	return bz, nil
 }
@@ -863,12 +896,17 @@ func (cdc *Codec) MarshalJSONIndent(o interface{}, prefix, indent string) ([]byt
 	if err != nil {
 		return nil, err
 	}
-	var out bytes.Buffer
-	err = json.Indent(&out, bz, prefix, indent)
+	out := poolBytesBuffer.Get().(*bytes.Buffer)
+	defer func() {
+		out.Reset()
+		poolBytesBuffer.Put(out)
+	}()
+
+	err = json.Indent(out, bz, prefix, indent)
 	if err != nil {
 		return nil, err
 	}
-	return out.Bytes(), nil
+	return copyBytes(out.Bytes()), nil
 }
 
 // ----------------------------------------
diff --git a/tm2/pkg/amino/benchmark_test.go b/tm2/pkg/amino/benchmark_test.go
index a1489b36..047145a2 100644
--- a/tm2/pkg/amino/benchmark_test.go
+++ b/tm2/pkg/amino/benchmark_test.go
@@ -64,8 +64,6 @@ func BenchmarkBinaryPBBindings(b *testing.B) {
 func _benchmarkBinary(b *testing.B, cdc *amino.Codec, rt reflect.Type, codecType string, encode bool) {
 	b.Helper()
 
-	b.StopTimer()
-
 	err := error(nil)
 	bz := []byte{}
 	f := fuzz.New()
@@ -88,17 +86,18 @@ func _benchmarkBinary(b *testing.B, cdc *amino.Codec, rt reflect.Type, codecType
 		}
 	}()
 
-	for i := 0; i < b.N; i++ {
+	for i := 0; i < 10; i++ {
 		f.Fuzz(ptr)
+	}
+
+	b.ResetTimer()
+
+	for i := 0; i < b.N; i++ {
 
 		// Reset, which makes debugging decoding easier.
 		rv2 = reflect.New(rt)
 		ptr2 = rv2.Interface()
 
-		// Encode to bz.
-		if encode {
-			b.StartTimer()
-		}
 		switch codecType {
 		case "binary":
 			bz, err = cdc.Marshal(ptr)
@@ -112,9 +111,6 @@ func _benchmarkBinary(b *testing.B, cdc *amino.Codec, rt reflect.Type, codecType
 		default:
 			panic("should not happen")
 		}
-		if encode {
-			b.StopTimer()
-		}
 
 		// Check for errors
 		require.Nil(b, err,
@@ -122,9 +118,6 @@ func _benchmarkBinary(b *testing.B, cdc *amino.Codec, rt reflect.Type, codecType
 			spw(ptr), err)
 
 		// Decode from bz.
-		if !encode {
-			b.StartTimer()
-		}
 		switch codecType {
 		case "binary":
 			err = cdc.Unmarshal(bz, ptr2)
@@ -137,9 +130,6 @@ func _benchmarkBinary(b *testing.B, cdc *amino.Codec, rt reflect.Type, codecType
 		default:
 			panic("should not happen")
 		}
-		if !encode {
-			b.StopTimer()
-		}
 
 		if codecType != "binary_pb_translate_only" {
 			// Decode for completeness and check for errors,
diff --git a/tm2/pkg/amino/binary_encode.go b/tm2/pkg/amino/binary_encode.go
index 426cc520..6f7adc6a 100644
--- a/tm2/pkg/amino/binary_encode.go
+++ b/tm2/pkg/amino/binary_encode.go
@@ -7,6 +7,7 @@ import (
 	"fmt"
 	"io"
 	"reflect"
+	"sync"
 )
 
 const beOptionByte = 0x01
@@ -209,6 +210,10 @@ func (cdc *Codec) encodeReflectBinary(w io.Writer, info *TypeInfo, rv reflect.Va
 	return err
 }
 
+var poolBytesBuffer = &sync.Pool{
+	New: func() any { return new(bytes.Buffer) },
+}
+
 func (cdc *Codec) encodeReflectBinaryInterface(w io.Writer, iinfo *TypeInfo, rv reflect.Value,
 	fopts FieldOptions, bare bool,
 ) (err error) {
@@ -250,7 +255,12 @@ func (cdc *Codec) encodeReflectBinaryInterface(w io.Writer, iinfo *TypeInfo, rv
 
 	// For Proto3 compatibility, encode interfaces as google.protobuf.Any
 	// Write field #1, TypeURL
-	buf := bytes.NewBuffer(nil)
+	buf := poolBytesBuffer.Get().(*bytes.Buffer)
+	defer func() {
+		buf.Reset()
+		poolBytesBuffer.Put(buf)
+	}()
+
 	{
 		fnum := uint32(1)
 		err = encodeFieldNumberAndTyp3(buf, fnum, Typ3ByteLength)
@@ -269,7 +279,12 @@ func (cdc *Codec) encodeReflectBinaryInterface(w io.Writer, iinfo *TypeInfo, rv
 	{
 		// google.protobuf.Any values must be a struct, or an unpacked list which
 		// is indistinguishable from a struct.
-		buf2 := bytes.NewBuffer(nil)
+		buf2 := poolBytesBuffer.Get().(*bytes.Buffer)
+		defer func() {
+			buf2.Reset()
+			poolBytesBuffer.Put(buf2)
+		}()
+
 		if !cinfo.IsStructOrUnpacked(fopts) {
 			writeEmpty := false
 			// Encode with an implicit struct, with a single field with number 1.
@@ -356,7 +371,11 @@ func (cdc *Codec) encodeReflectBinaryList(w io.Writer, info *TypeInfo, rv reflec
 
 	// Proto3 byte-length prefixing incurs alloc cost on the encoder.
 	// Here we incur it for unpacked form for ease of dev.
-	buf := bytes.NewBuffer(nil)
+	buf := poolBytesBuffer.Get().(*bytes.Buffer)
+	defer func() {
+		buf.Reset()
+		poolBytesBuffer.Put(buf)
+	}()
 
 	// If elem is not already a ByteLength type, write in packed form.
 	// This is a Proto wart due to Proto backwards compatibility issues.
@@ -431,9 +450,15 @@ func (cdc *Codec) encodeReflectBinaryList(w io.Writer, info *TypeInfo, rv reflec
 				// form) are represented as lists of implicit structs.
 				if writeImplicit {
 					// Write field key for Value field of implicit struct.
-					buf2 := new(bytes.Buffer)
+					buf2 := poolBytesBuffer.Get().(*bytes.Buffer)
+					buf2Done := func() {
+						buf2.Reset()
+						poolBytesBuffer.Put(buf2)
+					}
+
 					err = encodeFieldNumberAndTyp3(buf2, 1, Typ3ByteLength)
 					if err != nil {
+						buf2Done()
 						return
 					}
 					// Write field value of implicit struct to buf2.
@@ -441,10 +466,12 @@ func (cdc *Codec) encodeReflectBinaryList(w io.Writer, info *TypeInfo, rv reflec
 					efopts.BinFieldNum = 0 // dontcare
 					err = cdc.encodeReflectBinary(buf2, einfo, derv, efopts, false, 0)
 					if err != nil {
+						buf2Done()
 						return
 					}
 					// Write implicit struct to buf.
 					err = EncodeByteSlice(buf, buf2.Bytes())
+					buf2Done()
 					if err != nil {
 						return
 					}
@@ -497,7 +524,11 @@ func (cdc *Codec) encodeReflectBinaryStruct(w io.Writer, info *TypeInfo, rv refl
 
 	// Proto3 incurs a cost in writing non-root structs.
 	// Here we incur it for root structs as well for ease of dev.
-	buf := bytes.NewBuffer(nil)
+	buf := poolBytesBuffer.Get().(*bytes.Buffer)
+	defer func() {
+		buf.Reset()
+		poolBytesBuffer.Put(buf)
+	}()
 
 	for _, field := range info.Fields {
 		// Get type info for field.
diff --git a/tm2/pkg/amino/codec.go b/tm2/pkg/amino/codec.go
index 3fa7634e..c3952468 100644
--- a/tm2/pkg/amino/codec.go
+++ b/tm2/pkg/amino/codec.go
@@ -113,7 +113,12 @@ func (info *TypeInfo) String() string {
 		// before it's fully populated.
 		return "<new TypeInfo>"
 	}
-	buf := new(bytes.Buffer)
+	buf := poolBytesBuffer.Get().(*bytes.Buffer)
+	defer func() {
+		buf.Reset()
+		poolBytesBuffer.Put(buf)
+	}()
+
 	buf.Write([]byte("TypeInfo{"))
 	buf.Write([]byte(fmt.Sprintf("Type:%v,", info.Type)))
 	if info.ConcreteInfo.Registered {
diff --git a/tm2/pkg/amino/json_encode.go b/tm2/pkg/amino/json_encode.go
index 113c3486..3cbd5e2f 100644
--- a/tm2/pkg/amino/json_encode.go
+++ b/tm2/pkg/amino/json_encode.go
@@ -156,7 +156,12 @@ func (cdc *Codec) encodeReflectJSONInterface(w io.Writer, iinfo *TypeInfo, rv re
 	}
 
 	// Write Value to buffer
-	buf := new(bytes.Buffer)
+	buf := poolBytesBuffer.Get().(*bytes.Buffer)
+	defer func() {
+		buf.Reset()
+		poolBytesBuffer.Put(buf)
+	}()
+
 	cdc.encodeReflectJSON(buf, cinfo, crv, fopts)
 	value := buf.Bytes()
 	if len(value) == 0 {
diff --git a/tm2/pkg/amino/wellknown.go b/tm2/pkg/amino/wellknown.go
index 7720c289..2f5d9ef6 100644
--- a/tm2/pkg/amino/wellknown.go
+++ b/tm2/pkg/amino/wellknown.go
@@ -342,7 +342,12 @@ func encodeReflectBinaryWellKnown(w io.Writer, info *TypeInfo, rv reflect.Value,
 	}
 	// Maybe recurse with length-prefixing.
 	if !bare {
-		buf := bytes.NewBuffer(nil)
+		buf := poolBytesBuffer.Get().(*bytes.Buffer)
+		defer func() {
+			buf.Reset()
+			poolBytesBuffer.Put(buf)
+		}()
+
 		ok, err = encodeReflectBinaryWellKnown(buf, info, rv, fopts, true)
 		if err != nil {
 			return false, err

Performance improvements inside amino

name                                   old time/op    new time/op    delta
Binary/EmptyStruct:encode-8              3.86µs ± 5%    3.92µs ± 5%     ~     (p=0.548 n=5+5)
Binary/EmptyStruct:decode-8              3.79µs ± 5%    3.79µs ± 6%     ~     (p=0.690 n=5+5)
Binary/PrimitivesStruct:encode-8         35.5µs ± 2%    36.5µs ± 5%     ~     (p=0.151 n=5+5)
Binary/PrimitivesStruct:decode-8         35.0µs ± 2%    38.6µs ±11%  +10.17%  (p=0.016 n=5+5)
Binary/ShortArraysStruct:encode-8        5.91µs ± 6%    6.36µs ± 8%   +7.61%  (p=0.032 n=5+5)
Binary/ShortArraysStruct:decode-8        6.07µs ±21%    6.39µs ± 8%     ~     (p=0.151 n=5+5)
Binary/ArraysStruct:encode-8             95.1µs ± 8%   100.6µs ± 7%     ~     (p=0.222 n=5+5)
Binary/ArraysStruct:decode-8             91.3µs ± 5%    98.5µs ±12%     ~     (p=0.222 n=5+5)
Binary/ArraysArraysStruct:encode-8        131µs ± 3%     132µs ± 6%     ~     (p=0.841 n=5+5)
Binary/ArraysArraysStruct:decode-8        136µs ± 9%     134µs ± 3%     ~     (p=0.548 n=5+5)
Binary/SlicesStruct:encode-8             85.4µs ± 1%    92.3µs ± 9%   +8.15%  (p=0.008 n=5+5)
Binary/SlicesStruct:decode-8             87.1µs ± 8%    94.8µs ± 7%     ~     (p=0.056 n=5+5)
Binary/SlicesSlicesStruct:encode-8        506µs ± 2%     545µs ± 9%     ~     (p=0.151 n=5+5)
Binary/SlicesSlicesStruct:decode-8        506µs ± 3%     523µs ± 3%     ~     (p=0.095 n=5+5)
Binary/PointersStruct:encode-8           56.8µs ± 4%    65.5µs ±20%  +15.43%  (p=0.016 n=5+5)
Binary/PointersStruct:decode-8           57.5µs ± 3%    55.9µs ± 3%     ~     (p=0.095 n=5+5)
Binary/PointerSlicesStruct:encode-8       162µs ± 4%     172µs ±21%     ~     (p=0.841 n=5+5)
Binary/PointerSlicesStruct:decode-8       163µs ± 5%     185µs ±13%     ~     (p=0.095 n=5+5)
Binary/ComplexSt:encode-8                 314µs ± 3%     354µs ±11%  +12.90%  (p=0.008 n=5+5)
Binary/ComplexSt:decode-8                 319µs ± 2%     338µs ± 4%   +5.87%  (p=0.008 n=5+5)
Binary/EmbeddedSt1:encode-8              39.8µs ± 7%    39.3µs ± 8%     ~     (p=1.000 n=5+5)
Binary/EmbeddedSt1:decode-8              37.0µs ± 4%    37.8µs ± 6%     ~     (p=0.690 n=5+5)
Binary/EmbeddedSt2:encode-8               316µs ± 7%     307µs ± 3%     ~     (p=0.222 n=5+5)
Binary/EmbeddedSt2:decode-8               316µs ± 3%     306µs ± 2%     ~     (p=0.095 n=5+5)
Binary/EmbeddedSt3:encode-8               217µs ± 7%     201µs ± 1%   -7.26%  (p=0.008 n=5+5)
Binary/EmbeddedSt3:decode-8               222µs ±10%     204µs ± 2%   -8.50%  (p=0.032 n=5+5)
Binary/EmbeddedSt4:encode-8               332µs ± 4%     325µs ± 3%     ~     (p=0.421 n=5+5)
Binary/EmbeddedSt4:decode-8               332µs ± 4%     324µs ± 5%     ~     (p=0.095 n=5+5)
Binary/EmbeddedSt5:encode-8               218µs ± 2%     212µs ± 3%     ~     (p=0.056 n=5+5)
Binary/EmbeddedSt5:decode-8               224µs ± 8%     209µs ± 1%   -6.85%  (p=0.008 n=5+5)
Binary/AminoMarshalerStruct1:encode-8    9.03µs ± 6%    8.97µs ±12%     ~     (p=0.841 n=5+5)
Binary/AminoMarshalerStruct1:decode-8    8.91µs ± 5%    8.81µs ± 4%     ~     (p=0.841 n=5+5)
Binary/AminoMarshalerStruct2:encode-8    13.2µs ±10%    12.2µs ± 2%   -7.26%  (p=0.008 n=5+5)
Binary/AminoMarshalerStruct2:decode-8    13.2µs ± 6%    12.5µs ± 5%     ~     (p=0.095 n=5+5)
Binary/AminoMarshalerStruct3:encode-8    7.17µs ± 3%    7.50µs ± 8%     ~     (p=0.548 n=5+5)
Binary/AminoMarshalerStruct3:decode-8    7.12µs ± 4%    7.84µs ±10%  +10.12%  (p=0.016 n=5+5)
Binary/AminoMarshalerInt4:encode-8       6.60µs ± 5%    6.96µs ±11%     ~     (p=0.421 n=5+5)
Binary/AminoMarshalerInt4:decode-8       6.79µs ±12%    7.04µs ±15%     ~     (p=0.690 n=5+5)
Binary/AminoMarshalerInt5:encode-8       6.64µs ± 4%    6.92µs ± 5%   +4.09%  (p=0.032 n=5+5)
Binary/AminoMarshalerInt5:decode-8       6.55µs ± 3%    7.76µs ±10%  +18.44%  (p=0.008 n=5+5)
Binary/AminoMarshalerStruct6:encode-8    11.7µs ± 5%    13.2µs ±10%  +13.09%  (p=0.008 n=5+5)
Binary/AminoMarshalerStruct6:decode-8    11.4µs ± 3%    11.6µs ± 2%     ~     (p=0.222 n=5+5)
Binary/AminoMarshalerStruct7:encode-8    9.86µs ± 1%   10.10µs ±19%     ~     (p=0.310 n=5+5)
Binary/AminoMarshalerStruct7:decode-8    9.55µs ± 3%    9.75µs ±10%     ~     (p=0.690 n=5+5)

name                                   old alloc/op   new alloc/op   delta
Binary/EmptyStruct:encode-8              1.50kB ± 0%    1.41kB ± 0%   -6.32%  (p=0.008 n=5+5)
Binary/EmptyStruct:decode-8              1.50kB ± 0%    1.41kB ± 0%   -6.32%  (p=0.008 n=5+5)
Binary/PrimitivesStruct:encode-8         10.4kB ± 0%     9.6kB ± 0%   -7.82%  (p=0.008 n=5+5)
Binary/PrimitivesStruct:decode-8         10.4kB ± 0%     9.6kB ± 0%   -7.82%  (p=0.000 n=4+5)
Binary/ShortArraysStruct:encode-8        2.11kB ± 0%    1.92kB ± 0%   -9.04%  (p=0.008 n=5+5)
Binary/ShortArraysStruct:decode-8        2.11kB ± 0%    1.92kB ± 0%   -9.04%  (p=0.008 n=5+5)
Binary/ArraysStruct:encode-8             25.9kB ± 0%    22.0kB ± 0%  -15.04%  (p=0.008 n=5+5)
Binary/ArraysStruct:decode-8             25.9kB ± 0%    22.0kB ± 0%  -15.04%  (p=0.008 n=5+5)
Binary/ArraysArraysStruct:encode-8       37.7kB ± 0%    25.3kB ± 0%  -33.07%  (p=0.008 n=5+5)
Binary/ArraysArraysStruct:decode-8       37.7kB ± 0%    25.3kB ± 0%  -33.07%  (p=0.008 n=5+5)
Binary/SlicesStruct:encode-8             28.2kB ± 0%    25.1kB ± 0%  -10.96%  (p=0.008 n=5+5)
Binary/SlicesStruct:decode-8             28.2kB ± 0%    25.1kB ± 0%  -10.97%  (p=0.008 n=5+5)
Binary/SlicesSlicesStruct:encode-8        183kB ± 0%     147kB ± 0%  -19.92%  (p=0.008 n=5+5)
Binary/SlicesSlicesStruct:decode-8        183kB ± 0%     147kB ± 0%  -19.92%  (p=0.008 n=5+5)
Binary/PointersStruct:encode-8           14.4kB ± 0%    13.6kB ± 0%   -5.64%  (p=0.008 n=5+5)
Binary/PointersStruct:decode-8           14.4kB ± 0%    13.6kB ± 0%   -5.64%  (p=0.008 n=5+5)
Binary/PointerSlicesStruct:encode-8      43.9kB ± 0%    40.2kB ± 0%   -8.49%  (p=0.008 n=5+5)
Binary/PointerSlicesStruct:decode-8      43.9kB ± 0%    40.2kB ± 0%   -8.49%  (p=0.008 n=5+5)
Binary/ComplexSt:encode-8                95.3kB ± 0%    78.2kB ± 0%  -17.97%  (p=0.008 n=5+5)
Binary/ComplexSt:decode-8                95.3kB ± 0%    78.2kB ± 0%  -17.97%  (p=0.008 n=5+5)
Binary/EmbeddedSt1:encode-8              11.3kB ± 0%    10.2kB ± 0%   -9.62%  (p=0.000 n=5+4)
Binary/EmbeddedSt1:decode-8              11.3kB ± 0%    10.2kB ± 0%   -9.61%  (p=0.000 n=5+4)
Binary/EmbeddedSt2:encode-8              95.5kB ± 0%    78.3kB ± 0%  -17.96%  (p=0.008 n=5+5)
Binary/EmbeddedSt2:decode-8              95.5kB ± 0%    78.4kB ± 0%  -17.94%  (p=0.008 n=5+5)
Binary/EmbeddedSt3:encode-8              68.3kB ± 0%    56.6kB ± 0%  -17.22%  (p=0.008 n=5+5)
Binary/EmbeddedSt3:decode-8              68.3kB ± 0%    56.6kB ± 0%  -17.21%  (p=0.008 n=5+5)
Binary/EmbeddedSt4:encode-8              97.2kB ± 0%    82.3kB ± 0%  -15.32%  (p=0.008 n=5+5)
Binary/EmbeddedSt4:decode-8              97.2kB ± 0%    82.3kB ± 0%  -15.31%  (p=0.008 n=5+5)
Binary/EmbeddedSt5:encode-8              65.9kB ± 0%    55.3kB ± 0%  -16.19%  (p=0.008 n=5+5)
Binary/EmbeddedSt5:decode-8              66.0kB ± 0%    55.3kB ± 0%  -16.18%  (p=0.008 n=5+5)
Binary/AminoMarshalerStruct1:encode-8    2.87kB ± 0%    2.66kB ± 0%   -7.23%  (p=0.008 n=5+5)
Binary/AminoMarshalerStruct1:decode-8    2.87kB ± 0%    2.66kB ± 0%   -7.23%  (p=0.008 n=5+5)
Binary/AminoMarshalerStruct2:encode-8    4.58kB ± 0%    3.62kB ± 0%  -20.95%  (p=0.008 n=5+5)
Binary/AminoMarshalerStruct2:decode-8    4.58kB ± 0%    3.62kB ± 0%  -20.95%  (p=0.008 n=5+5)
Binary/AminoMarshalerStruct3:encode-8    2.42kB ± 0%    2.31kB ± 0%   -4.62%  (p=0.008 n=5+5)
Binary/AminoMarshalerStruct3:decode-8    2.42kB ± 0%    2.31kB ± 0%   -4.62%  (p=0.008 n=5+5)
Binary/AminoMarshalerInt4:encode-8       2.38kB ± 0%    2.15kB ± 0%   -9.38%  (p=0.008 n=5+5)
Binary/AminoMarshalerInt4:decode-8       2.38kB ± 0%    2.15kB ± 0%   -9.38%  (p=0.008 n=5+5)
Binary/AminoMarshalerInt5:encode-8       2.36kB ± 0%    2.27kB ± 0%   -4.07%  (p=0.008 n=5+5)
Binary/AminoMarshalerInt5:decode-8       2.36kB ± 0%    2.27kB ± 0%   -4.07%  (p=0.008 n=5+5)
Binary/AminoMarshalerStruct6:encode-8    3.51kB ± 0%    3.19kB ± 0%   -9.05%  (p=0.008 n=5+5)
Binary/AminoMarshalerStruct6:decode-8    3.51kB ± 0%    3.19kB ± 0%   -9.05%  (p=0.008 n=5+5)
Binary/AminoMarshalerStruct7:encode-8    2.89kB ± 0%    2.67kB ± 0%   -7.72%  (p=0.008 n=5+5)
Binary/AminoMarshalerStruct7:decode-8    2.89kB ± 0%    2.67kB ± 0%   -7.72%  (p=0.008 n=5+5)

name                                   old allocs/op  new allocs/op  delta
Binary/EmptyStruct:encode-8                38.0 ± 0%      36.0 ± 0%   -5.26%  (p=0.008 n=5+5)
Binary/EmptyStruct:decode-8                38.0 ± 0%      36.0 ± 0%   -5.26%  (p=0.008 n=5+5)
Binary/PrimitivesStruct:encode-8            439 ± 0%       429 ± 0%   -2.28%  (p=0.008 n=5+5)
Binary/PrimitivesStruct:decode-8            439 ± 0%       429 ± 0%   -2.28%  (p=0.008 n=5+5)
Binary/ShortArraysStruct:encode-8          56.0 ± 0%      52.0 ± 0%   -7.14%  (p=0.008 n=5+5)
Binary/ShortArraysStruct:decode-8          56.0 ± 0%      52.0 ± 0%   -7.14%  (p=0.008 n=5+5)
Binary/ArraysStruct:encode-8                977 ± 0%       919 ± 0%   -5.94%  (p=0.008 n=5+5)
Binary/ArraysStruct:decode-8                977 ± 0%       919 ± 0%   -5.94%  (p=0.008 n=5+5)
Binary/ArraysArraysStruct:encode-8        1.28k ± 0%     1.08k ± 0%  -15.05%  (p=0.008 n=5+5)
Binary/ArraysArraysStruct:decode-8        1.28k ± 0%     1.08k ± 0%  -15.05%  (p=0.008 n=5+5)
Binary/SlicesStruct:encode-8              1.01k ± 0%     0.97k ± 0%   -3.77%  (p=0.008 n=5+5)
Binary/SlicesStruct:decode-8              1.01k ± 0%     0.97k ± 0%   -3.77%  (p=0.008 n=5+5)
Binary/SlicesSlicesStruct:encode-8        6.33k ± 0%     5.95k ± 0%   -5.90%  (p=0.008 n=5+5)
Binary/SlicesSlicesStruct:decode-8        6.33k ± 0%     5.95k ± 0%   -5.90%  (p=0.008 n=5+5)
Binary/PointersStruct:encode-8              637 ± 0%       627 ± 0%   -1.57%  (p=0.008 n=5+5)
Binary/PointersStruct:decode-8              637 ± 0%       627 ± 0%   -1.57%  (p=0.008 n=5+5)
Binary/PointerSlicesStruct:encode-8       1.62k ± 0%     1.56k ± 0%   -3.28%  (p=0.008 n=5+5)
Binary/PointerSlicesStruct:decode-8       1.62k ± 0%     1.56k ± 0%   -3.28%  (p=0.008 n=5+5)
Binary/ComplexSt:encode-8                 3.37k ± 0%     3.22k ± 0%   -4.62%  (p=0.008 n=5+5)
Binary/ComplexSt:decode-8                 3.37k ± 0%     3.22k ± 0%   -4.62%  (p=0.008 n=5+5)
Binary/EmbeddedSt1:encode-8                 453 ± 0%       440 ± 0%   -2.87%  (p=0.008 n=5+5)
Binary/EmbeddedSt1:decode-8                 453 ± 0%       440 ± 0%   -2.87%  (p=0.008 n=5+5)
Binary/EmbeddedSt2:encode-8               3.37k ± 0%     3.22k ± 0%   -4.62%  (p=0.008 n=5+5)
Binary/EmbeddedSt2:decode-8               3.37k ± 0%     3.22k ± 0%   -4.62%  (p=0.008 n=5+5)
Binary/EmbeddedSt3:encode-8               2.32k ± 0%     2.20k ± 0%   -5.38%  (p=0.008 n=5+5)
Binary/EmbeddedSt3:decode-8               2.32k ± 0%     2.20k ± 0%   -5.38%  (p=0.008 n=5+5)
Binary/EmbeddedSt4:encode-8               3.67k ± 0%     3.54k ± 0%   -3.73%  (p=0.008 n=5+5)
Binary/EmbeddedSt4:decode-8               3.67k ± 0%     3.54k ± 0%   -3.73%  (p=0.008 n=5+5)
Binary/EmbeddedSt5:encode-8               2.32k ± 0%     2.20k ± 0%   -5.00%  (p=0.008 n=5+5)
Binary/EmbeddedSt5:decode-8               2.32k ± 0%     2.20k ± 0%   -5.00%  (p=0.008 n=5+5)
Binary/AminoMarshalerStruct1:encode-8      97.0 ± 0%      94.0 ± 0%   -3.09%  (p=0.008 n=5+5)
Binary/AminoMarshalerStruct1:decode-8      97.0 ± 0%      94.0 ± 0%   -3.09%  (p=0.008 n=5+5)
Binary/AminoMarshalerStruct2:encode-8       149 ± 0%       133 ± 0%  -10.74%  (p=0.008 n=5+5)
Binary/AminoMarshalerStruct2:decode-8       149 ± 0%       133 ± 0%  -10.74%  (p=0.008 n=5+5)
Binary/AminoMarshalerStruct3:encode-8      77.0 ± 0%      76.0 ± 0%   -1.30%  (p=0.008 n=5+5)
Binary/AminoMarshalerStruct3:decode-8      77.0 ± 0%      76.0 ± 0%   -1.30%  (p=0.008 n=5+5)
Binary/AminoMarshalerInt4:encode-8         71.0 ± 0%      68.0 ± 0%   -4.23%  (p=0.008 n=5+5)
Binary/AminoMarshalerInt4:decode-8         71.0 ± 0%      68.0 ± 0%   -4.23%  (p=0.008 n=5+5)
Binary/AminoMarshalerInt5:encode-8         74.0 ± 0%      73.0 ± 0%   -1.35%  (p=0.008 n=5+5)
Binary/AminoMarshalerInt5:decode-8         74.0 ± 0%      73.0 ± 0%   -1.35%  (p=0.008 n=5+5)
Binary/AminoMarshalerStruct6:encode-8       122 ± 0%       117 ± 0%   -4.10%  (p=0.008 n=5+5)
Binary/AminoMarshalerStruct6:decode-8       122 ± 0%       117 ± 0%   -4.10%  (p=0.008 n=5+5)
Binary/AminoMarshalerStruct7:encode-8       101 ± 0%        98 ± 0%   -2.97%  (p=0.008 n=5+5)
Binary/AminoMarshalerStruct7:decode-8       101 ± 0%        98 ± 0%   -2.97%  (p=0.008 n=5+5)

RAM improvements inside gnovm/pkg/gnolang.TestStdlibs

(pprof) tree MarshalAny
Active filters:
   focus=MarshalAny
Showing nodes accounting for 560.45MB, 0.26% of 215641.47MB total
Dropped 4 nodes (cum <= 1078.21MB)
----------------------------------------------------------+-------------
      flat  flat%   sum%        cum   cum%   calls calls% + context 	 	 
----------------------------------------------------------+-------------
                                              43MB 53.75% |   github.com/gnolang/gno/tm2/pkg/amino.(*Codec).writeFieldIfNotEmpty
                                           29.50MB 36.88% |   github.com/gnolang/gno/tm2/pkg/amino.(*Codec).encodeReflectBinaryInterface
                                            7.50MB  9.38% |   github.com/gnolang/gno/tm2/pkg/amino.(*Codec).encodeReflectBinaryList
      80MB 0.037% 0.037%       80MB 0.037%                | github.com/gnolang/gno/tm2/pkg/amino.encodeFieldNumberAndTyp3
----------------------------------------------------------+-------------
                                              70MB   100% |   github.com/gnolang/gno/tm2/pkg/amino.EncodeByteSlice
      70MB 0.032%  0.07%       70MB 0.032%                | github.com/gnolang/gno/tm2/pkg/amino.EncodeUvarint
----------------------------------------------------------+-------------
                                          560.95MB   100% |   github.com/gnolang/gno/tm2/pkg/amino.(*Codec).MustMarshalAny
   69.27MB 0.032%   0.1%   560.95MB  0.26%                | github.com/gnolang/gno/tm2/pkg/amino.(*Codec).MarshalAny
                                          471.67MB 84.09% |   github.com/gnolang/gno/tm2/pkg/amino.(*Codec).encodeReflectBinaryInterface
                                              20MB  3.57% |   reflect.Value.Interface (inline)
----------------------------------------------------------+-------------
                                           33.50MB 53.60% |   github.com/gnolang/gno/gnovm/pkg/gnolang.ObjectID.MarshalAmino (inline)
                                              29MB 46.40% |   github.com/gnolang/gno/gnovm/pkg/gnolang.ValueHash.MarshalAmino (inline)
   62.50MB 0.029%  0.13%    62.50MB 0.029%                | encoding/hex.EncodeToString
----------------------------------------------------------+-------------
                                              53MB   100% |   reflect.FuncOf
      53MB 0.025%  0.16%       53MB 0.025%                | reflect.New
----------------------------------------------------------+-------------
                                              49MB 65.77% |   github.com/gnolang/gno/tm2/pkg/amino.(*Codec).encodeReflectBinary (inline)
                                           25.50MB 34.23% |   github.com/gnolang/gno/tm2/pkg/amino.(*Codec).encodeReflectBinaryInterface (inline)
      50MB 0.023%  0.18%    74.50MB 0.035%                | github.com/gnolang/gno/tm2/pkg/amino.EncodeString
                                           24.50MB 32.89% |   github.com/gnolang/gno/tm2/pkg/amino.EncodeByteSlice
----------------------------------------------------------+-------------
                                          134.51MB   100% |   reflect.Value.Call
   43.50MB  0.02%   0.2%   134.51MB 0.062%                | reflect.Value.call
                                              58MB 43.12% |   github.com/gnolang/gno/gnovm/pkg/gnolang.ObjectID.MarshalAmino
                                              29MB 21.56% |   github.com/gnolang/gno/gnovm/pkg/gnolang.ValueHash.MarshalAmino
                                               4MB  2.97% |   reflect.methodReceiver
----------------------------------------------------------+-------------
                                              87MB   100% |   reflect.(*rtype).MethodByName
      34MB 0.016%  0.21%       87MB  0.04%                | reflect.(*rtype).Method
                                              53MB 60.92% |   reflect.FuncOf
----------------------------------------------------------+-------------
                                           31.16MB   100% |   bytes.(*Buffer).grow
   31.16MB 0.014%  0.23%    31.16MB 0.014%                | bytes.growSlice
----------------------------------------------------------+-------------
                                              20MB   100% |   reflect.valueInterface
      20MB 0.0093%  0.24%       20MB 0.0093%                | reflect.packEface
----------------------------------------------------------+-------------
                                           16.50MB   100% |   github.com/gnolang/gno/gnovm/pkg/gnolang.ObjectID.MarshalAmino
   16.50MB 0.0077%  0.25%    16.50MB 0.0077%                | fmt.Sprintf
----------------------------------------------------------+-------------
                                              10MB   100% |   github.com/gnolang/gno/tm2/pkg/amino.(*Codec).encodeReflectBinaryByteArray
      10MB 0.0046%  0.25%       10MB 0.0046%                | reflect.Value.Slice
----------------------------------------------------------+-------------
                                            8.50MB   100% |   github.com/gnolang/gno/tm2/pkg/amino.(*Codec).encodeReflectBinary
    8.50MB 0.0039%  0.25%     8.50MB 0.0039%                | github.com/gnolang/gno/tm2/pkg/amino.EncodeVarint

Kindly cc-ing @notJoon @n2p5 @jaekwon @moul @thehowl