Conservative bloomfilter #42
Conversation
|
Note to self: Lines 27 to 37 in e3cc080 The block size setting may relieve pressure on raid zero, make lvm unnecessary and improve cloud perf. |
|
This is now amended so that it shouldn't be able to blow things up. (very conservative) |
goleveldb.go
Outdated
| options := &opt.Options{ | ||
| Filter: filter.NewBloomFilter(10), | ||
| OpenFilesCacheCapacity: 8192, | ||
| BlockCacheCapacity: opt.GiB, |
There was a problem hiding this comment.
200 on mac, 500 on other.
There was a problem hiding this comment.
Can we add comments to say what the normal default was.
There was a problem hiding this comment.
absolutely.
Sorry for delay on this, been real busy.
goleveldb.go
Outdated
| BlockCacheCapacity: opt.GiB, // The default value is 8MiB. | ||
| WriteBuffer: 64 * opt.MiB, // The default value is 4MiB. |
There was a problem hiding this comment.
these are hefty jumps, what is the reasoning?
There was a problem hiding this comment.
I think these are way to high and we should delete them and opt to the default then figure out how to proceed instead of throwing random big numbers out there
There was a problem hiding this comment.
I'd be okay with that. The bloom filter should be a pretty big boost by itself. 1 moment please...
| @@ -24,7 +25,10 @@ type GoLevelDB struct { | |||
| var _ DB = (*GoLevelDB)(nil) | |||
|
|
|||
| func NewGoLevelDB(name string, dir string) (*GoLevelDB, error) { | |||
| return NewGoLevelDBWithOpts(name, dir, nil) | |||
| options := &opt.Options{ | |||
| Filter: filter.NewBloomFilter(10), // by default, goleveldb doesn't use a bloom filter. | |||
There was a problem hiding this comment.
should we also disable seek compaction? im not sure how this effects things though
There was a problem hiding this comment.
will look it up this week. Maybe we start here, and start to feed things into the iavl benchmarks-- they work now
|
feel like should benchmark those changes to know the difference and any other impact. Otherwise, we should think about setting options from the SDK side like: |
it did reduce compaction by quite a bit but I wasn't able to see the growth in disk space difference. |
|
@JayT106 I agree with you, but would like to mention that benchmarks for this library should properly run in iavl. |
This takes #3 and fumpts it, and adds a few other known-safe tweaks.