Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: A fully featured btree implementation #3126

Merged
merged 15 commits into from
Dec 5, 2024

Conversation

wyhaines
Copy link
Contributor

This is a fully featured btree implementation. A friend gave me some incomplete (and broken) btree code for Go, and when I started reworking it, I discovered that it was a broken semi-copy of an old version of Google's btree for Go.

I finished reworking it so that it adhere's to that original Google version's API, though there are some differences internally in places, and I think that my version is much easier to follow and to understand.

This implementation is quite a bit faster than the AVL tree. I will add links to some benchmarks that I did in a comment. This implementation supports copy-on-write for the trees, for inexpensively creating copies of a tree that are effectively isolated from each other with respect to changes that happen after the fork.

Contributors' checklist...
  • Added new tests, or not needed, or not feasible
  • Provided an example (e.g. screenshot) to aid review or the PR is self-explanatory
  • Updated the official documentation or not needed
  • No breaking changes were made, or a BREAKING CHANGE: xxx message was included in the description
  • Added references to related issues and PRs
  • Provided any useful hints for running manual tests

@github-actions github-actions bot added the 🧾 package/realm Tag used for new Realms or Packages. label Nov 14, 2024
@n0izn0iz
Copy link
Contributor

Thanks! I need this for that https://github.com/TERITORI/teritori-dapp/blob/2a5c8390438e6aad332f71410bcf8aabaaecab4b/gno/p/havl/havl.gno#L18
:D

@wyhaines wyhaines changed the title A a full features btree implementation A fully featured btree implementation Nov 15, 2024
@wyhaines wyhaines changed the title A fully featured btree implementation feat: A fully featured btree implementation Nov 15, 2024
@salmad3 salmad3 mentioned this pull request Nov 17, 2024
33 tasks
@Kouteki Kouteki requested a review from ajnavarro November 17, 2024 04:06
@Kouteki Kouteki added this to the 🚀 Mainnet launch milestone Nov 17, 2024
@Kouteki Kouteki added the in focus Core team is prioritizing this work label Nov 17, 2024
Copy link

codecov bot commented Nov 17, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

📢 Thoughts on this report? Let us know!

@notJoon
Copy link
Member

notJoon commented Nov 19, 2024

related with #3021

examples/gno.land/p/demo/btree/gno.mod Outdated Show resolved Hide resolved
examples/gno.land/p/demo/btree/btree.gno Outdated Show resolved Hide resolved
@leohhhn
Copy link
Contributor

leohhhn commented Nov 26, 2024

Would be really cool to see a mini tutorial in r/docs about how to use this. Also, a burning question for newbies will be why use this over map or AVL. the r/docs corresponding to this pkg would be a good place to explain this.

@wyhaines
Copy link
Contributor Author

Would be really cool to see a mini tutorial in r/docs about how to use this. Also, a burning question for newbies will be why use this over map or AVL. the r/docs corresponding to this pkg would be a good place to explain this.

I will plan on writing one. Thanks for the suggestion.

examples/gno.land/p/demo/btree/btree.gno Outdated Show resolved Hide resolved
examples/gno.land/p/demo/btree/btree.gno Outdated Show resolved Hide resolved
examples/gno.land/p/demo/btree/btree.gno Outdated Show resolved Hide resolved
examples/gno.land/p/demo/btree/btree.gno Outdated Show resolved Hide resolved
@Gno2D2
Copy link
Collaborator

Gno2D2 commented Nov 30, 2024

I'm a bot that assists the Gno Core team in maintaining this repository. My role is to ensure that contributors understand and follow our guidelines, helping to streamline the development process.

The following requirements must be fulfilled before a pull request can be merged.
Some requirement checks are automated and can be verified by the CI, while others need manual verification by a staff member.

These requirements are defined in this configuration file.

Automated Checks

🟢 Maintainers must be able to edit this pull request
🟢 The pull request head branch must be up-to-date with its base

Manual Checks

  • The pull request description provides enough details
Debug
Automated Checks
Maintainers must be able to edit this pull request

If

🟢 Condition met
└── 🟢 On every pull request

Then

🟢 Requirement satisfied
└── 🟢 Maintainer can modify this pull request

The pull request head branch must be up-to-date with its base

If

🟢 Condition met
└── 🟢 On every pull request

Then

🟢 Requirement satisfied
└── 🟢 Head branch (wyhaines:kh.add-btree) is up to date with base (master): behind by 0 / ahead by 15

Manual Checks
The pull request description provides enough details

If

🟢 Condition met
└── 🟢 Not (🔴 Pull request author is a member of the team: core-contributors)

Can be checked by

  • team core-contributors

@wyhaines
Copy link
Contributor Author

wyhaines commented Dec 5, 2024

In order to persist the link in the issue, for posterity, here are some timings that I did with the AVL tree, an AVL tree with one (of two that I have done) COW implementation, and the btree.

https://docs.google.com/spreadsheets/d/1_Q6Mas-IYs4REu6MZrDTcywAGn8EVAbNOX9CDw01WGk/edit?usp=sharing

I ran each test 9 times, and captured both the startup time for each test, and the total run time, so that the effective run time for each iteration can be determined. I also captures max RSS.

So, for instance, if the startup time is 0.81 seconds, and the run time is 35.83 seconds, then the time spent actually performing the benchmark actions was 35.02.

The tl;dr is that the btree is a lot faster than the AVL tree, at the cost of slightly more memory usage, unless trees are copied, in which case the memory usage from the btree is substantially less than the AVL tree.

@wyhaines wyhaines merged commit 7ce29ff into gnolang:master Dec 5, 2024
102 of 104 checks passed
@aeddi
Copy link
Contributor

aeddi commented Dec 5, 2024

@wyhaines this CI check was red when you merged this PR, that's why the CI is red on master right now.
Do you know how to fix it?

The error is:

Linting "./gno.land/p/demo/btree"...
gno.land/p/demo/btree/btree_test.gno:79:3: ./gno.land/p/demo/btree.Content does not implement gno.land/p/demo/btree.Record (wrong type for method Less) (code=2).

n2p5 added a commit that referenced this pull request Dec 5, 2024
@moul
Copy link
Member

moul commented Dec 6, 2024

Why was this PR merged?

omarsy pushed a commit to TERITORI/gno that referenced this pull request Dec 7, 2024
This is a fully featured btree implementation. A friend gave me some
incomplete (and broken) btree code for Go, and when I started reworking
it, I discovered that it was a broken semi-copy of an old version of
Google's btree for Go.

I finished reworking it so that it adhere's to that original Google
version's API, though there are some differences internally in places,
and I think that my version is much easier to follow and to understand.

This implementation is quite a bit faster than the AVL tree. I will add
links to some benchmarks that I did in a comment. This implementation
supports copy-on-write for the trees, for inexpensively creating copies
of a tree that are effectively isolated from each other with respect to
changes that happen after the fork.

<details><summary>Contributors' checklist...</summary>

- [x] Added new tests, or not needed, or not feasible
- [x] Provided an example (e.g. screenshot) to aid review or the PR is
self-explanatory
- [x] Updated the official documentation or not needed
- [x] No breaking changes were made, or a `BREAKING CHANGE: xxx` message
was included in the description
- [ ] Added references to related issues and PRs
- [ ] Provided any useful hints for running manual tests
</details>
r3v4s pushed a commit to gnoswap-labs/gno that referenced this pull request Dec 10, 2024
This is a fully featured btree implementation. A friend gave me some
incomplete (and broken) btree code for Go, and when I started reworking
it, I discovered that it was a broken semi-copy of an old version of
Google's btree for Go.

I finished reworking it so that it adhere's to that original Google
version's API, though there are some differences internally in places,
and I think that my version is much easier to follow and to understand.

This implementation is quite a bit faster than the AVL tree. I will add
links to some benchmarks that I did in a comment. This implementation
supports copy-on-write for the trees, for inexpensively creating copies
of a tree that are effectively isolated from each other with respect to
changes that happen after the fork.

<details><summary>Contributors' checklist...</summary>

- [x] Added new tests, or not needed, or not feasible
- [x] Provided an example (e.g. screenshot) to aid review or the PR is
self-explanatory
- [x] Updated the official documentation or not needed
- [x] No breaking changes were made, or a `BREAKING CHANGE: xxx` message
was included in the description
- [ ] Added references to related issues and PRs
- [ ] Provided any useful hints for running manual tests
</details>
@Kouteki Kouteki removed the in focus Core team is prioritizing this work label Dec 16, 2024
albttx pushed a commit that referenced this pull request Jan 10, 2025
This is a fully featured btree implementation. A friend gave me some
incomplete (and broken) btree code for Go, and when I started reworking
it, I discovered that it was a broken semi-copy of an old version of
Google's btree for Go.

I finished reworking it so that it adhere's to that original Google
version's API, though there are some differences internally in places,
and I think that my version is much easier to follow and to understand.

This implementation is quite a bit faster than the AVL tree. I will add
links to some benchmarks that I did in a comment. This implementation
supports copy-on-write for the trees, for inexpensively creating copies
of a tree that are effectively isolated from each other with respect to
changes that happen after the fork.

<details><summary>Contributors' checklist...</summary>

- [x] Added new tests, or not needed, or not feasible
- [x] Provided an example (e.g. screenshot) to aid review or the PR is
self-explanatory
- [x] Updated the official documentation or not needed
- [x] No breaking changes were made, or a `BREAKING CHANGE: xxx` message
was included in the description
- [ ] Added references to related issues and PRs
- [ ] Provided any useful hints for running manual tests
</details>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
🧾 package/realm Tag used for new Realms or Packages.
Projects
Development

Successfully merging this pull request may close these issues.

10 participants