Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(gno.land): improve test speed #3396

Merged
merged 31 commits into from
Jan 6, 2025

Conversation

gfanton
Copy link
Member

@gfanton gfanton commented Dec 20, 2024

This PR significantly improves the testing speed of the gno.land integration tests. This is mainly achieved through two changes:

  • First, I set an interval between empty blocks greater than 0. Having a duration of 0 between empty blocks overwhelmed the CPU and slowed down the tests considerably.
  • Second, I executed the integration tests in parallel. However, due to the current implementation (not sure if it's the VM or something else), it is currently impossible to run two (or more) nodes simultaneously in the same process. To address this, I had to rework the integration tests to spawn nodes in separate processes. I also wanted to maintain a way to run everything in memory and in the same process, so I left the option to run tests sequentially in the same process. This is primarily for debugging purposes, such as using pprof. This also explains why the changes are a bit more complex, as I aimed to maintain both workflows.

As a result of this PR, all the gno.land tests now run on the CI in approximately 3m, compared to about 20m on the master branch.


Once we can run multiple nodes in the same process, we will simplify this part of the codebase. In the meantime, if you are brave enough to try to fix this, you can attempt to run all the tests in memory AND in parallel (and watch them fail).

NO_SEQ_TS=true INMEMORY_TS=true go test -v -count=1 ./pkg/integration/...

@github-actions github-actions bot added 📦 🤖 gnovm Issues or PRs gnovm related 📦 🌐 tendermint v2 Issues or PRs tm2 related 📦 ⛰️ gno.land Issues or PRs gno.land package related labels Dec 20, 2024
@Gno2D2
Copy link
Collaborator

Gno2D2 commented Dec 20, 2024

🛠 PR Checks Summary

All Automated Checks passed. ✅

Manual Checks (for Reviewers):
  • IGNORE the bot requirements for this PR (force green CI check)
Read More

🤖 This bot helps streamline PR reviews by verifying automated checks and providing guidance for contributors and reviewers.

✅ Automated Checks (for Contributors):

🟢 Maintainers must be able to edit this pull request (more info)

☑️ Contributor Actions:
  1. Fix any issues flagged by automated checks.
  2. Follow the Contributor Checklist to ensure your PR is ready for review.
    • Add new tests, or document why they are unnecessary.
    • Provide clear examples/screenshots, if necessary.
    • Update documentation, if required.
    • Ensure no breaking changes, or include BREAKING CHANGE notes.
    • Link related issues/PRs, where applicable.
☑️ Reviewer Actions:
  1. Complete manual checks for the PR, including the guidelines and additional checks if applicable.
📚 Resources:
Debug
Automated Checks
Maintainers must be able to edit this pull request (more info)

If

🟢 Condition met
└── 🟢 The pull request was created from a fork (head branch repo: gfanton/gno)

Then

🟢 Requirement satisfied
└── 🟢 Maintainer can modify this pull request

Manual Checks
**IGNORE** the bot requirements for this PR (force green CI check)

If

🟢 Condition met
└── 🟢 On every pull request

Can be checked by

  • Any user with comment edit permission

Signed-off-by: gfanton <[email protected]>
@gfanton gfanton force-pushed the fix/gnland/test-speed branch from c5c5faa to ddef133 Compare December 20, 2024 23:21
Signed-off-by: gfanton <[email protected]>
Copy link

codecov bot commented Dec 20, 2024

Codecov Report

Attention: Patch coverage is 70.17544% with 289 lines in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
gno.land/pkg/integration/testscript_gnoland.go 68.76% 137 Missing and 32 partials ⚠️
gno.land/pkg/integration/process.go 69.53% 62 Missing and 23 partials ⚠️
gno.land/pkg/integration/pkgloader.go 65.68% 26 Missing and 9 partials ⚠️

📢 Thoughts on this report? Let us know!

@gfanton gfanton changed the title wip: improve gno.land test speed fix(gno.land): improve test speed Dec 29, 2024
@gfanton gfanton marked this pull request as ready for review December 30, 2024 18:03
@zivkovicmilos zivkovicmilos self-requested a review December 31, 2024 14:56
Copy link
Member

@zivkovicmilos zivkovicmilos left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Amazing work for the speed up 👏

I almost can't believe we can run this integration suite so quickly now (< 1 min).
This isn't a step forward, but an olympic leap :)

gno.land/pkg/gnoclient/integration_test.go Show resolved Hide resolved
gno.land/pkg/gnoclient/integration_test.go Show resolved Hide resolved
gno.land/pkg/integration/testscript_seqs.go Outdated Show resolved Hide resolved
gno.land/pkg/integration/pkgloader.go Show resolved Hide resolved
gno.land/pkg/integration/node_testing.go Show resolved Hide resolved
gno.land/pkg/integration/process.go Show resolved Hide resolved
@zivkovicmilos
Copy link
Member

Just a note that running the integration package tests generates a wal directory

cc @gfanton

Signed-off-by: gfanton <[email protected]>
@Kouteki Kouteki added the in focus Core team is prioritizing this work label Jan 6, 2025
@gfanton
Copy link
Member Author

gfanton commented Jan 6, 2025

@zivkovicmilos After some tests, I didn't have the wal file, and I double-checked that the WALFile is correctly disabled by default for integration tests, so I don't think it's related to integration tests.

@gfanton gfanton merged commit 74b077f into gnolang:master Jan 6, 2025
63 checks passed
x1unix pushed a commit to gnostudio/gno that referenced this pull request Jan 7, 2025
albttx pushed a commit that referenced this pull request Jan 10, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
in focus Core team is prioritizing this work 📦 🌐 tendermint v2 Issues or PRs tm2 related 📦 ⛰️ gno.land Issues or PRs gno.land package related 📦 🤖 gnovm Issues or PRs gnovm related
Projects
Development

Successfully merging this pull request may close these issues.

5 participants