Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Introduce RetryAttribute for test methods #4586

Open
wants to merge 9 commits into
base: main
Choose a base branch
from

Conversation

Youssef1313
Copy link
Member

Fixes #3161

@nohwnd
Copy link
Member

nohwnd commented Jan 9, 2025

FYI From previous talks with Playwright where they have good retries, that imho were attempted to be duplicated in mstest.playwright, Max also mentioned that they wanted:

  • max retries to be configurable via config
  • see what max retries value is in assembly initialize (might be specific to them because they are in position of test framework in vstest, so that story might not be relevant here, where they have way more power via extensions easily)
  • have good reporting of the retries, to be able to show retries that were not passing

@Youssef1313
Copy link
Member Author

Youssef1313 commented Jan 9, 2025

max retries to be configurable via config

Not sure what shape would you be expecting. Currently, every application of the attribute is specifying the number of retries


see what max retries value is in assembly initialize (might be specific to them because they are in position of test framework in vstest, so that story might not be relevant here, where they have way more power via extensions easily)

This is currently info specific to the test method itself and is irrelevant to assembly initialize. I think you are talking of a completely different design that allows a "global" retry count, not a Retry attribute on test methods which I'm implementing here.


have good reporting of the retries, to be able to show retries that were not passing

We are going to have a design that allows that, but that is unlikely to be implemented in this PR though, unfortunately. We need to know better how to communicate that information to tools that will make use of it.

@Youssef1313 Youssef1313 force-pushed the retry-attr branch 7 times, most recently from aeccb48 to 077b03b Compare January 14, 2025 04:22
src/Adapter/MSTest.TestAdapter/Resources/Resource.resx Outdated Show resolved Hide resolved
/// The other results are currently not used, but may be used in the future for tooling to show the
/// state of the failed attempts.
/// </returns>
protected internal override async Task<RetryResult> ExecuteAsync(RetryContext retryContext)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shall we use an interface instead to give us more freedom in future without breaking change? I don't know if that's support useful or not given roslyn mainly goes with struct.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Develop Retry Attribute for TestMethod
4 participants