Avoid long-running tests in GitHub Actions

[rnugent3]

Hi All. We have some tests among fda-model-tests that take several minutes to run. I don't want to lose these tests, but I am sure it has annoyed us all to wait on them.

What's more - I plan to write many more of these tests to have representation of every kind of study configuration (e.g. lp3 or graphical, with or without inflow-outflow, fragility or no fragility, so on and so forth).

What can we do to keep these tests running but not hold up our builds? Some ideas include marking the tests to not run in GitHub actions or moving the tests to a separate repository.

Thoughts?

[Brennan1994]

I can update our scripts to be selective of tests. These resources are just from an initial investigation, mostly saved here for my benefit. I'd like to separate integration tests (which I think is the proper term for what you're proposing), and unit tests. I'll use attributes to assign the catagory of the tests at the class level.

We'll also have to decide when we want to run the integration tests. My initial proposal would be to continue running unit tests automatically on a pull request or merge into main.

I can also make a new workflow that will run every Friday on main, running just the integration tests.
https://docs.microsoft.com/en-us/dotnet/core/tools/dotnet-test#filter-option-details
https://harrybellamy.com/posts/adding-test-categories-with-xunit/

I'm open to other suggestions, but I think that would be a reasonable place to start.

[rnugent3]

I agree every Friday seems reasonable, you are right, these longer running tests are definitely integration tests. How would you like to mark tests as integration to be run once a week?

…

On Jan 18, 2022, at 4:59 PM, Brennan Beam ***@***.***> wrote:  I can update our scripts to be selective of tests. These resources are just from an initial investigation, mostly saved here for my benefit. I'd like to separate integration tests (which I think is the proper term for what you're proposing), and unit tests. I'll use attributes to assign the catagory of the tests at the class level. We'll also have to decide when we want to run the integration tests. My initial proposal would be to continue running unit tests automatically on a pull request or merge into main. I can also make a new workflow that will run every Friday on main, running just the integration tests. https://docs.microsoft.com/en-us/dotnet/core/tools/dotnet-test#filter-option-details https://harrybellamy.com/posts/adding-test-categories-with-xunit/ I'm open to other suggestions, but I think that would be a reasonable place to start. — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you authored the thread.

[Brennan1994]

I think we follow the pattern on the second link up there. At the top of the class, just above the class declaration:

[Trait("Category", "Integration")]