feat(gno test): implement failfast flag #3866

Villaquiranm · 2025-03-02T08:04:43Z

Overview

This PR introduces support for the -failfast flag in gno test. When enabled, test execution stops as soon as any test fails. However, tests that were already running in parallel will be allowed to complete.

Motivation

In large-scale test executions, test failures can get lost in a flood of output, making it difficult to identify the faulty test. By using -failfast, we can quickly pinpoint failing tests and iterate faster, improving debugging efficiency.

Changes

Implemented -failfast behavior in test execution.

Sources

From Go 1.10 documentation:

The new go test -failfast flag disables running additional tests after any test fails. Note that tests running in parallel with the failing test are allowed to complete.

Gno2D2 · 2025-03-02T08:05:37Z

🛠 PR Checks Summary

All Automated Checks passed. ✅

Manual Checks (for Reviewers):

IGNORE the bot requirements for this PR (force green CI check)

✅ Automated Checks (for Contributors):

🟢 Maintainers must be able to edit this pull request (more info)
🟢 Pending initial approval by a review team member, or review from tech-staff

☑️ Contributor Actions:

Fix any issues flagged by automated checks.
Follow the Contributor Checklist to ensure your PR is ready for review.
- Add new tests, or document why they are unnecessary.
- Provide clear examples/screenshots, if necessary.
- Update documentation, if required.
- Ensure no breaking changes, or include BREAKING CHANGE notes.
- Link related issues/PRs, where applicable.

☑️ Reviewer Actions:

Complete manual checks for the PR, including the guidelines and additional checks if applicable.

📚 Resources:

Debug

Automated Checks

Maintainers must be able to edit this pull request (more info)

If

🟢 Condition met
└── 🟢 And
    ├── 🟢 The base branch matches this pattern: ^master$
    └── 🟢 The pull request was created from a fork (head branch repo: Villaquiranm/gno)

Then

🟢 Requirement satisfied
└── 🟢 Maintainer can modify this pull request

Pending initial approval by a review team member, or review from tech-staff

If

🟢 Condition met
└── 🟢 And
    ├── 🟢 The base branch matches this pattern: ^master$
    └── 🟢 Not (🔴 Pull request author is a member of the team: tech-staff)

Then

🟢 Requirement satisfied
└── 🟢 If
    ├── 🟢 Condition
    │   └── 🟢 Or
    │       ├── 🟢 At least 1 user(s) of the organization reviewed the pull request (with state "APPROVED")
    │       ├── 🟢 At least 1 user(s) of the team tech-staff reviewed pull request
    │       └── 🔴 This pull request is a draft
    └── 🟢 Then
        └── 🟢 Not (🔴 This label is applied to pull request: review/triage-pending)

Manual Checks

**IGNORE** the bot requirements for this PR (force green CI check)
If
🟢 Condition met
└── 🟢 On every pull request
Can be checked by

Any user with comment edit permission

codecov · 2025-03-02T08:09:10Z

Codecov Report

All modified and coverable lines are covered by tests ✅

📢 Thoughts on this report? Let us know!

notJoon

It will be helpful for gno-base project CI pipeline. LGTM 👍

remove: review/triage-pending flag

thehowl · 2025-03-07T13:36:07Z

gnovm/cmd/gno/test.go

+			if cfg.failfast {
+				io.ErrPrintfln("FAIL")
+				return fmt.Errorf("FAIL: %d build errors, %d test errors", buildErrCount, testErrCount)
+			}


-failfast does not fail just on the first failing package, but on the first failing test, so this implementation is incomplete

Fair enough, good catch :)
fixed here I think 2be439e

thehowl · 2025-03-10T15:31:38Z

gnovm/pkg/test/test.go

+			if opts.FailfastFlag {
+				return errs
+			}


Sorry, needs another round...

This won't stop execution for a subtest, ie. if I run a test using t.Run inside a test.

What do you think of 79fa228 ?

thehowl · 2025-03-10T15:31:56Z

gnovm/cmd/gno/testdata/test/flag_fail-fast.txtar

Also this should be named flag_failfast

fixed here 747e41c

examples/Makefile

gnovm/cmd/gno/test.go

moul · 2025-03-11T17:00:31Z

gnovm/cmd/gno/test.go

@@ -246,6 +255,10 @@ func execTest(cfg *testCfg, args []string, io commands.IO) error {
 			io.ErrPrintfln("FAIL    %s \t%s", pkg.Dir, dstr)
 			io.ErrPrintfln("FAIL")
 			testErrCount++
+			if cfg.failfast {
+				io.ErrPrintfln("FAIL")


there is already a printed FAIL two lines ago.

Hey thanks a lot for your review :)
I think there was some mix with the files diffs but I'm doing the same behavior that on lines
https://github.com/gnolang/gno/pull/3866/files#diff-2e39c5441249018f694183a5710aa7adcad09ec17e870507d20a42184b07e1a7R278-R281

Co-authored-by: Manfred Touron <[email protected]>

Villaquiranm · 2025-03-11T19:01:21Z

Sorry rebased because they were several files not belonging to this PR

github-actions bot assigned Villaquiranm Mar 2, 2025

github-actions bot added 🧾 package/realm Tag used for new Realms or Packages. 📦 🤖 gnovm Issues or PRs gnovm related labels Mar 2, 2025

Gno2D2 requested a review from a team March 2, 2025 08:05

Villaquiranm marked this pull request as ready for review March 6, 2025 17:11

Gno2D2 added the review/triage-pending PRs opened by external contributors that are waiting for the 1st review label Mar 6, 2025

notJoon approved these changes Mar 7, 2025

View reviewed changes

notJoon removed the review/triage-pending PRs opened by external contributors that are waiting for the 1st review label Mar 7, 2025

thehowl reviewed Mar 7, 2025

View reviewed changes

Kouteki mentioned this pull request Mar 10, 2025

Minutes: Core Staff Weekly Syncs [every Monday] gnolang/meetings#36

Open

thehowl reviewed Mar 10, 2025

View reviewed changes

zxxma mentioned this pull request Mar 11, 2025

Teritori's Builder Journey gnolang/hackerspace#7

Open

moul reviewed Mar 11, 2025

View reviewed changes

examples/Makefile Outdated Show resolved Hide resolved

moul reviewed Mar 11, 2025

View reviewed changes

gnovm/cmd/gno/test.go Outdated Show resolved Hide resolved

moul reviewed Mar 11, 2025

View reviewed changes

moul approved these changes Mar 11, 2025

View reviewed changes

github-actions bot added the 📦 ⛰️ gno.land Issues or PRs gno.land package related label Mar 11, 2025

Your Name and others added 7 commits March 11, 2025 19:58

feat: add failfast flag to gno test command

7dc977a

add test

0bed5f4

complete failfast implementation

8251e8b

implement failfast on t.Run

c4d8487

Update examples/Makefile

a9c65cf

Co-authored-by: Manfred Touron <[email protected]>

Update gnovm/cmd/gno/test.go

3d88daf

Co-authored-by: Manfred Touron <[email protected]>

rename test file

50da6e8

Villaquiranm force-pushed the feat/test-failfast branch from 747e41c to 50da6e8 Compare March 11, 2025 19:00

Merge branch 'master' into feat/test-failfast

fb7d40e

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat(gno test): implement failfast flag #3866

feat(gno test): implement failfast flag #3866

Villaquiranm commented Mar 2, 2025 •

edited

Loading

Gno2D2 commented Mar 2, 2025 •

edited

Loading

✅ Automated Checks (for Contributors):

☑️ Contributor Actions:

☑️ Reviewer Actions:

📚 Resources:

If

Then

If

Then

If

Can be checked by

codecov bot commented Mar 2, 2025 •

edited

Loading

notJoon left a comment

thehowl Mar 7, 2025

Villaquiranm Mar 8, 2025

thehowl Mar 10, 2025

Villaquiranm Mar 11, 2025

thehowl Mar 10, 2025

Villaquiranm Mar 11, 2025

moul Mar 11, 2025

Villaquiranm Mar 11, 2025

Villaquiranm commented Mar 11, 2025

feat(gno test): implement failfast flag #3866

Are you sure you want to change the base?

feat(gno test): implement failfast flag #3866

Conversation

Villaquiranm commented Mar 2, 2025 • edited Loading

Overview

Motivation

Changes

Sources

Gno2D2 commented Mar 2, 2025 • edited Loading

🛠 PR Checks Summary

Manual Checks (for Reviewers):

✅ Automated Checks (for Contributors):

☑️ Contributor Actions:

☑️ Reviewer Actions:

📚 Resources:

If

Then

If

Then

If

Can be checked by

codecov bot commented Mar 2, 2025 • edited Loading

Codecov Report

notJoon left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Villaquiranm commented Mar 11, 2025

Villaquiranm commented Mar 2, 2025 •

edited

Loading

Gno2D2 commented Mar 2, 2025 •

edited

Loading

codecov bot commented Mar 2, 2025 •

edited

Loading