x/tools/go/analysis/analysistest: analysistest.RunWithSuggestedFixes doesn't fail when a golden file is provided without suggested fixes

[ldez]

Go version

go version go1.23.4 linux/amd64

Output of go env in your module/workspace:

GO111MODULE='on'
GOARCH='amd64'
GOBIN=''
GOCACHE='/home/ldez/.cache/go-build'
GOENV='/home/ldez/.config/go/env'
GOEXE=''
GOEXPERIMENT=''
GOFLAGS=''
GOHOSTARCH='amd64'
GOHOSTOS='linux'
GOINSECURE=''
GOMODCACHE='/home/ldez/sources/go/pkg/mod'
GONOPROXY=''
GONOSUMDB=''
GOOS='linux'
GOPATH='/home/ldez/sources/go'
GOPRIVATE=''
GOPROXY='https://proxy.golang.org,direct'
GOROOT='/usr/lib/go'
GOSUMDB='sum.golang.org'
GOTMPDIR=''
GOTOOLCHAIN='auto'
GOTOOLDIR='/usr/lib/go/pkg/tool/linux_amd64'
GOVCS=''
GOVERSION='go1.23.4'
GODEBUG=''
GOTELEMETRY='on'
GOTELEMETRYDIR='/home/ldez/.config/go/telemetry'
GCCGO='gccgo'
GOAMD64='v1'
AR='ar'
CC='gcc'
CXX='g++'
CGO_ENABLED='1'
GOMOD='/home/ldez/sources/golangci/sandbox/go.mod'
GOWORK=''
CGO_CFLAGS='-O2 -g'
CGO_CPPFLAGS=''
CGO_CXXFLAGS='-O2 -g'
CGO_FFLAGS='-O2 -g'
CGO_LDFLAGS='-O2 -g'
PKG_CONFIG='pkg-config'
GOGCCFLAGS='-fPIC -m64 -pthread -Wl,--no-gc-sections -fmessage-length=0 -ffile-prefix-map=/tmp/go-build2319871150=/tmp/go-build -gno-record-gcc-switches'

What did you do?

In the following example, the suggested fixes are not created for all reports (this is expected).

The golden files and the sources are different.

The Analyzer implementation should be considered as bugged: i == 0 should be i == 1.

When you run the test (analysistest.RunWithSuggestedFixes), it doesn't fail because the golden file is only checked if there are suggested fixes.

This hides the bug inside the Analyzer implementation.

.
├── go.mod
├── go.sum
├── main.go
├── main_test.go
└── testdata
    └── src
        └── foo
            ├── foo.go
            └── foo.go.golden

main.go

package main

import (
	"strings"

	"golang.org/x/tools/go/analysis"
	"golang.org/x/tools/go/analysis/singlechecker"
)

var Analyzer = &analysis.Analyzer{
	Name: "example",
	Doc:  "foo",
	Run:  run,
}

func run(pass *analysis.Pass) (interface{}, error) {
	for _, file := range pass.Files {
		for _, group := range file.Comments {
			for i, comment := range group.List {
				originalText := comment.Text
				newText := strings.ReplaceAll(originalText, "foo", "bar")

				if originalText == newText {
					continue
				}

				diagnostic := analysis.Diagnostic{
					Pos:     comment.Pos(),
					Message: "Word detected",
				}

				if i == 0 {
					diagnostic.SuggestedFixes = append(diagnostic.SuggestedFixes, analysis.SuggestedFix{
						Message: "Replace 'foo' with 'bar'",
						TextEdits: []analysis.TextEdit{{
							Pos:     comment.Pos(),
							End:     comment.End(),
							NewText: []byte(newText),
						}},
					})
				}

				pass.Report(diagnostic)
			}
		}
	}

	return nil, nil
}

func main() {
	singlechecker.Main(Analyzer)
}

main_test.go

package main

import (
	"testing"

	"golang.org/x/tools/go/analysis/analysistest"
)

func TestAnalyzer(t *testing.T) {
	analysistest.RunWithSuggestedFixes(t, analysistest.TestData(), Analyzer, "foo")
}

testdata/src/foo/foo.go

package foo

import "fmt"

// want +3 "Word detected"

// This is an example.
// Here is a word foo.
func _() {
	fmt.Println("foo")
}

testdata/src/foo/foo.go.golden

package foo

import "fmt"

// want +3 "Word detected"

// This is an example.
// Here is a word bar.
func _() {
	fmt.Println("foo")
}

What did you see happen?

The test doesn't fail.

What did you expect to see?

A failure of the test.

[gabyhelp]

Related Issues

• x/tools/go/analysis/analysistest: output and golden file differ, but RunWithSuggestedFixes does not report failure #47118 (closed)
• x/tools/go/analysis/analysistest: RunWithSuggestedFixes should check against golden file even if the Go file has no suggested fixes #47128 (closed)
• x/tools/go/analysis/analysistest: Functionality for verifying SuggestedFixes and TextEdits #38044 (closed)
• x/tools/go/analysis: Generated test code passed in `pass.Files` #64436 (closed)
• x/tools/go/analysis/analysistest: Run will pass generated test files to analyzer #41771 (closed)
• x/tools/go/analysis: The first time singlechecker -fix parameter is executed, it does not fix the problem in the _test.go file #54740 (closed)
• x/tools/go/analysis/analysistest: buggy `// want` behavior with block comments #65003 (closed)
• x/tools/go/analysis/analysistest: facts are sometimes not inherited from dependent analyzers #59164 (closed)
• x/tools/go/analysis/internal/checker: if `offsetedit`'s start is 0, it can't be apply #54774 (closed)
• x/tools/go/analysis/analysistest: analysistest.Run failing with modules #46041

_{(Emoji vote if this was helpful or unhelpful; more detailed feedback welcome in this discussion.)}

[ldez]

Duplicate of #47128