Enable Dependabot service & update dependency versions

[atodorov]

Note: I get an error from go mod vendor, not sure how to solve it.

[atodorov]

@bcl, @ondrejbudai I'm not quite sure how to resolve the errors here. Can you take a look?

[bcl]

You didn't update the vendor files. I've added a makefile target, update-mods to do this in #23

I'm not too sure about this dependabot thing though. Why do we really need it, and is it going to help or just be one more thing to fail and debug on a regular basis?

We should be checking for updated modules, maybe after each release so there is time to deal with any problems they introduce. But I'm worried about this thing becoming a constant stream of unexpected changes that interrupt other work.

[atodorov]

You didn't update the vendor files. I've added a makefile target, update-mods to do this in #23

That's exactly what I did but go mod vendor dies with an error:

$ go get -u ./...
go: gopkg.in/yaml.v2 upgrade => v2.4.0
go: github.com/cpuguy83/go-md2man/v2 upgrade => v2.0.1
go: github.com/russross/blackfriday/v2 upgrade => v2.1.0
go: github.com/BurntSushi/toml upgrade => v0.4.1

$ go mod vendor
github.com/osbuild/weldr-client/cmd/composer-cli/blueprints imports
	github.com/BurntSushi/toml imports
	io/fs: package io/fs is not in GOROOT (/usr/lib/golang/src/io/fs)

For all I can tell this is an upstream issue but I'm not qualified enough to figure out how to resolve it.

I'm not too sure about this dependabot thing though. Why do we really need it

Because of the fact that we're pulling 3rd party libraries directly from the Internet and compiling them into our own products which then get shipped to customers there is some level of risk involved, in particular security risk. Hence the desire to keep all dependencies up to date.

Dependabot is just a nice service which can take care of some of the tasks around updating dependencies.

Whether or not the risks are real enough or something that "may happen but we aren't sure" I can't tell. That's why in addition to updating the dependencies we're also enabling services like Snyk and CoverityScan. In particular Snyk reports 20 high severity issues for dependencies in weldr-client, see https://app.snyk.io/org/osbuild/projects (ping me for invite link or see Slack).

, and is it going to help or just be one more thing to fail and debug on a regular basis?

That we don't know. I've used all of these services for quite some time on other open source repositories and rarely had issues with them. However these other repositories aren't written in Go so mileage may vary. In osbuild-composer we didn't have issues IIRC, you can check https://github.com/osbuild/osbuild-composer/pulls/app%2Fdependabot

We should be checking for updated modules, maybe after each release so there is time to deal with any problems they introduce. But I'm worried about this thing becoming a constant stream of unexpected changes that interrupt other work.

I agree that it's a valid concern. The best strategy IMO is enable, measure & readjust if/when necessary. I also agree that this adds additional pressure on devel engineers to review the updates.

OTOH I don't support updating modules after releases because we could miss an opportunity to resolve critical issues or even situations in which we may need to perform an urgent security release. Of course without actual data we can argue all day about this but the practice of updating your dependencies as soon as possible is relatively widespread IMO. There are many services like Dependabot.

[ondrejbudai]

That's exactly what I did but go mod vendor dies with an error:

Have you tried https://github.com/osbuild/weldr-client/blob/main/tools/prepare-source.sh ?

[atodorov]

That's exactly what I did but go mod vendor dies with an error:

Have you tried https://github.com/osbuild/weldr-client/blob/main/tools/prepare-source.sh ?

I haven't before your comment. However it results in the same error:

Success. You may now run 'go1.14.14'
+ /home/atodorov/go/bin/go1.14.14 mod tidy
github.com/osbuild/weldr-client/cmd/composer-cli/blueprints imports
	github.com/BurntSushi/toml imports
	io/fs: package io/fs is not in GOROOT (/home/atodorov/sdk/go1.14.14/src/io/fs)
github.com/osbuild/weldr-client/cmd/composer-cli/blueprints imports
	github.com/BurntSushi/toml tested by
	github.com/BurntSushi/toml.test imports
	testing/fstest: package testing/fstest is not in GOROOT (/home/atodorov/sdk/go1.14.14/src/testing/fstest)
github.com/osbuild/weldr-client/cmd/composer-cli/blueprints imports
	github.com/BurntSushi/toml tested by
	github.com/BurntSushi/toml.test imports
	github.com/BurntSushi/toml/internal/toml-test imports
	embed: package embed is not in GOROOT (/home/atodorov/sdk/go1.14.14/src/embed)

There are also few more changes to the source files:

diff --git a/go.mod b/go.mod
index 7d25a78..57f9798 100644
--- a/go.mod
+++ b/go.mod
@@ -26,6 +26,7 @@ require (
        github.com/xiang90/probing v0.0.0-20190116061207-43a291ad63a2 // indirect
        github.com/xordataexchange/crypt v0.0.3-0.20170626215501-b2862e3d0a77 // indirect
        go.etcd.io/bbolt v1.3.2 // indirect
+       golang.org/dl v0.0.0-20210831190321-294a5db02644 // indirect
        gopkg.in/check.v1 v1.0.0-20200227125254-8fa46927fb4f // indirect
        gopkg.in/resty.v1 v1.12.0 // indirect
 )
diff --git a/go.sum b/go.sum
index a47905d..a0906a8 100644
--- a/go.sum
+++ b/go.sum
@@ -321,6 +321,8 @@ go.uber.org/multierr v1.1.0/go.mod h1:wR5kodmAFQ0UK8QlbwjlSNy0Z68gJhDJUG5sjR94q/
 go.uber.org/multierr v1.6.0/go.mod h1:cdWPpRnG4AhwMwsgIHip0KRBQjJy5kYEpYjJxpXp9iU=
 go.uber.org/zap v1.10.0/go.mod h1:vwi/ZaCAaUcBkycHslxD9B2zi4UTXhF60s6SWpuDF0Q=
 go.uber.org/zap v1.17.0/go.mod h1:MXVU+bhUf/A7Xi2HNOnopQOrmycQ5Ih87HtOu4q5SSo=
+golang.org/dl v0.0.0-20210831190321-294a5db02644 h1:b5YUr4Rh1r5vQ4ThcVf37NB4wYFgpgeTyIZ1Dzw/yV0=
+golang.org/dl v0.0.0-20210831190321-294a5db02644/go.mod h1:IUMfjQLJQd4UTqG1Z90tenwKoCX93Gn3MAQJMOSBsDQ=
 golang.org/x/crypto v0.0.0-20180904163835-0709b304e793/go.mod h1:6SG95UA2DQfeDnfUPMdvaQW0Q7yPrPDi9nlGo2tz2b4=
 golang.org/x/crypto v0.0.0-20181029021203-45a5f77698d3/go.mod h1:6SG95UA2DQfeDnfUPMdvaQW0Q7yPrPDi9nlGo2tz2b4=
 golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2/go.mod h1:djNgcEr1/C05ACkg1iLfiJU5Ep61QUkGW8qpdssI0+w=

[bcl]

OTOH I don't support updating modules after releases because we could miss an opportunity to resolve critical issues or even situations in which we may need to perform an urgent security release.

I think those are 2 separate situations. As far as I can tell dependabot just updates to the newest, which has a high possibility of breaking things. Doing that kind of update after a release leaves time to adjust for new APIs, etc.

A security issue is different, and should be a different scan and different level of priority (I'll take a look at snyk.io that you mentioned) and should be done on a regular basis with a release following soon after.

It's odd that I'm not seeing the same errors you are. One difference I see is that on my F34 system I have go 1.15.14 instead of go 1.14 -- I'll try doing a similar PR from my system with new vendor files and see if the tests pass.

[bcl]

In #24 I bumped the go version to 1.15 (matching what osbuild-composer is targeting) and reran prepare-source and go get and it looks ok. I was able to duplicate your error locally when I tried running prepare-source with 1.14 twice so I think it was a mismatch of the go version.

I also changed the dependabot pr count to 1 instead of 5.

ETA: I also found that most of the extra packages that showed up in go.mod were due to cobra, but they don't end up in the vendor directory since weldr-client doesn't actually use them. See spf13/cobra#1435