[RFE] Ship multiple files with one update, let client pick which files to download

[t-lo]

Current situation

From a Nebraska client perspective, any given version of an application relates to a single file. This implies that updates to an application must consist of a single file only (commonly referred to as the "update payload").

Impact

It's not possible to ship multiple files with the same update, resulting in the need to bundle all files in a single archive even if the client only requires a subset of files to fully apply the update.
More specifically, this prevents the Flatcar Container Linux project from efficiently shipping OEM partition updates ("vendor tools") alongside OS image updates.

Ideal future situation

Instead of a single file per version update providers optionally can provide a list of files. All files are cached by Nebraska (stand-alone use case for e.g. air-gapped environments) and a list of files is sent to clients querying for new versions. Clients will then decide which files to download and to install in order to fully update to the new version.

It is desirable to retain the current behaviour for single-file updates for backwards compatibility.

Additional information

The Flatcar feature is tracked in flatcar/Flatcar#60 .

[joaquimrocha]

We met this morning (@iaguis , myself, @pothos , @illume and @t-lo ), and even though having several packages would cover this as mentioned in this issue description, that would need to be implemented in Nebraska and brings more complexity to it for users.
I thus suggested that we can leverage the newly added omaha protocol extension that includes a metadata field in the Package entity.
This is backward compatible in the way that it shouldn't break existing instances (they will just not see this new field), and can be used to describe any data the updater logic needs.

In this specific case, we could add the OEM partition updates' tarballs by describing their location, name, and hashes.

One downside is that Nebraska will not download these packages like it does to the FCL update payloads, but we've also agreed that this special cases in Nebraska should be deprecated as we turn it more into a generic omaha update manager.