Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support to download from a remote registry #27

Merged

Conversation

tkashem
Copy link
Contributor

@tkashem tkashem commented Feb 12, 2019

Add a new binary 'appregistry-server' that can be used to download operator manifest(s) from a remote app registry and generate a sqlite database to be served using grpc.

https://jira.coreos.com/browse/MKTPLC-108
https://jira.coreos.com/browse/MKTPLC-240

TBD:

  • e2e test(s) for the binary
  • I am running into error while building upstream docker files for both configmap-server and the appregistry-server binaries. This needs to be fixed in a separate PR later.

Unverified

This user has not yet uploaded their public signing key.
The goal is to reuse 'conFigMap loader' to load operator manifest(s)
downloaded from a remote registry (ie. quay.io) into a sqlite
database.

- Refactor ConfigMapLoader to use map[string]string instead of a
  `configMap` object to refer to data section.
- Parameterize the logger so that we can pass in `WithFields` logger
  specific to appregistry or ConfigMap.
@openshift-ci-robot openshift-ci-robot added the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Feb 12, 2019
@openshift-ci-robot openshift-ci-robot added the size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. label Feb 12, 2019
@tkashem tkashem changed the title [WIP] Add support to download from a remote registry Add support to download from a remote registry Feb 13, 2019
@openshift-ci-robot openshift-ci-robot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Feb 13, 2019

appr "github.com/operator-framework/go-appr/appregistry"
appr_blobs "github.com/operator-framework/go-appr/appregistry/blobs"
appr_package "github.com/operator-framework/go-appr/appregistry/package_appr"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Removed underscores

)

const (
mediaType string = "helm"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: no need for explicit typing here

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Removed explicit typing

Unverified

This user has not yet uploaded their public signing key.
apprclient package is an adapter that can talk to appregistry server.
The new binary we are adding will use this go package to download
operator manifest(s) from remote app registry.

Unverified

This user has not yet uploaded their public signing key.
Add a new binary 'appregistry-server' that can be used to download
operator manifest(s) from a remote app registry and generate a
sqlite database to be served using grpc.

Unverified

This user has not yet uploaded their public signing key.
- Add appregistry-server binary into image for OpenShift.
- Add a new docker file for upstream
@tkashem
Copy link
Contributor Author

tkashem commented Feb 14, 2019

@njhale Addressed the comments you posted, let me know if you have more questions.

@tkashem
Copy link
Contributor Author

tkashem commented Feb 14, 2019

/test e2e-aws

Copy link
Member

@njhale njhale left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for adding this. Great Job! I think in the next sprint we'll want to decouple appregistry-server from marketplace CRDs though.

/approve
/lgtm

"github.com/go-openapi/runtime"
httptransport "github.com/go-openapi/runtime/client"
"github.com/go-openapi/strfmt"
apprclient "github.com/operator-framework/go-appr/appregistry"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should the import be aliased with the same name as the enclosing package?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

you are right, didn't realize this at all. I will fix this in a separate PR next sprint.

Decode(encoded []byte) ([]byte, error)
}

type blobDecoderImpl struct {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just for fun, the other way you could do this is by declaring a function type that implements blobDecoder:

type DecoderFunc func(encoded []byte) ([]byte, error)

func (d DecoderFunc) Decode(encoded []byte) ([]byte, error) {
    return d.Decode(encoded)
}

func Decode(encoded []byte) ([]byte, error) {
    ....
}

Then the function Decode of type DecoderFunc implements blobDecoder

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yeah, I like it, makes it concise.

Packages []string
}

func (i *Input) PackagesToMap() map[string]bool {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The native golang set is map[<type>]struct{}.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was thinking of using truct{}, but then came across this in effective go -
A set can be implemented as a map with value type bool. Set the map entry to true to put the value in the set, and then test it by simple indexing

attended := map[string]bool{
    "Ann": true,
    "Joe": true,
    ...
}

if attended[person] { // will be false if person is not in the map
    fmt.Println(person, "was at the meeting")
}

@openshift-ci-robot openshift-ci-robot added the lgtm Indicates that a PR is ready to be merged. label Feb 14, 2019
@openshift-ci-robot
Copy link

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: njhale, tkashem

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@openshift-ci-robot openshift-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Feb 14, 2019
@tkashem
Copy link
Contributor Author

tkashem commented Feb 14, 2019

/test e2e-aws

@openshift-merge-robot openshift-merge-robot merged commit 83a23fa into operator-framework:master Feb 14, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. lgtm Indicates that a PR is ready to be merged. size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants