[feature request] make runtime advantages support in CRI managers for Kubernetes

[allencloud]

Ⅰ. Issue Description

Currently, PouchContainer has lots of advantages over other container runtimes, such as:

• resource review isolation via lxcfs,
• diskquota for multi-directory in container,
• runtime choosing for runc-based container or runv-based containers,
• rich container mode
• and so on

While actually in kubernetes, there is no related API to support these feature.

While for these extended features in Kubernetes, Kubernetes has a hiden way to support this: make user-defined parameters in annotations field in pod's definition.

However, in order to implement this, I think we have to support these parameters in annotations in CRI manager(CRI implementation of PouchContainer).

When CRI manager deals the annotation details, it could pass these parameters to container manager, and container manager definitely implement these features very well.

So, we need to accomplish the following things:

• define the specific naming in annotations for each feature;
• implement the transformation in CRI manager and pass them to container manager.

[pouchrobot]

Thanks for your report, @allencloud
😱 This is a priority/P1 issue which is highest.
Seems to be severe enough.
ping @alibaba/pouch , PTAL.

[allencloud]

also cc @fanux

[fanux]

I think this will be useful, for example support the runtime (runv or runc) in kubernetes annotations.

[allencloud]

I think this will be useful, for example support the runtime (runv or runc) in kubernetes annotations.

We could implement this ASAP. and runtime choosing could be the highest priority. @fanux

[YaoZengzeng]

@allencloud Please define the name for specific feature ASAP. It's easy for us to implement it in CRI manager 😄

[allencloud]

If the annotation has a type of map[string]string, we have the following naming definitions:

runtime type

{"runtime":"kata"} for choosing katacontainer as the runtime for a container. Currently we just support one pod on katacontainer. Does this meet your demand? @fanux

{"runtime":"runc"} for normal runc container creation. We could support one pod multi-containers which are all created via runc

resource view isolation via lxcfs

To be compatible with pouchd's API, we could have the following definition:

{"enablelxcfs": "true"} means we need to enable lxcfs for a container; obviously, enablelxcfs and runtime are exclusive when runtime is runv.

diskquota type

I think this is quite easy, we should just use

{"diskquota": "/=10gb"}: key is diskquota and value should be dir=size such as /=10gb. And this is quite compatible with pouchd's API.

Rich container mode

to be decided this mode's user case.

/cc @YaoZengzeng @starnop

[fanux]

How about add a runtime {"runtime":"runv"}? One pod can meet demand at present. @allencloud

[YaoZengzeng]

@allencloud The name should be more kubernetes native, like {"io.kubernetes.runtime":"runv"} 😄

[YaoZengzeng]

cri-containerd has already supported hypervisor-based runtime, and we will follow this approach as well.

containerd/cri#546