Support bufferizable in okl

[howin98]

No description provided.

[mosout]

TensorManager和TmpBufferManager这两个类型包括其内置类的的写法我感觉有点混乱，我建议把内置类都独立出来，然后把声明都写到同一个头文件里面（或者把这两个类放一个头文件，内置类们放一个头文件，没必要把每个类都放一个头文件）

这几个类型也是有交叉的，有些操作不需要写两遍，建议抽出基类使用继承或者使用Mixin。

实现统一放到cpp里，尽量别在头文件里搞。

[howin98]

TensorManager和TmpBufferManager这两个类型包括其内置类的的写法我感觉有点混乱，我建议把内置类都独立出来，然后把声明都写到同一个头文件里面（或者把这两个类放一个头文件，内置类们放一个头文件，没必要把每个类都放一个头文件）

这几个类型也是有交叉的，有些操作不需要写两遍，建议抽出基类使用继承或者使用Mixin。

实现统一放到cpp里，尽量别在头文件里搞。
嗯嗯。其实只有tmp buffer manager，tensor manager是之前tmp buffer manager的改名，但是防止改名对定义产生误解，所以又改回去了，但当时应该是忘记删除tensormanger的头文件了。tmp buffer manger的内置类主要只服务于tmp buffer manager的内部，不对外暴露。

[github-actions]

Code got formatted by CI. Please request CI again if you still want to have this PR merged. If the PR is from a forked repo, please download the patch files from the GitHub Actions web page and apply them locally.

[github-actions]

CI failed when running job: Build cpu. PR label automerge has been removed

[github-actions]

Speed stats:

GPU Name: GeForce GTX 1080 

❌ OneFlow resnet50 time: 141.0ms (= 14103.0ms / 100, input_shape=[16, 3, 224, 224])
PyTorch resnet50 time: 141.5ms (= 14154.6ms / 100, input_shape=[16, 3, 224, 224])
❌ Relative speed: 1.00 (= 141.5ms / 141.0ms)

OneFlow resnet50 time: 81.4ms (= 8141.1ms / 100, input_shape=[8, 3, 224, 224])
PyTorch resnet50 time: 87.3ms (= 8729.3ms / 100, input_shape=[8, 3, 224, 224])
✔️ Relative speed: 1.07 (= 87.3ms / 81.4ms)

OneFlow resnet50 time: 50.6ms (= 10116.0ms / 200, input_shape=[4, 3, 224, 224])
PyTorch resnet50 time: 59.7ms (= 11931.6ms / 200, input_shape=[4, 3, 224, 224])
✔️ Relative speed: 1.18 (= 59.7ms / 50.6ms)

OneFlow resnet50 time: 33.4ms (= 6671.2ms / 200, input_shape=[2, 3, 224, 224])
PyTorch resnet50 time: 43.4ms (= 8687.3ms / 200, input_shape=[2, 3, 224, 224])
✔️ Relative speed: 1.30 (= 43.4ms / 33.4ms)

OneFlow resnet50 time: 25.1ms (= 5010.6ms / 200, input_shape=[1, 3, 224, 224])
PyTorch resnet50 time: 38.5ms (= 7691.0ms / 200, input_shape=[1, 3, 224, 224])
✔️ Relative speed: 1.53 (= 38.5ms / 25.1ms)

OneFlow swin dataloader time: 0.234s (= 46.763s / 200, num_workers=1)
PyTorch swin dataloader time: 0.153s (= 30.626s / 200, num_workers=1)
Relative speed: 0.655 (= 0.153s / 0.234s)

OneFlow swin dataloader time: 0.069s (= 13.826s / 200, num_workers=4)
PyTorch swin dataloader time: 0.043s (= 8.589s / 200, num_workers=4)
Relative speed: 0.621 (= 0.043s / 0.069s)

OneFlow swin dataloader time: 0.041s (= 8.153s / 200, num_workers=8)
PyTorch swin dataloader time: 0.023s (= 4.528s / 200, num_workers=8)
Relative speed: 0.555 (= 0.023s / 0.041s)

❌ OneFlow resnet50 time: 152.9ms (= 15286.7ms / 100, input_shape=[16, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 163.3ms (= 16327.2ms / 100, input_shape=[16, 3, 224, 224], ddp, world size=2)
❌ Relative speed: 1.07 (= 163.3ms / 152.9ms)

OneFlow resnet50 time: 92.4ms (= 9239.0ms / 100, input_shape=[8, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 103.2ms (= 10320.9ms / 100, input_shape=[8, 3, 224, 224], ddp, world size=2)
✔️ Relative speed: 1.12 (= 103.2ms / 92.4ms)

OneFlow resnet50 time: 60.6ms (= 12121.6ms / 200, input_shape=[4, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 78.4ms (= 15676.1ms / 200, input_shape=[4, 3, 224, 224], ddp, world size=2)
✔️ Relative speed: 1.29 (= 78.4ms / 60.6ms)

OneFlow resnet50 time: 42.2ms (= 8445.2ms / 200, input_shape=[2, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 69.4ms (= 13883.6ms / 200, input_shape=[2, 3, 224, 224], ddp, world size=2)
✔️ Relative speed: 1.64 (= 69.4ms / 42.2ms)

OneFlow resnet50 time: 36.2ms (= 7231.2ms / 200, input_shape=[1, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 68.5ms (= 13704.1ms / 200, input_shape=[1, 3, 224, 224], ddp, world size=2)
✔️ Relative speed: 1.90 (= 68.5ms / 36.2ms)

[github-actions]

View latest API docs preview at: https://staging.oneflow.info/docs/Oneflow-Inc/oneflow/pr/9787/

[github-actions]

Speed stats:

GPU Name: GeForce GTX 1080 

❌ OneFlow resnet50 time: 141.1ms (= 14110.7ms / 100, input_shape=[16, 3, 224, 224])
PyTorch resnet50 time: 142.9ms (= 14286.9ms / 100, input_shape=[16, 3, 224, 224])
❌ Relative speed: 1.01 (= 142.9ms / 141.1ms)

OneFlow resnet50 time: 81.1ms (= 8114.5ms / 100, input_shape=[8, 3, 224, 224])
PyTorch resnet50 time: 85.1ms (= 8505.3ms / 100, input_shape=[8, 3, 224, 224])
✔️ Relative speed: 1.05 (= 85.1ms / 81.1ms)

OneFlow resnet50 time: 50.9ms (= 10185.7ms / 200, input_shape=[4, 3, 224, 224])
PyTorch resnet50 time: 54.8ms (= 10958.1ms / 200, input_shape=[4, 3, 224, 224])
✔️ Relative speed: 1.08 (= 54.8ms / 50.9ms)

OneFlow resnet50 time: 33.6ms (= 6722.2ms / 200, input_shape=[2, 3, 224, 224])
PyTorch resnet50 time: 41.8ms (= 8360.5ms / 200, input_shape=[2, 3, 224, 224])
✔️ Relative speed: 1.24 (= 41.8ms / 33.6ms)

OneFlow resnet50 time: 25.2ms (= 5040.4ms / 200, input_shape=[1, 3, 224, 224])
PyTorch resnet50 time: 36.0ms (= 7207.2ms / 200, input_shape=[1, 3, 224, 224])
✔️ Relative speed: 1.43 (= 36.0ms / 25.2ms)

OneFlow swin dataloader time: 0.245s (= 49.093s / 200, num_workers=1)
PyTorch swin dataloader time: 0.148s (= 29.700s / 200, num_workers=1)
Relative speed: 0.605 (= 0.148s / 0.245s)

OneFlow swin dataloader time: 0.067s (= 13.342s / 200, num_workers=4)
PyTorch swin dataloader time: 0.042s (= 8.336s / 200, num_workers=4)
Relative speed: 0.625 (= 0.042s / 0.067s)

OneFlow swin dataloader time: 0.044s (= 8.799s / 200, num_workers=8)
PyTorch swin dataloader time: 0.022s (= 4.487s / 200, num_workers=8)
Relative speed: 0.510 (= 0.022s / 0.044s)

❌ OneFlow resnet50 time: 152.7ms (= 15271.7ms / 100, input_shape=[16, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 163.2ms (= 16324.8ms / 100, input_shape=[16, 3, 224, 224], ddp, world size=2)
❌ Relative speed: 1.07 (= 163.2ms / 152.7ms)

OneFlow resnet50 time: 92.0ms (= 9196.2ms / 100, input_shape=[8, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 104.2ms (= 10419.9ms / 100, input_shape=[8, 3, 224, 224], ddp, world size=2)
✔️ Relative speed: 1.13 (= 104.2ms / 92.0ms)

OneFlow resnet50 time: 60.2ms (= 12044.7ms / 200, input_shape=[4, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 77.3ms (= 15467.4ms / 200, input_shape=[4, 3, 224, 224], ddp, world size=2)
✔️ Relative speed: 1.28 (= 77.3ms / 60.2ms)

OneFlow resnet50 time: 42.2ms (= 8435.6ms / 200, input_shape=[2, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 68.6ms (= 13723.2ms / 200, input_shape=[2, 3, 224, 224], ddp, world size=2)
✔️ Relative speed: 1.63 (= 68.6ms / 42.2ms)

OneFlow resnet50 time: 36.4ms (= 7289.9ms / 200, input_shape=[1, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 67.5ms (= 13496.1ms / 200, input_shape=[1, 3, 224, 224], ddp, world size=2)
✔️ Relative speed: 1.85 (= 67.5ms / 36.4ms)

[github-actions]

View latest API docs preview at: https://staging.oneflow.info/docs/Oneflow-Inc/oneflow/pr/9787/