Skip to content

Commit 5a1bd04

Browse files
dustymabecgwalters
authored andcommittedOct 28, 2023
add proof of concept for building with osbuild
This is proof of concept code with many things hardcoded in the coreos.osbuild.mpp.yaml that need to become more dynamically defined. To use this you can set the COSA_USE_OSBUILD env var to have a value. COSA_USE_OSBUILD=1 should work just fine.
1 parent 138e2df commit 5a1bd04

9 files changed

+422
-13
lines changed
 

‎Dockerfile

+1
Original file line numberDiff line numberDiff line change
@@ -23,6 +23,7 @@ COPY ./ /root/containerbuild/
2323
RUN ./build.sh write_archive_info
2424
RUN ./build.sh make_and_makeinstall
2525
RUN ./build.sh configure_user
26+
RUN ./build.sh patch_osbuild
2627

2728
# clean up scripts (it will get cached in layers, but oh well)
2829
WORKDIR /srv/

‎build.sh

+8
Original file line numberDiff line numberDiff line change
@@ -22,6 +22,7 @@ if [ $# -gt 1 ]; then
2222
echo " configure_yum_repos"
2323
echo " install_rpms"
2424
echo " make_and_makeinstall"
25+
echo " patch_osbuild"
2526
exit 1
2627
fi
2728

@@ -168,6 +169,12 @@ write_archive_info() {
168169
prepare_git_artifacts "${srcdir}" /cosa/coreos-assembler-git.json /cosa/coreos-assembler-git.tar.gz
169170
}
170171

172+
patch_osbuild() {
173+
# A few patches that either haven't made it into a release or
174+
# that will be obsoleted with other work that will be done soon.
175+
cat /usr/lib/coreos-assembler/*.patch | patch -p1 -d /usr/lib/python3.11/site-packages/
176+
}
177+
171178
if [ $# -ne 0 ]; then
172179
# Run the function specified by the calling script
173180
${1}
@@ -182,4 +189,5 @@ else
182189
install_ocp_tools
183190
trust_redhat_gpg_keys
184191
configure_user
192+
patch_osbuild
185193
fi
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,92 @@
1+
From d4b3e3655deb7d55792e52fe6a11c609fb24e3b8 Mon Sep 17 00:00:00 2001
2+
From: Dusty Mabe <dusty@dustymabe.com>
3+
Date: Tue, 24 Oct 2023 14:08:44 -0400
4+
Subject: [PATCH] objectstore: also mount /etc/containers for "host" buildroot
5+
6+
In the case we are not using a buildroot (i.e. we are using
7+
the host as the buildroot) let's also mount in /etc/containers
8+
into the environment. There are sometimes where software running
9+
from /usr can't operate without configuration in /etc and this
10+
will allow it to work.
11+
12+
An example of software hitting this problem is skopeo. With a
13+
simple config like:
14+
15+
```
16+
version: '2'
17+
mpp-vars:
18+
release: 38
19+
pipelines:
20+
- name: skopeo-tree
21+
# build: name:build
22+
source-epoch: 1659397331
23+
stages:
24+
- type: org.osbuild.skopeo
25+
inputs:
26+
images:
27+
type: org.osbuild.containers
28+
origin: org.osbuild.source
29+
mpp-resolve-images:
30+
images:
31+
- source: quay.io/fedora/fedora-coreos
32+
tag: stable
33+
name: localhost/fcos
34+
options:
35+
destination:
36+
type: containers-storage
37+
storage-path: /usr/share/containers/storage
38+
```
39+
40+
We end up hitting an error like this:
41+
42+
```
43+
time="2023-10-24T18:27:14Z" level=fatal msg="Error loading trust policy: open /etc/containers/policy.json: no such file or directory"
44+
Traceback (most recent call last):
45+
File "/run/osbuild/bin/org.osbuild.skopeo", line 90, in <module>
46+
r = main(args["inputs"], args["tree"], args["options"])
47+
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
48+
File "/run/osbuild/bin/org.osbuild.skopeo", line 73, in main
49+
subprocess.run(["skopeo", "copy", image_source, dest], check=True)
50+
File "/usr/lib64/python3.11/subprocess.py", line 571, in run
51+
raise CalledProcessError(retcode, process.args,
52+
subprocess.CalledProcessError: Command '['skopeo', 'copy', 'dir:/tmp/tmp5_qcng99/image', 'containers-storage:[overlay@/run/osbuild/tree/usr/share/containers/storage+/run/containers/storage]localhost/fcos']' returned non-zero exit status 1.
53+
```
54+
55+
This PR adds in a mount for /etc/containers from the host so that
56+
/etc/containers/policy.json can be accessed.
57+
---
58+
osbuild/objectstore.py | 12 ++++++++++--
59+
1 file changed, 10 insertions(+), 2 deletions(-)
60+
61+
diff --git a/osbuild/objectstore.py b/osbuild/objectstore.py
62+
index 4a19ce9..922d5ee 100644
63+
--- a/osbuild/objectstore.py
64+
+++ b/osbuild/objectstore.py
65+
@@ -283,14 +283,22 @@ class HostTree:
66+
self._root = self.store.tempdir(prefix="host")
67+
68+
root = self._root.name
69+
- # Create a bare bones root file system
70+
- # with just /usr mounted from the host
71+
+ # Create a bare bones root file system. Starting with just
72+
+ # /usr mounted from the host.
73+
usr = os.path.join(root, "usr")
74+
os.makedirs(usr)
75+
+ # Also add in /etc/containers, which will allow us to access
76+
+ # /etc/containers/policy.json and enable moving containers
77+
+ # (skopeo): https://github.com/osbuild/osbuild/pull/1410
78+
+ # If https://github.com/containers/image/issues/2157 ever gets
79+
+ # fixed we can probably remove this bind mount.
80+
+ etc_containers = os.path.join(root, "etc", "containers")
81+
+ os.makedirs(etc_containers)
82+
83+
# ensure / is read-only
84+
mount(root, root)
85+
mount("/usr", usr)
86+
+ mount("/etc/containers", etc_containers)
87+
88+
@property
89+
def tree(self) -> os.PathLike:
90+
--
91+
2.41.0
92+
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,45 @@
1+
From 2e34303f2e9ef1d48b965703976ef1029d7309f1 Mon Sep 17 00:00:00 2001
2+
From: Dusty Mabe <dusty@dustymabe.com>
3+
Date: Fri, 1 Sep 2023 12:18:25 -0400
4+
Subject: [PATCH] Mount boot from host in host builder case
5+
6+
---
7+
osbuild/buildroot.py | 2 +-
8+
osbuild/objectstore.py | 3 +++
9+
2 files changed, 4 insertions(+), 1 deletion(-)
10+
11+
diff --git a/osbuild/buildroot.py b/osbuild/buildroot.py
12+
index 5b47d70..a0f654d 100644
13+
--- a/osbuild/buildroot.py
14+
+++ b/osbuild/buildroot.py
15+
@@ -196,7 +196,7 @@ class BuildRoot(contextlib.AbstractContextManager):
16+
17+
# Import directories from the caller-provided root.
18+
imports = ["usr"]
19+
- if self.mount_boot:
20+
+ if True:
21+
imports.insert(0, "boot")
22+
23+
for p in imports:
24+
diff --git a/osbuild/objectstore.py b/osbuild/objectstore.py
25+
index 922d5ee..6a3f89a 100644
26+
--- a/osbuild/objectstore.py
27+
+++ b/osbuild/objectstore.py
28+
@@ -294,11 +294,14 @@ class HostTree:
29+
# fixed we can probably remove this bind mount.
30+
etc_containers = os.path.join(root, "etc", "containers")
31+
os.makedirs(etc_containers)
32+
+ boot = os.path.join(root, "boot")
33+
+ os.makedirs(boot)
34+
35+
# ensure / is read-only
36+
mount(root, root)
37+
mount("/usr", usr)
38+
mount("/etc/containers", etc_containers)
39+
+ mount("/boot", boot)
40+
41+
@property
42+
def tree(self) -> os.PathLike:
43+
--
44+
2.41.0
45+

‎src/cmd-buildextend-metal

+15-7
Original file line numberDiff line numberDiff line change
@@ -261,13 +261,21 @@ EOF
261261
cat "${image_json}" image-dynamic.json | jq -s add > image-for-disk.json
262262
platforms_json="${workdir}/tmp/platforms.json"
263263
yaml2json "${configdir}/platforms.yaml" "${platforms_json}"
264-
runvm "${qemu_args[@]}" -- \
265-
/usr/lib/coreos-assembler/create_disk.sh \
266-
--config "$(pwd)"/image-for-disk.json \
267-
--kargs "${kargs}" \
268-
--platform "${ignition_platform_id}" \
269-
--platforms-json "${platforms_json}" \
270-
"${disk_args[@]}"
264+
265+
if [ "${image_type}" == "qemu" ] && [ "${COSA_USE_OSBUILD:-}" != "" ]; then
266+
runvm -- /usr/lib/coreos-assembler/runvm-osbuild \
267+
"${ostree_repo}" "${ref}" \
268+
/usr/lib/coreos-assembler/coreos.osbuild.mpp.yaml \
269+
"${path}.tmp"
270+
else
271+
runvm "${qemu_args[@]}" -- \
272+
/usr/lib/coreos-assembler/create_disk.sh \
273+
--config "$(pwd)"/image-for-disk.json \
274+
--kargs "${kargs}" \
275+
--platform "${ignition_platform_id}" \
276+
--platforms-json "${platforms_json}" \
277+
"${disk_args[@]}"
278+
fi
271279

272280
if [[ $secure_execution -eq "1" && -z "${hostkey}" ]]; then
273281
/usr/lib/coreos-assembler/secex-genprotimgvm-scripts/runvm.sh \

‎src/coreos.osbuild.mpp.yaml

+228
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,228 @@
1+
version: '2'
2+
mpp-vars:
3+
release: 38
4+
mpp-define-image:
5+
id: image
6+
#10G
7+
size: '10737418240'
8+
table:
9+
uuid: 00000000-0000-4000-a000-000000000001
10+
label: gpt
11+
partitions:
12+
- id: BIOS-BOOT
13+
size: 2048
14+
type: 21686148-6449-6E6F-744E-656564454649
15+
bootable: true
16+
uuid: FAC7F1FB-3E8D-4137-A512-961DE09A5549
17+
- id: EFI-SYSTEM
18+
size: 260096
19+
type: C12A7328-F81F-11D2-BA4B-00A0C93EC93B
20+
uuid: 68B2905B-DF3E-4FB3-80FA-49D1E773AA33
21+
- id: boot
22+
size: 786432
23+
type: 0FC63DAF-8483-4772-8E79-3D69D8477DE4
24+
uuid: 61B2905B-DF3E-4FB3-80FA-49D1E773AA32
25+
- id: root
26+
# XXX: Dynamically set this size in the future
27+
size: 4194304
28+
type: 0FC63DAF-8483-4772-8E79-3D69D8477DE4
29+
uuid: CA7D7CCB-63ED-4C53-861C-1742536059CC
30+
pipelines:
31+
- name: image-tree
32+
source-epoch: 1659397331
33+
stages:
34+
- type: org.osbuild.ostree.init-fs
35+
- type: org.osbuild.ostree.os-init
36+
options:
37+
osname: fedora-coreos
38+
- type: org.osbuild.ostree.config
39+
options:
40+
repo: /ostree/repo
41+
config:
42+
sysroot:
43+
readonly: false
44+
bootloader: none
45+
- type: org.osbuild.mkdir
46+
options:
47+
paths:
48+
- path: /boot/efi
49+
mode: 448
50+
- type: org.osbuild.ignition
51+
- type: org.osbuild.ostree.deploy
52+
options:
53+
osname: fedora-coreos
54+
remote: fedora
55+
mounts:
56+
- /boot
57+
- /boot/efi
58+
kernel_opts:
59+
- rw
60+
- console=tty0
61+
- console=ttyS0
62+
- ignition.platform.id=qemu
63+
- '$ignition_firstboot'
64+
inputs:
65+
commits:
66+
type: org.osbuild.ostree
67+
origin: org.osbuild.source
68+
mpp-resolve-ostree-commits:
69+
commits:
70+
- ref: $ref
71+
remote:
72+
url: $repourl
73+
- type: org.osbuild.ostree.selinux
74+
options:
75+
deployment:
76+
osname: fedora-coreos
77+
ref: ostree/1/1/0
78+
- type: org.osbuild.grub2
79+
options:
80+
rootfs:
81+
label: root
82+
bootfs:
83+
label: boot
84+
uefi:
85+
vendor: fedora
86+
install: true
87+
legacy: i386-pc
88+
write_defaults: false
89+
greenboot: false
90+
ignition: true
91+
- name: image
92+
stages:
93+
- type: org.osbuild.truncate
94+
options:
95+
filename: disk.img
96+
size:
97+
mpp-format-string: '{image.size}'
98+
- type: org.osbuild.sfdisk
99+
devices:
100+
device:
101+
type: org.osbuild.loopback
102+
options:
103+
filename: disk.img
104+
options:
105+
mpp-format-json: '{image.layout}'
106+
- type: org.osbuild.mkfs.fat
107+
devices:
108+
device:
109+
type: org.osbuild.loopback
110+
options:
111+
filename: disk.img
112+
start:
113+
mpp-format-int: '{image.layout[''EFI-SYSTEM''].start}'
114+
size:
115+
mpp-format-int: '{image.layout[''EFI-SYSTEM''].size}'
116+
lock: true
117+
options:
118+
label: EFI-SYSTEM
119+
volid: 7B7795E7
120+
- type: org.osbuild.mkfs.ext4
121+
devices:
122+
device:
123+
type: org.osbuild.loopback
124+
options:
125+
filename: disk.img
126+
start:
127+
mpp-format-int: '{image.layout[''boot''].start}'
128+
size:
129+
mpp-format-int: '{image.layout[''boot''].size}'
130+
lock: true
131+
options:
132+
uuid: 96d15588-3596-4b3c-adca-a2ff7279ea63
133+
label: boot
134+
- type: org.osbuild.mkfs.xfs
135+
devices:
136+
device:
137+
type: org.osbuild.loopback
138+
options:
139+
filename: disk.img
140+
start:
141+
mpp-format-int: '{image.layout[''root''].start}'
142+
size:
143+
mpp-format-int: '{image.layout[''root''].size}'
144+
lock: true
145+
options:
146+
uuid: 910678ff-f77e-4a7d-8d53-86f2ac47a823
147+
label: root
148+
- type: org.osbuild.copy
149+
inputs:
150+
tree:
151+
type: org.osbuild.tree
152+
origin: org.osbuild.pipeline
153+
references:
154+
- name:image-tree
155+
options:
156+
paths:
157+
- from: input://tree/
158+
to: mount://root/
159+
devices:
160+
efi:
161+
type: org.osbuild.loopback
162+
options:
163+
filename: disk.img
164+
start:
165+
mpp-format-int: '{image.layout[''EFI-SYSTEM''].start}'
166+
size:
167+
mpp-format-int: '{image.layout[''EFI-SYSTEM''].size}'
168+
boot:
169+
type: org.osbuild.loopback
170+
options:
171+
filename: disk.img
172+
start:
173+
mpp-format-int: '{image.layout[''boot''].start}'
174+
size:
175+
mpp-format-int: '{image.layout[''boot''].size}'
176+
root:
177+
type: org.osbuild.loopback
178+
options:
179+
filename: disk.img
180+
start:
181+
mpp-format-int: '{image.layout[''root''].start}'
182+
size:
183+
mpp-format-int: '{image.layout[''root''].size}'
184+
mounts:
185+
- name: root
186+
type: org.osbuild.xfs
187+
source: root
188+
target: /
189+
- name: boot
190+
type: org.osbuild.ext4
191+
source: boot
192+
target: /boot
193+
- name: efi
194+
type: org.osbuild.fat
195+
source: efi
196+
target: /boot/efi
197+
- type: org.osbuild.grub2.inst
198+
options:
199+
platform: i386-pc
200+
filename: disk.img
201+
location:
202+
mpp-format-int: '{image.layout[''BIOS-BOOT''].start}'
203+
core:
204+
type: mkimage
205+
partlabel: gpt
206+
filesystem: ext4
207+
prefix:
208+
type: partition
209+
partlabel:
210+
mpp-format-string: '{image.layout.label}'
211+
number:
212+
mpp-format-int: '{image.layout[''boot''].index}'
213+
path: /grub2
214+
- name: qcow2
215+
stages:
216+
- type: org.osbuild.qemu
217+
inputs:
218+
image:
219+
type: org.osbuild.files
220+
origin: org.osbuild.pipeline
221+
references:
222+
name:image:
223+
file: disk.img
224+
options:
225+
filename: disk.qcow2
226+
format:
227+
type: qcow2
228+
compat: '1.1'

‎src/deps.txt

-6
Original file line numberDiff line numberDiff line change
@@ -16,12 +16,6 @@ dumb-init
1616
rpm-ostree createrepo_c openssh-clients python3-createrepo_c
1717
dnf-utils
1818

19-
# We don't actually use this right now but we intend to share
20-
# code in the future.
21-
# XXX: temporarily disabled until we fix
22-
# https://github.com/osbuild/osbuild-composer/issues/1915
23-
# osbuild-composer
24-
2519
# For generating ISO images
2620
genisoimage
2721

‎src/runvm-osbuild

+30
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,30 @@
1+
#!/bin/bash
2+
set -eux -o pipefail
3+
4+
repo=$1
5+
ref=$2
6+
mppyaml=$3
7+
path=$4
8+
9+
# Since it doesn't exist create loop-control
10+
mknod /dev/loop-control c 10 237
11+
12+
# get away from the virtiofs share because the xattrs that
13+
# are written out by the ostree deploy will cause SELinux denials.
14+
mkdir /root/osbuild && cd /root/osbuild
15+
16+
# Run through the preprocessor
17+
osbuild-mpp \
18+
-D ref=\""${ref}"\" \
19+
-D repourl=\""file://${repo}"\" \
20+
"${mppyaml}" \
21+
processed.json
22+
23+
# Build the image
24+
osbuild --store store/ \
25+
--output-directory out/ \
26+
--export qcow2 processed.json
27+
28+
29+
# Copy it out to the specified location
30+
cp out/qcow2/disk.qcow2 "${path}"

‎src/vmdeps.txt

+3
Original file line numberDiff line numberDiff line change
@@ -35,3 +35,6 @@ tar
3535

3636
# needed for extensions container build
3737
podman
38+
39+
# For running osbuild
40+
osbuild osbuild-ostree osbuild-selinux osbuild-tools python3-pyrsistent

0 commit comments

Comments
 (0)
Please sign in to comment.