Can't load or unload module

[kaaax0815]

I am using Podman 5.2.2 with quadlet

# /etc/containers/systemd/wireguard/wireguard.container
[Container]
ContainerName=wireguard
Image=ghcr.io/wg-easy/wg-easy:development

Volume=/etc/containers/volumes/wireguard:/etc/wireguard:Z
Volume=/lib/modules:/lib/modules:ro,z
Network=wireguard.network
PublishPort=51820:51820/udp

AddCapability=NET_ADMIN
AddCapability=SYS_MODULE
AddCapability=NET_RAW
Sysctl=net.ipv4.ip_forward=1
Sysctl=net.ipv4.conf.all.src_valid_mark=1
Sysctl=net.ipv6.conf.all.disable_ipv6=0
Sysctl=net.ipv6.conf.all.forwarding=1
Sysctl=net.ipv6.conf.default.forwarding=1

Network=traefik.network
Label="traefik.enable=true"
# ... more traefik labels

[Install]
# WantedBy=default.target

When I run

sudo podman exec -it wireguard /bin/sh

i can't load or unload the wireguard module. (even tho i have the SYS_MODULE cap)

modprobe wireguard
-> modprobe: ERROR: could not insert 'wireguard': Operation not permitted
modprobe -r wireguard
-> modprobe: ERROR: could not remove 'wireguard': Operation not permitted

I also have a problem with nftables, as I can't run

nft add rule inet my_table postrouting ip saddr 10.8.0.0/24 oifname eth0 masquerade
-> Error: Could not process rule: No such file or directory
-> add rule inet my_table postrouting ip saddr 10.8.0.0/24 oifname eth0 masquerade
->                                                                      ^^^^^^^^^^

but when I use the --privileged flag nftables works again

I was able to make nftables work by loading nft_masq