| Message ID | 20231214124940.3281278-1-dimitri.ledkov@canonical.com |
|---|---|
| Headers | show |
| Series | Enforce RETPOLINE and SLS mitigrations | expand |
Some more background information about retpoline check: After like 2 years under embargo, Spectre Meltdown were a set of vulnerabilities that were disclosed in January 2018 and have caused quite a havoc. It has thrashed performance of multiple generations of hardware, and even today causes issues of unable to security support hardware that users and customers are still running despite these vulnerabilities. More information on https://meltdownattack.com/ and also this excellent FOSDEM talk https://www.youtube.com/watch?v=vOHtDey8wqI and also https://wiki.ubuntu.com/SecurityTeam/KnowledgeBase/SpectreAndMeltdown (especially for the Timeline as has been known to us). As part of these mitigations, a retpoline toolchain & kernel code support was added in the kernel to partially mitigate the attack. Initially blindly, and eventually very nuanced. In Ubuntu kernels, one can see that retpoline unsafe call site validation was started to be tracked with ABI checker functionality. This was added in 4.15.0-10.11 kernel or thereabouts see https://git.launchpad.net/~ubuntu-kernel/ubuntu/+source/linux/+git/bionic/tree/debian.master/changelog?h=Ubuntu-4.15.0-10.11#n68 And for example the next upload did show over 300 unsafe call sites in the 10.11 abi - as we can see in the 11.12 upload's vendored abi files. https://git.launchpad.net/~ubuntu-kernel/ubuntu/+source/linux/+git/bionic/tree/debian.master/abi/4.15.0-10.11/amd64/generic.retpoline?h=Ubuntu-4.15.0-11.12 Later, the upstream kernel in objtool added the ability to track unsafe call sites, reduce the number of them, and mark remaining ones as safe. See these commits in the 4.15.0-13.14 kernel https://git.launchpad.net/~ubuntu-kernel/ubuntu/+source/linux/+git/bionic/tree/debian.master/changelog?h=Ubuntu-4.15.0-13.14#n348 This resulted in retpoline ABI file becoming empty, and it has remained empty ever since. https://git.launchpad.net/~ubuntu-kernel/ubuntu/+source/linux/+git/bionic/tree/debian.master/abi/4.15.0-13.14/amd64/generic.retpoline?h=Ubuntu-4.15.0-14.15 because kernel build and objtool enforce this. It has remained empty ever since. At the same time better ways to mitigate the attacks have become available in hardware - with microcode updates, CPU improvements, development of Intel CET technologies Indirect Branch Tracking & Shadow Stack, encrypted memory, and now Confidential computing with SEV and TDX. If I fake a regression in toolchain (by commenting out) KBUILD_CFLAGS += $(RETPOLINE_CFLAGS), the build should fail even before getting to the abi-checks, as kernel's objtool catches this..... .... Hang on a minute, why did the build pass?! And that is the motivation for these patch series.
On 12/14/23 5:49 AM, Dimitri John Ledkov wrote: > [ Impact ] > > Enforce RETPOLINE and SLS mitigrations > > Currently retpoline ABI checks in the kernel build do nothing. They > produce no output, as if everything is fine. And if one manually hacks > makefile to "forget" retpoline & SLS mitigration flags, objtool prints > lots of warnings, retpoline ABI check passes and the build is > succesful. Yet totally vulnerable. > > Proposal is to enforce objtool warnings as fatal errors for RETPOLINE > and SLS, as tested to be passed on mantic for both kernel and all > available dkms. And otherwise rip out custom Ubuntu retpoline abi > checks. > > I have prepared this for noble v6.7 kernel, once this lands, I will > make appropriate backports for earlier series as we likely want usable > retpoline build time enforcement in earlier series too where possible. > > [ Test Plan ] > > Hack arch/x86/Makefile and comment out KBUILD_CFLAGS += $(RETPOLINE_CFLAGS) > > This simulate a build infrastructure, or toolchain regression, or > hand-written assembly code that is susceptible to speculative attacks. > > Attempt to build the kernel. > > The kernel build must fail. Currently it doesn't, and retpoline ABI > checks do not catch it. > > Another approach is to build a known buggy dkms modules on x86 - for > example zfs-dkms with ret -> RET changes reverted in assembly > accelerated code. > > [ Where problems could occur ] > > This change will make our kernel build more strict, especially for > dkms packages. dkms packages that ship in Ubuntu archive have been > build tested to pass with these more strict requirements in > place. Other external modules that fail with such strict configuration > should either fix their code to be retpoline/redbleed safe - or use a > config override CONFIG_RETPOLINE=n to disable retpoline during their > build, or otherwise use one of the OBJTOOL_ settings in their dkms > Makefiles to skip objtool on given portions of code, or otherwise mark > things as retpoline_safe / noreturn / etc. See examples in the linux > upstream source code. > > [ Other Info ] > > This work was done as part of hackathon questioning abi checks > usefulness, given I have never experienced retpoline check > failure. And they have always been empty since early v4.15 days > https://git.launchpad.net/~ubuntu-kernel/ubuntu/+source/linux/+git/bionic/tree/debian.master/abi/4.15.0-13.14/amd64/generic.retpoline?h=Ubuntu-4.15.0-14.15 > > Gitea review URL: > https://kernel.ubuntu.com/gitea/kernel/noble-linux-unstable/pulls/15 > > > Dimitri John Ledkov (5): > UBUNTU: SAUCE: objtool: Make objtool check actually fatal upon fatal > errors > UBUNTU: SAUCE: objtool: make objtool SLS validation fatal when > building with CONFIG_SLS=y > UBUNTU: SAUCE: objtool: make objtool RETPOLINE validation fatal when > building with CONFIG_RETPOLINE=y > UBUNTU: SAUCE: scripts: remove generating .o-ur objects > UBUNTU: [Packaging] Remove all custom retpoline-extract code > > debian.master/abi/amd64/generic.retpoline | 1 - > debian.master/abi/arm64/generic-64k.retpoline | 1 - > debian.master/abi/arm64/generic.retpoline | 1 - > debian.master/abi/armhf/generic.retpoline | 1 - > debian.master/abi/ppc64el/generic.retpoline | 1 - > debian.master/abi/riscv64/generic.retpoline | 0 > debian.master/abi/riscv64/ignore.retpoline | 1 - > debian.master/abi/s390x/generic.retpoline | 1 - > debian/rules | 6 +- > debian/rules.d/2-binary-arch.mk | 15 - > debian/rules.d/4-checks.mk | 8 +- > debian/scripts/checks/final-checks | 7 - > debian/scripts/checks/retpoline-check | 52 ---- > debian/scripts/dkms-build | 2 +- > debian/scripts/dkms-build--nvidia-N | 7 +- > debian/scripts/helpers/open | 3 +- > debian/scripts/misc/getabis | 7 +- > debian/scripts/retpoline-extract | 23 -- > debian/scripts/retpoline-extract-one | 270 ------------------ > scripts/Makefile.build | 8 - > snapcraft.yaml | 4 - > tools/objtool/check.c | 26 +- > 22 files changed, 22 insertions(+), 423 deletions(-) > delete mode 100644 debian.master/abi/amd64/generic.retpoline > delete mode 100644 debian.master/abi/arm64/generic-64k.retpoline > delete mode 100644 debian.master/abi/arm64/generic.retpoline > delete mode 100644 debian.master/abi/armhf/generic.retpoline > delete mode 100644 debian.master/abi/ppc64el/generic.retpoline > delete mode 100644 debian.master/abi/riscv64/generic.retpoline > delete mode 100644 debian.master/abi/riscv64/ignore.retpoline > delete mode 100644 debian.master/abi/s390x/generic.retpoline > delete mode 100755 debian/scripts/checks/retpoline-check > delete mode 100755 debian/scripts/retpoline-extract > delete mode 100755 debian/scripts/retpoline-extract-one > Acked-by: Tim Gardner <tim.gardner@canonical.com>
[ Impact ] Enforce RETPOLINE and SLS mitigrations Currently retpoline ABI checks in the kernel build do nothing. They produce no output, as if everything is fine. And if one manually hacks makefile to "forget" retpoline & SLS mitigration flags, objtool prints lots of warnings, retpoline ABI check passes and the build is succesful. Yet totally vulnerable. Proposal is to enforce objtool warnings as fatal errors for RETPOLINE and SLS, as tested to be passed on mantic for both kernel and all available dkms. And otherwise rip out custom Ubuntu retpoline abi checks. I have prepared this for noble v6.7 kernel, once this lands, I will make appropriate backports for earlier series as we likely want usable retpoline build time enforcement in earlier series too where possible. [ Test Plan ] Hack arch/x86/Makefile and comment out KBUILD_CFLAGS += $(RETPOLINE_CFLAGS) This simulate a build infrastructure, or toolchain regression, or hand-written assembly code that is susceptible to speculative attacks. Attempt to build the kernel. The kernel build must fail. Currently it doesn't, and retpoline ABI checks do not catch it. Another approach is to build a known buggy dkms modules on x86 - for example zfs-dkms with ret -> RET changes reverted in assembly accelerated code. [ Where problems could occur ] This change will make our kernel build more strict, especially for dkms packages. dkms packages that ship in Ubuntu archive have been build tested to pass with these more strict requirements in place. Other external modules that fail with such strict configuration should either fix their code to be retpoline/redbleed safe - or use a config override CONFIG_RETPOLINE=n to disable retpoline during their build, or otherwise use one of the OBJTOOL_ settings in their dkms Makefiles to skip objtool on given portions of code, or otherwise mark things as retpoline_safe / noreturn / etc. See examples in the linux upstream source code. [ Other Info ] This work was done as part of hackathon questioning abi checks usefulness, given I have never experienced retpoline check failure. And they have always been empty since early v4.15 days https://git.launchpad.net/~ubuntu-kernel/ubuntu/+source/linux/+git/bionic/tree/debian.master/abi/4.15.0-13.14/amd64/generic.retpoline?h=Ubuntu-4.15.0-14.15 Gitea review URL: https://kernel.ubuntu.com/gitea/kernel/noble-linux-unstable/pulls/15 Dimitri John Ledkov (5): UBUNTU: SAUCE: objtool: Make objtool check actually fatal upon fatal errors UBUNTU: SAUCE: objtool: make objtool SLS validation fatal when building with CONFIG_SLS=y UBUNTU: SAUCE: objtool: make objtool RETPOLINE validation fatal when building with CONFIG_RETPOLINE=y UBUNTU: SAUCE: scripts: remove generating .o-ur objects UBUNTU: [Packaging] Remove all custom retpoline-extract code debian.master/abi/amd64/generic.retpoline | 1 - debian.master/abi/arm64/generic-64k.retpoline | 1 - debian.master/abi/arm64/generic.retpoline | 1 - debian.master/abi/armhf/generic.retpoline | 1 - debian.master/abi/ppc64el/generic.retpoline | 1 - debian.master/abi/riscv64/generic.retpoline | 0 debian.master/abi/riscv64/ignore.retpoline | 1 - debian.master/abi/s390x/generic.retpoline | 1 - debian/rules | 6 +- debian/rules.d/2-binary-arch.mk | 15 - debian/rules.d/4-checks.mk | 8 +- debian/scripts/checks/final-checks | 7 - debian/scripts/checks/retpoline-check | 52 ---- debian/scripts/dkms-build | 2 +- debian/scripts/dkms-build--nvidia-N | 7 +- debian/scripts/helpers/open | 3 +- debian/scripts/misc/getabis | 7 +- debian/scripts/retpoline-extract | 23 -- debian/scripts/retpoline-extract-one | 270 ------------------ scripts/Makefile.build | 8 - snapcraft.yaml | 4 - tools/objtool/check.c | 26 +- 22 files changed, 22 insertions(+), 423 deletions(-) delete mode 100644 debian.master/abi/amd64/generic.retpoline delete mode 100644 debian.master/abi/arm64/generic-64k.retpoline delete mode 100644 debian.master/abi/arm64/generic.retpoline delete mode 100644 debian.master/abi/armhf/generic.retpoline delete mode 100644 debian.master/abi/ppc64el/generic.retpoline delete mode 100644 debian.master/abi/riscv64/generic.retpoline delete mode 100644 debian.master/abi/riscv64/ignore.retpoline delete mode 100644 debian.master/abi/s390x/generic.retpoline delete mode 100755 debian/scripts/checks/retpoline-check delete mode 100755 debian/scripts/retpoline-extract delete mode 100755 debian/scripts/retpoline-extract-one