mbox series

[v2,0/5] Add support for memory sealing

Message ID 20240731200307.2269811-1-adhemerval.zanella@linaro.org
Headers show
Series Add support for memory sealing | expand

Message

Adhemerval Zanella Netto July 31, 2024, 8:02 p.m. UTC
The Linux 6.10 (8be7258aad44b5e25977a98db136f677fa6f4370) added the mseal
syscall that allows blocking some memory operations on the VMA range:

 * Unmapping, moving to another location, extending or shrinking the
   size, munmap, and mremap.
 * Moving or expanding a different VMA into the current location, via
   mremap.
 * Modifying the memory range with mmap along with flag MAP_FIXED.
 * Expanding the size with mremap.
 * Change the protection flags with mprotect or pkey_mprotect.
 * Destructive behaviors on anonymous memory, such as madvice with
   MADV_DONTNEED.

Memory sealing might be useful as a hardening mechanism to avoid either
remapping the memory segments or changing the memory protection segments
layout by the dynamic loader (for instance, the RELRO hardening).
A similar hardening is done by OpenBSD with the mimmutable syscall [1].

The first patch expands the use of RTLD_NODELETE for objects that should
not be deallocated during process execution.  The flag will be used later
to decide whether to seal the link_map:

 * on dlopen dependencies for objects opened with RTLD_NODELETE
 * For the main map executable.
 * For the main map DT_NEEDED dependencies.
 * On __libc_unwind_link_get for libgcc_s.so (used for backtrace and
   unwind).

The second patch adds the mseal support for Linux.  Although most
programs will not use it directly, some specific ones, like Chrome,
intend to use it.

The third patch adds memory sealing in multiple places where the memory
is supposed to be immutable over program execution:

 * All shared library dependencies from the binary, including the
   read-only segments after PT_GNU_RELRO setup.
 * The binary itself, including dynamic and static links. In both cases,
   it is up either to binary or the loader to set up the sealing.
 * Any preload libraries.
 * Any library loaded with dlopen with RTLD_NODELETE flag (including
   libgcc.so loaded to enable process unwind and thread cancellation).
 * Audit modules.
 * The loader bump allocator.

For binary dependencies, the RTLD_NODELETE signals the link_map should be
sealed. It also makes dlopen objects with the flag sealed as well.

The sealing is also controlled by a new tunable, glibc.rtld.seal, with
three different states:

 0. Disabled, where no memory sealing is done.
 1. Enabled, where the loader will issue the mseal syscall on the
    memory mappings, and any failure is ignored. This is the default.
 2. Enforce, similar to Enabled and any failure from the mseal terminates
    the process.

The final patch adds support for GNU_PROPERTY_NO_MEMORY_SEAL, where the
binary can be marked to avoid sealing (i.e., on Firefox hack to bypass the
dynamic loader and enable DT_RELR on older glibc [3]).  In this case, it is
up to the module to apply memory sealing itself.  I have a patch to
binutils to add -Wl,-z,no-memory-seal option to enable it.

This patchset does not delay RELRO activation until after their ELF
constructors have been executed, as suggested on the previous RFC for
mseal support. It is not strictly required, and it requires extensive
changes on_dl_start_user to either make _dl_init call RELRO/sealing setup
after ctor/initarray is done, or call it after _dl_init. There is also the
question of whether to apply RELRO/sealing per module after dtor/initarray
or in bulk after _dt_init.

I tested on both x86_64-linux-gnu and aarch64-linux-gnu with Linux
6.11-rc1, along with some testing on a powerpc64le-linux-gnu VM. I also
enabled glibc.rtld.seal=2 to check for possible mseal failures.

[1] https://man.openbsd.org/mimmutable.2
[2] https://docs.google.com/document/d/1O2jwK4dxI3nRcOJuPYkonhTkNQfbmwdvxQMyXgeaRHo/edit#heading=h.bvaojj9fu6hc
[3] https://glandium.org/blog/?p=4297

Adhemerval Zanella (5):
  elf: Use RTLD_NODELETE is more places
  linux: Add mseal syscall support
  elf: Add support to memory sealing
  elf: Also parse gnu properties for static linked binaries
  elf: Add support for GNU_PROPERTY_NO_MEMORY_SEAL

 NEWS                                          |  15 +
 configure                                     |  35 +++
 configure.ac                                  |   5 +
 elf/dl-load.c                                 |   4 +
 elf/dl-map-segments.h                         |   5 +
 elf/dl-minimal-malloc.c                       |   2 +
 elf/dl-mseal-mode.h                           |  29 ++
 elf/dl-open.c                                 |   7 +-
 elf/dl-reloc.c                                |  47 +++
 elf/dl-support.c                              |  20 ++
 elf/dl-tunables.list                          |   6 +
 elf/elf.h                                     |   2 +
 elf/rtld.c                                    |  13 +-
 elf/setup-vdso.h                              |   2 +
 elf/tst-rtld-list-tunables.exp                |   1 +
 include/dlfcn.h                               |   2 +
 include/link.h                                |   7 +
 manual/memory.texi                            |  66 +++++
 manual/tunables.texi                          |  42 +++
 misc/unwind-link.c                            |   5 +-
 sysdeps/aarch64/dl-prop.h                     |   5 +
 sysdeps/generic/dl-mseal.h                    |  23 ++
 sysdeps/generic/dl-prop-mseal.h               |  38 +++
 sysdeps/generic/dl-prop.h                     |   5 +
 sysdeps/generic/ldsodefs.h                    |   9 +
 sysdeps/unix/sysv/linux/Makefile              |  94 ++++++
 sysdeps/unix/sysv/linux/Versions              |   3 +
 sysdeps/unix/sysv/linux/aarch64/libc.abilist  |   1 +
 sysdeps/unix/sysv/linux/alpha/libc.abilist    |   1 +
 sysdeps/unix/sysv/linux/arc/libc.abilist      |   1 +
 sysdeps/unix/sysv/linux/arm/be/libc.abilist   |   1 +
 sysdeps/unix/sysv/linux/arm/le/libc.abilist   |   1 +
 sysdeps/unix/sysv/linux/bits/mman-shared.h    |   8 +
 sysdeps/unix/sysv/linux/csky/libc.abilist     |   1 +
 sysdeps/unix/sysv/linux/dl-mseal.c            |  51 ++++
 sysdeps/unix/sysv/linux/dl-mseal.h            |  27 ++
 sysdeps/unix/sysv/linux/hppa/libc.abilist     |   1 +
 sysdeps/unix/sysv/linux/i386/libc.abilist     |   1 +
 sysdeps/unix/sysv/linux/kernel-features.h     |   8 +
 .../sysv/linux/loongarch/lp64/libc.abilist    |   1 +
 .../sysv/linux/m68k/coldfire/libc.abilist     |   1 +
 .../unix/sysv/linux/m68k/m680x0/libc.abilist  |   1 +
 .../sysv/linux/microblaze/be/libc.abilist     |   1 +
 .../sysv/linux/microblaze/le/libc.abilist     |   1 +
 .../sysv/linux/mips/mips32/fpu/libc.abilist   |   1 +
 .../sysv/linux/mips/mips64/n32/libc.abilist   |   1 +
 .../sysv/linux/mips/mips64/n64/libc.abilist   |   1 +
 sysdeps/unix/sysv/linux/nios2/libc.abilist    |   1 +
 sysdeps/unix/sysv/linux/or1k/libc.abilist     |   1 +
 .../linux/powerpc/powerpc32/fpu/libc.abilist  |   1 +
 .../powerpc/powerpc32/nofpu/libc.abilist      |   1 +
 .../linux/powerpc/powerpc64/be/libc.abilist   |   1 +
 .../linux/powerpc/powerpc64/le/libc.abilist   |   1 +
 .../unix/sysv/linux/riscv/rv32/libc.abilist   |   1 +
 .../unix/sysv/linux/riscv/rv64/libc.abilist   |   1 +
 .../unix/sysv/linux/s390/s390-32/libc.abilist |   1 +
 .../unix/sysv/linux/s390/s390-64/libc.abilist |   1 +
 sysdeps/unix/sysv/linux/sh/be/libc.abilist    |   1 +
 sysdeps/unix/sysv/linux/sh/le/libc.abilist    |   1 +
 .../sysv/linux/sparc/sparc32/libc.abilist     |   1 +
 .../sysv/linux/sparc/sparc64/libc.abilist     |   1 +
 sysdeps/unix/sysv/linux/syscalls.list         |   1 +
 .../unix/sysv/linux/tst-dl_mseal-auditmod.c   |  23 ++
 .../unix/sysv/linux/tst-dl_mseal-dlopen-1-1.c |  19 ++
 .../unix/sysv/linux/tst-dl_mseal-dlopen-1.c   |  19 ++
 .../unix/sysv/linux/tst-dl_mseal-dlopen-2-1.c |  19 ++
 .../unix/sysv/linux/tst-dl_mseal-dlopen-2.c   |  19 ++
 .../tst-dl_mseal-dlopen-no-memory-seal-2-1.c  |  19 ++
 .../tst-dl_mseal-dlopen-no-memory-seal-2.c    |  19 ++
 sysdeps/unix/sysv/linux/tst-dl_mseal-mod-1.c  |  19 ++
 sysdeps/unix/sysv/linux/tst-dl_mseal-mod-2.c  |  19 ++
 .../linux/tst-dl_mseal-mod-no-memory-seal-1.c |  19 ++
 .../linux/tst-dl_mseal-mod-no-memory-seal-2.c |  19 ++
 .../tst-dl_mseal-no-memory-seal-auditmod.c    |   1 +
 .../tst-dl_mseal-no-memory-seal-preload.c     |   1 +
 .../sysv/linux/tst-dl_mseal-no-memory-seal.c  |  65 +++++
 .../unix/sysv/linux/tst-dl_mseal-preload.c    |  19 ++
 .../unix/sysv/linux/tst-dl_mseal-skeleton.c   | 272 ++++++++++++++++++
 .../tst-dl_mseal-static-no-memory-seal.c      |  38 +++
 sysdeps/unix/sysv/linux/tst-dl_mseal-static.c |  36 +++
 sysdeps/unix/sysv/linux/tst-dl_mseal.c        |  67 +++++
 sysdeps/unix/sysv/linux/tst-mseal.c           |  67 +++++
 .../unix/sysv/linux/x86_64/64/libc.abilist    |   1 +
 .../unix/sysv/linux/x86_64/x32/libc.abilist   |   1 +
 sysdeps/x86/dl-prop.h                         |   4 +
 85 files changed, 1396 insertions(+), 6 deletions(-)
 create mode 100644 elf/dl-mseal-mode.h
 create mode 100644 sysdeps/generic/dl-mseal.h
 create mode 100644 sysdeps/generic/dl-prop-mseal.h
 create mode 100644 sysdeps/unix/sysv/linux/dl-mseal.c
 create mode 100644 sysdeps/unix/sysv/linux/dl-mseal.h
 create mode 100644 sysdeps/unix/sysv/linux/tst-dl_mseal-auditmod.c
 create mode 100644 sysdeps/unix/sysv/linux/tst-dl_mseal-dlopen-1-1.c
 create mode 100644 sysdeps/unix/sysv/linux/tst-dl_mseal-dlopen-1.c
 create mode 100644 sysdeps/unix/sysv/linux/tst-dl_mseal-dlopen-2-1.c
 create mode 100644 sysdeps/unix/sysv/linux/tst-dl_mseal-dlopen-2.c
 create mode 100644 sysdeps/unix/sysv/linux/tst-dl_mseal-dlopen-no-memory-seal-2-1.c
 create mode 100644 sysdeps/unix/sysv/linux/tst-dl_mseal-dlopen-no-memory-seal-2.c
 create mode 100644 sysdeps/unix/sysv/linux/tst-dl_mseal-mod-1.c
 create mode 100644 sysdeps/unix/sysv/linux/tst-dl_mseal-mod-2.c
 create mode 100644 sysdeps/unix/sysv/linux/tst-dl_mseal-mod-no-memory-seal-1.c
 create mode 100644 sysdeps/unix/sysv/linux/tst-dl_mseal-mod-no-memory-seal-2.c
 create mode 100644 sysdeps/unix/sysv/linux/tst-dl_mseal-no-memory-seal-auditmod.c
 create mode 100644 sysdeps/unix/sysv/linux/tst-dl_mseal-no-memory-seal-preload.c
 create mode 100644 sysdeps/unix/sysv/linux/tst-dl_mseal-no-memory-seal.c
 create mode 100644 sysdeps/unix/sysv/linux/tst-dl_mseal-preload.c
 create mode 100644 sysdeps/unix/sysv/linux/tst-dl_mseal-skeleton.c
 create mode 100644 sysdeps/unix/sysv/linux/tst-dl_mseal-static-no-memory-seal.c
 create mode 100644 sysdeps/unix/sysv/linux/tst-dl_mseal-static.c
 create mode 100644 sysdeps/unix/sysv/linux/tst-dl_mseal.c
 create mode 100644 sysdeps/unix/sysv/linux/tst-mseal.c

Comments

Mike Hommey July 31, 2024, 8:48 p.m. UTC | #1
Thanks for the Cc.

On Wed, Jul 31, 2024 at 05:02:03PM -0300, Adhemerval Zanella wrote:
> The final patch adds support for GNU_PROPERTY_NO_MEMORY_SEAL, where the
> binary can be marked to avoid sealing (i.e., on Firefox hack to bypass the
> dynamic loader and enable DT_RELR on older glibc [3]).  In this case, it is
> up to the module to apply memory sealing itself.  I have a patch to
> binutils to add -Wl,-z,no-memory-seal option to enable it.

Somehow I missed your message from the previous thread where you were
talking about .note.gnu.property. It is rather inconvenient that this
requires a patched or very new linker, and even more inconvenient that
current linkers don't preserve .gnu.note.property from object files,
so it can't even be added "manually" via current linkers.

Mike
Jeff Xu July 31, 2024, 11:36 p.m. UTC | #2
On Wed, Jul 31, 2024 at 1:03 PM Adhemerval Zanella
<adhemerval.zanella@linaro.org> wrote:
>
> The Linux 6.10 (8be7258aad44b5e25977a98db136f677fa6f4370) added the mseal
> syscall that allows blocking some memory operations on the VMA range:
>
>  * Unmapping, moving to another location, extending or shrinking the
>    size, munmap, and mremap.
>  * Moving or expanding a different VMA into the current location, via
>    mremap.
>  * Modifying the memory range with mmap along with flag MAP_FIXED.
>  * Expanding the size with mremap.
>  * Change the protection flags with mprotect or pkey_mprotect.
>  * Destructive behaviors on anonymous memory, such as madvice with
>    MADV_DONTNEED.
>
> Memory sealing might be useful as a hardening mechanism to avoid either
> remapping the memory segments or changing the memory protection segments
> layout by the dynamic loader (for instance, the RELRO hardening).
> A similar hardening is done by OpenBSD with the mimmutable syscall [1].
>
> The first patch expands the use of RTLD_NODELETE for objects that should
> not be deallocated during process execution.  The flag will be used later
> to decide whether to seal the link_map:
>
>  * on dlopen dependencies for objects opened with RTLD_NODELETE
>  * For the main map executable.
>  * For the main map DT_NEEDED dependencies.
>  * On __libc_unwind_link_get for libgcc_s.so (used for backtrace and
>    unwind).
>
> The second patch adds the mseal support for Linux.  Although most
> programs will not use it directly, some specific ones, like Chrome,
> intend to use it.
>
> The third patch adds memory sealing in multiple places where the memory
> is supposed to be immutable over program execution:
>
>  * All shared library dependencies from the binary, including the
>    read-only segments after PT_GNU_RELRO setup.
>  * The binary itself, including dynamic and static links. In both cases,
>    it is up either to binary or the loader to set up the sealing.
>  * Any preload libraries.
>  * Any library loaded with dlopen with RTLD_NODELETE flag (including
>    libgcc.so loaded to enable process unwind and thread cancellation).
>  * Audit modules.
>  * The loader bump allocator.
>
> For binary dependencies, the RTLD_NODELETE signals the link_map should be
> sealed. It also makes dlopen objects with the flag sealed as well.
>
> The sealing is also controlled by a new tunable, glibc.rtld.seal, with
> three different states:
>
>  0. Disabled, where no memory sealing is done.
>  1. Enabled, where the loader will issue the mseal syscall on the
>     memory mappings, and any failure is ignored. This is the default.
>  2. Enforce, similar to Enabled and any failure from the mseal terminates
>     the process.
>
Will those states affect the dlopen() or just the main exec and its
dependency .so.
For example:
If glibc.rtld.seal=1, will ld.so also seal the .so even when
RTLD_NODELETE is not set for ldopen(). I'm thinking about the scenario
where the application didn't migrate to use RTLD_NODELETE, and the
sandboxer wants to seal the .so for all ldopen() calls anyway.


> The final patch adds support for GNU_PROPERTY_NO_MEMORY_SEAL, where the
> binary can be marked to avoid sealing (i.e., on Firefox hack to bypass the
> dynamic loader and enable DT_RELR on older glibc [3]).  In this case, it is
> up to the module to apply memory sealing itself.  I have a patch to
> binutils to add -Wl,-z,no-memory-seal option to enable it.
>
> This patchset does not delay RELRO activation until after their ELF
> constructors have been executed, as suggested on the previous RFC for
> mseal support. It is not strictly required, and it requires extensive
> changes on_dl_start_user to either make _dl_init call RELRO/sealing setup
> after ctor/initarray is done, or call it after _dl_init. There is also the
> question of whether to apply RELRO/sealing per module after dtor/initarray
> or in bulk after _dt_init.
>
> I tested on both x86_64-linux-gnu and aarch64-linux-gnu with Linux
> 6.11-rc1, along with some testing on a powerpc64le-linux-gnu VM. I also
> enabled glibc.rtld.seal=2 to check for possible mseal failures.
>
> [1] https://man.openbsd.org/mimmutable.2
> [2] https://docs.google.com/document/d/1O2jwK4dxI3nRcOJuPYkonhTkNQfbmwdvxQMyXgeaRHo/edit#heading=h.bvaojj9fu6hc
> [3] https://glandium.org/blog/?p=4297
>
> Adhemerval Zanella (5):
>   elf: Use RTLD_NODELETE is more places
>   linux: Add mseal syscall support
>   elf: Add support to memory sealing
>   elf: Also parse gnu properties for static linked binaries
>   elf: Add support for GNU_PROPERTY_NO_MEMORY_SEAL
>
>  NEWS                                          |  15 +
>  configure                                     |  35 +++
>  configure.ac                                  |   5 +
>  elf/dl-load.c                                 |   4 +
>  elf/dl-map-segments.h                         |   5 +
>  elf/dl-minimal-malloc.c                       |   2 +
>  elf/dl-mseal-mode.h                           |  29 ++
>  elf/dl-open.c                                 |   7 +-
>  elf/dl-reloc.c                                |  47 +++
>  elf/dl-support.c                              |  20 ++
>  elf/dl-tunables.list                          |   6 +
>  elf/elf.h                                     |   2 +
>  elf/rtld.c                                    |  13 +-
>  elf/setup-vdso.h                              |   2 +
>  elf/tst-rtld-list-tunables.exp                |   1 +
>  include/dlfcn.h                               |   2 +
>  include/link.h                                |   7 +
>  manual/memory.texi                            |  66 +++++
>  manual/tunables.texi                          |  42 +++
>  misc/unwind-link.c                            |   5 +-
>  sysdeps/aarch64/dl-prop.h                     |   5 +
>  sysdeps/generic/dl-mseal.h                    |  23 ++
>  sysdeps/generic/dl-prop-mseal.h               |  38 +++
>  sysdeps/generic/dl-prop.h                     |   5 +
>  sysdeps/generic/ldsodefs.h                    |   9 +
>  sysdeps/unix/sysv/linux/Makefile              |  94 ++++++
>  sysdeps/unix/sysv/linux/Versions              |   3 +
>  sysdeps/unix/sysv/linux/aarch64/libc.abilist  |   1 +
>  sysdeps/unix/sysv/linux/alpha/libc.abilist    |   1 +
>  sysdeps/unix/sysv/linux/arc/libc.abilist      |   1 +
>  sysdeps/unix/sysv/linux/arm/be/libc.abilist   |   1 +
>  sysdeps/unix/sysv/linux/arm/le/libc.abilist   |   1 +
>  sysdeps/unix/sysv/linux/bits/mman-shared.h    |   8 +
>  sysdeps/unix/sysv/linux/csky/libc.abilist     |   1 +
>  sysdeps/unix/sysv/linux/dl-mseal.c            |  51 ++++
>  sysdeps/unix/sysv/linux/dl-mseal.h            |  27 ++
>  sysdeps/unix/sysv/linux/hppa/libc.abilist     |   1 +
>  sysdeps/unix/sysv/linux/i386/libc.abilist     |   1 +
>  sysdeps/unix/sysv/linux/kernel-features.h     |   8 +
>  .../sysv/linux/loongarch/lp64/libc.abilist    |   1 +
>  .../sysv/linux/m68k/coldfire/libc.abilist     |   1 +
>  .../unix/sysv/linux/m68k/m680x0/libc.abilist  |   1 +
>  .../sysv/linux/microblaze/be/libc.abilist     |   1 +
>  .../sysv/linux/microblaze/le/libc.abilist     |   1 +
>  .../sysv/linux/mips/mips32/fpu/libc.abilist   |   1 +
>  .../sysv/linux/mips/mips64/n32/libc.abilist   |   1 +
>  .../sysv/linux/mips/mips64/n64/libc.abilist   |   1 +
>  sysdeps/unix/sysv/linux/nios2/libc.abilist    |   1 +
>  sysdeps/unix/sysv/linux/or1k/libc.abilist     |   1 +
>  .../linux/powerpc/powerpc32/fpu/libc.abilist  |   1 +
>  .../powerpc/powerpc32/nofpu/libc.abilist      |   1 +
>  .../linux/powerpc/powerpc64/be/libc.abilist   |   1 +
>  .../linux/powerpc/powerpc64/le/libc.abilist   |   1 +
>  .../unix/sysv/linux/riscv/rv32/libc.abilist   |   1 +
>  .../unix/sysv/linux/riscv/rv64/libc.abilist   |   1 +
>  .../unix/sysv/linux/s390/s390-32/libc.abilist |   1 +
>  .../unix/sysv/linux/s390/s390-64/libc.abilist |   1 +
>  sysdeps/unix/sysv/linux/sh/be/libc.abilist    |   1 +
>  sysdeps/unix/sysv/linux/sh/le/libc.abilist    |   1 +
>  .../sysv/linux/sparc/sparc32/libc.abilist     |   1 +
>  .../sysv/linux/sparc/sparc64/libc.abilist     |   1 +
>  sysdeps/unix/sysv/linux/syscalls.list         |   1 +
>  .../unix/sysv/linux/tst-dl_mseal-auditmod.c   |  23 ++
>  .../unix/sysv/linux/tst-dl_mseal-dlopen-1-1.c |  19 ++
>  .../unix/sysv/linux/tst-dl_mseal-dlopen-1.c   |  19 ++
>  .../unix/sysv/linux/tst-dl_mseal-dlopen-2-1.c |  19 ++
>  .../unix/sysv/linux/tst-dl_mseal-dlopen-2.c   |  19 ++
>  .../tst-dl_mseal-dlopen-no-memory-seal-2-1.c  |  19 ++
>  .../tst-dl_mseal-dlopen-no-memory-seal-2.c    |  19 ++
>  sysdeps/unix/sysv/linux/tst-dl_mseal-mod-1.c  |  19 ++
>  sysdeps/unix/sysv/linux/tst-dl_mseal-mod-2.c  |  19 ++
>  .../linux/tst-dl_mseal-mod-no-memory-seal-1.c |  19 ++
>  .../linux/tst-dl_mseal-mod-no-memory-seal-2.c |  19 ++
>  .../tst-dl_mseal-no-memory-seal-auditmod.c    |   1 +
>  .../tst-dl_mseal-no-memory-seal-preload.c     |   1 +
>  .../sysv/linux/tst-dl_mseal-no-memory-seal.c  |  65 +++++
>  .../unix/sysv/linux/tst-dl_mseal-preload.c    |  19 ++
>  .../unix/sysv/linux/tst-dl_mseal-skeleton.c   | 272 ++++++++++++++++++
>  .../tst-dl_mseal-static-no-memory-seal.c      |  38 +++
>  sysdeps/unix/sysv/linux/tst-dl_mseal-static.c |  36 +++
>  sysdeps/unix/sysv/linux/tst-dl_mseal.c        |  67 +++++
>  sysdeps/unix/sysv/linux/tst-mseal.c           |  67 +++++
>  .../unix/sysv/linux/x86_64/64/libc.abilist    |   1 +
>  .../unix/sysv/linux/x86_64/x32/libc.abilist   |   1 +
>  sysdeps/x86/dl-prop.h                         |   4 +
>  85 files changed, 1396 insertions(+), 6 deletions(-)
>  create mode 100644 elf/dl-mseal-mode.h
>  create mode 100644 sysdeps/generic/dl-mseal.h
>  create mode 100644 sysdeps/generic/dl-prop-mseal.h
>  create mode 100644 sysdeps/unix/sysv/linux/dl-mseal.c
>  create mode 100644 sysdeps/unix/sysv/linux/dl-mseal.h
>  create mode 100644 sysdeps/unix/sysv/linux/tst-dl_mseal-auditmod.c
>  create mode 100644 sysdeps/unix/sysv/linux/tst-dl_mseal-dlopen-1-1.c
>  create mode 100644 sysdeps/unix/sysv/linux/tst-dl_mseal-dlopen-1.c
>  create mode 100644 sysdeps/unix/sysv/linux/tst-dl_mseal-dlopen-2-1.c
>  create mode 100644 sysdeps/unix/sysv/linux/tst-dl_mseal-dlopen-2.c
>  create mode 100644 sysdeps/unix/sysv/linux/tst-dl_mseal-dlopen-no-memory-seal-2-1.c
>  create mode 100644 sysdeps/unix/sysv/linux/tst-dl_mseal-dlopen-no-memory-seal-2.c
>  create mode 100644 sysdeps/unix/sysv/linux/tst-dl_mseal-mod-1.c
>  create mode 100644 sysdeps/unix/sysv/linux/tst-dl_mseal-mod-2.c
>  create mode 100644 sysdeps/unix/sysv/linux/tst-dl_mseal-mod-no-memory-seal-1.c
>  create mode 100644 sysdeps/unix/sysv/linux/tst-dl_mseal-mod-no-memory-seal-2.c
>  create mode 100644 sysdeps/unix/sysv/linux/tst-dl_mseal-no-memory-seal-auditmod.c
>  create mode 100644 sysdeps/unix/sysv/linux/tst-dl_mseal-no-memory-seal-preload.c
>  create mode 100644 sysdeps/unix/sysv/linux/tst-dl_mseal-no-memory-seal.c
>  create mode 100644 sysdeps/unix/sysv/linux/tst-dl_mseal-preload.c
>  create mode 100644 sysdeps/unix/sysv/linux/tst-dl_mseal-skeleton.c
>  create mode 100644 sysdeps/unix/sysv/linux/tst-dl_mseal-static-no-memory-seal.c
>  create mode 100644 sysdeps/unix/sysv/linux/tst-dl_mseal-static.c
>  create mode 100644 sysdeps/unix/sysv/linux/tst-dl_mseal.c
>  create mode 100644 sysdeps/unix/sysv/linux/tst-mseal.c
>
> --
> 2.43.0
>
Adhemerval Zanella Netto Aug. 1, 2024, 12:26 p.m. UTC | #3
On 31/07/24 17:48, Mike Hommey wrote:
> Thanks for the Cc.
> 
> On Wed, Jul 31, 2024 at 05:02:03PM -0300, Adhemerval Zanella wrote:
>> The final patch adds support for GNU_PROPERTY_NO_MEMORY_SEAL, where the
>> binary can be marked to avoid sealing (i.e., on Firefox hack to bypass the
>> dynamic loader and enable DT_RELR on older glibc [3]).  In this case, it is
>> up to the module to apply memory sealing itself.  I have a patch to
>> binutils to add -Wl,-z,no-memory-seal option to enable it.
> 
> Somehow I missed your message from the previous thread where you were
> talking about .note.gnu.property. It is rather inconvenient that this
> requires a patched or very new linker, and even more inconvenient that
> current linkers don't preserve .gnu.note.property from object files,
> so it can't even be added "manually" via current linkers.

The .note.gnu.property gives us more freedom to define the required, and
it is used on other configuration for security hardening that requires
loading action, like AArch64 BTI or Intel CET (as I noted [1]).

And the linker support was already raised by Florian on same thread [2].  
At least with newer binutils versions, the '.gnu.note.property' are merged
(but I will double checked if it were the case for GNU_PROPERTY_NO_MEMORY_SEAL
with my patch).

I am not sure how to accomplish all the requirements your are imposing for
a very specific hack without adding an ad-hoc solutions (like a
new ELF mark just for memory sealing disable) or some hacks to make it work
with old linker (specially when recent work is being moving toward to use 
.note.gnu.property for such support).

At least with GNU_PROPERTY_NO_MEMORY_SEAL, you can still mseal your binaries
if requires after you relocate it (you will need to handle the zero padding
as done in elf/dl-map-segments.h).  This is slight better than disable the
sealing altogether with the tunable, and I think it is not that troublesome
to require a linker update to accomplish it.

[1] https://sourceware.org/pipermail/libc-alpha/2024-June/157769.html
[2] https://sourceware.org/pipermail/libc-alpha/2024-June/157853.html
Adhemerval Zanella Netto Aug. 1, 2024, 12:38 p.m. UTC | #4
On 31/07/24 20:36, Jeff Xu wrote:
> On Wed, Jul 31, 2024 at 1:03 PM Adhemerval Zanella
> <adhemerval.zanella@linaro.org> wrote:
>>
>> The Linux 6.10 (8be7258aad44b5e25977a98db136f677fa6f4370) added the mseal
>> syscall that allows blocking some memory operations on the VMA range:
>>
>>  * Unmapping, moving to another location, extending or shrinking the
>>    size, munmap, and mremap.
>>  * Moving or expanding a different VMA into the current location, via
>>    mremap.
>>  * Modifying the memory range with mmap along with flag MAP_FIXED.
>>  * Expanding the size with mremap.
>>  * Change the protection flags with mprotect or pkey_mprotect.
>>  * Destructive behaviors on anonymous memory, such as madvice with
>>    MADV_DONTNEED.
>>
>> Memory sealing might be useful as a hardening mechanism to avoid either
>> remapping the memory segments or changing the memory protection segments
>> layout by the dynamic loader (for instance, the RELRO hardening).
>> A similar hardening is done by OpenBSD with the mimmutable syscall [1].
>>
>> The first patch expands the use of RTLD_NODELETE for objects that should
>> not be deallocated during process execution.  The flag will be used later
>> to decide whether to seal the link_map:
>>
>>  * on dlopen dependencies for objects opened with RTLD_NODELETE
>>  * For the main map executable.
>>  * For the main map DT_NEEDED dependencies.
>>  * On __libc_unwind_link_get for libgcc_s.so (used for backtrace and
>>    unwind).
>>
>> The second patch adds the mseal support for Linux.  Although most
>> programs will not use it directly, some specific ones, like Chrome,
>> intend to use it.
>>
>> The third patch adds memory sealing in multiple places where the memory
>> is supposed to be immutable over program execution:
>>
>>  * All shared library dependencies from the binary, including the
>>    read-only segments after PT_GNU_RELRO setup.
>>  * The binary itself, including dynamic and static links. In both cases,
>>    it is up either to binary or the loader to set up the sealing.
>>  * Any preload libraries.
>>  * Any library loaded with dlopen with RTLD_NODELETE flag (including
>>    libgcc.so loaded to enable process unwind and thread cancellation).
>>  * Audit modules.
>>  * The loader bump allocator.
>>
>> For binary dependencies, the RTLD_NODELETE signals the link_map should be
>> sealed. It also makes dlopen objects with the flag sealed as well.
>>
>> The sealing is also controlled by a new tunable, glibc.rtld.seal, with
>> three different states:
>>
>>  0. Disabled, where no memory sealing is done.
>>  1. Enabled, where the loader will issue the mseal syscall on the
>>     memory mappings, and any failure is ignored. This is the default.
>>  2. Enforce, similar to Enabled and any failure from the mseal terminates
>>     the process.
>>
> Will those states affect the dlopen() or just the main exec and its
> dependency .so.
> For example:
> If glibc.rtld.seal=1, will ld.so also seal the .so even when
> RTLD_NODELETE is not set for ldopen(). I'm thinking about the scenario
> where the application didn't migrate to use RTLD_NODELETE, and the
> sandboxer wants to seal the .so for all ldopen() calls anyway.

The tunable only changes the internal mseal behavior, it does not change
dlopen semantic regarding RTLD_NODELETE.  And I am not sure if we can, 
without also changing the process behavior (since RTLD_NOTDELETE changes
how ctors/dtors are called).
Jeff Xu Aug. 1, 2024, 7:24 p.m. UTC | #5
On Thu, Aug 1, 2024 at 5:38 AM Adhemerval Zanella Netto
<adhemerval.zanella@linaro.org> wrote:
>
>
>
> On 31/07/24 20:36, Jeff Xu wrote:
> > On Wed, Jul 31, 2024 at 1:03 PM Adhemerval Zanella
> > <adhemerval.zanella@linaro.org> wrote:
> >>
> >> The Linux 6.10 (8be7258aad44b5e25977a98db136f677fa6f4370) added the mseal
> >> syscall that allows blocking some memory operations on the VMA range:
> >>
> >>  * Unmapping, moving to another location, extending or shrinking the
> >>    size, munmap, and mremap.
> >>  * Moving or expanding a different VMA into the current location, via
> >>    mremap.
> >>  * Modifying the memory range with mmap along with flag MAP_FIXED.
> >>  * Expanding the size with mremap.
> >>  * Change the protection flags with mprotect or pkey_mprotect.
> >>  * Destructive behaviors on anonymous memory, such as madvice with
> >>    MADV_DONTNEED.
> >>
> >> Memory sealing might be useful as a hardening mechanism to avoid either
> >> remapping the memory segments or changing the memory protection segments
> >> layout by the dynamic loader (for instance, the RELRO hardening).
> >> A similar hardening is done by OpenBSD with the mimmutable syscall [1].
> >>
> >> The first patch expands the use of RTLD_NODELETE for objects that should
> >> not be deallocated during process execution.  The flag will be used later
> >> to decide whether to seal the link_map:
> >>
> >>  * on dlopen dependencies for objects opened with RTLD_NODELETE
> >>  * For the main map executable.
> >>  * For the main map DT_NEEDED dependencies.
> >>  * On __libc_unwind_link_get for libgcc_s.so (used for backtrace and
> >>    unwind).
> >>
> >> The second patch adds the mseal support for Linux.  Although most
> >> programs will not use it directly, some specific ones, like Chrome,
> >> intend to use it.
> >>
> >> The third patch adds memory sealing in multiple places where the memory
> >> is supposed to be immutable over program execution:
> >>
> >>  * All shared library dependencies from the binary, including the
> >>    read-only segments after PT_GNU_RELRO setup.
> >>  * The binary itself, including dynamic and static links. In both cases,
> >>    it is up either to binary or the loader to set up the sealing.
> >>  * Any preload libraries.
> >>  * Any library loaded with dlopen with RTLD_NODELETE flag (including
> >>    libgcc.so loaded to enable process unwind and thread cancellation).
> >>  * Audit modules.
> >>  * The loader bump allocator.
> >>
> >> For binary dependencies, the RTLD_NODELETE signals the link_map should be
> >> sealed. It also makes dlopen objects with the flag sealed as well.
> >>
> >> The sealing is also controlled by a new tunable, glibc.rtld.seal, with
> >> three different states:
> >>
> >>  0. Disabled, where no memory sealing is done.
> >>  1. Enabled, where the loader will issue the mseal syscall on the
> >>     memory mappings, and any failure is ignored. This is the default.
> >>  2. Enforce, similar to Enabled and any failure from the mseal terminates
> >>     the process.
> >>
> > Will those states affect the dlopen() or just the main exec and its
> > dependency .so.
> > For example:
> > If glibc.rtld.seal=1, will ld.so also seal the .so even when
> > RTLD_NODELETE is not set for ldopen(). I'm thinking about the scenario
> > where the application didn't migrate to use RTLD_NODELETE, and the
> > sandboxer wants to seal the .so for all ldopen() calls anyway.
>
> The tunable only changes the internal mseal behavior, it does not change
> dlopen semantic regarding RTLD_NODELETE.  And I am not sure if we can,
> without also changing the process behavior (since RTLD_NOTDELETE changes
> how ctors/dtors are called).

Thanks for clarifying.

I'm not sure how widely RTLD_NODELETE is used,  it might be
interesting to have glibc.rtld.seal_dlopen = 1 to enforce dlopen  when
RTLD_NODELETE is not set.

However, that also potentially breaks the process's expectation on
dlclose(). So I'm not suggesting adding this into this patch.  It can
wait for a real-world use case and if people are interested.

PS: I tested the latest patch on the dynamic linker side (not the ld),
all functional as expected.

[55a5feb15000-55a5feb16000], [rd mr mw me sl      ] [test_glibc.out]
[55a5feb16000-55a5feb17000], [rd ex mr mw me sl   ] [test_glibc.out]
[55a5feb17000-55a5feb18000], [rd mr mw me sl      ] [test_glibc.out]
[55a5feb18000-55a5feb19000], [rd mr mw me ac sl   ] [test_glibc.out]
[55a5feb19000-55a5feb1a000], [rd wr mr mw me ac sl] [test_glibc.out]
[55a623a6b000-55a623a8c000], [rd wr mr mw me ac   ] [[heap]]
[7f572dbd3000-7f572dbe1000], [rd mr mw me sl      ] [libm.so.6]
[7f572dbe1000-7f572dc59000], [rd ex mr mw me sl   ] [libm.so.6]
[7f572dc59000-7f572dcb3000], [rd mr mw me sl      ] [libm.so.6]
[7f572dcb3000-7f572dcb4000], [rd mr mw me ac sl   ] [libm.so.6]
[7f572dcb4000-7f572dcb5000], [rd wr mr mw me ac sl] [libm.so.6]
[7f572dcb5000-7f572dcb8000], [rd wr mr mw me ac sl] [[anonymous]]
[7f572dcb8000-7f572dcdc000], [rd mr mw me sl      ] [libc.so.6]
[7f572dcdc000-7f572de36000], [rd ex mr mw me sl   ] [libc.so.6]
[7f572de36000-7f572de8c000], [rd mr mw me sl      ] [libc.so.6]
[7f572de8c000-7f572de90000], [rd mr mw me ac sl   ] [libc.so.6]
[7f572de90000-7f572de92000], [rd wr mr mw me ac sl] [libc.so.6]
[7f572de92000-7f572dea1000], [rd wr mr mw me ac sl] [[anonymous]]
[7f572dea1000-7f572dea5000], [rd mr pf io de dd   ] [[vvar]]
[7f572dea5000-7f572dea7000], [rd ex mr mw me de   ] [[vdso]]
[7f572dea7000-7f572dea8000], [rd mr mw me sl      ] [ld-linux-x86-64.so.2]
[7f572dea8000-7f572decf000], [rd ex mr mw me sl   ] [ld-linux-x86-64.so.2]
[7f572decf000-7f572deda000], [rd mr mw me sl      ] [ld-linux-x86-64.so.2]
[7f572deda000-7f572dedc000], [rd mr mw me ac sl   ] [ld-linux-x86-64.so.2]
[7f572dedc000-7f572dede000], [rd wr mr mw me ac sl] [ld-linux-x86-64.so.2]
[7ffe7a64b000-7ffe7a66c000], [rd wr mr mw me gd ac] [[stack]]

parsed from /proc/pid/smaps
"sl" - sealed
libm.so is called with dlopen(RTLD_NODELETE)

This looks great, Thanks!


-Jeff