[v2] manual: Describe struct link_map

---
v2: Fix Texinfo issue.
 manual/dynlink.texi | 76 ++++++++++++++++++++++++++++++++++++++++++++++++++++-
 1 file changed, 75 insertions(+), 1 deletion(-)


base-commit: 7a630f7d3392ca391a399486ce2846f9e4b4ee63

Various comments and questions from an invested user, as requested by Ben Woodard.

On Mon, 2024-08-05 at 16:12 +0200, Florian Weimer wrote:
> ```
> +@deftp {Data Type} {struct link_map}
> +
> +@cindex link map
> +A @dfn{link map} is associated with the main executable and each shared
> +object.  Some fields of the link map are accesible to applications and
> +exposed through the @code{stuct link_map}.  Applications must not modify
> +the link map directly.
> ```

The interfaces that give a link map write it as `struct link_map*`. If applications "must not" modify it (i.e. it is a bug to do so) can these interfaces be adjusted to use `const struct link_map*` instead?

> ```
> +@item l_addr
> +@cindex load bias
> +This field contains the @dfn{load bias} of the object: the difference
> +between the actual load address of the object, and the virtual addresses
> +found in the program header.  For position-dependent executables, this
> ```

This paragraph does not clearly define the "load address of an object," nor does it use "load bias" aside from defining it at the start. Other parts of the documentation call this difference the "base address", e.g. from `man dl_iterate_phdr(1)`:

> The dlpi_addr  field  indicates  the  base address of the shared object (i.e., the difference between the virtual memory address of the
> shared object and the offset of that object in the file from which it was loaded).

The documentation would greatly benefit from using a consistent name and rich description for this value.

> ```
> +difference is zero because they start at virtual address zero in the
> +program headers and are loaded at address zero, so @code{l_addr} is zero
> +as well.  Shared objects and position-independent executables (PIEs) use
> +virtual address zero in the program headers, but are loaded at some
> +other address.  This is the reason that for them, @code{l_addr} usually
> +equals the load address of the object.
> ```

These two sentences seem to contradict, "shared objects use virtual address zero" but `l_addr` only "usually equals" the load address. When is it the case that `l_addr != 0` does NOT imply that `l_addr` is the "load address?" Is a shared object with a non-zero virtual address valid?

Can `l_addr` be used as a unique identifier for a `link_map` in the process or namespace? The text would benefit by adding a short answer/discussion here.

> ```
> +On Linux, to obtain the lowest loaded address of the main program, use
> +@code{getauxval} to obtain the @code{AT_PHDR}, @code{AT_PHENT},
> +@code{AT_PHNUM} values for the current process.  These values allow
> +processing the array of program headers, and the addressing information
> +in its @code{PT_LOAD} entries.  This works even when the program was
> +started with an explicit loader invocation.
> ```

A more portable method than `getauxval` is to use `dl_iterate_phdr`. The documentation would greatly benefit from mentioning or replacing this text with a `dl_iterate_phdr`-based workaround.

I find it extremely problematic and unfortunate that there is not a simple public method to obtain the program headers for a shared object from a `link_map*`.

> ```
> +@item l_name
> +For a shared object, this field contains the file name that the
> +@theglibc{} dynamic loader used when opening the object.  This can be a
> +relative path (relative the current directory at process start).
> ```

This comment is incorrect: if the application calls `dlopen()` with a relative path after `chdir()`, the path is relative to the new working directory and not the working directory at process start.

It would help to have some instruction here on how to acquire the absolute path for an object when `l_name` is a relative path, specifically in the case where `chdir()` was called unknowingly or in parallel.

> ```
> +Symbolic links are not necessarily resolved.
> ```

Are symlinks ever resolved by ld.so? Or is this a comment on future compatibility?

> ```
> +For the main executable, @code{l_name} is @samp{""} (the empty string).
> +(The main executable is not loaded by @theglibc{}, so its file name is
> +not available.)  On Linux, the main executable is available as
> +@file{/proc/self/exe} (unless an explicit loader invocation was used to
> ```

IIRC on Linux `/proc` need not be mounted (or may be mounted somewhere other than `/proc`). The text should make it clear that this file may not exist.

> ```
> +start the program).  The file name @file{/proc/self/exe} continues to
> +resolve to the same file even if it is moved within or deleted from the
> +file system.  Its current location can be read using @code{readlink}.
> +@xref{Symbolic Links}.  (Although @file{/proc/self/exe} is not actually
> +a symbol link, it is only presented as one.)
> ```

A portable solution is to use `dladdr` (e.g. `dladdr(l_ld)`), although it returns `argv[0]` which may not match `/proc/self/exe`. The text here would greatly benefit by including an `dladdr`-based workaround and discussing the subtle differences between these two workarounds.

> ```
> +
> +If an explicit loader invocation is used (such as @samp{ld.so
> +/usr/bin/emacs}), the @file{/proc/self/exe} approach does not work
> +because the file name refers to the dynamic linker @code{ld.so}, and not
> +the @code{/usr/bin/emacs} program.
> ```

Is there any method to safely identify when an explicit loader invocation was used?

> ```
> +@item l_ld
> +This is a pointer to the ELF dynamic segment, an array of tag/value
> +pairs that provide various pieces of information that the dynamic
> +linking process uses.  On most architectures, addreses in the dynamic
> +segment are relocated at run time, but on some targets, it is necessary
> +to add the @code{l_addr} field value to obtain a proper address.
> ```

Which architectures require adding `l_addr`? Will that list remain stable for future versions of Glibc? Is there a query we can use to determine if we're on one of these architectures?

In the context of LD_AUDIT, when is this relocation guaranteed to have occurred? Before `la_objopen()` or before a following `la_activity(CONSISTENT)` call?

> ```
> +@item l_prev
> +@itemx l_next
> +These fields are used to main a double-linked linked list of all link
> +maps within on @code{dlmopen} namespace.  Note that there is currently
> +no thread-safe way to iteratoe over this list.  The callback-based
> +@code{dl_iterate_phdr} interface can be used instead.
> ```

`dl_iterate_phdr` is a very imperfect replacement, it only iterates through the `link_map`s of the caller's namespace. The text here would greatly benefit by adding some instruction on a thread-safe way to iterate over an arbitrary `dlmopen`'d namespace.

> ```
> +@strong{Portability note:} In current versions of @theglibc{}, handles
> +returned by @code{dlopen} and @code{dlmopen} are pointers to link maps.
> +However, this is not a portable assumption, and may even change in
> +future versions of @theglibc{}.  To obtain the link map associated with
> +a handle, see @code{dlinfo} and @code{RTLD_DI_LINKMAP} below.  To obtain
> +a handle for a link map, it is possible to use the @code{l_name}
> +argument in a @code{dlopen} call, preferably with the @code{RTLD_NOLOAD}
> +flag.
> ```

Today a `link_map*` can be passed into e.g. `dlinfo` as if it was a `dlopen` handle. Is this also subject to change? If so, is there a portable way to obtain a handle from a `link_map*` *without* triggering ELF constructors?

If `l_name` is relative and e.g. `chdir("subdir")` was called, is it portable to expect `dlopen(l_name)` to refer to the original object? Today it seems to, but that also feels very much like a bug and shouldn't be relied on.

-Jonathon

On 8/6/24 7:47 AM, Jonathon Anderson wrote:
>> |+On Linux, to obtain the lowest loaded address of the main program, 
>> use +@code{getauxval} to obtain the @code{AT_PHDR}, @code{AT_PHENT}, 
>> +@code{AT_PHNUM} values for the current process. These values allow 
>> +processing the array of program headers, and the addressing 
>> information +in its @code{PT_LOAD} entries. This works even when the 
>> program was +started with an explicit loader invocation. |
>
> A more portable method than |getauxval| is to use |dl_iterate_phdr|. 
> The documentation would greatly benefit from mentioning or replacing 
> this text with a |dl_iterate_phdr|-based workaround.
>
> I find it extremely problematic and unfortunate that there is not a 
> simple public method to obtain the program headers for a shared object 
> from a |link_map*|.
>
I know that there is the portability note further down in the document 
saying that a dl handle is not interchangeable with a link_map* pointer 
but for the moment it is. Would calling dlinfo with RTLD_DI_PHDR meet 
your needs?

 From /usr/include/dlfcn.h:

     /* Treat ARG as const ElfW(Phdr) **, and store the address of the
        program header array at that location.  The dlinfo call returns
        the number of program headers in the array.  */
     RTLD_DI_PHDR = 11,

This request was so recently added to glibc that the man pages have not 
yet caught up.

<snip>

>     |+@strong{Portability note:} In current versions of @theglibc{},
>     handles +returned by @code{dlopen} and @code{dlmopen} are pointers
>     to link maps. +However, this is not a portable assumption, and may
>     even change in +future versions of @theglibc{}. To obtain the link
>     map associated with +a handle, see @code{dlinfo} and
>     @code{RTLD_DI_LINKMAP} below. To obtain +a handle for a link map,
>     it is possible to use the @code{l_name} +argument in a
>     @code{dlopen} call, preferably with the @code{RTLD_NOLOAD} +flag. |
>
>     Today a |link_map*| can be passed into e.g. |dlinfo| as if it was
>     a |dlopen| handle. Is this also subject to change? If so, is there
>     a portable way to obtain a handle from a |link_map*| /without/
>     triggering ELF constructors?
>
If there were an audit function like:

void *la_dlhandle( struct link_map *map);

which portably converted from a link_map* to a dl handle would you 
expect to be able to call dlsym() on the resulting handle? or would 
being able to portably call dlinfo() be sufficient? Evidently, there is 
some additional setup above and beyond the pointer conversion which 
would be needed if you needs included being able to call dlsym(). on the 
resulting handle.

-ben

* Jonathon Anderson:

> On Mon, 2024-08-05 at 16:12 +0200, Florian Weimer wrote:
>
>  +@deftp {Data Type} {struct link_map}
> +
> +@cindex link map
> +A @dfn{link map} is associated with the main executable and each shared
> +object.  Some fields of the link map are accesible to applications and
> +exposed through the @code{stuct link_map}.  Applications must not modify
> +the link map directly.
>
> The interfaces that give a link map write it as struct link_map*. If
> applications "must not" modify it (i.e. it is a bug to do so) can
> these interfaces be adjusted to use const struct link_map* instead?

It's probably too late for that.  We generally do not make such changes
because they can impact application ABIs if they use C++.  Maybe the
risk is fairly minimal here.  I would welcome more feedback.

> +@item l_addr
> +@cindex load bias
> +This field contains the @dfn{load bias} of the object: the difference
> +between the actual load address of the object, and the virtual addresses
> +found in the program header.  For position-dependent executables, this
>
> This paragraph does not clearly define the "load address of an
> object," nor does it use "load bias" aside from defining it at the
> start. Other parts of the documentation call this difference the "base
> address", e.g. from man dl_iterate_phdr(1):
>
>    The dlpi_addr field indicates the base address of the shared object
>    (i.e., the difference between the virtual memory address of the
>    shared object and the offset of that object in the file from which
>    it was loaded).

I think this isn't quite right.  The file offset of the ELF header is
always zero, and it's unclear whether “virtual memory address” refers to
the p_vaddr member or address as seen by the CPU, in the in-memory
process image.

> The documentation would greatly benefit from using a consistent name
> and rich description for this value.
>
> +difference is zero because they start at virtual address zero in the
> +program headers and are loaded at address zero, so @code{l_addr} is zero
> +as well.  Shared objects and position-independent executables (PIEs) use
> +virtual address zero in the program headers, but are loaded at some
> +other address.  This is the reason that for them, @code{l_addr} usually
> +equals the load address of the object.
>
> These two sentences seem to contradict, "shared objects use virtual
> address zero" but l_addr only "usually equals" the load address. When
> is it the case that l_addr != 0 does NOT imply that l_addr is the
> "load address?" Is a shared object with a non-zero virtual address
> valid?

I'm not sure if we can correctly load shared objects which have p_vaddr
!= 0.  There was some discussion recently concerning the handling of
ET_DYN main programs with p_vaddr != 0 (in bug 31799
<https://sourceware.org/bugzilla/show_bug.cgi?id=31799>).  The ELF
format has a lot of redundant data in this area.

I'll try to reword all this and resubmit.

> Can l_addr be used as a unique identifier for a link_map in the
> process or namespace? The text would benefit by adding a short
> answer/discussion here.

The unique identifier is the address of the link map (the struct
link_map * value).  Secondary namespaces can contain proxy link maps
that share l_addr with the main namespace, but which have distinct link
maps and link map addresses.  Currently, this is only used for ld.so.

> +On Linux, to obtain the lowest loaded address of the main program, use
> +@code{getauxval} to obtain the @code{AT_PHDR}, @code{AT_PHENT},
> +@code{AT_PHNUM} values for the current process.  These values allow
> +processing the array of program headers, and the addressing information
> +in its @code{PT_LOAD} entries.  This works even when the program was
> +started with an explicit loader invocation.
>
> A more portable method than getauxval is to use dl_iterate_phdr. The
> documentation would greatly benefit from mentioning or replacing this
> text with a dl_iterate_phdr-based workaround.

Does the main program show up in the dl_iterate_phdr iteration if you
call it from a secondary namespace?  I'm kind of surprised this actually
works.  I think using dlinfo with RTLD_DI_PHDR on the link map from
_r_debug is probably the way to go (with a documentation update to
clarify that this is expected to work going forward).

> +@item l_name
> +For a shared object, this field contains the file name that the
> +@theglibc{} dynamic loader used when opening the object.  This can be a
> +relative path (relative the current directory at process start).
>
> This comment is incorrect: if the application calls dlopen() with a
> relative path after chdir(), the path is relative to the new working
> directory and not the working directory at process start.

Good point, will fix.

> It would help to have some instruction here on how to acquire the
> absolute path for an object when l_name is a relative path,
> specifically in the case where chdir() was called unknowingly or in
> parallel.

I think if chdir is called in parallel, the dynamic linker currently
does not necessarily record the correct directory.  (It does not use
/proc/self/fd to read the full path of the opened file.)

Currently, there is no direct way to obtain the path that I can see.  It
seems possible to reconstruct it from the RTLD_DI_ORIGIN dlinfo
operation (with the buffer overflow caveat) and the final file name
component in l_name.  If that's a useful operation, we should add a more
direct way to do this, with the caveat that of course the file at the
path might be different once this function is called.

If you need a file descriptor from an la_objopen callback, we can
directly give you the descriptor that the dynamic linker used.
Currently the file is already closed at the la_objopen event, but that
doesn't look too hard to change.

>  +Symbolic links are not necessarily resolved.
>
> Are symlinks ever resolved by ld.so Or is this a comment on future
> compatibility?

They are resolved (by the kernel) if we use /proc/self/exe to obtain the
$ORIGIN dynamic string token for the main program.

>  +For the main executable, @code{l_name} is @samp{""} (the empty string).
> +(The main executable is not loaded by @theglibc{}, so its file name is
> +not available.)  On Linux, the main executable is available as
> +@file{/proc/self/exe} (unless an explicit loader invocation was used to
>
> IIRC on Linux /proc need not be mounted (or may be mounted somewhere
> other than /proc). The text should make it clear that this file may
> not exist.

I'll add something.

>  +start the program).  The file name @file{/proc/self/exe} continues to
> +resolve to the same file even if it is moved within or deleted from the
> +file system.  Its current location can be read using @code{readlink}.
> +@xref{Symbolic Links}.  (Although @file{/proc/self/exe} is not actually
> +a symbol link, it is only presented as one.)
>
> A portable solution is to use dladdr (e.g. dladdr(l_ld)), although it
> returns argv[0] which may not match /proc/self/exe. The text here
> would greatly benefit by including an dladdr-based workaround and
> discussing the subtle differences between these two workarounds.

Again, if this is important functionality, we should a more direct
approach (perhaps again using dlinfo).

> +If an explicit loader invocation is used (such as @samp{ld.so
> +/usr/bin/emacs}), the @file{/proc/self/exe} approach does not work
> +because the file name refers to the dynamic linker @code{ld.so}, and not
> +the @code{/usr/bin/emacs} program.
>
> Is there any method to safely identify when an explicit loader
> invocation was used?

The file /proc/self/auxv has a copy of the original auxiliary vector.
The return value from getauxval (AT_PHDR) has been adjusted by glibc,
but the value /proc/self/auxv is unchanged.  Therefore, if the values
are different, an explicit loader invocation occurred.

In the future, we might use the kernel's checkpoint/restore
functionality to preserve /proc/self/auxv and /proc/self/exe, but it's
not trivial to implement, and I don't know if we'll ever get to it.  It
could completely hide that an explicit loader invocation has happened,
helping with tools compatibility.

> +@item l_ld
> +This is a pointer to the ELF dynamic segment, an array of tag/value
> +pairs that provide various pieces of information that the dynamic
> +linking process uses.  On most architectures, addreses in the dynamic
> +segment are relocated at run time, but on some targets, it is necessary
> +to add the @code{l_addr} field value to obtain a proper address.
>
> Which architectures require adding l_addr? Will that list remain
> stable for future versions of Glibc? Is there a query we can use to
> determine if we're on one of these architectures?

I think it's currently MIPS and RISC-V.  On MIPS, the DYNAMIC segment
falls into a read-only LOAD segment, so it's easily detectable in
generic code.  On RISC-V, the behavior was incorrectly copied over from
the MIPS glibc implementation, but the linkers do not actually place the
DYNAMIC segment into a read-only LOAD segment because on the binutils
side, MIPS apparently wasn't used at the starting point.  We maintain an
internal macro definition with this information.  Maybe we should turn
this into an installed header?

The RISC-V accident is quite unfortunate, but too late to correct now
for the existing ABIs.  It's now more explicitly handled in the glibc
sources, and I hope we're going to catch it in future port submissions
before it becomes an official part of the ABI.

> In the context of LD_AUDIT, when is this relocation guaranteed to have
> occurred? Before la_objopen () or before a following
> la_activity(CONSISTENT) call?

Great question.

The object is definitely unrelocated at the point of the la_objopen
call.  I believe the intent is that at LA_ACT_CONSISTENT event, the
intent is that objects are relocated (but constructors have not run
yet).  This is what happens with the main program.  But for dlopen, we
currently signal LA_ACT_CONSISTENT before relocation processing.  This
is in part what this bug is about (which I think is familiar to you):

  Loading the same library within an audit library and within an
  application can cause ld.so to crash with an assert.
  <https://sourceware.org/bugzilla/show_bug.cgi?id=31986>

My posted patches change the dlopen behavior to match the initial load
behavior, i.e., at the LA_ACT_CONSISTENT event, objects have been
relocated.

> +@item l_prev
> +@itemx l_next
> +These fields are used to main a double-linked linked list of all link
> +maps within on @code{dlmopen} namespace.  Note that there is currently
> +no thread-safe way to iteratoe over this list.  The callback-based
> +@code{dl_iterate_phdr} interface can be used instead.
>
> dl_iterate_phdr is a very imperfect replacement, it only iterates
> through the link_maps of the caller's namespace. The text here would
> greatly benefit by adding some instruction on a thread-safe way to
> iterate over an arbitrary dlmopen'd namespace.

It's really ugly right now: call dl_iterate_phdr to obtain the loader
lock, and then use struct r_debug_extended from the callback function
(support for which we have not backported yet, though).

The locking in dl_iterate_phdr will remain for backwards compatibility
(older libgcc_s depends on it), so this will keep working.

I'm not sure what a better interface would look like.  If we make just a
shallow copy of the lists, we'd need to ensure that link map pointers
get never invalidated.  (Not very hard to do.)  But the application
might want to make sure it operates on a consistent snapshot, and that
won't be the case if we reuse the link maps for newly loaded objects.

We should probably add some tests that we signal LA_ACT_CONSISTENT when
the loader lock is active, so that the _r_debug link map list cannot
change underneath the auditor.  I don't think this locking behavior is
likely to change.

> +@strong{Portability note:} In current versions of @theglibc{}, handles
> +returned by @code{dlopen} and @code{dlmopen} are pointers to link maps.
> +However, this is not a portable assumption, and may even change in
> +future versions of @theglibc{}.  To obtain the link map associated with
> +a handle, see @code{dlinfo} and @code{RTLD_DI_LINKMAP} below.  To obtain
> +a handle for a link map, it is possible to use the @code{l_name}
> +argument in a @code{dlopen} call, preferably with the @code{RTLD_NOLOAD}
> +flag.
>
> Today a link_map* can be passed into e.g. dlinfo as if it was a dlopen
> handle. Is this also subject to change? If so, is there a portable way
> to obtain a handle from a link_map* without triggering ELF
> constructors?

There currently isn't.  If you are mainly concerned about dlinfo, we can
document it as an extension that it accepts both (struct link_map * and
dlopen handles).  I think we also need to add proxy map handling, which
is currently missing.  Looking at the code, I expect RTLD_DI_PHDR
currently returns a null pointer for the ld.so proxy map in a dlmopen
namespace.

> If l_name is relative and e.g. chdir("subdir") was called, is it
> portable to expect dlopen(l_name) to refer to the original object?
> Today it seems to, but that also feels very much like a bug and
> shouldn't be relied on.

I don't see a way around it.  File system stress from dlopen is already
a thing, and we can't really revisit the file system each time dlopen is
called for a known path.

Thanks,
Florian

On Wed, 2024-08-07 at 08:49 -0700, Ben Woodard wrote:
> > On 8/6/24 7:47 AM, Jonathon Anderson       wrote:  
> > > ```
> > > +On Linux, to obtain the lowest loaded address of the main program, use
> > > +@code{getauxval} to obtain the @code{AT_PHDR}, @code{AT_PHENT},
> > > +@code{AT_PHNUM} values for the current process.  These values allow
> > > +processing the array of program headers, and the addressing information
> > > +in its @code{PT_LOAD} entries.  This works even when the program was
> > > +started with an explicit loader invocation.
> > > ```
> > A more portable method than `getauxval` is to use `dl_iterate_phdr`.         The documentation would greatly benefit from mentioning or         replacing this text with a `dl_iterate_phdr`-based         workaround.
> > I find it extremely problematic and unfortunate that there is         not a simple public method to obtain the program headers for a         shared object from a `link_map*`.
> 
> I know that there is the portability note further down in the       document saying that a dl handle is not interchangeable with a       link_map* pointer but for the moment it is. Would calling dlinfo       with RTLD_DI_PHDR meet your needs?  
> From /usr/include/dlfcn.h:  
> 
>     /* Treat ARG as const ElfW(Phdr) **, and store the address of       the  
>              program header array at that location.  The dlinfo call       returns  
>              the number of program headers in the array.  */  
>           RTLD_DI_PHDR = 11,  
> 
> This request was so recently added to glibc that the man pages       have not yet caught up.  
> <snip>  

Aside from the portability issue `RTLD_DI_PHDR` is fine for our purposes. I had no idea this request was here. Great! What version was it added?

> > > ```
> > > +@strong{Portability note:} In current versions of @theglibc{}, handles
> > > +returned by @code{dlopen} and @code{dlmopen} are pointers to link maps.
> > > +However, this is not a portable assumption, and may even change in
> > > +future versions of @theglibc{}.  To obtain the link map associated with
> > > +a handle, see @code{dlinfo} and @code{RTLD_DI_LINKMAP} below.  To obtain
> > > +a handle for a link map, it is possible to use the @code{l_name}
> > > +argument in a @code{dlopen} call, preferably with the @code{RTLD_NOLOAD}
> > > +flag.
> > > ```
> > Today a `link_map*` can be passed into e.g. `dlinfo`           as if it was a `dlopen` handle. Is this also           subject to change? If so, is there a portable way to obtain a           handle from a `link_map*` *without*           triggering ELF constructors?
> 
> If there were an audit function like:  
> void *la_dlhandle( struct link_map *map);  
> which portably converted from a link_map* to a dl handle would       you expect to be able to call dlsym() on the resulting handle? or       would being able to portably call dlinfo() be sufficient?       Evidently, there is some additional setup above and beyond the       pointer conversion which would be needed if you needs included       being able to call dlsym(). on the resulting handle.  

We definitely will need `dlinfo()`. I can make an argument against being able to call `dlsym()`, but honestly having different "kinds" of dl handles is going to be a headache to remember.

-Jonathon

* Jonathon Anderson:

> Aside from the portability issue RTLD_DI_PHDR is fine for our
> purposes.

That's good to know.  If I recall correctly, we added it specifically
for your use case. 8-)

> I had no idea this request was here. Great! What version was it added?

It went into glibc 2.36 initially, but we backported it all the way to
glibc 2.34 upstream, so various older distributions have it, too,
including RHEL 9.1.

We also did a downstream-specific backport to RHEL 8.7 and later.

> We definitely will need dlinfo(). I can make an argument against being
> able to call dlsym(), but honestly having different "kinds" of dl
> handles is going to be a headache to remember.

We should probably return a nice error message when we detect an
un-dlopen'ed link map in dlsym/dlvsym, instead of crashing.  That should
help to reduce confusion.

Thanks,
Florian

On 8/7/24 10:43 AM, Florian Weimer wrote:
> * Jonathon Anderson:
>
>> On Mon, 2024-08-05 at 16:12 +0200, Florian Weimer wrote:
>>
>>   +@deftp {Data Type} {struct link_map}
>> +
>> +@cindex link map
>> +A @dfn{link map} is associated with the main executable and each shared
>> +object.  Some fields of the link map are accesible to applications and
>> +exposed through the @code{stuct link_map}.  Applications must not modify
>> +the link map directly.
>>
>> The interfaces that give a link map write it as struct link_map*. If
>> applications "must not" modify it (i.e. it is a bug to do so) can
>> these interfaces be adjusted to use const struct link_map* instead?
> It's probably too late for that.  We generally do not make such changes
> because they can impact application ABIs if they use C++.  Maybe the
> risk is fairly minimal here.  I would welcome more feedback.

Since the functions that provide the link_map* pointer to tools are 
exported with C linkage and therefore the symbol is not mangled, how 
would changing the parameter type to const link_map* impact tools 
written in C++?

it seems like we are really only talking about changing:

unsigned int la_objopen (struct link_map *__map, Lmid_t __lmid, 
uintptr_t *__cookie);
to:
unsigned int la_objopen (const struct link_map *__map, Lmid_t __lmid, 
uintptr_t *__cookie);

when an app calls dlinfo( handle, RTLD_DI_LINKMAP, info) they are going 
to be casting the void* info into something anyway and so it seems like 
for this case it would just be a documentation change from:

      RTLD_DI_LINKMAP (struct link_map **)

to:

      RTLD_DI_LINKMAP ( const struct link_map **)

reminding them to not to mess with any values in the link_map structure 
that they got back.

-ben

On Wed, 2024-08-07 at 19:43 +0200, Florian Weimer wrote:
> ```
> > +On Linux, to obtain the lowest loaded address of the main program, use
> > +@code{getauxval} to obtain the @code{AT_PHDR}, @code{AT_PHENT},
> > +@code{AT_PHNUM} values for the current process.  These values allow
> > +processing the array of program headers, and the addressing information
> > +in its @code{PT_LOAD} entries.  This works even when the program was
> > +started with an explicit loader invocation.
> > 
> > A more portable method than getauxval is to use dl_iterate_phdr. The
> > documentation would greatly benefit from mentioning or replacing this
> > text with a dl_iterate_phdr-based workaround.
> 
> 
> Does the main program show up in the dl_iterate_phdr iteration if you
> call it from a secondary namespace?  I'm kind of surprised this actually
> works.
> ```

No, it doesn't, I meant you can call `dl_iterate_phdr` in the main namespace. (I usually `dlmopen()` a shim library into the main namespace from the secondary namespace.)

> ```
> I think using dlinfo with RTLD_DI_PHDR on the link map from
> _r_debug is probably the way to go (with a documentation update to
> clarify that this is expected to work going forward).
> ```

When is the earliest we can expect `dlinfo(_r_debug.r_map)` to work in an auditor? During init constructors or do we need to wait until `la_objopen` or `la_activity`? (This should be also documented somewhere in the man pages.)

> ```
> > It would help to have some instruction here on how to acquire the
> > absolute path for an object when l_name is a relative path,
> > specifically in the case where chdir() was called unknowingly or in
> > parallel.
> 
> 
> I think if chdir is called in parallel, the dynamic linker currently
> does not necessarily record the correct directory.  (It does not use
> /proc/self/fd to read the full path of the opened file.)
> ```

Ah, right, parallel has that issue too. I was only thinking about the auditor side. Let's just stick with "unknowingly" (i.e. the working directory is different when reading `l_name` from what it was when it was loaded).

> ```
> Currently, there is no direct way to obtain the path that I can see.  It
> seems possible to reconstruct it from the RTLD_DI_ORIGIN dlinfo
> operation (with the buffer overflow caveat) and the final file name
> component in l_name.  If that's a useful operation, we should add a more
> direct way to do this,
> ```

This would be really helpful for us, we record paths for most shared objects in our output (for further processing post-mortem) so a direct route to get the file path is definitely helpful.

> ```
> with the caveat that of course the file at the
> path might be different once this function is called.
>
> If you need a file descriptor from an la_objopen callback, we can
> directly give you the descriptor that the dynamic linker used.
> Currently the file is already closed at the la_objopen event, but that
> doesn't look too hard to change.
> ```

I consider access to this file descriptor an important feature. It wouldn't be mission-critical for us right now, but I don't see a way around the "file might have changed on-disk" caveat without it.

> ```
> > +If an explicit loader invocation is used (such as @samp{ld.so
> > +/usr/bin/emacs}), the @file{/proc/self/exe} approach does not work
> > +because the file name refers to the dynamic linker @code{ld.so}, and not
> > +the @code{/usr/bin/emacs} program.
> >
> > Is there any method to safely identify when an explicit loader
> > invocation was used?
> 
> The file /proc/self/auxv has a copy of the original auxiliary vector.
> The return value from getauxval (AT_PHDR) has been adjusted by glibc,
> but the value /proc/self/auxv is unchanged.  Therefore, if the values
> are different, an explicit loader invocation occurred.
> 
> In the future, we might use the kernel's checkpoint/restore
> functionality to preserve /proc/self/auxv and /proc/self/exe, but it's
> not trivial to implement, and I don't know if we'll ever get to it.  It
> could completely hide that an explicit loader invocation has happened,
> helping with tools compatibility.
> ```

The only reason I ask is because it prevents the `/proc/self/exe` solution from working. If we can get a `dlinfo` request to get the path from the executable's `link_map*` we would just use that and this will be moot.

> ```
> > +@item l_ld
> > +This is a pointer to the ELF dynamic segment, an array of tag/value
> > +pairs that provide various pieces of information that the dynamic
> > +linking process uses.  On most architectures, addreses in the dynamic
> > +segment are relocated at run time, but on some targets, it is necessary
> > +to add the @code{l_addr} field value to obtain a proper address.
> > 
> > Which architectures require adding l_addr? Will that list remain
> > stable for future versions of Glibc? Is there a query we can use to
> > determine if we're on one of these architectures?
> 
> 
> I think it's currently MIPS and RISC-V.  On MIPS, the DYNAMIC segment
> falls into a read-only LOAD segment, so it's easily detectable in
> generic code.  On RISC-V, the behavior was incorrectly copied over from
> the MIPS glibc implementation, but the linkers do not actually place the
> DYNAMIC segment into a read-only LOAD segment because on the binutils
> side, MIPS apparently wasn't used at the starting point.  We maintain an
> internal macro definition with this information.  Maybe we should turn
> this into an installed header?
> ```

Please do.

> ```
> > In the context of LD_AUDIT, when is this relocation guaranteed to have
> > occurred? Before la_objopen () or before a following
> > la_activity(CONSISTENT) call?
> 
> 
> Great question.
> 
> The object is definitely unrelocated at the point of the la_objopen
> call.  I believe the intent is that at LA_ACT_CONSISTENT event, the
> intent is that objects are relocated (but constructors have not run
> yet).  This is what happens with the main program.  But for dlopen, we
> currently signal LA_ACT_CONSISTENT before relocation processing.  This
> is in part what this bug is about (which I think is familiar to you):
> 
>   Loading the same library within an audit library and within an
>   application can cause ld.so to crash with an assert.
>   <[https://sourceware.org/bugzilla/show_bug.cgi?id=31986](https://sourceware.org/bugzilla/show_bug.cgi?id=31986)>
> 
> My posted patches change the dlopen behavior to match the initial load
> behavior, i.e., at the LA_ACT_CONSISTENT event, objects have been
> relocated.
> ```

I haven't tested the patches yet, but the behavior described here is what I was expecting. Great! It would be nice to have this detail confirmed in the man pages.

> ```
> > +@item l_prev
> > +@itemx l_next
> > +These fields are used to main a double-linked linked list of all link
> > +maps within on @code{dlmopen} namespace.  Note that there is currently
> > +no thread-safe way to iteratoe over this list.  The callback-based
> > +@code{dl_iterate_phdr} interface can be used instead.
> > 
> > dl_iterate_phdr is a very imperfect replacement, it only iterates
> > through the link_maps of the caller's namespace. The text here would
> > greatly benefit by adding some instruction on a thread-safe way to
> > iterate over an arbitrary dlmopen'd namespace.
> 
> 
> It's really ugly right now: call dl_iterate_phdr to obtain the loader
> lock, and then use struct r_debug_extended from the callback function
> (support for which we have not backported yet, though).
> 
> The locking in dl_iterate_phdr will remain for backwards compatibility
> (older libgcc_s depends on it), so this will keep working.
> ```

I don't see where the `r_debug_extended` needs to come in, so long as you have the `link_map*` you should be able to iterate it with the lock held:

```
int cb(struct dl_phdr_info*, size_t, void* data) {
  for (const struct link_map* lm = data; lm != NULL; lm = lm->l_next) {
    // do something
  }
  return -1;
}
int main() {
  void* some_handle = dlmopen(LM_ID_NEWLM, "libfoo.so", ...);
  const struct link_map* lm;
  dlinfo(some_handle, RTLD_DI_LINKMAP, &lm);
  dl_iterate_phdr(cb, lm);
}
```

I guess my comment here is really: there is a thread-safe way to iterate over the `l_prev`/`l_next` list, so the text here should include an example of how to correctly (and thread-safely) iterate over the list, something like the above code fragment.

> ```
> > +@strong{Portability note:} In current versions of @theglibc{}, handles
> > +returned by @code{dlopen} and @code{dlmopen} are pointers to link maps.
> > +However, this is not a portable assumption, and may even change in
> > +future versions of @theglibc{}.  To obtain the link map associated with
> > +a handle, see @code{dlinfo} and @code{RTLD_DI_LINKMAP} below.  To obtain
> > +a handle for a link map, it is possible to use the @code{l_name}
> > +argument in a @code{dlopen} call, preferably with the @code{RTLD_NOLOAD}
> > +flag.
> > 
> > Today a link_map* can be passed into e.g. dlinfo as if it was a dlopen
> > handle. Is this also subject to change? If so, is there a portable way
> > to obtain a handle from a link_map* without triggering ELF
> > constructors?
> 
> 
> There currently isn't.  If you are mainly concerned about dlinfo, we can
> document it as an extension that it accepts both (struct link_map * and
> dlopen handles).  I think we also need to add proxy map handling, which
> is currently missing.  Looking at the code, I expect RTLD_DI_PHDR
> currently returns a null pointer for the ld.so proxy map in a dlmopen
> namespace.
> ```

Yes, I'm mostly concerned about `dlinfo` here. Improving the proxy map handling would be useful but I don't think is absolutely necessary, so long as `dlinfo` doesn't crash in this case.

> ```
> > If l_name is relative and e.g. chdir("subdir") was called, is it
> > portable to expect dlopen(l_name) to refer to the original object?
> > Today it seems to, but that also feels very much like a bug and
> > shouldn't be relied on.
> 
> 
> I don't see a way around it.  File system stress from dlopen is already
> a thing, and we can't really revisit the file system each time dlopen is
> called for a known path.
> ```

You could compose the $ORIGIN string first and then only reuse an object if both it and `l_name` match. That should avoid most of the file system stress but would also change the behavior of `dlopen(l_name)` after a `chdir`. Not saying the behavior should change today, but this is a quirk that IMHO could be seen as a POSIX non-compliance and I would rather avoid relying on it not changing.

-Jonathon