| Message ID | ZvZj2jkpJjhaP-Yc@gardel-login |
|---|---|
| State | New |
| Headers | show |
| Series | nss: look for databases in /usr/share/ if they don't exist in /etc/ | expand |
* Lennart Poettering: > In order to improve compatibility with systems that implement a > hermetic /usr/, this changes glibc NSS code to always look for its > databases files in /usr/share/ too in case they don't exist (ENOENT) in > /etc/. This allows distributions to move /etc/protocols and similar > data files into /usr/share/. These days various of these files are kinda > static anyway, and hence in many cases are better placed below the /usr/ > hierarchy than below the configurable /etc/. We don't really know what's in /usr/share. There's some evidence that /usr/share/services/ was used as a directory at one point. The patch isn't directly incompatible with that, it's just that it can't be used to move /etc/services to /usr/share/services on such systems. Can we use a subdirectory of /usr/share instead, or perhaps /usr/etc? Thanks, Florian
On Fr, 27.09.24 10:54, Florian Weimer (fweimer@redhat.com) wrote: > * Lennart Poettering: > > > In order to improve compatibility with systems that implement a > > hermetic /usr/, this changes glibc NSS code to always look for its > > databases files in /usr/share/ too in case they don't exist (ENOENT) in > > /etc/. This allows distributions to move /etc/protocols and similar > > data files into /usr/share/. These days various of these files are kinda > > static anyway, and hence in many cases are better placed below the /usr/ > > hierarchy than below the configurable /etc/. > > We don't really know what's in /usr/share. There's some evidence that > /usr/share/services/ was used as a directory at one point. The patch > isn't directly incompatible with that, it's just that it can't be used > to move /etc/services to /usr/share/services on such systems. > > Can we use a subdirectory of /usr/share instead, or perhaps /usr/etc? Hmm, maybe it makes sense to treat NSS databases as something slightly distinct from other config files in /etc/, hence, maybe let's use /usr/share/nss/ for this, to indicate this is about nss databases? After all these files are similarly structured, have similar (sometimes identical) parsers, but have no file suffix that would tell us that. By placing them in a new subdir of /usr/share/ called "nss" we could communicate this similarity in structure a bit? (Not too keen on /usr/etc/ because that introduces yet another high-level hierarchy in /usr/. Maybe /usr/share/etc/ might be a middle ground?) Ultimately I don't care too much. As long as the files can be moved somewhere below /usr/ I am fine with either. My personal preference though would be in this order (if you already rule out /usr/share/ itself): 1. /usr/share/nss/ (clear preference from my side!) 2. /usr/share/etc/ 3. /usr/etc/ Just let me know what to use, I'll rearrange the patch accordingly. Lennart -- Lennart Poettering, Berlin
* Lennart Poettering: > On Fr, 27.09.24 10:54, Florian Weimer (fweimer@redhat.com) wrote: > >> * Lennart Poettering: >> >> > In order to improve compatibility with systems that implement a >> > hermetic /usr/, this changes glibc NSS code to always look for its >> > databases files in /usr/share/ too in case they don't exist (ENOENT) in >> > /etc/. This allows distributions to move /etc/protocols and similar >> > data files into /usr/share/. These days various of these files are kinda >> > static anyway, and hence in many cases are better placed below the /usr/ >> > hierarchy than below the configurable /etc/. >> >> We don't really know what's in /usr/share. There's some evidence that >> /usr/share/services/ was used as a directory at one point. The patch >> isn't directly incompatible with that, it's just that it can't be used >> to move /etc/services to /usr/share/services on such systems. >> >> Can we use a subdirectory of /usr/share instead, or perhaps /usr/etc? > > Hmm, maybe it makes sense to treat NSS databases as something slightly > distinct from other config files in /etc/, hence, maybe let's use > /usr/share/nss/ for this, to indicate this is about nss databases? > After all these files are similarly structured, have similar > (sometimes identical) parsers, but have no file suffix that would tell > us that. By placing them in a new subdir of /usr/share/ called "nss" > we could communicate this similarity in structure a bit? The Netscape Security Services do not use /usr/share/nss, so that should work. > (Not too keen on /usr/etc/ because that introduces yet another > high-level hierarchy in /usr/. Maybe /usr/share/etc/ might be a middle > ground?) I think openSUSE has been using /usr/etc for many years. See “System Databases (rpc, services, protocols)” in <https://kubic.opensuse.org/blog/2019-12-05-usr-etc/>, and <https://github.com/openSUSE/libnss_usrfiles>, particularly list of pathnames there. Cc:ing Thorsten. I don't particularly like /usr/etc because it is inconsistent with UsrMove (from a certain perspective), but this dislike doesn't go to the level that we need to be incompatible with existing deployments. Thanks, Florian
On Fri, Sep 27, 2024 at 11:26 AM Florian Weimer <fweimer@redhat.com> wrote: > > * Lennart Poettering: > > > Hmm, maybe it makes sense to treat NSS databases as something slightly > > distinct from other config files in /etc/, hence, maybe let's use > > /usr/share/nss/ for this, to indicate this is about nss databases? > > After all these files are similarly structured, have similar > > (sometimes identical) parsers, but have no file suffix that would tell > > us that. By placing them in a new subdir of /usr/share/ called "nss" > > we could communicate this similarity in structure a bit? > > The Netscape Security Services do not use /usr/share/nss, so that should > work. > > > (Not too keen on /usr/etc/ because that introduces yet another > > high-level hierarchy in /usr/. Maybe /usr/share/etc/ might be a middle > > ground?) > > I think openSUSE has been using /usr/etc for many years. See “System > Databases (rpc, services, protocols)” in > <https://kubic.opensuse.org/blog/2019-12-05-usr-etc/>, and > <https://github.com/openSUSE/libnss_usrfiles>, particularly list of > pathnames there. Cc:ing Thorsten. > > I don't particularly like /usr/etc because it is inconsistent with > UsrMove (from a certain perspective), but this dislike doesn't go to the > level that we need to be incompatible with existing deployments. We (openSUSE/SUSE) use /usr/etc only for files where there is no upstream location. If there is one, we will move the files to that location. So if we agree on /usr/share/nss or /usr/share/etc (I personally have no preference here), we would move the files from /usr/etc to this /usr/share/... directory. Since no application accesses this files directly, it's no problem for us. Thorsten
* Thorsten Kukuk: > We (openSUSE/SUSE) use /usr/etc only for files where there is no > upstream location. > If there is one, we will move the files to that location. Thank you for offering that. > So if we agree on /usr/share/nss or /usr/share/etc (I personally have > no preference here), we would move the files from /usr/etc to this > /usr/share/... directory. Since no application accesses this files > directly, it's no problem for us. The Go runtime needs to be taught about the alternative locations. There are parallel consumers of the NSS configuration, such as libsubid and sudo. Likely there are more. And of course we have a couple of alternate libcs now. It's not *that* much, but it's not just glibc either. Thanks, Florian
* Florian Weimer: > * Thorsten Kukuk: > >> We (openSUSE/SUSE) use /usr/etc only for files where there is no >> upstream location. >> If there is one, we will move the files to that location. > > Thank you for offering that. Feedback from Colin Walters on the Fedora development list: | Please avoid having projects unconditionally read `/usr/etc` | because on ostree based systems `/usr/etc/services` will always | exist; we use it for our 3-way merge of /etc. | (It's actually also really nice to have the defaults always | visible, that's how `ostree admin config-diff` works) | | It will start to conflict if other projects put things there. | | Now our technical direction may go in a place where we | move away from `/usr/etc` and do things differently | (and this is a whole *other* big thing to align in api | probably). <https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/message/LQQ3X3QI4USLV6B6SSNURLU3GVY5EKVE/> So I assume it should be /usr/share/nss then. Thanks, Florian
* Lennart Poettering: > /* Open PATH for reading, as a data source for nss_files. */ > -FILE *__nss_files_fopen (const char *path); > +FILE *__nss_files_fopen (const char *path, const char *fallback_path); > libc_hidden_proto (__nss_files_fopen) I suggest to create a new function that takes just the database name as a string (say "group") and constructs the file name from that. We already have some :include: processing (and we probably want more), so you probably should not change __nss_file_fopen itself. Thanks, Florian
On Mo, 30.09.24 17:17, DJ Delorie (dj@redhat.com) wrote: > Lennart Poettering <lennart@poettering.net> writes: > > In order to improve compatibility with systems that implement a > > hermetic /usr/, this changes glibc NSS code to always look for its > > databases files in /usr/share/ too in case they don't exist (ENOENT) in > > /etc/. This allows distributions to move /etc/protocols and similar > > data files into /usr/share/. These days various of these files are kinda > > static anyway, and hence in many cases are better placed below the /usr/ > > hierarchy than below the configurable /etc/. > > I'm opposed to the glibc sources knowing more about the OS choices than > is needed. The thing is that this really shouldn't be an OS choice. The files themselves are API to some point, are being read (and manipulated) by numerous tools. I think it's really important we push people towards common paths for the fallback versions, to make this workable. > We have existing ways of handling this: > > 1. Symbolic links (or other aggregators) This contradicts the concept of hermetic /usr/, i.e. that you really only need /usr/ to boot up, which is the point of the whole excercise. if you require symlinks to be placed in /etc/, then well, you didn't solve the problem that everyone currently tries to solve independently, and differently. in the discussions it cam up that at least we at microsoft, opensuse, fedora coreos, solus us came up with different approaches to the same problem because they didn't want to touch glibc so far. > 2. ./configure options Most installations don't want to simply move the files (i.e. have one place for them), but want a "fallback" concept, i.e. (i.e. have two places for them, one checked after the other). A configure option alone wouldn't do. > I note that glibc is lacking in #2 for NSS files, but I would not be > opposed to adding support for such. Perhaps, by definition, those > options could be a list of files to search. > > $ ./configure --services-path=/etc/services:/usr/share/systemd/whatever/services > > Solving this problem by hard-coding *yet another* path choice is a step > backwards. I think this would be overkill: I am not aware of any of the existing approaches needing a whole series of paths to check, they are all fine with just one in /usr/. Lennart -- Lennart Poettering, Berlin
diff --git a/include/nss_files.h b/include/nss_files.h index adf934f3ea..852c64da99 100644 --- a/include/nss_files.h +++ b/include/nss_files.h @@ -26,7 +26,7 @@ #endif /* Open PATH for reading, as a data source for nss_files. */ -FILE *__nss_files_fopen (const char *path); +FILE *__nss_files_fopen (const char *path, const char *fallback_path); libc_hidden_proto (__nss_files_fopen) /* Read a line from FP, storing it BUF. Strip leading blanks and skip @@ -89,7 +89,7 @@ enum nss_files_file null. */ enum nss_status __nss_files_data_open (struct nss_files_per_file_data **pdata, enum nss_files_file file, - const char *path, + const char *path, const char *fallback_path, int *errnop, int *herrnop); libc_hidden_proto (__nss_files_data_open) @@ -101,7 +101,7 @@ libc_hidden_proto (__nss_files_data_put) /* Performs the set*ent operation for FILE. PATH is the file to open. */ enum nss_status __nss_files_data_setent (enum nss_files_file file, - const char *path); + const char *path, const char *fallback_path); libc_hidden_proto (__nss_files_data_setent) /* Performs the end*ent operation for FILE. */ diff --git a/nss/nss_compat/compat-grp.c b/nss/nss_compat/compat-grp.c index ede7503de7..bacad5b583 100644 --- a/nss/nss_compat/compat-grp.c +++ b/nss/nss_compat/compat-grp.c @@ -108,7 +108,7 @@ internal_setgrent (ent_t *ent, int stayopen, int needent) if (ent->stream == NULL) { - ent->stream = __nss_files_fopen ("/etc/group"); + ent->stream = __nss_files_fopen ("/etc/group", "/usr/share/group"); if (ent->stream == NULL) status = errno == EAGAIN ? NSS_STATUS_TRYAGAIN : NSS_STATUS_UNAVAIL; diff --git a/nss/nss_compat/compat-initgroups.c b/nss/nss_compat/compat-initgroups.c index 2598cfbc9a..37eb4b17af 100644 --- a/nss/nss_compat/compat-initgroups.c +++ b/nss/nss_compat/compat-initgroups.c @@ -122,7 +122,7 @@ internal_setgrent (ent_t *ent) else ent->blacklist.current = 0; - ent->stream = __nss_files_fopen ("/etc/group"); + ent->stream = __nss_files_fopen ("/etc/group", "/usr/share/group"); if (ent->stream == NULL) status = errno == EAGAIN ? NSS_STATUS_TRYAGAIN : NSS_STATUS_UNAVAIL; diff --git a/nss/nss_compat/compat-pwd.c b/nss/nss_compat/compat-pwd.c index 2f37e0f621..2a3b012c33 100644 --- a/nss/nss_compat/compat-pwd.c +++ b/nss/nss_compat/compat-pwd.c @@ -223,7 +223,7 @@ internal_setpwent (ent_t *ent, int stayopen, int needent) if (ent->stream == NULL) { - ent->stream = __nss_files_fopen ("/etc/passwd"); + ent->stream = __nss_files_fopen ("/etc/passwd", "/usr/share/passwd"); if (ent->stream == NULL) status = errno == EAGAIN ? NSS_STATUS_TRYAGAIN : NSS_STATUS_UNAVAIL; diff --git a/nss/nss_compat/compat-spwd.c b/nss/nss_compat/compat-spwd.c index fac1e766cb..fc6984bcb4 100644 --- a/nss/nss_compat/compat-spwd.c +++ b/nss/nss_compat/compat-spwd.c @@ -178,7 +178,7 @@ internal_setspent (ent_t *ent, int stayopen, int needent) if (ent->stream == NULL) { - ent->stream = __nss_files_fopen ("/etc/shadow"); + ent->stream = __nss_files_fopen ("/etc/shadow", "/usr/share/shadow"); if (ent->stream == NULL) status = errno == EAGAIN ? NSS_STATUS_TRYAGAIN : NSS_STATUS_UNAVAIL; diff --git a/nss/nss_files/files-XXX.c b/nss/nss_files/files-XXX.c index 558b2c364f..fa47dec277 100644 --- a/nss/nss_files/files-XXX.c +++ b/nss/nss_files/files-XXX.c @@ -39,7 +39,8 @@ #define ENTNAME_r CONCAT(ENTNAME,_r) -#define DATAFILE "/etc/" DATABASE +#define DATAFILE "/etc/" DATABASE +#define DATAFILE_FALLBACK "/usr/share/" DATABASE #ifdef NEED_H_ERRNO # include <netdb.h> @@ -73,7 +74,7 @@ internal_setent (FILE **stream) if (*stream == NULL) { - *stream = __nss_files_fopen (DATAFILE); + *stream = __nss_files_fopen (DATAFILE, DATAFILE_FALLBACK); if (*stream == NULL) status = errno == EAGAIN ? NSS_STATUS_TRYAGAIN : NSS_STATUS_UNAVAIL; @@ -89,7 +90,7 @@ internal_setent (FILE **stream) enum nss_status CONCAT(_nss_files_set,ENTNAME) (int stayopen) { - return __nss_files_data_setent (CONCAT (nss_file_, ENTNAME), DATAFILE); + return __nss_files_data_setent (CONCAT (nss_file_, ENTNAME), DATAFILE, DATAFILE_FALLBACK); } libc_hidden_def (CONCAT (_nss_files_set,ENTNAME)) @@ -170,7 +171,7 @@ CONCAT(_nss_files_get,ENTNAME_r) (struct STRUCTURE *result, char *buffer, struct nss_files_per_file_data *data; enum nss_status status = __nss_files_data_open (&data, CONCAT (nss_file_, ENTNAME), - DATAFILE, + DATAFILE, DATAFILE_FALLBACK, errnop, H_ERRNO_ARG_OR_NULL); if (status != NSS_STATUS_SUCCESS) return status; diff --git a/nss/nss_files/files-alias.c b/nss/nss_files/files-alias.c index 14a59b4655..d0b1ce2a8e 100644 --- a/nss/nss_files/files-alias.c +++ b/nss/nss_files/files-alias.c @@ -42,7 +42,7 @@ internal_setent (FILE **stream) if (*stream == NULL) { - *stream = __nss_files_fopen ("/etc/aliases"); + *stream = __nss_files_fopen ("/etc/aliases", "/usr/share/aliases"); if (*stream == NULL) status = errno == EAGAIN ? NSS_STATUS_TRYAGAIN : NSS_STATUS_UNAVAIL; @@ -58,7 +58,7 @@ internal_setent (FILE **stream) enum nss_status _nss_files_setaliasent (void) { - return __nss_files_data_setent (nss_file_aliasent, "/etc/aliases"); + return __nss_files_data_setent (nss_file_aliasent, "/etc/aliases", "/usr/share/aliases"); } libc_hidden_def (_nss_files_setaliasent) @@ -182,7 +182,7 @@ get_next_alias (FILE *stream, const char *match, struct aliasent *result, first_unused = cp; - listfile = __nss_files_fopen (&cp[9]); + listfile = __nss_files_fopen (&cp[9], NULL); /* If the file does not exist we simply ignore the statement. */ if (listfile != NULL @@ -338,7 +338,8 @@ _nss_files_getaliasent_r (struct aliasent *result, char *buffer, size_t buflen, struct nss_files_per_file_data *data; enum nss_status status = __nss_files_data_open (&data, nss_file_aliasent, - "/etc/aliases", errnop, NULL); + "/etc/aliases", "/usr/share/aliases", + errnop, NULL); if (status != NSS_STATUS_SUCCESS) return status; diff --git a/nss/nss_files/files-initgroups.c b/nss/nss_files/files-initgroups.c index 65189d3929..896fa50c5a 100644 --- a/nss/nss_files/files-initgroups.c +++ b/nss/nss_files/files-initgroups.c @@ -33,7 +33,7 @@ _nss_files_initgroups_dyn (const char *user, gid_t group, long int *start, long int *size, gid_t **groupsp, long int limit, int *errnop) { - FILE *stream = __nss_files_fopen ("/etc/group"); + FILE *stream = __nss_files_fopen ("/etc/group", "/usr/share/group"); if (stream == NULL) { *errnop = errno; diff --git a/nss/nss_files/files-netgrp.c b/nss/nss_files/files-netgrp.c index 92d8062e43..7f71eebf60 100644 --- a/nss/nss_files/files-netgrp.c +++ b/nss/nss_files/files-netgrp.c @@ -27,7 +27,8 @@ #include "netgroup.h" #include <nss_files.h> -#define DATAFILE "/etc/netgroup" +#define DATAFILE "/etc/netgroup" +#define DATAFILE_FALLBACK "/usr/share/netgroup" libc_hidden_proto (_nss_files_endnetgrent) @@ -62,7 +63,7 @@ _nss_files_setnetgrent (const char *group, struct __netgrent *result) return NSS_STATUS_UNAVAIL; /* Find the netgroups file and open it. */ - fp = __nss_files_fopen (DATAFILE); + fp = __nss_files_fopen (DATAFILE, DATAFILE_FALLBACK); if (fp == NULL) status = errno == EAGAIN ? NSS_STATUS_TRYAGAIN : NSS_STATUS_UNAVAIL; else diff --git a/nss/nss_files_data.c b/nss/nss_files_data.c index 261a01a775..c6af527e6d 100644 --- a/nss/nss_files_data.c +++ b/nss/nss_files_data.c @@ -74,13 +74,13 @@ __nss_files_data_get (struct nss_files_per_file_data **pdata, /* Helper function for opening the backing file at PATH. */ static enum nss_status __nss_files_data_internal_open (struct nss_files_per_file_data *data, - const char *path) + const char *path, const char *fallback_path) { enum nss_status status = NSS_STATUS_SUCCESS; if (data->stream == NULL) { - data->stream = __nss_files_fopen (path); + data->stream = __nss_files_fopen (path, fallback_path); if (data->stream == NULL) status = errno == EAGAIN ? NSS_STATUS_TRYAGAIN : NSS_STATUS_UNAVAIL; @@ -93,6 +93,7 @@ __nss_files_data_internal_open (struct nss_files_per_file_data *data, enum nss_status __nss_files_data_open (struct nss_files_per_file_data **pdata, enum nss_files_file file, const char *path, + const char *fallback_path, int *errnop, int *herrnop) { enum nss_status status = __nss_files_data_get (pdata, file, errnop, herrnop); @@ -103,7 +104,7 @@ __nss_files_data_open (struct nss_files_per_file_data **pdata, if ((*pdata)->stream == NULL) { int saved_errno = errno; - status = __nss_files_data_internal_open (*pdata, path); + status = __nss_files_data_internal_open (*pdata, path, fallback_path); __set_errno (saved_errno); if (status != NSS_STATUS_SUCCESS) __nss_files_data_put (*pdata); @@ -122,7 +123,7 @@ __nss_files_data_put (struct nss_files_per_file_data *data) libc_hidden_def (__nss_files_data_put) enum nss_status -__nss_files_data_setent (enum nss_files_file file, const char *path) +__nss_files_data_setent (enum nss_files_file file, const char *path, const char *fallback_path) { struct nss_files_per_file_data *data; enum nss_status status = __nss_files_data_get (&data, file, NULL, NULL); @@ -130,7 +131,7 @@ __nss_files_data_setent (enum nss_files_file file, const char *path) return status; if (data->stream == NULL) - status = __nss_files_data_internal_open (data, path); + status = __nss_files_data_internal_open (data, path, fallback_path); else rewind (data->stream); diff --git a/nss/nss_files_fopen.c b/nss/nss_files_fopen.c index e7c48d7bd8..a2b7affdb4 100644 --- a/nss/nss_files_fopen.c +++ b/nss/nss_files_fopen.c @@ -22,11 +22,18 @@ #include <stdio_ext.h> FILE * -__nss_files_fopen (const char *path) +__nss_files_fopen (const char *path, const char *fallback_path) { FILE *fp = fopen (path, "rce"); if (fp == NULL) - return NULL; + { + if (errno != ENOENT || fallback_path == NULL) + return NULL; + + fp = fopen (fallback_path, "rce"); + if (fp == NULL) + return NULL; + } /* The stream is not shared across threads. */ __fsetlocking (fp, FSETLOCKING_BYCALLER);