Message ID | 1521736598-12812-1-git-send-email-blackzert@gmail.com |
---|---|
Headers | show |
Series | Randomization of address chosen by mmap. | expand |
On Thu, 22 Mar 2018 19:36:36 +0300 Ilya Smith <blackzert@gmail.com> wrote: > Current implementation doesn't randomize address returned by mmap. > All the entropy ends with choosing mmap_base_addr at the process > creation. After that mmap build very predictable layout of address > space. It allows to bypass ASLR in many cases. Perhaps some more effort on the problem description would help. *Are* people predicting layouts at present? What problems does this cause? How are they doing this and are there other approaches to solving the problem? Mainly: what value does this patchset have to our users? This reader is unable to determine that from the information which you have provided. Full details, please.
On Thu, Mar 22, 2018 at 07:36:36PM +0300, Ilya Smith wrote: > Current implementation doesn't randomize address returned by mmap. > All the entropy ends with choosing mmap_base_addr at the process > creation. After that mmap build very predictable layout of address > space. It allows to bypass ASLR in many cases. This patch make > randomization of address on any mmap call. Why should this be done in the kernel rather than libc? libc is perfectly capable of specifying random numbers in the first argument of mmap.
Hello, Andrew Thanks for reading this patch. > On 22 Mar 2018, at 23:57, Andrew Morton <akpm@linux-foundation.org> wrote: > > On Thu, 22 Mar 2018 19:36:36 +0300 Ilya Smith <blackzert@gmail.com> wrote: > >> Current implementation doesn't randomize address returned by mmap. >> All the entropy ends with choosing mmap_base_addr at the process >> creation. After that mmap build very predictable layout of address >> space. It allows to bypass ASLR in many cases. > > Perhaps some more effort on the problem description would help. *Are* > people predicting layouts at present? What problems does this cause? > How are they doing this and are there other approaches to solving the > problem? > Sorry, I’ve lost it in first version. In short - memory layout could be easily repaired by single leakage. Also any Out of Bounds error may easily be exploited according to current implementation. All because mmap choose address just before previously allocated segment. You can read more about it here: http://www.openwall.com/lists/oss-security/2018/02/27/5 Some test are available here https://github.com/blackzert/aslur. To solve the problem Kernel should randomize address on any mmap so attacker could never easily gain needed addresses. > Mainly: what value does this patchset have to our users? This reader > is unable to determine that from the information which you have > provided. Full details, please. The value of this patch is to decrease successful rate of exploitation vulnerable applications.These could be either remote or local vectors.
> On 23 Mar 2018, at 15:48, Matthew Wilcox <willy@infradead.org> wrote: > > On Thu, Mar 22, 2018 at 07:36:36PM +0300, Ilya Smith wrote: >> Current implementation doesn't randomize address returned by mmap. >> All the entropy ends with choosing mmap_base_addr at the process >> creation. After that mmap build very predictable layout of address >> space. It allows to bypass ASLR in many cases. This patch make >> randomization of address on any mmap call. > > Why should this be done in the kernel rather than libc? libc is perfectly > capable of specifying random numbers in the first argument of mmap. Well, there is following reasons: 1. It should be done in any libc implementation, what is not possible IMO; 2. User mode is not that layer which should be responsible for choosing random address or handling entropy; 3. Memory fragmentation is unpredictable in this case Off course user mode could use random ‘hint’ address, but kernel may discard this address if it is occupied for example and allocate just before closest vma. So this solution doesn’t give that much security like randomization address inside kernel.
On Fri, Mar 23, 2018 at 05:48:06AM -0700, Matthew Wilcox wrote: > On Thu, Mar 22, 2018 at 07:36:36PM +0300, Ilya Smith wrote: > > Current implementation doesn't randomize address returned by mmap. > > All the entropy ends with choosing mmap_base_addr at the process > > creation. After that mmap build very predictable layout of address > > space. It allows to bypass ASLR in many cases. This patch make > > randomization of address on any mmap call. > > Why should this be done in the kernel rather than libc? libc is perfectly > capable of specifying random numbers in the first argument of mmap. Generally libc does not have a view of the current vm maps, and thus in passing "random numbers", they would have to be uniform across the whole vm space and thus non-uniform once the kernel rounds up to avoid existing mappings. Also this would impose requirements that libc be aware of the kernel's use of the virtual address space and what's available to userspace -- for example, on 32-bit archs whether 2GB, 3GB, or full 4GB (for 32-bit-user-on-64-bit-kernel) is available, and on 64-bit archs where fewer than the full 64 bits are actually valid in addresses, what the actual usable pointer size is. There is currently no clean way of conveying this information to userspace. Rich
On Fri, Mar 23, 2018 at 02:00:24PM -0400, Rich Felker wrote: > On Fri, Mar 23, 2018 at 05:48:06AM -0700, Matthew Wilcox wrote: > > On Thu, Mar 22, 2018 at 07:36:36PM +0300, Ilya Smith wrote: > > > Current implementation doesn't randomize address returned by mmap. > > > All the entropy ends with choosing mmap_base_addr at the process > > > creation. After that mmap build very predictable layout of address > > > space. It allows to bypass ASLR in many cases. This patch make > > > randomization of address on any mmap call. > > > > Why should this be done in the kernel rather than libc? libc is perfectly > > capable of specifying random numbers in the first argument of mmap. > > Generally libc does not have a view of the current vm maps, and thus > in passing "random numbers", they would have to be uniform across the > whole vm space and thus non-uniform once the kernel rounds up to avoid > existing mappings. I'm aware that you're the musl author, but glibc somehow manages to provide etext, edata and end, demonstrating that it does know where at least some of the memory map lies. Virtually everything after that is brought into the address space via mmap, which at least glibc intercepts, so it's entirely possible for a security-conscious libc to know where other things are in the memory map. Not to mention that what we're primarily talking about here are libraries which are dynamically linked and are loaded by ld.so before calling main(); not dlopen() or even regular user mmaps. > Also this would impose requirements that libc be > aware of the kernel's use of the virtual address space and what's > available to userspace -- for example, on 32-bit archs whether 2GB, > 3GB, or full 4GB (for 32-bit-user-on-64-bit-kernel) is available, and > on 64-bit archs where fewer than the full 64 bits are actually valid > in addresses, what the actual usable pointer size is. There is > currently no clean way of conveying this information to userspace. Huh, I thought libc was aware of this. Also, I'd expect a libc-based implementation to restrict itself to, eg, only loading libraries in the bottom 1GB to avoid applications who want to map huge things from running out of unfragmented address space.
On Fri, Mar 23, 2018 at 12:06:18PM -0700, Matthew Wilcox wrote: > On Fri, Mar 23, 2018 at 02:00:24PM -0400, Rich Felker wrote: > > On Fri, Mar 23, 2018 at 05:48:06AM -0700, Matthew Wilcox wrote: > > > On Thu, Mar 22, 2018 at 07:36:36PM +0300, Ilya Smith wrote: > > > > Current implementation doesn't randomize address returned by mmap. > > > > All the entropy ends with choosing mmap_base_addr at the process > > > > creation. After that mmap build very predictable layout of address > > > > space. It allows to bypass ASLR in many cases. This patch make > > > > randomization of address on any mmap call. > > > > > > Why should this be done in the kernel rather than libc? libc is perfectly > > > capable of specifying random numbers in the first argument of mmap. > > > > Generally libc does not have a view of the current vm maps, and thus > > in passing "random numbers", they would have to be uniform across the > > whole vm space and thus non-uniform once the kernel rounds up to avoid > > existing mappings. > > I'm aware that you're the musl author, but glibc somehow manages to > provide etext, edata and end, demonstrating that it does know where at > least some of the memory map lies. Yes, but that's pretty minimal info. > Virtually everything after that is > brought into the address space via mmap, which at least glibc intercepts, There's also vdso, the program interpreter (ldso), and theoretically other things the kernel might add. I agree you _could_ track most of this (and all if you want to open /proc/self/maps), but it seems hackish and wrong (violating clean boundaries between userspace and kernel responsibility). > > Also this would impose requirements that libc be > > aware of the kernel's use of the virtual address space and what's > > available to userspace -- for example, on 32-bit archs whether 2GB, > > 3GB, or full 4GB (for 32-bit-user-on-64-bit-kernel) is available, and > > on 64-bit archs where fewer than the full 64 bits are actually valid > > in addresses, what the actual usable pointer size is. There is > > currently no clean way of conveying this information to userspace. > > Huh, I thought libc was aware of this. Also, I'd expect a libc-based > implementation to restrict itself to, eg, only loading libraries in > the bottom 1GB to avoid applications who want to map huge things from > running out of unfragmented address space. That seems like a rather arbitrary expectation and I'm not sure why you'd expect it to result in less fragmentation rather than more. For example if it started from 1GB and worked down, you'd immediately reduce the contiguous free space from ~3GB to ~2GB, and if it started from the bottom and worked up, brk would immediately become unavailable, increasing mmap pressure elsewhere. Rich
On Fri, Mar 23, 2018 at 03:16:21PM -0400, Rich Felker wrote: > > Huh, I thought libc was aware of this. Also, I'd expect a libc-based > > implementation to restrict itself to, eg, only loading libraries in > > the bottom 1GB to avoid applications who want to map huge things from > > running out of unfragmented address space. > > That seems like a rather arbitrary expectation and I'm not sure why > you'd expect it to result in less fragmentation rather than more. For > example if it started from 1GB and worked down, you'd immediately > reduce the contiguous free space from ~3GB to ~2GB, and if it started > from the bottom and worked up, brk would immediately become > unavailable, increasing mmap pressure elsewhere. By *not* limiting yourself to the bottom 1GB, you'll almost immediately fragment the address space even worse. Just looking at 'ls' as a hopefully-good example of a typical app, it maps: linux-vdso.so.1 (0x00007ffef5eef000) libselinux.so.1 => /lib/x86_64-linux-gnu/libselinux.so.1 (0x00007fb3657f5000) libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007fb36543b000) libpcre.so.3 => /lib/x86_64-linux-gnu/libpcre.so.3 (0x00007fb3651c9000) libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007fb364fc5000) /lib64/ld-linux-x86-64.so.2 (0x00007fb365c3f000) libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007fb364da7000) The VDSO wouldn't move, but look at the distribution of mapping 6 things into a 3GB address space in random locations. What are the odds you have a contiguous 1GB chunk of address space? If you restrict yourself to the bottom 1GB before running out of room and falling back to a sequential allocation, you'll prevent a lot of fragmentation.
On Fri, Mar 23, 2018 at 12:29:52PM -0700, Matthew Wilcox wrote: > On Fri, Mar 23, 2018 at 03:16:21PM -0400, Rich Felker wrote: > > > Huh, I thought libc was aware of this. Also, I'd expect a libc-based > > > implementation to restrict itself to, eg, only loading libraries in > > > the bottom 1GB to avoid applications who want to map huge things from > > > running out of unfragmented address space. > > > > That seems like a rather arbitrary expectation and I'm not sure why > > you'd expect it to result in less fragmentation rather than more. For > > example if it started from 1GB and worked down, you'd immediately > > reduce the contiguous free space from ~3GB to ~2GB, and if it started > > from the bottom and worked up, brk would immediately become > > unavailable, increasing mmap pressure elsewhere. > > By *not* limiting yourself to the bottom 1GB, you'll almost immediately > fragment the address space even worse. Just looking at 'ls' as a > hopefully-good example of a typical app, it maps: > > linux-vdso.so.1 (0x00007ffef5eef000) > libselinux.so.1 => /lib/x86_64-linux-gnu/libselinux.so.1 (0x00007fb3657f5000) > libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007fb36543b000) > libpcre.so.3 => /lib/x86_64-linux-gnu/libpcre.so.3 (0x00007fb3651c9000) > libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007fb364fc5000) > /lib64/ld-linux-x86-64.so.2 (0x00007fb365c3f000) > libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007fb364da7000) > > The VDSO wouldn't move, but look at the distribution of mapping 6 things > into a 3GB address space in random locations. What are the odds you have > a contiguous 1GB chunk of address space? If you restrict yourself to the > bottom 1GB before running out of room and falling back to a sequential > allocation, you'll prevent a lot of fragmentation. Oh, you're talking about "with random locations" case. Randomizing each map just hopelessly fragments things no matter what you do on 32-bit. If you reduce the space over which you randomize to the point where it's not fragmenting/killing your available vm space, there are so few degrees of freedom left that it's trivial to brute-force. Maybe "libs randomized in low 1GB, everything else near-sequential in high addresses" works half decently, but I have a hard time believing you can get any ASLR that's significantly better than snake oil in a 32-bit address space, and you certainly do pay a high price in total available vm space. Rich
On Fri 23-03-18 20:55:49, Ilya Smith wrote: > > > On 23 Mar 2018, at 15:48, Matthew Wilcox <willy@infradead.org> wrote: > > > > On Thu, Mar 22, 2018 at 07:36:36PM +0300, Ilya Smith wrote: > >> Current implementation doesn't randomize address returned by mmap. > >> All the entropy ends with choosing mmap_base_addr at the process > >> creation. After that mmap build very predictable layout of address > >> space. It allows to bypass ASLR in many cases. This patch make > >> randomization of address on any mmap call. > > > > Why should this be done in the kernel rather than libc? libc is perfectly > > capable of specifying random numbers in the first argument of mmap. > Well, there is following reasons: > 1. It should be done in any libc implementation, what is not possible IMO; Is this really so helpful? > 2. User mode is not that layer which should be responsible for choosing > random address or handling entropy; Why? > 3. Memory fragmentation is unpredictable in this case > > Off course user mode could use random ‘hint’ address, but kernel may > discard this address if it is occupied for example and allocate just before > closest vma. So this solution doesn’t give that much security like > randomization address inside kernel. The userspace can use the new MAP_FIXED_NOREPLACE to probe for the address range atomically and chose a different range on failure.
> On 26 Mar 2018, at 11:46, Michal Hocko <mhocko@kernel.org> wrote: > > On Fri 23-03-18 20:55:49, Ilya Smith wrote: >> >>> On 23 Mar 2018, at 15:48, Matthew Wilcox <willy@infradead.org> wrote: >>> >>> On Thu, Mar 22, 2018 at 07:36:36PM +0300, Ilya Smith wrote: >>>> Current implementation doesn't randomize address returned by mmap. >>>> All the entropy ends with choosing mmap_base_addr at the process >>>> creation. After that mmap build very predictable layout of address >>>> space. It allows to bypass ASLR in many cases. This patch make >>>> randomization of address on any mmap call. >>> >>> Why should this be done in the kernel rather than libc? libc is perfectly >>> capable of specifying random numbers in the first argument of mmap. >> Well, there is following reasons: >> 1. It should be done in any libc implementation, what is not possible IMO; > > Is this really so helpful? Yes, ASLR is one of very important mitigation techniques which are really used to protect applications. If there is no ASLR, it is very easy to exploit vulnerable application and compromise the system. We can’t just fix all the vulnerabilities right now, thats why we have mitigations - techniques which are makes exploitation more hard or impossible in some cases. Thats why it is helpful. > >> 2. User mode is not that layer which should be responsible for choosing >> random address or handling entropy; > > Why? Because of the following reasons: 1. To get random address you should have entropy. These entropy shouldn’t be exposed to attacker anyhow, the best case is to get it from kernel. So this is a syscall. 2. You should have memory map of your process to prevent remapping or big fragmentation. Kernel already has this map. You will got another one in libc. And any non-libc user of mmap (via syscall, etc) will make hole in your map. This one also decrease performance cause you any way call syscall_mmap which will try to find some address for you in worst case, but after you already did some computing on it. 3. The more memory you use in userland for these proposal, the easier for attacker to leak it or use in exploitation techniques. 4. It is so easy to fix Kernel function and so hard to support memory management from userspace. > >> 3. Memory fragmentation is unpredictable in this case >> >> Off course user mode could use random ‘hint’ address, but kernel may >> discard this address if it is occupied for example and allocate just before >> closest vma. So this solution doesn’t give that much security like >> randomization address inside kernel. > > The userspace can use the new MAP_FIXED_NOREPLACE to probe for the > address range atomically and chose a different range on failure. > This algorithm should track current memory. If he doesn’t he may cause infinite loop while trying to choose memory. And each iteration increase time needed on allocation new memory, what is not preferred by any libc library developer. Thats why I did this patch. Thanks, Ilya
On Mon 26-03-18 22:45:31, Ilya Smith wrote: > > > On 26 Mar 2018, at 11:46, Michal Hocko <mhocko@kernel.org> wrote: > > > > On Fri 23-03-18 20:55:49, Ilya Smith wrote: > >> > >>> On 23 Mar 2018, at 15:48, Matthew Wilcox <willy@infradead.org> wrote: > >>> > >>> On Thu, Mar 22, 2018 at 07:36:36PM +0300, Ilya Smith wrote: > >>>> Current implementation doesn't randomize address returned by mmap. > >>>> All the entropy ends with choosing mmap_base_addr at the process > >>>> creation. After that mmap build very predictable layout of address > >>>> space. It allows to bypass ASLR in many cases. This patch make > >>>> randomization of address on any mmap call. > >>> > >>> Why should this be done in the kernel rather than libc? libc is perfectly > >>> capable of specifying random numbers in the first argument of mmap. > >> Well, there is following reasons: > >> 1. It should be done in any libc implementation, what is not possible IMO; > > > > Is this really so helpful? > > Yes, ASLR is one of very important mitigation techniques which are really used > to protect applications. If there is no ASLR, it is very easy to exploit > vulnerable application and compromise the system. We can’t just fix all the > vulnerabilities right now, thats why we have mitigations - techniques which are > makes exploitation more hard or impossible in some cases. > > Thats why it is helpful. I am not questioning ASLR in general. I am asking whether we really need per mmap ASLR in general. I can imagine that some environments want to pay the additional price and other side effects, but considering this can be achieved by libc, why to add more code to the kernel? > > > >> 2. User mode is not that layer which should be responsible for choosing > >> random address or handling entropy; > > > > Why? > > Because of the following reasons: > 1. To get random address you should have entropy. These entropy shouldn’t be > exposed to attacker anyhow, the best case is to get it from kernel. So this is > a syscall. /dev/[u]random is not sufficient? > 2. You should have memory map of your process to prevent remapping or big > fragmentation. Kernel already has this map. /proc/self/maps? > You will got another one in libc. > And any non-libc user of mmap (via syscall, etc) will make hole in your map. > This one also decrease performance cause you any way call syscall_mmap > which will try to find some address for you in worst case, but after you already > did some computing on it. I do not understand. a) you should be prepared to pay an additional price for an additional security measures and b) how would anybody punch a hole into your mapping? > 3. The more memory you use in userland for these proposal, the easier for > attacker to leak it or use in exploitation techniques. This is true in general, isn't it? I fail to see how kernel chosen and user chosen ranges would make any difference. > 4. It is so easy to fix Kernel function and so hard to support memory > management from userspace. Well, on the other hand the new layout mode will add a maintenance burden on the kernel and will have to be maintained for ever because it is a user visible ABI. > >> 3. Memory fragmentation is unpredictable in this case > >> > >> Off course user mode could use random ‘hint’ address, but kernel may > >> discard this address if it is occupied for example and allocate just before > >> closest vma. So this solution doesn’t give that much security like > >> randomization address inside kernel. > > > > The userspace can use the new MAP_FIXED_NOREPLACE to probe for the > > address range atomically and chose a different range on failure. > > > > This algorithm should track current memory. If he doesn’t he may cause > infinite loop while trying to choose memory. And each iteration increase time > needed on allocation new memory, what is not preferred by any libc library > developer. Well, I am pretty sure userspace can implement proper free ranges tracking...
> On 27 Mar 2018, at 10:24, Michal Hocko <mhocko@kernel.org> wrote: > > On Mon 26-03-18 22:45:31, Ilya Smith wrote: >> >>> On 26 Mar 2018, at 11:46, Michal Hocko <mhocko@kernel.org> wrote: >>> >>> On Fri 23-03-18 20:55:49, Ilya Smith wrote: >>>> >>>>> On 23 Mar 2018, at 15:48, Matthew Wilcox <willy@infradead.org> wrote: >>>>> >>>>> On Thu, Mar 22, 2018 at 07:36:36PM +0300, Ilya Smith wrote: >>>>>> Current implementation doesn't randomize address returned by mmap. >>>>>> All the entropy ends with choosing mmap_base_addr at the process >>>>>> creation. After that mmap build very predictable layout of address >>>>>> space. It allows to bypass ASLR in many cases. This patch make >>>>>> randomization of address on any mmap call. >>>>> >>>>> Why should this be done in the kernel rather than libc? libc is perfectly >>>>> capable of specifying random numbers in the first argument of mmap. >>>> Well, there is following reasons: >>>> 1. It should be done in any libc implementation, what is not possible IMO; >>> >>> Is this really so helpful? >> >> Yes, ASLR is one of very important mitigation techniques which are really used >> to protect applications. If there is no ASLR, it is very easy to exploit >> vulnerable application and compromise the system. We can’t just fix all the >> vulnerabilities right now, thats why we have mitigations - techniques which are >> makes exploitation more hard or impossible in some cases. >> >> Thats why it is helpful. > > I am not questioning ASLR in general. I am asking whether we really need > per mmap ASLR in general. I can imagine that some environments want to > pay the additional price and other side effects, but considering this > can be achieved by libc, why to add more code to the kernel? I believe this is the only one right place for it. Adding these 200+ lines of code we give this feature for any user - on desktop, on server, on IoT device, on SCADA, etc. But if only glibc will implement ‘user-mode-aslr’ IoT and SCADA devices will never get it. >>> >>>> 2. User mode is not that layer which should be responsible for choosing >>>> random address or handling entropy; >>> >>> Why? >> >> Because of the following reasons: >> 1. To get random address you should have entropy. These entropy shouldn’t be >> exposed to attacker anyhow, the best case is to get it from kernel. So this is >> a syscall. > > /dev/[u]random is not sufficient? Using /dev/[u]random makes 3 syscalls - open, read, close. This is a performance issue. > >> 2. You should have memory map of your process to prevent remapping or big >> fragmentation. Kernel already has this map. > > /proc/self/maps? Not any system has /proc and parsing /proc/self/maps is robust so it is the performance issue. libc will have to do it on any mmap. And there is a possible race here - application may mmap/unmap memory with native syscall during other thread reading maps. >> You will got another one in libc. >> And any non-libc user of mmap (via syscall, etc) will make hole in your map. >> This one also decrease performance cause you any way call syscall_mmap >> which will try to find some address for you in worst case, but after you already >> did some computing on it. > > I do not understand. a) you should be prepared to pay an additional > price for an additional security measures and b) how would anybody punch > a hole into your mapping? > I was talking about any code that call mmap directly without libc wrapper. >> 3. The more memory you use in userland for these proposal, the easier for >> attacker to leak it or use in exploitation techniques. > > This is true in general, isn't it? I fail to see how kernel chosen and > user chosen ranges would make any difference. My point here was that libc will have to keep memory representation as a tree and this tree increase attack surface. It could be hidden in kernel as it is right now. > >> 4. It is so easy to fix Kernel function and so hard to support memory >> management from userspace. > > Well, on the other hand the new layout mode will add a maintenance > burden on the kernel and will have to be maintained for ever because it > is a user visible ABI. Thats why I made this patch as RFC and would like to discuss this ABI here. I made randomize_va_space parameter to allow disable randomisation per whole system. PF_RANDOMIZE flag may disable randomization for concrete process (or process groups?). For architecture I’ve made info.random_shift = 0 , so if your arch has small address space you may disable shifting. I also would like to add some sysctl to allow process/groups to change this value and allow some processes to have shifts bigger then another. Lets discuss it, please. > >>>> 3. Memory fragmentation is unpredictable in this case >>>> >>>> Off course user mode could use random ‘hint’ address, but kernel may >>>> discard this address if it is occupied for example and allocate just before >>>> closest vma. So this solution doesn’t give that much security like >>>> randomization address inside kernel. >>> >>> The userspace can use the new MAP_FIXED_NOREPLACE to probe for the >>> address range atomically and chose a different range on failure. >>> >> >> This algorithm should track current memory. If he doesn’t he may cause >> infinite loop while trying to choose memory. And each iteration increase time >> needed on allocation new memory, what is not preferred by any libc library >> developer. > > Well, I am pretty sure userspace can implement proper free ranges > tracking… I think we need to know what libc developers will say on implementing ASLR in user-mode. I am pretty sure they will say ‘nether’ or ‘some-day’. And problem of ASLR will stay forever. Thanks, Ilya
On Tue 27-03-18 16:51:08, Ilya Smith wrote: > > > On 27 Mar 2018, at 10:24, Michal Hocko <mhocko@kernel.org> wrote: > > > > On Mon 26-03-18 22:45:31, Ilya Smith wrote: > >> > >>> On 26 Mar 2018, at 11:46, Michal Hocko <mhocko@kernel.org> wrote: > >>> > >>> On Fri 23-03-18 20:55:49, Ilya Smith wrote: > >>>> > >>>>> On 23 Mar 2018, at 15:48, Matthew Wilcox <willy@infradead.org> wrote: > >>>>> > >>>>> On Thu, Mar 22, 2018 at 07:36:36PM +0300, Ilya Smith wrote: > >>>>>> Current implementation doesn't randomize address returned by mmap. > >>>>>> All the entropy ends with choosing mmap_base_addr at the process > >>>>>> creation. After that mmap build very predictable layout of address > >>>>>> space. It allows to bypass ASLR in many cases. This patch make > >>>>>> randomization of address on any mmap call. > >>>>> > >>>>> Why should this be done in the kernel rather than libc? libc is perfectly > >>>>> capable of specifying random numbers in the first argument of mmap. > >>>> Well, there is following reasons: > >>>> 1. It should be done in any libc implementation, what is not possible IMO; > >>> > >>> Is this really so helpful? > >> > >> Yes, ASLR is one of very important mitigation techniques which are really used > >> to protect applications. If there is no ASLR, it is very easy to exploit > >> vulnerable application and compromise the system. We can’t just fix all the > >> vulnerabilities right now, thats why we have mitigations - techniques which are > >> makes exploitation more hard or impossible in some cases. > >> > >> Thats why it is helpful. > > > > I am not questioning ASLR in general. I am asking whether we really need > > per mmap ASLR in general. I can imagine that some environments want to > > pay the additional price and other side effects, but considering this > > can be achieved by libc, why to add more code to the kernel? > > I believe this is the only one right place for it. Adding these 200+ lines of > code we give this feature for any user - on desktop, on server, on IoT device, > on SCADA, etc. But if only glibc will implement ‘user-mode-aslr’ IoT and SCADA > devices will never get it. I guess it would really help if you could be more specific about the class of security issues this would help to mitigate. My first understanding was that we we need some randomization between program executable segments to reduce the attack space when a single address leaks and you know the segments layout (ordering). But why do we need _all_ mmaps to be randomized. Because that complicates the implementation consirably for different reasons you have mentioned earlier. Do you have any specific CVE that would be mitigated by this randomization approach? I am sorry, I am not a security expert to see all the cosequences but a vague - the more randomization the better - sounds rather weak to me.
On Tue, Mar 27, 2018 at 6:51 AM, Ilya Smith <blackzert@gmail.com> wrote: > >> On 27 Mar 2018, at 10:24, Michal Hocko <mhocko@kernel.org> wrote: >> >> On Mon 26-03-18 22:45:31, Ilya Smith wrote: >>> >>>> On 26 Mar 2018, at 11:46, Michal Hocko <mhocko@kernel.org> wrote: >>>> >>>> On Fri 23-03-18 20:55:49, Ilya Smith wrote: >>>>> >>>>>> On 23 Mar 2018, at 15:48, Matthew Wilcox <willy@infradead.org> wrote: >>>>>> >>>>>> On Thu, Mar 22, 2018 at 07:36:36PM +0300, Ilya Smith wrote: >>>>>>> Current implementation doesn't randomize address returned by mmap. >>>>>>> All the entropy ends with choosing mmap_base_addr at the process >>>>>>> creation. After that mmap build very predictable layout of address >>>>>>> space. It allows to bypass ASLR in many cases. This patch make >>>>>>> randomization of address on any mmap call. >>>>>> >>>>>> Why should this be done in the kernel rather than libc? libc is perfectly >>>>>> capable of specifying random numbers in the first argument of mmap. >>>>> Well, there is following reasons: >>>>> 1. It should be done in any libc implementation, what is not possible IMO; >>>> >>>> Is this really so helpful? >>> >>> Yes, ASLR is one of very important mitigation techniques which are really used >>> to protect applications. If there is no ASLR, it is very easy to exploit >>> vulnerable application and compromise the system. We can’t just fix all the >>> vulnerabilities right now, thats why we have mitigations - techniques which are >>> makes exploitation more hard or impossible in some cases. >>> >>> Thats why it is helpful. >> >> I am not questioning ASLR in general. I am asking whether we really need >> per mmap ASLR in general. I can imagine that some environments want to >> pay the additional price and other side effects, but considering this >> can be achieved by libc, why to add more code to the kernel? > > I believe this is the only one right place for it. Adding these 200+ lines of > code we give this feature for any user - on desktop, on server, on IoT device, > on SCADA, etc. But if only glibc will implement ‘user-mode-aslr’ IoT and SCADA > devices will never get it. I agree: pushing this off to libc leaves a lot of things unprotected. I think this should live in the kernel. The question I have is about making it maintainable/readable/etc. The state-of-the-art for ASLR is moving to finer granularity (over just base-address offset), so I'd really like to see this supported in the kernel. We'll be getting there for other things in the future, and I'd like to have a working production example for researchers to study, etc. -Kees
On Tue, Mar 27, 2018 at 03:53:53PM -0700, Kees Cook wrote: > I agree: pushing this off to libc leaves a lot of things unprotected. > I think this should live in the kernel. The question I have is about > making it maintainable/readable/etc. > > The state-of-the-art for ASLR is moving to finer granularity (over > just base-address offset), so I'd really like to see this supported in > the kernel. We'll be getting there for other things in the future, and > I'd like to have a working production example for researchers to > study, etc. One thing we need is to limit the fragmentation of this approach. Even on 64-bit systems, we can easily get into a situation where there isn't space to map a contiguous terabyte.
On Tue, Mar 27, 2018 at 4:49 PM, Matthew Wilcox <willy@infradead.org> wrote: > On Tue, Mar 27, 2018 at 03:53:53PM -0700, Kees Cook wrote: >> I agree: pushing this off to libc leaves a lot of things unprotected. >> I think this should live in the kernel. The question I have is about >> making it maintainable/readable/etc. >> >> The state-of-the-art for ASLR is moving to finer granularity (over >> just base-address offset), so I'd really like to see this supported in >> the kernel. We'll be getting there for other things in the future, and >> I'd like to have a working production example for researchers to >> study, etc. > > One thing we need is to limit the fragmentation of this approach. > Even on 64-bit systems, we can easily get into a situation where there isn't > space to map a contiguous terabyte. FWIW, I wouldn't expect normal systems to use this. I am curious about fragmentation vs entropy though. Are workloads with a mis of lots of tiny allocations and TB-allocations? AIUI, glibc uses larger mmap() regions for handling tiny mallocs(). -Kees
On Tue, Mar 27, 2018 at 06:16:35PM -0400, Theodore Y. Ts'o wrote: > On Tue, Mar 27, 2018 at 04:51:08PM +0300, Ilya Smith wrote: > > > /dev/[u]random is not sufficient? > > > > Using /dev/[u]random makes 3 syscalls - open, read, close. This is a performance > > issue. > > You may want to take a look at the getrandom(2) system call, which is > the recommended way getting secure random numbers from the kernel. Yes, while opening /dev/urandom is not acceptable due to needing an fd, getrandom and existing fallbacks for it have this covered if needed. > > > Well, I am pretty sure userspace can implement proper free ranges > > > tracking… > > > > I think we need to know what libc developers will say on implementing ASLR in > > user-mode. I am pretty sure they will say ‘nether’ or ‘some-day’. And problem > > of ASLR will stay forever. > > Why can't you send patches to the libc developers? I can tell you right now that any patch submitted for musl that depended on trying to duplicate knowledge of the entire virtual address space layout in userspace as part of mmap would be rejected, and I would recommend glibc do the same. Not only does it vastly increase complexity; it also has all sorts of failure modes (fd exhastion, etc.) which would either introduce new and unwanted ways for mmap to fail, or would force fallback to the normal (no extra randomization) strategy under conditions an attacker could potentially control, defeating the whole purpose. It would also potentially make it easier for an attacker to examine the vm layout for attacks, since it would be recorded in userspace. There's also the issue of preserving AS-safety of mmap. POSIX does not actually require mmap to be AS-safe, and on musl munmap is not fully AS-safe anyway because of some obscure issues it compensates for, but we may be able to make it AS-safe (this is a low-priority open issue). If mmap were manipulating data structures representing the vm space in userspace, though, the only way to make it anywhere near AS-safe would be to block all signals and take a lock every time mmap or munmap is called. This would significantly increase the cost of each call, especially now that meltdown/spectre mitigations have greatly increased the overhead of each syscall. Overall, asking userspace to take a lead role in management of process vm space is a radical change in the split of what user and kernel are responsible for, and it really does not make sense as part of a dubious hardening measure. Something this big would need to be really well-motivated. Rich
On Tue, Mar 27, 2018 at 04:49:04PM -0700, Matthew Wilcox wrote: > On Tue, Mar 27, 2018 at 03:53:53PM -0700, Kees Cook wrote: > > I agree: pushing this off to libc leaves a lot of things unprotected. > > I think this should live in the kernel. The question I have is about > > making it maintainable/readable/etc. > > > > The state-of-the-art for ASLR is moving to finer granularity (over > > just base-address offset), so I'd really like to see this supported in > > the kernel. We'll be getting there for other things in the future, and > > I'd like to have a working production example for researchers to > > study, etc. > > One thing we need is to limit the fragmentation of this approach. > Even on 64-bit systems, we can easily get into a situation where there isn't > space to map a contiguous terabyte. The default limit of only 65536 VMAs will also quickly come into play if consecutive anon mmaps don't get merged. Of course this can be raised, but it has significant resource and performance (fork) costs. Rich
On 03/23/2018 02:06 PM, Matthew Wilcox wrote: > On Fri, Mar 23, 2018 at 02:00:24PM -0400, Rich Felker wrote: >> On Fri, Mar 23, 2018 at 05:48:06AM -0700, Matthew Wilcox wrote: >>> On Thu, Mar 22, 2018 at 07:36:36PM +0300, Ilya Smith wrote: >>>> Current implementation doesn't randomize address returned by mmap. >>>> All the entropy ends with choosing mmap_base_addr at the process >>>> creation. After that mmap build very predictable layout of address >>>> space. It allows to bypass ASLR in many cases. This patch make >>>> randomization of address on any mmap call. >>> >>> Why should this be done in the kernel rather than libc? libc is perfectly >>> capable of specifying random numbers in the first argument of mmap. >> >> Generally libc does not have a view of the current vm maps, and thus >> in passing "random numbers", they would have to be uniform across the >> whole vm space and thus non-uniform once the kernel rounds up to avoid >> existing mappings. > > I'm aware that you're the musl author, but glibc somehow manages to > provide etext, edata and end, demonstrating that it does know where at > least some of the memory map lies. You can parse /proc/self/maps, but it's really expensive and disgusting. Rob
> On 27 Mar 2018, at 17:38, Michal Hocko <mhocko@kernel.org> wrote: > > On Tue 27-03-18 16:51:08, Ilya Smith wrote: >> >>> On 27 Mar 2018, at 10:24, Michal Hocko <mhocko@kernel.org> wrote: >>> >>> On Mon 26-03-18 22:45:31, Ilya Smith wrote: >>>> >>>>> On 26 Mar 2018, at 11:46, Michal Hocko <mhocko@kernel.org> wrote: >>>>> >>>>> On Fri 23-03-18 20:55:49, Ilya Smith wrote: >>>>>> >>>>>>> On 23 Mar 2018, at 15:48, Matthew Wilcox <willy@infradead.org> wrote: >>>>>>> >>>>>>> On Thu, Mar 22, 2018 at 07:36:36PM +0300, Ilya Smith wrote: >>>>>>>> Current implementation doesn't randomize address returned by mmap. >>>>>>>> All the entropy ends with choosing mmap_base_addr at the process >>>>>>>> creation. After that mmap build very predictable layout of address >>>>>>>> space. It allows to bypass ASLR in many cases. This patch make >>>>>>>> randomization of address on any mmap call. >>>>>>> >>>>>>> Why should this be done in the kernel rather than libc? libc is perfectly >>>>>>> capable of specifying random numbers in the first argument of mmap. >>>>>> Well, there is following reasons: >>>>>> 1. It should be done in any libc implementation, what is not possible IMO; >>>>> >>>>> Is this really so helpful? >>>> >>>> Yes, ASLR is one of very important mitigation techniques which are really used >>>> to protect applications. If there is no ASLR, it is very easy to exploit >>>> vulnerable application and compromise the system. We can’t just fix all the >>>> vulnerabilities right now, thats why we have mitigations - techniques which are >>>> makes exploitation more hard or impossible in some cases. >>>> >>>> Thats why it is helpful. >>> >>> I am not questioning ASLR in general. I am asking whether we really need >>> per mmap ASLR in general. I can imagine that some environments want to >>> pay the additional price and other side effects, but considering this >>> can be achieved by libc, why to add more code to the kernel? >> >> I believe this is the only one right place for it. Adding these 200+ lines of >> code we give this feature for any user - on desktop, on server, on IoT device, >> on SCADA, etc. But if only glibc will implement ‘user-mode-aslr’ IoT and SCADA >> devices will never get it. > > I guess it would really help if you could be more specific about the > class of security issues this would help to mitigate. My first > understanding was that we we need some randomization between program > executable segments to reduce the attack space when a single address > leaks and you know the segments layout (ordering). But why do we need > _all_ mmaps to be randomized. Because that complicates the > implementation consirably for different reasons you have mentioned > earlier. > There are following reasons: 1) To protect layout if one region was leaked (as you said). 2) To protect against exploitation of Out-of-bounds vulnerabilities in some cases (CWE-125 , CWE-787) 3) To protect against exploitation of Buffer Overflows in some cases (CWE-120) 4) To protect application in cases when attacker need to guess the address (paper ASLR-NG by Hector Marco-Gisbert and Ismael Ripoll-Ripoll) And may be more cases. > Do you have any specific CVE that would be mitigated by this > randomization approach? > I am sorry, I am not a security expert to see all the cosequences but a > vague - the more randomization the better - sounds rather weak to me. It is hard to name concrete CVE number, sorry. Mitigations are made to prevent exploitation but not to fix vulnerabilities. It means good mitigation will make vulnerable application crash but not been compromised in most cases. This means the better randomization, the less successful exploitation rate. Thanks, Ilya
> On 28 Mar 2018, at 01:16, Theodore Y. Ts'o <tytso@mit.edu> wrote: > > On Tue, Mar 27, 2018 at 04:51:08PM +0300, Ilya Smith wrote: >>> /dev/[u]random is not sufficient? >> >> Using /dev/[u]random makes 3 syscalls - open, read, close. This is a performance >> issue. > > You may want to take a look at the getrandom(2) system call, which is > the recommended way getting secure random numbers from the kernel. > >>> Well, I am pretty sure userspace can implement proper free ranges >>> tracking… >> >> I think we need to know what libc developers will say on implementing ASLR in >> user-mode. I am pretty sure they will say ‘nether’ or ‘some-day’. And problem >> of ASLR will stay forever. > > Why can't you send patches to the libc developers? > > Regards, > > - Ted I still believe the issue is on kernel side, not in library. Best regards, Ilya
> The default limit of only 65536 VMAs will also quickly come into play > if consecutive anon mmaps don't get merged. Of course this can be > raised, but it has significant resource and performance (fork) costs. Could the random mmap address chooser look for how many existing VMAs have space before/after and the right attributes to merge with the new one you want to create? If this is above some threshold (100?) then pick one of them randomly and allocate the new address so that it will merge from below/above with an existing one. That should still give you a very high degree of randomness, but prevent out of control numbers of VMAs from being created. -Tony
> On 28 Mar 2018, at 02:49, Matthew Wilcox <willy@infradead.org> wrote: > > On Tue, Mar 27, 2018 at 03:53:53PM -0700, Kees Cook wrote: >> I agree: pushing this off to libc leaves a lot of things unprotected. >> I think this should live in the kernel. The question I have is about >> making it maintainable/readable/etc. >> >> The state-of-the-art for ASLR is moving to finer granularity (over >> just base-address offset), so I'd really like to see this supported in >> the kernel. We'll be getting there for other things in the future, and >> I'd like to have a working production example for researchers to >> study, etc. > > One thing we need is to limit the fragmentation of this approach. > Even on 64-bit systems, we can easily get into a situation where there isn't > space to map a contiguous terabyte. As I wrote before, shift_random is introduced to be fragmentation limit. Even without it, the main question here is ‘if we can’t allocate memory with N size bytes, how many bytes we already allocated?’. From these point of view I already showed in previous version of patch that if application uses not so big memory allocations, it will have enough memory to use. If it uses XX Gigabytes or Terabytes memory, this application has all chances to be exploited with fully randomization or without. Since it is much easier to find(or guess) any usable pointer, etc. For the instance you have only 128 terabytes of memory for user space, so probability to exploit this application is 1/128 what is not secure at all. This is very rough estimate but I try to make things easier to understand. Best regards, Ilya
Hi! > Current implementation doesn't randomize address returned by mmap. > All the entropy ends with choosing mmap_base_addr at the process > creation. After that mmap build very predictable layout of address > space. It allows to bypass ASLR in many cases. This patch make > randomization of address on any mmap call. How will this interact with people debugging their application, and getting different behaviours based on memory layout? strace, strace again, get different results? Pavel
Hi > On 30 Mar 2018, at 10:55, Pavel Machek <pavel@ucw.cz> wrote: > > Hi! > >> Current implementation doesn't randomize address returned by mmap. >> All the entropy ends with choosing mmap_base_addr at the process >> creation. After that mmap build very predictable layout of address >> space. It allows to bypass ASLR in many cases. This patch make >> randomization of address on any mmap call. > > How will this interact with people debugging their application, and > getting different behaviours based on memory layout? > > strace, strace again, get different results? > Honestly I’m confused about your question. If the only one way for debugging application is to use predictable mmap behaviour, then something went wrong in this live and we should stop using computers at all. Thanks, Ilya
On Fri 2018-03-30 12:07:58, Ilya Smith wrote: > Hi > > > On 30 Mar 2018, at 10:55, Pavel Machek <pavel@ucw.cz> wrote: > > > > Hi! > > > >> Current implementation doesn't randomize address returned by mmap. > >> All the entropy ends with choosing mmap_base_addr at the process > >> creation. After that mmap build very predictable layout of address > >> space. It allows to bypass ASLR in many cases. This patch make > >> randomization of address on any mmap call. > > > > How will this interact with people debugging their application, and > > getting different behaviours based on memory layout? > > > > strace, strace again, get different results? > > > > Honestly I’m confused about your question. If the only one way for debugging > application is to use predictable mmap behaviour, then something went wrong in > this live and we should stop using computers at all. I'm not saying "only way". I'm saying one way, and you are breaking that. There's advanced stuff like debuggers going "back in time". Pavel
> On 30 Mar 2018, at 12:57, Pavel Machek <pavel@ucw.cz> wrote: > > On Fri 2018-03-30 12:07:58, Ilya Smith wrote: >> Hi >> >>> On 30 Mar 2018, at 10:55, Pavel Machek <pavel@ucw.cz> wrote: >>> >>> Hi! >>> >>>> Current implementation doesn't randomize address returned by mmap. >>>> All the entropy ends with choosing mmap_base_addr at the process >>>> creation. After that mmap build very predictable layout of address >>>> space. It allows to bypass ASLR in many cases. This patch make >>>> randomization of address on any mmap call. >>> >>> How will this interact with people debugging their application, and >>> getting different behaviours based on memory layout? >>> >>> strace, strace again, get different results? >>> >> >> Honestly I’m confused about your question. If the only one way for debugging >> application is to use predictable mmap behaviour, then something went wrong in >> this live and we should stop using computers at all. > > I'm not saying "only way". I'm saying one way, and you are breaking > that. There's advanced stuff like debuggers going "back in time". > Correct me if I wrong, when you run gdb for instance and try to debug some application, gdb will disable randomization. This behaviour works with gdb command: set disable-randomization on. As I know, gdb remove flag PF_RANDOMIZE from current personality thats how it disables ASLR for debugging process. According to my patch, flag PF_RANDOMIZE is checked before calling unmapped_area_random. So I don’t breaking debugging. If you talking about the case, when your application crashes under customer environment and you want to debug it; in this case layout of memory is what you don’t control at all and you have to understand what is where. So for debugging memory process layout is not what you should care of. Thanks, Ilya
On Fri, Mar 30, 2018 at 09:55:08AM +0200, Pavel Machek wrote: > Hi! > > > Current implementation doesn't randomize address returned by mmap. > > All the entropy ends with choosing mmap_base_addr at the process > > creation. After that mmap build very predictable layout of address > > space. It allows to bypass ASLR in many cases. This patch make > > randomization of address on any mmap call. > > How will this interact with people debugging their application, and > getting different behaviours based on memory layout? > > strace, strace again, get different results? Normally gdb disables ASLR for the process when invoking a program to debug. I don't see why that would be terribly useful with strace but you can do the same if you want. Rich
> On 29 Mar 2018, at 00:07, Luck, Tony <tony.luck@intel.com> wrote: > >> The default limit of only 65536 VMAs will also quickly come into play >> if consecutive anon mmaps don't get merged. Of course this can be >> raised, but it has significant resource and performance (fork) costs. > > Could the random mmap address chooser look for how many existing > VMAs have space before/after and the right attributes to merge with the > new one you want to create? If this is above some threshold (100?) then > pick one of them randomly and allocate the new address so that it will > merge from below/above with an existing one. > > That should still give you a very high degree of randomness, but prevent > out of control numbers of VMAs from being created. I think this wouldn’t work. For example these 100 allocation may happened on process initialization. But when attacker come to the server all his allocations would be made on the predictable offsets from each other. So in result we did nothing just decrease performance of first 100 allocations. I think I can make ioctl to turn off this randomization per process and it could be used if needed. For example if application going to allocate big chunk or make big memory pressure, etc. Best regards, Ilya