diff mbox

pci: Reserve free space at the end of bridge windows instead of at the beginning

Message ID 1494848732-16497-1-git-send-email-thuth@redhat.com
State Accepted
Headers show

Commit Message

Thomas Huth May 15, 2017, 11:45 a.m. UTC
This reverts commit e53c2541784fba7951c8aa6ccdbe4412fb03fca6
("pci-scan: reserve memory for pci-bridge without devices").
That commit reserved some free space on PCI bridges at the beginning
of the bridges' memory windows (by adjusting the pci-next-[mem|mmio|io]
variables during the pci-bridge-set-[mem|mmio|io]-base functions).

While this was basically a good idea, this way also had two drawbacks:

1) There also might be free space at the end of the window (since the
base of the next bridge window has to be aligned, too), so the free
space on the bridge is non-contiguous.

2) As soon as there was at least one device on the bridge that uses
at least some few byte in the I/O space, SLOF reserved at least 8k
of I/O space on the bridge - which is a *lot* in the scarce I/O space,
so that for example it was not possible anymore to next 8 PCI bridges
with devices attached to them (see the buglink below for details).

It's better to reserve the free space at the end of the memory windows
instead (in the pci-bridge-set-[mem|mmio|io]-limit functions), and
with regards to the scarce I/O space, we should also reserve less
I/O memory on each bridge, so we use a limit of 2k (plus alignment)
here now.

Buglink: https://bugzilla.redhat.com/show_bug.cgi?id=1443433
Signed-off-by: Thomas Huth <thuth@redhat.com>
---
 slof/fs/pci-scan.fs | 15 +++++++++------
 1 file changed, 9 insertions(+), 6 deletions(-)

Comments

David Gibson May 16, 2017, 5:32 a.m. UTC | #1
On Mon, May 15, 2017 at 01:45:32PM +0200, Thomas Huth wrote:
> This reverts commit e53c2541784fba7951c8aa6ccdbe4412fb03fca6
> ("pci-scan: reserve memory for pci-bridge without devices").
> That commit reserved some free space on PCI bridges at the beginning
> of the bridges' memory windows (by adjusting the pci-next-[mem|mmio|io]
> variables during the pci-bridge-set-[mem|mmio|io]-base functions).
> 
> While this was basically a good idea, this way also had two drawbacks:
> 
> 1) There also might be free space at the end of the window (since the
> base of the next bridge window has to be aligned, too), so the free
> space on the bridge is non-contiguous.
> 
> 2) As soon as there was at least one device on the bridge that uses
> at least some few byte in the I/O space, SLOF reserved at least 8k
> of I/O space on the bridge - which is a *lot* in the scarce I/O space,
> so that for example it was not possible anymore to next 8 PCI bridges
> with devices attached to them (see the buglink below for details).
> 
> It's better to reserve the free space at the end of the memory windows
> instead (in the pci-bridge-set-[mem|mmio|io]-limit functions), and
> with regards to the scarce I/O space, we should also reserve less
> I/O memory on each bridge, so we use a limit of 2k (plus alignment)
> here now.
> 
> Buglink: https://bugzilla.redhat.com/show_bug.cgi?id=1443433
> Signed-off-by: Thomas Huth <thuth@redhat.com>

I don't speak Forth, but from the description it sounds like a good idea.

> ---
>  slof/fs/pci-scan.fs | 15 +++++++++------
>  1 file changed, 9 insertions(+), 6 deletions(-)
> 
> diff --git a/slof/fs/pci-scan.fs b/slof/fs/pci-scan.fs
> index c0dbbed..9578189 100644
> --- a/slof/fs/pci-scan.fs
> +++ b/slof/fs/pci-scan.fs
> @@ -81,7 +81,7 @@ here 100 allot CONSTANT pci-device-vec
>  \ needed for scanning possible devices behind the bridge
>  : pci-bridge-set-mmio-base ( addr -- )
>          pci-next-mmio @ 100000 #aligned         \ read the current Value and align to 1MB boundary
> -        dup 100000 + pci-next-mmio !            \ and write back with 1MB for bridge
> +        dup pci-next-mmio !                     \ and write it back
>          10 rshift                               \ mmio-base reg is only the upper 16 bits
>          pci-max-mmio @ 1- FFFF0000 and or       \ and Insert mmio Limit (set it to max)
>          swap 20 + rtas-config-l!                \ and write it into the bridge
> @@ -91,7 +91,8 @@ here 100 allot CONSTANT pci-device-vec
>  \ The Limit Value is one less then the upper boundary
>  \ If the limit is less than the base the mmio is disabled
>  : pci-bridge-set-mmio-limit ( addr -- )
> -        pci-next-mmio @ 100000 #aligned         \ fetch current value and align to 1MB
> +        pci-next-mmio @ 100000 +                \ add space for hot-plugging
> +        100000 #aligned                         \ align to 1MB boundary
>          dup pci-next-mmio !                     \ and write it back
>          1- FFFF0000 and                         \ make it one less and keep upper 16 bits
>          over 20 + rtas-config-l@ 0000FFFF and   \ fetch original value
> @@ -103,7 +104,7 @@ here 100 allot CONSTANT pci-device-vec
>  \ needed for scanning possible devices behind the bridge
>  : pci-bridge-set-mem-base ( addr -- )
>          pci-next-mem @ 100000 #aligned          \ read the current Value and align to 1MB boundary
> -        dup 100000 + pci-next-mem !             \ and write back with 1MB for bridge
> +        dup pci-next-mem !                      \ and write it back
>          over 24 + rtas-config-w@                \ check if 64bit support
>          1 and IF                                \ IF 64 bit support
>                  pci-next-mem64 @ 100000000 #aligned \ | read the current Value of 64-bit and align to 4GB boundary
> @@ -123,7 +124,8 @@ here 100 allot CONSTANT pci-device-vec
>  \ The Limit Value is one less then the upper boundary
>  \ If the limit is less than the base the mem is disabled
>  : pci-bridge-set-mem-limit ( addr -- )
> -        pci-next-mem @ 100000 #aligned          \ read the current Value and align to 1MB boundary
> +        pci-next-mem @ 100000 +                 \ add space for hot-plugging
> +        100000 #aligned                         \ align to 1MB boundary
>          dup pci-next-mem !                      \ and write it back
>          1-                                      \ make limit one less than boundary
>          over 24 + rtas-config-w@                \ check if 64bit support
> @@ -145,7 +147,7 @@ here 100 allot CONSTANT pci-device-vec
>  \ needed for scanning possible devices behind the bridge
>  : pci-bridge-set-io-base ( addr -- )
>          pci-next-io @ 1000 #aligned             \ read the current Value and align to 4KB boundary
> -        dup 1000 + pci-next-io !                \ and write back with 4K for bridge
> +        dup pci-next-io !                       \ and write it back
>          over 1C + rtas-config-l@                \ check if 32bit support
>          1 and IF                                \ IF 32 bit support
>                  2dup 10 rshift                  \ | keep upper 16 bits
> @@ -162,7 +164,8 @@ here 100 allot CONSTANT pci-device-vec
>  \ The Limit Value is one less then the upper boundary
>  \ If the limit is less than the base the io is disabled
>  : pci-bridge-set-io-limit ( addr -- )
> -        pci-next-io @ 1000 #aligned             \ read the current Value and align to 4KB boundary
> +        pci-next-io @ 800 +                     \ add space for hot-plugging
> +        1000 #aligned                           \ align to 4KB boundary
>          dup pci-next-io !                       \ and write it back
>          1-                                      \ make limit one less than boundary
>          over 1D + rtas-config-b@                \ check if 32bit support
Nikunj A Dadhania May 16, 2017, 6:47 a.m. UTC | #2
Thomas Huth <thuth@redhat.com> writes:

> This reverts commit e53c2541784fba7951c8aa6ccdbe4412fb03fca6
> ("pci-scan: reserve memory for pci-bridge without devices").

The above patch was added for an issue that was reported when an empty
pci-bridge was added along with the network adapter, rhel6.5/sles11sp3
was not able to get the dhcp address for the network.

The actual problem was while enumerating an empty pci-bridge, stale
range value programmed before the bridge probing stayed. Which then
caused overlapping pci address allocation.

This does not seem to be an issue any more.

> That commit reserved some free space on PCI bridges at the beginning
> of the bridges' memory windows (by adjusting the pci-next-[mem|mmio|io]
> variables during the pci-bridge-set-[mem|mmio|io]-base functions).
>
> While this was basically a good idea, this way also had two drawbacks:
>
> 1) There also might be free space at the end of the window (since the
> base of the next bridge window has to be aligned, too), so the free
> space on the bridge is non-contiguous.
>
> 2) As soon as there was at least one device on the bridge that uses
> at least some few byte in the I/O space, SLOF reserved at least 8k
> of I/O space on the bridge - which is a *lot* in the scarce I/O space,
> so that for example it was not possible anymore to next 8 PCI bridges
> with devices attached to them (see the buglink below for details).
>
> It's better to reserve the free space at the end of the memory windows
> instead (in the pci-bridge-set-[mem|mmio|io]-limit functions), and
> with regards to the scarce I/O space, we should also reserve less
> I/O memory on each bridge, so we use a limit of 2k (plus alignment)
> here now.
>
> Buglink: https://bugzilla.redhat.com/show_bug.cgi?id=1443433
> Signed-off-by: Thomas Huth <thuth@redhat.com>

Reviewed-by: Nikunj A Dadhania <nikunj@linux.vnet.ibm.com>

> ---
>  slof/fs/pci-scan.fs | 15 +++++++++------
>  1 file changed, 9 insertions(+), 6 deletions(-)
>
> diff --git a/slof/fs/pci-scan.fs b/slof/fs/pci-scan.fs
> index c0dbbed..9578189 100644
> --- a/slof/fs/pci-scan.fs
> +++ b/slof/fs/pci-scan.fs
> @@ -81,7 +81,7 @@ here 100 allot CONSTANT pci-device-vec
>  \ needed for scanning possible devices behind the bridge
>  : pci-bridge-set-mmio-base ( addr -- )
>          pci-next-mmio @ 100000 #aligned         \ read the current Value and align to 1MB boundary
> -        dup 100000 + pci-next-mmio !            \ and write back with 1MB for bridge
> +        dup pci-next-mmio !                     \ and write it back
>          10 rshift                               \ mmio-base reg is only the upper 16 bits
>          pci-max-mmio @ 1- FFFF0000 and or       \ and Insert mmio Limit (set it to max)
>          swap 20 + rtas-config-l!                \ and write it into the bridge
> @@ -91,7 +91,8 @@ here 100 allot CONSTANT pci-device-vec
>  \ The Limit Value is one less then the upper boundary
>  \ If the limit is less than the base the mmio is disabled
>  : pci-bridge-set-mmio-limit ( addr -- )
> -        pci-next-mmio @ 100000 #aligned         \ fetch current value and align to 1MB
> +        pci-next-mmio @ 100000 +                \ add space for hot-plugging
> +        100000 #aligned                         \ align to 1MB boundary
>          dup pci-next-mmio !                     \ and write it back
>          1- FFFF0000 and                         \ make it one less and keep upper 16 bits
>          over 20 + rtas-config-l@ 0000FFFF and   \ fetch original value
> @@ -103,7 +104,7 @@ here 100 allot CONSTANT pci-device-vec
>  \ needed for scanning possible devices behind the bridge
>  : pci-bridge-set-mem-base ( addr -- )
>          pci-next-mem @ 100000 #aligned          \ read the current Value and align to 1MB boundary
> -        dup 100000 + pci-next-mem !             \ and write back with 1MB for bridge
> +        dup pci-next-mem !                      \ and write it back
>          over 24 + rtas-config-w@                \ check if 64bit support
>          1 and IF                                \ IF 64 bit support
>                  pci-next-mem64 @ 100000000 #aligned \ | read the current Value of 64-bit and align to 4GB boundary
> @@ -123,7 +124,8 @@ here 100 allot CONSTANT pci-device-vec
>  \ The Limit Value is one less then the upper boundary
>  \ If the limit is less than the base the mem is disabled
>  : pci-bridge-set-mem-limit ( addr -- )
> -        pci-next-mem @ 100000 #aligned          \ read the current Value and align to 1MB boundary
> +        pci-next-mem @ 100000 +                 \ add space for hot-plugging
> +        100000 #aligned                         \ align to 1MB boundary
>          dup pci-next-mem !                      \ and write it back
>          1-                                      \ make limit one less than boundary
>          over 24 + rtas-config-w@                \ check if 64bit support
> @@ -145,7 +147,7 @@ here 100 allot CONSTANT pci-device-vec
>  \ needed for scanning possible devices behind the bridge
>  : pci-bridge-set-io-base ( addr -- )
>          pci-next-io @ 1000 #aligned             \ read the current Value and align to 4KB boundary
> -        dup 1000 + pci-next-io !                \ and write back with 4K for bridge
> +        dup pci-next-io !                       \ and write it back
>          over 1C + rtas-config-l@                \ check if 32bit support
>          1 and IF                                \ IF 32 bit support
>                  2dup 10 rshift                  \ | keep upper 16 bits
> @@ -162,7 +164,8 @@ here 100 allot CONSTANT pci-device-vec
>  \ The Limit Value is one less then the upper boundary
>  \ If the limit is less than the base the io is disabled
>  : pci-bridge-set-io-limit ( addr -- )
> -        pci-next-io @ 1000 #aligned             \ read the current Value and align to 4KB boundary
> +        pci-next-io @ 800 +                     \ add space for hot-plugging
> +        1000 #aligned                           \ align to 4KB boundary
>          dup pci-next-io !                       \ and write it back
>          1-                                      \ make limit one less than boundary
>          over 1D + rtas-config-b@                \ check if 32bit support
> -- 
> 1.8.3.1
Thomas Huth May 16, 2017, 7:05 a.m. UTC | #3
On 16.05.2017 08:47, Nikunj A Dadhania wrote:
> Thomas Huth <thuth@redhat.com> writes:
> 
>> This reverts commit e53c2541784fba7951c8aa6ccdbe4412fb03fca6
>> ("pci-scan: reserve memory for pci-bridge without devices").
> 
> The above patch was added for an issue that was reported when an empty
> pci-bridge was added along with the network adapter, rhel6.5/sles11sp3
> was not able to get the dhcp address for the network.
> 
> The actual problem was while enumerating an empty pci-bridge, stale
> range value programmed before the bridge probing stayed. Which then
> caused overlapping pci address allocation.
> 
> This does not seem to be an issue any more.

Ok, good to know what the original problem was. Looking at the git
history, it seems like the main fix for the problem was the commit
7cc0b6c5ce87549ba7bc396bfca1018c076a7351 already, and the "reserve
memory" patch was just the additional sugar (since hot-plugging on pci
bridges is not possible yet, as far as I know)?

>> It's better to reserve the free space at the end of the memory windows
>> instead (in the pci-bridge-set-[mem|mmio|io]-limit functions), and
>> with regards to the scarce I/O space, we should also reserve less
>> I/O memory on each bridge, so we use a limit of 2k (plus alignment)
>> here now.
>>
>> Buglink: https://bugzilla.redhat.com/show_bug.cgi?id=1443433
>> Signed-off-by: Thomas Huth <thuth@redhat.com>
> 
> Reviewed-by: Nikunj A Dadhania <nikunj@linux.vnet.ibm.com>

Thanks!

Any chance that you could commit this while Alexey is away?

 Thomas
Nikunj A Dadhania May 16, 2017, 8:07 a.m. UTC | #4
Thomas Huth <thuth@redhat.com> writes:

> On 16.05.2017 08:47, Nikunj A Dadhania wrote:
>> Thomas Huth <thuth@redhat.com> writes:
>> 
>>> This reverts commit e53c2541784fba7951c8aa6ccdbe4412fb03fca6
>>> ("pci-scan: reserve memory for pci-bridge without devices").
>> 
>> The above patch was added for an issue that was reported when an empty
>> pci-bridge was added along with the network adapter, rhel6.5/sles11sp3
>> was not able to get the dhcp address for the network.
>> 
>> The actual problem was while enumerating an empty pci-bridge, stale
>> range value programmed before the bridge probing stayed. Which then
>> caused overlapping pci address allocation.
>> 
>> This does not seem to be an issue any more.
>
> Ok, good to know what the original problem was. Looking at the git
> history, it seems like the main fix for the problem was the commit
> 7cc0b6c5ce87549ba7bc396bfca1018c076a7351 already, and the "reserve
> memory" patch was just the additional sugar

Right

> (since hot-plugging on pci
> bridges is not possible yet, as far as I know)?
>
>>> It's better to reserve the free space at the end of the memory windows
>>> instead (in the pci-bridge-set-[mem|mmio|io]-limit functions), and
>>> with regards to the scarce I/O space, we should also reserve less
>>> I/O memory on each bridge, so we use a limit of 2k (plus alignment)
>>> here now.
>>>
>>> Buglink: https://bugzilla.redhat.com/show_bug.cgi?id=1443433
>>> Signed-off-by: Thomas Huth <thuth@redhat.com>
>> 
>> Reviewed-by: Nikunj A Dadhania <nikunj@linux.vnet.ibm.com>
>
> Thanks!
>
> Any chance that you could commit this while Alexey is away?

Pushed, thanks.

Regards
Nikunj
Alexey Kardashevskiy May 18, 2017, 11:27 p.m. UTC | #5
On 16/05/17 17:05, Thomas Huth wrote:
> On 16.05.2017 08:47, Nikunj A Dadhania wrote:
>> Thomas Huth <thuth@redhat.com> writes:
>>
>>> This reverts commit e53c2541784fba7951c8aa6ccdbe4412fb03fca6
>>> ("pci-scan: reserve memory for pci-bridge without devices").
>>
>> The above patch was added for an issue that was reported when an empty
>> pci-bridge was added along with the network adapter, rhel6.5/sles11sp3
>> was not able to get the dhcp address for the network.
>>
>> The actual problem was while enumerating an empty pci-bridge, stale
>> range value programmed before the bridge probing stayed. Which then
>> caused overlapping pci address allocation.
>>
>> This does not seem to be an issue any more.
> 
> Ok, good to know what the original problem was. Looking at the git
> history, it seems like the main fix for the problem was the commit
> 7cc0b6c5ce87549ba7bc396bfca1018c076a7351 already, and the "reserve
> memory" patch was just the additional sugar (since hot-plugging on pci
> bridges is not possible yet, as far as I know)?
> 
>>> It's better to reserve the free space at the end of the memory windows
>>> instead (in the pci-bridge-set-[mem|mmio|io]-limit functions), and
>>> with regards to the scarce I/O space, we should also reserve less
>>> I/O memory on each bridge, so we use a limit of 2k (plus alignment)
>>> here now.
>>>
>>> Buglink: https://bugzilla.redhat.com/show_bug.cgi?id=1443433
>>> Signed-off-by: Thomas Huth <thuth@redhat.com>
>>
>> Reviewed-by: Nikunj A Dadhania <nikunj@linux.vnet.ibm.com>
> 
> Thanks!
> 
> Any chance that you could commit this while Alexey is away?

What is the hurry (just curious)?


> 
>  Thomas
> _______________________________________________
> SLOF mailing list
> SLOF@lists.ozlabs.org
> https://lists.ozlabs.org/listinfo/slof
>
David Gibson May 19, 2017, 5:47 a.m. UTC | #6
On Fri, May 19, 2017 at 09:27:44AM +1000, Alexey Kardashevskiy wrote:
> On 16/05/17 17:05, Thomas Huth wrote:
> > On 16.05.2017 08:47, Nikunj A Dadhania wrote:
> >> Thomas Huth <thuth@redhat.com> writes:
> >>
> >>> This reverts commit e53c2541784fba7951c8aa6ccdbe4412fb03fca6
> >>> ("pci-scan: reserve memory for pci-bridge without devices").
> >>
> >> The above patch was added for an issue that was reported when an empty
> >> pci-bridge was added along with the network adapter, rhel6.5/sles11sp3
> >> was not able to get the dhcp address for the network.
> >>
> >> The actual problem was while enumerating an empty pci-bridge, stale
> >> range value programmed before the bridge probing stayed. Which then
> >> caused overlapping pci address allocation.
> >>
> >> This does not seem to be an issue any more.
> > 
> > Ok, good to know what the original problem was. Looking at the git
> > history, it seems like the main fix for the problem was the commit
> > 7cc0b6c5ce87549ba7bc396bfca1018c076a7351 already, and the "reserve
> > memory" patch was just the additional sugar (since hot-plugging on pci
> > bridges is not possible yet, as far as I know)?
> > 
> >>> It's better to reserve the free space at the end of the memory windows
> >>> instead (in the pci-bridge-set-[mem|mmio|io]-limit functions), and
> >>> with regards to the scarce I/O space, we should also reserve less
> >>> I/O memory on each bridge, so we use a limit of 2k (plus alignment)
> >>> here now.
> >>>
> >>> Buglink: https://bugzilla.redhat.com/show_bug.cgi?id=1443433
> >>> Signed-off-by: Thomas Huth <thuth@redhat.com>
> >>
> >> Reviewed-by: Nikunj A Dadhania <nikunj@linux.vnet.ibm.com>
> > 
> > Thanks!
> > 
> > Any chance that you could commit this while Alexey is away?
> 
> What is the hurry (just curious)?

It'd be nice to fix for RHEL7.4 which is very close now.

That said, I just realized there's another wrinkle here - we're going
to need to introduce a compatibility mode for SLOF (presumably keyed
off a device tree variable) so that we don't introduce a guest-visible
change in older machine types.
Thomas Huth May 19, 2017, 6:28 a.m. UTC | #7
On 19.05.2017 07:47, David Gibson wrote:
[...]
> That said, I just realized there's another wrinkle here - we're going
> to need to introduce a compatibility mode for SLOF (presumably keyed
> off a device tree variable) so that we don't introduce a guest-visible
> change in older machine types.

I don't think so. It does not affect migration, you only get a different
layout once you reboot the guest - and then the OS should adapt to that
new layout automatically.

 Thomas
David Gibson May 19, 2017, 6:44 a.m. UTC | #8
On Fri, May 19, 2017 at 08:28:48AM +0200, Thomas Huth wrote:
> On 19.05.2017 07:47, David Gibson wrote:
> [...]
> > That said, I just realized there's another wrinkle here - we're going
> > to need to introduce a compatibility mode for SLOF (presumably keyed
> > off a device tree variable) so that we don't introduce a guest-visible
> > change in older machine types.
> 
> I don't think so. It does not affect migration, you only get a different
> layout once you reboot the guest - and then the OS should adapt to that
> new layout automatically.

Hm, according to Andrea at least, that's not sufficient - the same
argument applies to the CAS PVR negotiation changes.
Thomas Huth May 19, 2017, 7:14 a.m. UTC | #9
On 19.05.2017 08:44, David Gibson wrote:
> On Fri, May 19, 2017 at 08:28:48AM +0200, Thomas Huth wrote:
>> On 19.05.2017 07:47, David Gibson wrote:
>> [...]
>>> That said, I just realized there's another wrinkle here - we're going
>>> to need to introduce a compatibility mode for SLOF (presumably keyed
>>> off a device tree variable) so that we don't introduce a guest-visible
>>> change in older machine types.
>>
>> I don't think so. It does not affect migration, you only get a different
>> layout once you reboot the guest - and then the OS should adapt to that
>> new layout automatically.
> 
> Hm, according to Andrea at least, that's not sufficient - the same
> argument applies to the CAS PVR negotiation changes.

We've never added compatibility switches in the past that only affect
the boot process of SLOF ... for example, we also did not add anything
for the recent changes like 975b31f80aff26addee5d70c34de9cd1b0a204ca or
cbb69ccd92de3864ec14ab48ae256ce6876c244f. And I don't see a reason why
we should change this now, or is there a scenario where this could
really, really render problems for us?

 Thomas
diff mbox

Patch

diff --git a/slof/fs/pci-scan.fs b/slof/fs/pci-scan.fs
index c0dbbed..9578189 100644
--- a/slof/fs/pci-scan.fs
+++ b/slof/fs/pci-scan.fs
@@ -81,7 +81,7 @@  here 100 allot CONSTANT pci-device-vec
 \ needed for scanning possible devices behind the bridge
 : pci-bridge-set-mmio-base ( addr -- )
         pci-next-mmio @ 100000 #aligned         \ read the current Value and align to 1MB boundary
-        dup 100000 + pci-next-mmio !            \ and write back with 1MB for bridge
+        dup pci-next-mmio !                     \ and write it back
         10 rshift                               \ mmio-base reg is only the upper 16 bits
         pci-max-mmio @ 1- FFFF0000 and or       \ and Insert mmio Limit (set it to max)
         swap 20 + rtas-config-l!                \ and write it into the bridge
@@ -91,7 +91,8 @@  here 100 allot CONSTANT pci-device-vec
 \ The Limit Value is one less then the upper boundary
 \ If the limit is less than the base the mmio is disabled
 : pci-bridge-set-mmio-limit ( addr -- )
-        pci-next-mmio @ 100000 #aligned         \ fetch current value and align to 1MB
+        pci-next-mmio @ 100000 +                \ add space for hot-plugging
+        100000 #aligned                         \ align to 1MB boundary
         dup pci-next-mmio !                     \ and write it back
         1- FFFF0000 and                         \ make it one less and keep upper 16 bits
         over 20 + rtas-config-l@ 0000FFFF and   \ fetch original value
@@ -103,7 +104,7 @@  here 100 allot CONSTANT pci-device-vec
 \ needed for scanning possible devices behind the bridge
 : pci-bridge-set-mem-base ( addr -- )
         pci-next-mem @ 100000 #aligned          \ read the current Value and align to 1MB boundary
-        dup 100000 + pci-next-mem !             \ and write back with 1MB for bridge
+        dup pci-next-mem !                      \ and write it back
         over 24 + rtas-config-w@                \ check if 64bit support
         1 and IF                                \ IF 64 bit support
                 pci-next-mem64 @ 100000000 #aligned \ | read the current Value of 64-bit and align to 4GB boundary
@@ -123,7 +124,8 @@  here 100 allot CONSTANT pci-device-vec
 \ The Limit Value is one less then the upper boundary
 \ If the limit is less than the base the mem is disabled
 : pci-bridge-set-mem-limit ( addr -- )
-        pci-next-mem @ 100000 #aligned          \ read the current Value and align to 1MB boundary
+        pci-next-mem @ 100000 +                 \ add space for hot-plugging
+        100000 #aligned                         \ align to 1MB boundary
         dup pci-next-mem !                      \ and write it back
         1-                                      \ make limit one less than boundary
         over 24 + rtas-config-w@                \ check if 64bit support
@@ -145,7 +147,7 @@  here 100 allot CONSTANT pci-device-vec
 \ needed for scanning possible devices behind the bridge
 : pci-bridge-set-io-base ( addr -- )
         pci-next-io @ 1000 #aligned             \ read the current Value and align to 4KB boundary
-        dup 1000 + pci-next-io !                \ and write back with 4K for bridge
+        dup pci-next-io !                       \ and write it back
         over 1C + rtas-config-l@                \ check if 32bit support
         1 and IF                                \ IF 32 bit support
                 2dup 10 rshift                  \ | keep upper 16 bits
@@ -162,7 +164,8 @@  here 100 allot CONSTANT pci-device-vec
 \ The Limit Value is one less then the upper boundary
 \ If the limit is less than the base the io is disabled
 : pci-bridge-set-io-limit ( addr -- )
-        pci-next-io @ 1000 #aligned             \ read the current Value and align to 4KB boundary
+        pci-next-io @ 800 +                     \ add space for hot-plugging
+        1000 #aligned                           \ align to 4KB boundary
         dup pci-next-io !                       \ and write it back
         1-                                      \ make limit one less than boundary
         over 1D + rtas-config-b@                \ check if 32bit support