| Message ID | 20170512053957.10426-1-mrugiero@gmail.com |
|---|---|
| State | Rejected |
| Delegated to: | Boris Brezillon |
| Headers | show |
Mario, On Fri, May 12, 2017 at 7:39 AM, Mario J. Rugiero <mrugiero@gmail.com> wrote: > Some chips used under a custom vendor driver can get their blocks > incorrectly detected as bad blocks, out of incompatibilities > between such drivers and MTD drivers. > When there are too many misdetected bad blocks, the device becomes > unusable because a bad block table can't be allocated, aside from > all the legitimately good blocks which become unusable under these > conditions. > This adds a build option to workaround the issue by enabling the > user to free up space regardless of what the driver thinks about > the blocks. Hmm, this sounds like a gross hack. > Example usage: recovering NAND chips on sunxi devices, as explained > here: http://linux-sunxi.org/Mainline_NAND_Howto#Known_issues What this wiki suggests is not wise. How can you know which blocks are really bad and which not?
2017-05-12 5:12 GMT-03:00 Richard Weinberger <richard.weinberger@gmail.com>: > Mario, > > On Fri, May 12, 2017 at 7:39 AM, Mario J. Rugiero <mrugiero@gmail.com> wrote: >> Some chips used under a custom vendor driver can get their blocks >> incorrectly detected as bad blocks, out of incompatibilities >> between such drivers and MTD drivers. >> When there are too many misdetected bad blocks, the device becomes >> unusable because a bad block table can't be allocated, aside from >> all the legitimately good blocks which become unusable under these >> conditions. >> This adds a build option to workaround the issue by enabling the >> user to free up space regardless of what the driver thinks about >> the blocks. > > Hmm, this sounds like a gross hack. It is, but I see no other solution. The NAND chips were used in an incompatible way by a hack-n-slash driver made by allwinner, and trying to load them with a proper MTD driver fails miserably if this is not done. If anyone can propose a better solution I'll more than happily implement it. I'm open to suggestions, and of course I'm open to rejection of my patches if needed. > >> Example usage: recovering NAND chips on sunxi devices, as explained >> here: http://linux-sunxi.org/Mainline_NAND_Howto#Known_issues > > What this wiki suggests is not wise. > How can you know which blocks are really bad and which not? You don't, at least not without an even grosser hack implementing read support for their incompatible format. Would that be better? I might attempt it if desired. > > -- > Thanks, > //richard
On Fri, 12 May 2017 05:16:08 -0300 Mario Rugiero <mrugiero@gmail.com> wrote: > 2017-05-12 5:12 GMT-03:00 Richard Weinberger <richard.weinberger@gmail.com>: > > Mario, > > > > On Fri, May 12, 2017 at 7:39 AM, Mario J. Rugiero <mrugiero@gmail.com> wrote: > >> Some chips used under a custom vendor driver can get their blocks > >> incorrectly detected as bad blocks, out of incompatibilities > >> between such drivers and MTD drivers. > >> When there are too many misdetected bad blocks, the device becomes > >> unusable because a bad block table can't be allocated, aside from > >> all the legitimately good blocks which become unusable under these > >> conditions. > >> This adds a build option to workaround the issue by enabling the > >> user to free up space regardless of what the driver thinks about > >> the blocks. > > > > Hmm, this sounds like a gross hack. > It is, but I see no other solution. The NAND chips were used in an > incompatible way by a hack-n-slash driver made by allwinner, and > trying to load them with a proper MTD driver fails miserably if this > is not done. > If anyone can propose a better solution I'll more than happily implement it. > I'm open to suggestions, and of course I'm open to rejection of my > patches if needed. u-boot provides the nand.scrub command, which does exactly what you're looking for. And no, I don't think it's a good idea to allow erasing bad blocks, at least not by default. If we really want to support this feature in linux, this should be explicitly enabled through debugfs. > > > >> Example usage: recovering NAND chips on sunxi devices, as explained > >> here: http://linux-sunxi.org/Mainline_NAND_Howto#Known_issues > > > > What this wiki suggests is not wise. > > How can you know which blocks are really bad and which not? > You don't, at least not without an even grosser hack implementing read > support for their incompatible format. > Would that be better? I might attempt it if desired. No, please don't do that, at least not in the kernel. If you really want to parse the old format, you should develop a tool that reads NAND pages in raw mode, stores the list of bad blocks somewhere and then re-use this list to select which blocks should be forcibly erased. Not sure it's worth the pain :-).
Boris, Mario, Am 12.05.2017 um 10:24 schrieb Boris Brezillon: >>> Hmm, this sounds like a gross hack. >> It is, but I see no other solution. The NAND chips were used in an >> incompatible way by a hack-n-slash driver made by allwinner, and >> trying to load them with a proper MTD driver fails miserably if this >> is not done. >> If anyone can propose a better solution I'll more than happily implement it. >> I'm open to suggestions, and of course I'm open to rejection of my >> patches if needed. > > u-boot provides the nand.scrub command, which does exactly what you're > looking for. And no, I don't think it's a good idea to allow erasing > bad blocks, at least not by default. To make this very clear for all MTD users out there, scrubbing the NAND and losing the information which blocks are bad is awful. Bad blocks can work somehow and fail much later in funny ways. UBI/FS problems ahead... Do this only if you *really* know what you are doing. Thanks, //richard
2017-05-12 5:24 GMT-03:00 Boris Brezillon <boris.brezillon@free-electrons.com>: > On Fri, 12 May 2017 05:16:08 -0300 > Mario Rugiero <mrugiero@gmail.com> wrote: > >> 2017-05-12 5:12 GMT-03:00 Richard Weinberger <richard.weinberger@gmail.com>: >> > Mario, >> > >> > On Fri, May 12, 2017 at 7:39 AM, Mario J. Rugiero <mrugiero@gmail.com> wrote: >> >> Some chips used under a custom vendor driver can get their blocks >> >> incorrectly detected as bad blocks, out of incompatibilities >> >> between such drivers and MTD drivers. >> >> When there are too many misdetected bad blocks, the device becomes >> >> unusable because a bad block table can't be allocated, aside from >> >> all the legitimately good blocks which become unusable under these >> >> conditions. >> >> This adds a build option to workaround the issue by enabling the >> >> user to free up space regardless of what the driver thinks about >> >> the blocks. >> > >> > Hmm, this sounds like a gross hack. >> It is, but I see no other solution. The NAND chips were used in an >> incompatible way by a hack-n-slash driver made by allwinner, and >> trying to load them with a proper MTD driver fails miserably if this >> is not done. >> If anyone can propose a better solution I'll more than happily implement it. >> I'm open to suggestions, and of course I'm open to rejection of my >> patches if needed. > > u-boot provides the nand.scrub command, which does exactly what you're > looking for. And no, I don't think it's a good idea to allow erasing > bad blocks, at least not by default. > > If we really want to support this feature in linux, this should be > explicitly enabled through debugfs. If I do this, does it stand a chance at getting upstream? If so, I'll have it done soon. Note however that the build option is disabled by default. I get that there should also be one runtime option, disabled by default, exposed through debugfs. Does that sound right? > >> > >> >> Example usage: recovering NAND chips on sunxi devices, as explained >> >> here: http://linux-sunxi.org/Mainline_NAND_Howto#Known_issues >> > >> > What this wiki suggests is not wise. >> > How can you know which blocks are really bad and which not? >> You don't, at least not without an even grosser hack implementing read >> support for their incompatible format. >> Would that be better? I might attempt it if desired. > > No, please don't do that, at least not in the kernel. If you really > want to parse the old format, you should develop a tool that reads NAND > pages in raw mode, stores the list of bad blocks somewhere and then > re-use this list to select which blocks should be forcibly erased. > > Not sure it's worth the pain :-). It's worth the pain to me. I'm dealing with a bit rotten 3.4 based pile of cr*p on production because of this. Whatever I have to do to get those machines running the mainline kernel is worth it.
On Fri, 12 May 2017 10:33:27 +0200 Richard Weinberger <richard@nod.at> wrote: > Boris, Mario, > > Am 12.05.2017 um 10:24 schrieb Boris Brezillon: > >>> Hmm, this sounds like a gross hack. > >> It is, but I see no other solution. The NAND chips were used in an > >> incompatible way by a hack-n-slash driver made by allwinner, and > >> trying to load them with a proper MTD driver fails miserably if this > >> is not done. > >> If anyone can propose a better solution I'll more than happily implement it. > >> I'm open to suggestions, and of course I'm open to rejection of my > >> patches if needed. > > > > u-boot provides the nand.scrub command, which does exactly what you're > > looking for. And no, I don't think it's a good idea to allow erasing > > bad blocks, at least not by default. > > To make this very clear for all MTD users out there, scrubbing the NAND and > losing the information which blocks are bad is awful. > Bad blocks can work somehow and fail much later in funny ways. > UBI/FS problems ahead... > > Do this only if you *really* know what you are doing. I'm clearly not encouraging people to use nand.scrub, it's just that sometime you don't have a choice, and this is the case here: Allwinner is putting non-FF data in the BBM region, and when we switch from an Allwinner kernel to a mainline kernel, the mainline kernel considers all programmed blocks as bad.
Boris, Am 12.05.2017 um 10:44 schrieb Boris Brezillon: >> Do this only if you *really* know what you are doing. > > I'm clearly not encouraging people to use nand.scrub, it's just that > sometime you don't have a choice, and this is the case here: > Allwinner is putting non-FF data in the BBM region, and when we switch > from an Allwinner kernel to a mainline kernel, the mainline kernel > considers all programmed blocks as bad. This is the "...only if you *really* know what you are doing" part. :-) Thanks, //richard
On Fri, 12 May 2017 05:34:10 -0300 Mario Rugiero <mrugiero@gmail.com> wrote: > 2017-05-12 5:24 GMT-03:00 Boris Brezillon <boris.brezillon@free-electrons.com>: > > On Fri, 12 May 2017 05:16:08 -0300 > > Mario Rugiero <mrugiero@gmail.com> wrote: > > > >> 2017-05-12 5:12 GMT-03:00 Richard Weinberger <richard.weinberger@gmail.com>: > >> > Mario, > >> > > >> > On Fri, May 12, 2017 at 7:39 AM, Mario J. Rugiero <mrugiero@gmail.com> wrote: > >> >> Some chips used under a custom vendor driver can get their blocks > >> >> incorrectly detected as bad blocks, out of incompatibilities > >> >> between such drivers and MTD drivers. > >> >> When there are too many misdetected bad blocks, the device becomes > >> >> unusable because a bad block table can't be allocated, aside from > >> >> all the legitimately good blocks which become unusable under these > >> >> conditions. > >> >> This adds a build option to workaround the issue by enabling the > >> >> user to free up space regardless of what the driver thinks about > >> >> the blocks. > >> > > >> > Hmm, this sounds like a gross hack. > >> It is, but I see no other solution. The NAND chips were used in an > >> incompatible way by a hack-n-slash driver made by allwinner, and > >> trying to load them with a proper MTD driver fails miserably if this > >> is not done. > >> If anyone can propose a better solution I'll more than happily implement it. > >> I'm open to suggestions, and of course I'm open to rejection of my > >> patches if needed. > > > > u-boot provides the nand.scrub command, which does exactly what you're > > looking for. And no, I don't think it's a good idea to allow erasing > > bad blocks, at least not by default. > > > > If we really want to support this feature in linux, this should be > > explicitly enabled through debugfs. > If I do this, does it stand a chance at getting upstream? > If so, I'll have it done soon. > Note however that the build option is disabled by default. I get that > there should also be one runtime option, disabled by default, exposed > through debugfs. Does that sound right? > > > >> > > >> >> Example usage: recovering NAND chips on sunxi devices, as explained > >> >> here: http://linux-sunxi.org/Mainline_NAND_Howto#Known_issues > >> > > >> > What this wiki suggests is not wise. > >> > How can you know which blocks are really bad and which not? > >> You don't, at least not without an even grosser hack implementing read > >> support for their incompatible format. > >> Would that be better? I might attempt it if desired. > > > > No, please don't do that, at least not in the kernel. If you really > > want to parse the old format, you should develop a tool that reads NAND > > pages in raw mode, stores the list of bad blocks somewhere and then > > re-use this list to select which blocks should be forcibly erased. > > > > Not sure it's worth the pain :-). > It's worth the pain to me. I'm dealing with a bit rotten 3.4 based > pile of cr*p on production because of this. Whatever I have to do to > get those machines running the mainline kernel is worth it. No, I meant, doing that vs scrubbing the NAND. Note that MLC support is not reliable in mainline, so I'd strongly discourage to use a mainline kernel right now, unless you have an SLC NAND.
On Fri, 12 May 2017 05:56:40 -0300 Mario Rugiero <mrugiero@gmail.com> wrote: > El may. 12, 2017 5:46, "Boris Brezillon" <boris.brezillon@free-electrons.com> > escribió: > > On Fri, 12 May 2017 05:34:10 -0300 > Mario Rugiero <mrugiero@gmail.com> wrote: > > > 2017-05-12 5:24 GMT-03:00 Boris Brezillon <boris.brezillon@free- > electrons.com>: > > > On Fri, 12 May 2017 05:16:08 -0300 > > > Mario Rugiero <mrugiero@gmail.com> wrote: > > > > > >> 2017-05-12 5:12 GMT-03:00 Richard Weinberger < > richard.weinberger@gmail.com>: > > >> > Mario, > > >> > > > >> > On Fri, May 12, 2017 at 7:39 AM, Mario J. Rugiero <mrugiero@gmail.com> > wrote: > > >> >> Some chips used under a custom vendor driver can get their blocks > > >> >> incorrectly detected as bad blocks, out of incompatibilities > > >> >> between such drivers and MTD drivers. > > >> >> When there are too many misdetected bad blocks, the device becomes > > >> >> unusable because a bad block table can't be allocated, aside from > > >> >> all the legitimately good blocks which become unusable under these > > >> >> conditions. > > >> >> This adds a build option to workaround the issue by enabling the > > >> >> user to free up space regardless of what the driver thinks about > > >> >> the blocks. > > >> > > > >> > Hmm, this sounds like a gross hack. > > >> It is, but I see no other solution. The NAND chips were used in an > > >> incompatible way by a hack-n-slash driver made by allwinner, and > > >> trying to load them with a proper MTD driver fails miserably if this > > >> is not done. > > >> If anyone can propose a better solution I'll more than happily > implement it. > > >> I'm open to suggestions, and of course I'm open to rejection of my > > >> patches if needed. > > > > > > u-boot provides the nand.scrub command, which does exactly what you're > > > looking for. And no, I don't think it's a good idea to allow erasing > > > bad blocks, at least not by default. > > > > > > If we really want to support this feature in linux, this should be > > > explicitly enabled through debugfs. > > If I do this, does it stand a chance at getting upstream? > > If so, I'll have it done soon. > > Note however that the build option is disabled by default. I get that > > there should also be one runtime option, disabled by default, exposed > > through debugfs. Does that sound right? > > > > > >> > > > >> >> Example usage: recovering NAND chips on sunxi devices, as explained > > >> >> here: http://linux-sunxi.org/Mainline_NAND_Howto#Known_issues > > >> > > > >> > What this wiki suggests is not wise. > > >> > How can you know which blocks are really bad and which not? > > >> You don't, at least not without an even grosser hack implementing read > > >> support for their incompatible format. > > >> Would that be better? I might attempt it if desired. > > > > > > No, please don't do that, at least not in the kernel. If you really > > > want to parse the old format, you should develop a tool that reads NAND > > > pages in raw mode, stores the list of bad blocks somewhere and then > > > re-use this list to select which blocks should be forcibly erased. > > > > > > Not sure it's worth the pain :-). > > It's worth the pain to me. I'm dealing with a bit rotten 3.4 based > > pile of cr*p on production because of this. Whatever I have to do to > > get those machines running the mainline kernel is worth it. > > No, I meant, doing that vs scrubbing the NAND. Note that MLC support is > not reliable in mainline, so I'd strongly discourage to use a mainline > kernel right now, unless you have an SLC NAND. > > I know. Sunxi's driver doesn't seem stable either, though, and I've read > using an MLC chip as SLC by half The storage capacity was a viable > solution. Well, yes, but it's not supported either (at least not in mainline). > If it isn't implemented right now, I might implement that > solution in The meantime to a proper fix. Sadly, I'm not skilled enough for > that final solution. I have a branch containing the work we did we Richard to reliably support MLC NANDs. It's still WIP, but should give a rough idea of the solution we're heading to [1]. [1]https://github.com/bbrezillon/linux-sunxi/commits/bb/4.7/ubi-mlc
2017-05-12 6:02 GMT-03:00 Boris Brezillon <boris.brezillon@free-electrons.com>: > On Fri, 12 May 2017 05:56:40 -0300 > Mario Rugiero <mrugiero@gmail.com> wrote: > >> El may. 12, 2017 5:46, "Boris Brezillon" <boris.brezillon@free-electrons.com> >> escribió: >> >> On Fri, 12 May 2017 05:34:10 -0300 >> Mario Rugiero <mrugiero@gmail.com> wrote: >> >> > 2017-05-12 5:24 GMT-03:00 Boris Brezillon <boris.brezillon@free- >> electrons.com>: >> > > On Fri, 12 May 2017 05:16:08 -0300 >> > > Mario Rugiero <mrugiero@gmail.com> wrote: >> > > >> > >> 2017-05-12 5:12 GMT-03:00 Richard Weinberger < >> richard.weinberger@gmail.com>: >> > >> > Mario, >> > >> > >> > >> > On Fri, May 12, 2017 at 7:39 AM, Mario J. Rugiero <mrugiero@gmail.com> >> wrote: >> > >> >> Some chips used under a custom vendor driver can get their blocks >> > >> >> incorrectly detected as bad blocks, out of incompatibilities >> > >> >> between such drivers and MTD drivers. >> > >> >> When there are too many misdetected bad blocks, the device becomes >> > >> >> unusable because a bad block table can't be allocated, aside from >> > >> >> all the legitimately good blocks which become unusable under these >> > >> >> conditions. >> > >> >> This adds a build option to workaround the issue by enabling the >> > >> >> user to free up space regardless of what the driver thinks about >> > >> >> the blocks. >> > >> > >> > >> > Hmm, this sounds like a gross hack. >> > >> It is, but I see no other solution. The NAND chips were used in an >> > >> incompatible way by a hack-n-slash driver made by allwinner, and >> > >> trying to load them with a proper MTD driver fails miserably if this >> > >> is not done. >> > >> If anyone can propose a better solution I'll more than happily >> implement it. >> > >> I'm open to suggestions, and of course I'm open to rejection of my >> > >> patches if needed. >> > > >> > > u-boot provides the nand.scrub command, which does exactly what you're >> > > looking for. And no, I don't think it's a good idea to allow erasing >> > > bad blocks, at least not by default. >> > > >> > > If we really want to support this feature in linux, this should be >> > > explicitly enabled through debugfs. >> > If I do this, does it stand a chance at getting upstream? >> > If so, I'll have it done soon. >> > Note however that the build option is disabled by default. I get that >> > there should also be one runtime option, disabled by default, exposed >> > through debugfs. Does that sound right? >> > > >> > >> > >> > >> >> Example usage: recovering NAND chips on sunxi devices, as explained >> > >> >> here: http://linux-sunxi.org/Mainline_NAND_Howto#Known_issues >> > >> > >> > >> > What this wiki suggests is not wise. >> > >> > How can you know which blocks are really bad and which not? >> > >> You don't, at least not without an even grosser hack implementing read >> > >> support for their incompatible format. >> > >> Would that be better? I might attempt it if desired. >> > > >> > > No, please don't do that, at least not in the kernel. If you really >> > > want to parse the old format, you should develop a tool that reads NAND >> > > pages in raw mode, stores the list of bad blocks somewhere and then >> > > re-use this list to select which blocks should be forcibly erased. >> > > >> > > Not sure it's worth the pain :-). >> > It's worth the pain to me. I'm dealing with a bit rotten 3.4 based >> > pile of cr*p on production because of this. Whatever I have to do to >> > get those machines running the mainline kernel is worth it. >> >> No, I meant, doing that vs scrubbing the NAND. Note that MLC support is >> not reliable in mainline, so I'd strongly discourage to use a mainline >> kernel right now, unless you have an SLC NAND. >> >> I know. Sunxi's driver doesn't seem stable either, though, and I've read >> using an MLC chip as SLC by half The storage capacity was a viable >> solution. > > Well, yes, but it's not supported either (at least not in mainline). > >> If it isn't implemented right now, I might implement that >> solution in The meantime to a proper fix. Sadly, I'm not skilled enough for >> that final solution. > > I have a branch containing the work we did we Richard to reliably > support MLC NANDs. It's still WIP, but should give a rough idea of the > solution we're heading to [1]. > > [1]https://github.com/bbrezillon/linux-sunxi/commits/bb/4.7/ubi-mlc I'll read it carefully later. Is there any rough time estimate for it to hit mainline? I'm not expecting a date, but rather something in the lines of "several weeks, several months, several years". I think we can do with several months, and we'd be happy to start local experiments with that timeframe in mind. Several years might be more than the devices will live, though.
Also, if you have any pointers to where to study the issue, I might convince my boss to let me allocate time to lend a hand, if you need workers. 2017-05-12 6:15 GMT-03:00 Mario Rugiero <mrugiero@gmail.com>: > 2017-05-12 6:02 GMT-03:00 Boris Brezillon <boris.brezillon@free-electrons.com>: >> On Fri, 12 May 2017 05:56:40 -0300 >> Mario Rugiero <mrugiero@gmail.com> wrote: >> >>> El may. 12, 2017 5:46, "Boris Brezillon" <boris.brezillon@free-electrons.com> >>> escribió: >>> >>> On Fri, 12 May 2017 05:34:10 -0300 >>> Mario Rugiero <mrugiero@gmail.com> wrote: >>> >>> > 2017-05-12 5:24 GMT-03:00 Boris Brezillon <boris.brezillon@free- >>> electrons.com>: >>> > > On Fri, 12 May 2017 05:16:08 -0300 >>> > > Mario Rugiero <mrugiero@gmail.com> wrote: >>> > > >>> > >> 2017-05-12 5:12 GMT-03:00 Richard Weinberger < >>> richard.weinberger@gmail.com>: >>> > >> > Mario, >>> > >> > >>> > >> > On Fri, May 12, 2017 at 7:39 AM, Mario J. Rugiero <mrugiero@gmail.com> >>> wrote: >>> > >> >> Some chips used under a custom vendor driver can get their blocks >>> > >> >> incorrectly detected as bad blocks, out of incompatibilities >>> > >> >> between such drivers and MTD drivers. >>> > >> >> When there are too many misdetected bad blocks, the device becomes >>> > >> >> unusable because a bad block table can't be allocated, aside from >>> > >> >> all the legitimately good blocks which become unusable under these >>> > >> >> conditions. >>> > >> >> This adds a build option to workaround the issue by enabling the >>> > >> >> user to free up space regardless of what the driver thinks about >>> > >> >> the blocks. >>> > >> > >>> > >> > Hmm, this sounds like a gross hack. >>> > >> It is, but I see no other solution. The NAND chips were used in an >>> > >> incompatible way by a hack-n-slash driver made by allwinner, and >>> > >> trying to load them with a proper MTD driver fails miserably if this >>> > >> is not done. >>> > >> If anyone can propose a better solution I'll more than happily >>> implement it. >>> > >> I'm open to suggestions, and of course I'm open to rejection of my >>> > >> patches if needed. >>> > > >>> > > u-boot provides the nand.scrub command, which does exactly what you're >>> > > looking for. And no, I don't think it's a good idea to allow erasing >>> > > bad blocks, at least not by default. >>> > > >>> > > If we really want to support this feature in linux, this should be >>> > > explicitly enabled through debugfs. >>> > If I do this, does it stand a chance at getting upstream? >>> > If so, I'll have it done soon. >>> > Note however that the build option is disabled by default. I get that >>> > there should also be one runtime option, disabled by default, exposed >>> > through debugfs. Does that sound right? >>> > > >>> > >> > >>> > >> >> Example usage: recovering NAND chips on sunxi devices, as explained >>> > >> >> here: http://linux-sunxi.org/Mainline_NAND_Howto#Known_issues >>> > >> > >>> > >> > What this wiki suggests is not wise. >>> > >> > How can you know which blocks are really bad and which not? >>> > >> You don't, at least not without an even grosser hack implementing read >>> > >> support for their incompatible format. >>> > >> Would that be better? I might attempt it if desired. >>> > > >>> > > No, please don't do that, at least not in the kernel. If you really >>> > > want to parse the old format, you should develop a tool that reads NAND >>> > > pages in raw mode, stores the list of bad blocks somewhere and then >>> > > re-use this list to select which blocks should be forcibly erased. >>> > > >>> > > Not sure it's worth the pain :-). >>> > It's worth the pain to me. I'm dealing with a bit rotten 3.4 based >>> > pile of cr*p on production because of this. Whatever I have to do to >>> > get those machines running the mainline kernel is worth it. >>> >>> No, I meant, doing that vs scrubbing the NAND. Note that MLC support is >>> not reliable in mainline, so I'd strongly discourage to use a mainline >>> kernel right now, unless you have an SLC NAND. >>> >>> I know. Sunxi's driver doesn't seem stable either, though, and I've read >>> using an MLC chip as SLC by half The storage capacity was a viable >>> solution. >> >> Well, yes, but it's not supported either (at least not in mainline). >> >>> If it isn't implemented right now, I might implement that >>> solution in The meantime to a proper fix. Sadly, I'm not skilled enough for >>> that final solution. >> >> I have a branch containing the work we did we Richard to reliably >> support MLC NANDs. It's still WIP, but should give a rough idea of the >> solution we're heading to [1]. >> >> [1]https://github.com/bbrezillon/linux-sunxi/commits/bb/4.7/ubi-mlc > I'll read it carefully later. Is there any rough time estimate for it > to hit mainline? > I'm not expecting a date, but rather something in the lines of > "several weeks, several months, several years". > I think we can do with several months, and we'd be happy to start > local experiments with that timeframe in mind. > Several years might be more than the devices will live, though.
Am 12.05.2017 um 11:15 schrieb Mario Rugiero: > I'll read it carefully later. Is there any rough time estimate for it > to hit mainline? > I'm not expecting a date, but rather something in the lines of > "several weeks, several months, several years". > I think we can do with several months, and we'd be happy to start > local experiments with that timeframe in mind. > Several years might be more than the devices will live, though. Since there is currently no funding it might take much longer. Thanks, //richard
2017-05-12 6:19 GMT-03:00 Richard Weinberger <richard@nod.at>: > Am 12.05.2017 um 11:15 schrieb Mario Rugiero: >> I'll read it carefully later. Is there any rough time estimate for it >> to hit mainline? >> I'm not expecting a date, but rather something in the lines of >> "several weeks, several months, several years". >> I think we can do with several months, and we'd be happy to start >> local experiments with that timeframe in mind. >> Several years might be more than the devices will live, though. > > Since there is currently no funding it might take much longer. GSoC sounds like a viable workaround. It'd hahve to wait for next year, though. I think I'm elligible, so if I still have access to these boards and Boris can mentor me, we could get funding to speed up the development. I'd be happy to donate it to have another developer, as I'm already kinda being paid for this. What do you think of that? > > Thanks, > //richard
On Fri, 12 May 2017 06:16:33 -0300 Mario Rugiero <mrugiero@gmail.com> wrote: > Also, if you have any pointers to where to study the issue, I might > convince my boss to let me allocate time to lend a hand, if you need > workers. Here are the slides [1] the talk we gave at ELCE last year. Should be a good starting point to understand the problem and how we'd like to solve them. [1]http://events.linuxfoundation.org/sites/events/files/slides/ubi-mlc.pdf
On Fri, 12 May 2017 06:26:14 -0300 Mario Rugiero <mrugiero@gmail.com> wrote: > 2017-05-12 6:19 GMT-03:00 Richard Weinberger <richard@nod.at>: > > Am 12.05.2017 um 11:15 schrieb Mario Rugiero: > >> I'll read it carefully later. Is there any rough time estimate for it > >> to hit mainline? > >> I'm not expecting a date, but rather something in the lines of > >> "several weeks, several months, several years". > >> I think we can do with several months, and we'd be happy to start > >> local experiments with that timeframe in mind. > >> Several years might be more than the devices will live, though. > > > > Since there is currently no funding it might take much longer. > GSoC sounds like a viable workaround. > It'd hahve to wait for next year, though. > I think I'm elligible, so if I still have access to these boards and > Boris can mentor me, we could get funding to speed up the development. > I'd be happy to donate it to have another developer, as I'm already > kinda being paid for this. > What do you think of that? Yep, that's a possibility.
2017-05-12 6:34 GMT-03:00 Boris Brezillon <boris.brezillon@free-electrons.com>: > On Fri, 12 May 2017 06:26:14 -0300 > Mario Rugiero <mrugiero@gmail.com> wrote: > >> 2017-05-12 6:19 GMT-03:00 Richard Weinberger <richard@nod.at>: >> > Am 12.05.2017 um 11:15 schrieb Mario Rugiero: >> >> I'll read it carefully later. Is there any rough time estimate for it >> >> to hit mainline? >> >> I'm not expecting a date, but rather something in the lines of >> >> "several weeks, several months, several years". >> >> I think we can do with several months, and we'd be happy to start >> >> local experiments with that timeframe in mind. >> >> Several years might be more than the devices will live, though. >> > >> > Since there is currently no funding it might take much longer. >> GSoC sounds like a viable workaround. >> It'd hahve to wait for next year, though. >> I think I'm elligible, so if I still have access to these boards and >> Boris can mentor me, we could get funding to speed up the development. >> I'd be happy to donate it to have another developer, as I'm already >> kinda being paid for this. >> What do you think of that? > > Yep, that's a possibility. Great. Could you act as a mentor without GSoC for now? I think I might be able to help after finishing with the slides, they seem clear enough, with proper guidance, and that'd help in my current work. I think I've read some slides from a previous talk by you, before the way to go was as clearly defined. Regards, Mario.
On Fri, 12 May 2017 07:06:45 -0300 Mario Rugiero <mrugiero@gmail.com> wrote: > 2017-05-12 6:34 GMT-03:00 Boris Brezillon <boris.brezillon@free-electrons.com>: > > On Fri, 12 May 2017 06:26:14 -0300 > > Mario Rugiero <mrugiero@gmail.com> wrote: > > > >> 2017-05-12 6:19 GMT-03:00 Richard Weinberger <richard@nod.at>: > >> > Am 12.05.2017 um 11:15 schrieb Mario Rugiero: > >> >> I'll read it carefully later. Is there any rough time estimate for it > >> >> to hit mainline? > >> >> I'm not expecting a date, but rather something in the lines of > >> >> "several weeks, several months, several years". > >> >> I think we can do with several months, and we'd be happy to start > >> >> local experiments with that timeframe in mind. > >> >> Several years might be more than the devices will live, though. > >> > > >> > Since there is currently no funding it might take much longer. > >> GSoC sounds like a viable workaround. > >> It'd hahve to wait for next year, though. > >> I think I'm elligible, so if I still have access to these boards and > >> Boris can mentor me, we could get funding to speed up the development. > >> I'd be happy to donate it to have another developer, as I'm already > >> kinda being paid for this. > >> What do you think of that? > > > > Yep, that's a possibility. > Great. Could you act as a mentor without GSoC for now? Sure. > I think I might be able to help after finishing with the slides, they > seem clear enough, with proper guidance, and that'd help in my current > work. > I think I've read some slides from a previous talk by you, before the > way to go was as clearly defined. Actually, most of the development is done already, but we need to intensively test the implementation and cleanup the code. I also started to modify mtd-utils [1] to support the new format, but didn't have time to finish. [1]https://github.com/bbrezillon/mtd-utils/tree/ubi/mlc
2017-05-12 7:19 GMT-03:00 Boris Brezillon <boris.brezillon@free-electrons.com>: > On Fri, 12 May 2017 07:06:45 -0300 > Mario Rugiero <mrugiero@gmail.com> wrote: > >> 2017-05-12 6:34 GMT-03:00 Boris Brezillon <boris.brezillon@free-electrons.com>: >> > On Fri, 12 May 2017 06:26:14 -0300 >> > Mario Rugiero <mrugiero@gmail.com> wrote: >> > >> >> 2017-05-12 6:19 GMT-03:00 Richard Weinberger <richard@nod.at>: >> >> > Am 12.05.2017 um 11:15 schrieb Mario Rugiero: >> >> >> I'll read it carefully later. Is there any rough time estimate for it >> >> >> to hit mainline? >> >> >> I'm not expecting a date, but rather something in the lines of >> >> >> "several weeks, several months, several years". >> >> >> I think we can do with several months, and we'd be happy to start >> >> >> local experiments with that timeframe in mind. >> >> >> Several years might be more than the devices will live, though. >> >> > >> >> > Since there is currently no funding it might take much longer. >> >> GSoC sounds like a viable workaround. >> >> It'd hahve to wait for next year, though. >> >> I think I'm elligible, so if I still have access to these boards and >> >> Boris can mentor me, we could get funding to speed up the development. >> >> I'd be happy to donate it to have another developer, as I'm already >> >> kinda being paid for this. >> >> What do you think of that? >> > >> > Yep, that's a possibility. >> Great. Could you act as a mentor without GSoC for now? > > Sure. > >> I think I might be able to help after finishing with the slides, they >> seem clear enough, with proper guidance, and that'd help in my current >> work. >> I think I've read some slides from a previous talk by you, before the >> way to go was as clearly defined. > > Actually, most of the development is done already, but we need to > intensively test the implementation and cleanup the code. If you can describe the tests to me, I think I can have them running in at least three devices during next week. I can help with the cleanup, too. > > I also started to modify mtd-utils [1] to support the new format, but > didn't have time to finish. > > [1]https://github.com/bbrezillon/mtd-utils/tree/ubi/mlc I'll try to help with that.
2017-05-12 7:23 GMT-03:00 Mario Rugiero <mrugiero@gmail.com>: > 2017-05-12 7:19 GMT-03:00 Boris Brezillon <boris.brezillon@free-electrons.com>: >> On Fri, 12 May 2017 07:06:45 -0300 >> Mario Rugiero <mrugiero@gmail.com> wrote: >> >>> 2017-05-12 6:34 GMT-03:00 Boris Brezillon <boris.brezillon@free-electrons.com>: >>> > On Fri, 12 May 2017 06:26:14 -0300 >>> > Mario Rugiero <mrugiero@gmail.com> wrote: >>> > >>> >> 2017-05-12 6:19 GMT-03:00 Richard Weinberger <richard@nod.at>: >>> >> > Am 12.05.2017 um 11:15 schrieb Mario Rugiero: >>> >> >> I'll read it carefully later. Is there any rough time estimate for it >>> >> >> to hit mainline? >>> >> >> I'm not expecting a date, but rather something in the lines of >>> >> >> "several weeks, several months, several years". >>> >> >> I think we can do with several months, and we'd be happy to start >>> >> >> local experiments with that timeframe in mind. >>> >> >> Several years might be more than the devices will live, though. >>> >> > >>> >> > Since there is currently no funding it might take much longer. >>> >> GSoC sounds like a viable workaround. >>> >> It'd hahve to wait for next year, though. >>> >> I think I'm elligible, so if I still have access to these boards and >>> >> Boris can mentor me, we could get funding to speed up the development. >>> >> I'd be happy to donate it to have another developer, as I'm already >>> >> kinda being paid for this. >>> >> What do you think of that? >>> > >>> > Yep, that's a possibility. >>> Great. Could you act as a mentor without GSoC for now? >> >> Sure. >> >>> I think I might be able to help after finishing with the slides, they >>> seem clear enough, with proper guidance, and that'd help in my current >>> work. >>> I think I've read some slides from a previous talk by you, before the >>> way to go was as clearly defined. >> >> Actually, most of the development is done already, but we need to >> intensively test the implementation and cleanup the code. > If you can describe the tests to me, I think I can have them running > in at least three devices during next week. > I can help with the cleanup, too. >> >> I also started to modify mtd-utils [1] to support the new format, but >> didn't have time to finish. >> >> [1]https://github.com/bbrezillon/mtd-utils/tree/ubi/mlc > I'll try to help with that. Back to the original subject, scrubbing from Linux on demand. Is it a total no-go, or if I add the debugfs entry to make it opt-in on runtime too it might get accepted? Is it viable to add it as a new operation, so it can be more explicit about the risks? Where would be the proper place to put the debugfs entry? nand_base.c doesn't seem to deal with debugfs, and exporting the variable seems too risky IMO.
Mario, Am 12.05.2017 um 12:23 schrieb Mario Rugiero: >> Actually, most of the development is done already, but we need to >> intensively test the implementation and cleanup the code. > If you can describe the tests to me, I think I can have them running > in at least three devices during next week. > I can help with the cleanup, too. Well, the critical and most important task is understanding the existing issues, see slides, and find a bulletproof solution that scales. If you have questions, don't hesitate to ask. You can also find me and Boris on #mtd. Thanks, //richard
Sorry for the noise, but I'm resending since my phone is fixated on sending HTML mails, so it was rejected from the list: After some thought, I believe the proper solution is the one proposed by Boris: write a little program which classifies the false positives for bad blocks, and erase those blocks forcibly. What I'm currently wondering is if the incompatibilities come with false negatives too. In such a case, we should be able to provide the MTD driver with the information we just gathered, or is it viable to just program the BBT we just built? Regards, Mario. PD (not in the original mail): would mtd-tools the right place to do this, or rather sunxi-tools? I think converting from vendor-specific layouts to MTD could be a more common problem, so I thought of trying and making it generic enough to support several backends. 2017-05-13 6:17 GMT-03:00 Richard Weinberger <richard@nod.at>: > Mario, > > Am 12.05.2017 um 12:23 schrieb Mario Rugiero: >>> Actually, most of the development is done already, but we need to >>> intensively test the implementation and cleanup the code. >> If you can describe the tests to me, I think I can have them running >> in at least three devices during next week. >> I can help with the cleanup, too. > > Well, the critical and most important task is understanding the existing > issues, see slides, and find a bulletproof solution that scales. > > If you have questions, don't hesitate to ask. You can also find > me and Boris on #mtd. > > Thanks, > //richard
diff --git a/drivers/mtd/nand/Kconfig b/drivers/mtd/nand/Kconfig index c3029528063b..e5b29fab4a60 100644 --- a/drivers/mtd/nand/Kconfig +++ b/drivers/mtd/nand/Kconfig @@ -36,6 +36,15 @@ config MTD_NAND_ECC_BCH ECC codes. They are used with NAND devices requiring more than 1 bit of error correction. +config MTD_NAND_ERASE_BADBLOCKS + bool "Enable erasing of bad blocks (DANGEROUS)]" + default n + help + This enables support for attempting to erase bad blocks. + It is needed to workaround too many badblocks issue on chips used + under custom, incompatible vendor drivers. + Say N if unsure. + config MTD_SM_COMMON tristate default n diff --git a/drivers/mtd/nand/nand_base.c b/drivers/mtd/nand/nand_base.c index d474378ed810..0216dfc67976 100644 --- a/drivers/mtd/nand/nand_base.c +++ b/drivers/mtd/nand/nand_base.c @@ -3206,6 +3206,7 @@ int nand_erase_nand(struct mtd_info *mtd, struct erase_info *instr, instr->state = MTD_ERASING; while (len) { + #ifndef CONFIG_MTD_NAND_ERASE_BADBLOCKS /* Check if we have a bad block, we do not erase bad blocks! */ if (nand_block_checkbad(mtd, ((loff_t) page) << chip->page_shift, allowbbt)) { @@ -3214,6 +3215,7 @@ int nand_erase_nand(struct mtd_info *mtd, struct erase_info *instr, instr->state = MTD_ERASE_FAILED; goto erase_exit; } + #endif /* * Invalidate the page cache, if we erase the block which
Some chips used under a custom vendor driver can get their blocks incorrectly detected as bad blocks, out of incompatibilities between such drivers and MTD drivers. When there are too many misdetected bad blocks, the device becomes unusable because a bad block table can't be allocated, aside from all the legitimately good blocks which become unusable under these conditions. This adds a build option to workaround the issue by enabling the user to free up space regardless of what the driver thinks about the blocks. Example usage: recovering NAND chips on sunxi devices, as explained here: http://linux-sunxi.org/Mainline_NAND_Howto#Known_issues Signed-off-by: Mario J. Rugiero <mrugiero@gmail.com> --- drivers/mtd/nand/Kconfig | 9 +++++++++ drivers/mtd/nand/nand_base.c | 2 ++ 2 files changed, 11 insertions(+)