| Message ID | 20241008133342.1937674-1-b.spranger@linutronix.de |
|---|---|
| State | New |
| Headers | show |
| Series | [1/1] ubifs: Try to recover from missing znode | expand |
在 2024/10/8 21:33, Benedikt Spranger 写道: > After powercut on a system using ubifs mounting failed: > > 2024-09-30T12:38:26.880487+02:00 sonja kernel: UBIFS error (ubi0:0 pid 2178): ubifs_read_node [ubifs]: bad node type (255 but expected 9) > 2024-09-30T12:38:26.880506+02:00 sonja kernel: UBIFS error (ubi0:0 pid 2178): ubifs_read_node [ubifs]: bad node at LEB 103:46920, LEB mapping status 0 > 2024-09-30T12:38:26.880509+02:00 sonja kernel: Not a node, first 24 bytes: > 2024-09-30T12:38:26.880510+02:00 sonja kernel: 00000000: ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ........................ > > While traversing over zbranches during the journal replay one zbranch > points to a znode, which was not written to the flash and therefore the > flash is empty. UBIFS guarantees two things for znodes: 1) all index nodes(in commit seq N) are written on flash before master nodes(for commit seq N) are written. 2) all index nodes(in commit seq N) won't be erased from flash before master nodes(for commit seq N+1) are written. So, I don't understand that why znodes not exist during journal replaying? > > Try to recover from that by inserting an empty znode instead of failing. > > Signed-off-by: Benedikt Spranger <b.spranger@linutronix.de> > Reviewed-by: John Ogness <john.ogness@linutronix.de> > --- > fs/ubifs/io.c | 16 ++++++++++++++++ > fs/ubifs/tnc_misc.c | 6 +++++- > 2 files changed, 21 insertions(+), 1 deletion(-) > > diff --git a/fs/ubifs/io.c b/fs/ubifs/io.c > index 01d8eb170382..0bbb426f9006 100644 > --- a/fs/ubifs/io.c > +++ b/fs/ubifs/io.c > @@ -1110,6 +1110,22 @@ int ubifs_read_node(const struct ubifs_info *c, void *buf, int type, int len, > return err; > > if (type != ch->node_type) { > + /* > + * While recovering, we may face lost data i.e. empty flash. > + * Give callsites a hint by returning -ENODATA. > + */ > + if (c->replaying) { > + u8 *b = buf; > + > + for (l = 0; l < len; l++) { > + if (b[l] != 0xff) > + break; > + } > + if (l == len) { > + ubifs_errc(c, "no node, but empty flash"); > + return -ENODATA; > + } > + } > ubifs_errc(c, "bad node type (%d but expected %d)", > ch->node_type, type); > goto out; > diff --git a/fs/ubifs/tnc_misc.c b/fs/ubifs/tnc_misc.c > index d3f8a6aa1f49..4d085fc1300f 100644 > --- a/fs/ubifs/tnc_misc.c > +++ b/fs/ubifs/tnc_misc.c > @@ -300,7 +300,11 @@ static int read_znode(struct ubifs_info *c, struct ubifs_zbranch *zzbr, > err = ubifs_read_node(c, idx, UBIFS_IDX_NODE, len, lnum, offs); > if (err < 0) { > kfree(idx); > - return err; > + /* > + * While recovering we may face a non written znode. > + * Inject an empty znode in this case. > + */ > + return (err == -ENODATA) ? 0 : err; > } > > err = ubifs_node_check_hash(c, idx, zzbr->hash); >
----- Ursprüngliche Mail ----- > Von: "chengzhihao1" <chengzhihao1@huawei.com> > An: "Benedikt Spranger" <b.spranger@linutronix.de>, "linux-kernel" <linux-kernel@vger.kernel.org> > CC: "linux-mtd" <linux-mtd@lists.infradead.org>, "richard" <richard@nod.at> > Gesendet: Mittwoch, 9. Oktober 2024 04:23:02 > Betreff: Re: [PATCH 1/1] ubifs: Try to recover from missing znode > 在 2024/10/8 21:33, Benedikt Spranger 写道: >> After powercut on a system using ubifs mounting failed: >> >> 2024-09-30T12:38:26.880487+02:00 sonja kernel: UBIFS error (ubi0:0 pid 2178): >> ubifs_read_node [ubifs]: bad node type (255 but expected 9) >> 2024-09-30T12:38:26.880506+02:00 sonja kernel: UBIFS error (ubi0:0 pid 2178): >> ubifs_read_node [ubifs]: bad node at LEB 103:46920, LEB mapping status 0 >> 2024-09-30T12:38:26.880509+02:00 sonja kernel: Not a node, first 24 bytes: >> 2024-09-30T12:38:26.880510+02:00 sonja kernel: 00000000: ff ff ff ff ff ff ff ff >> ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff >> ........................ >> >> While traversing over zbranches during the journal replay one zbranch >> points to a znode, which was not written to the flash and therefore the >> flash is empty. > > UBIFS guarantees two things for znodes: > 1) all index nodes(in commit seq N) are written on flash before master > nodes(for commit seq N) are written. > 2) all index nodes(in commit seq N) won't be erased from flash before > master nodes(for commit seq N+1) are written. > So, I don't understand that why znodes not exist during journal replaying? In addition to that, is just one znode missing or are larger parts of the flash empty? Thanks, //richard
在 2024/10/9 14:03, Richard Weinberger 写道: > ----- Ursprüngliche Mail ----- >> Von: "chengzhihao1" <chengzhihao1@huawei.com> >> An: "Benedikt Spranger" <b.spranger@linutronix.de>, "linux-kernel" <linux-kernel@vger.kernel.org> >> CC: "linux-mtd" <linux-mtd@lists.infradead.org>, "richard" <richard@nod.at> >> Gesendet: Mittwoch, 9. Oktober 2024 04:23:02 >> Betreff: Re: [PATCH 1/1] ubifs: Try to recover from missing znode > >> 在 2024/10/8 21:33, Benedikt Spranger 写道: >>> After powercut on a system using ubifs mounting failed: >>> >>> 2024-09-30T12:38:26.880487+02:00 sonja kernel: UBIFS error (ubi0:0 pid 2178): >>> ubifs_read_node [ubifs]: bad node type (255 but expected 9) >>> 2024-09-30T12:38:26.880506+02:00 sonja kernel: UBIFS error (ubi0:0 pid 2178): >>> ubifs_read_node [ubifs]: bad node at LEB 103:46920, LEB mapping status 0 >>> 2024-09-30T12:38:26.880509+02:00 sonja kernel: Not a node, first 24 bytes: >>> 2024-09-30T12:38:26.880510+02:00 sonja kernel: 00000000: ff ff ff ff ff ff ff ff >>> ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff >>> ........................ >>> >>> While traversing over zbranches during the journal replay one zbranch >>> points to a znode, which was not written to the flash and therefore the >>> flash is empty. >> >> UBIFS guarantees two things for znodes: >> 1) all index nodes(in commit seq N) are written on flash before master >> nodes(for commit seq N) are written. >> 2) all index nodes(in commit seq N) won't be erased from flash before >> master nodes(for commit seq N+1) are written. >> So, I don't understand that why znodes not exist during journal replaying? > > In addition to that, is just one znode missing or are larger parts of the flash > empty? > After reviewing the TNC related code, I cannot find out any illegal behaviors to make znode lost or valid index LEB erased. Hi Benedikt, could you provide a corrupted UBIFS image? Let me try getting more information.
On Wed, 9 Oct 2024 18:46:43 +0800 Zhihao Cheng <chengzhihao1@huawei.com> wrote: > 在 2024/10/9 14:03, Richard Weinberger 写道: > > ----- Ursprüngliche Mail ----- > >> Von: "chengzhihao1" <chengzhihao1@huawei.com> > >> An: "Benedikt Spranger" <b.spranger@linutronix.de>, "linux-kernel" > >> <linux-kernel@vger.kernel.org> CC: "linux-mtd" > >> <linux-mtd@lists.infradead.org>, "richard" <richard@nod.at> > >> Gesendet: Mittwoch, 9. Oktober 2024 04:23:02 Betreff: Re: [PATCH > >> 1/1] ubifs: Try to recover from missing znode > > > >> 在 2024/10/8 21:33, Benedikt Spranger 写道: > >>> After powercut on a system using ubifs mounting failed: > >>> > >>> 2024-09-30T12:38:26.880487+02:00 sonja kernel: UBIFS error > >>> (ubi0:0 pid 2178): ubifs_read_node [ubifs]: bad node type (255 > >>> but expected 9) 2024-09-30T12:38:26.880506+02:00 sonja kernel: > >>> UBIFS error (ubi0:0 pid 2178): ubifs_read_node [ubifs]: bad node > >>> at LEB 103:46920, LEB mapping status 0 > >>> 2024-09-30T12:38:26.880509+02:00 sonja kernel: Not a node, first > >>> 24 bytes: 2024-09-30T12:38:26.880510+02:00 sonja kernel: > >>> 00000000: ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff > >>> ff ff ff ff ff ff ........................ > >>> > >>> While traversing over zbranches during the journal replay one > >>> zbranch points to a znode, which was not written to the flash and > >>> therefore the flash is empty. > >> > >> UBIFS guarantees two things for znodes: > >> 1) all index nodes(in commit seq N) are written on flash before > >> master nodes(for commit seq N) are written. > >> 2) all index nodes(in commit seq N) won't be erased from flash > >> before master nodes(for commit seq N+1) are written. > >> So, I don't understand that why znodes not exist during journal > >> replaying? > > > > In addition to that, is just one znode missing or are larger parts > > of the flash empty? Some background: The system producing the broken image runs on an older kernel version (4.9.286-rt189). Attaching UBI performs without any error. It seems, that there is only one znode missing. There are empty parts on the flash, but as far as I can see all PEBs have UBI header Information. The missing znode is somewhere in th middle of an PEB and other znodes are in that PEB. > After reviewing the TNC related code, I cannot find out any illegal > behaviors to make znode lost or valid index LEB erased. OK. Is it possible, that this behaviour is caused by a already fixed bug? This happend on a system running 4.9.286-rt189. > Hi Benedikt, could you provide a corrupted UBIFS image? Let me try > getting more information. I need to clarify this. Regards Benedikt Spranger
在 2024/10/9 20:49, Benedikt Spranger 写道: > On Wed, 9 Oct 2024 18:46:43 +0800 > Zhihao Cheng <chengzhihao1@huawei.com> wrote: > >> 在 2024/10/9 14:03, Richard Weinberger 写道: >>> ----- Ursprüngliche Mail ----- >>>> Von: "chengzhihao1" <chengzhihao1@huawei.com> >>>> An: "Benedikt Spranger" <b.spranger@linutronix.de>, "linux-kernel" >>>> <linux-kernel@vger.kernel.org> CC: "linux-mtd" >>>> <linux-mtd@lists.infradead.org>, "richard" <richard@nod.at> >>>> Gesendet: Mittwoch, 9. Oktober 2024 04:23:02 Betreff: Re: [PATCH >>>> 1/1] ubifs: Try to recover from missing znode >>> >>>> 在 2024/10/8 21:33, Benedikt Spranger 写道: >>>>> After powercut on a system using ubifs mounting failed: >>>>> >>>>> 2024-09-30T12:38:26.880487+02:00 sonja kernel: UBIFS error >>>>> (ubi0:0 pid 2178): ubifs_read_node [ubifs]: bad node type (255 >>>>> but expected 9) 2024-09-30T12:38:26.880506+02:00 sonja kernel: >>>>> UBIFS error (ubi0:0 pid 2178): ubifs_read_node [ubifs]: bad node >>>>> at LEB 103:46920, LEB mapping status 0 >>>>> 2024-09-30T12:38:26.880509+02:00 sonja kernel: Not a node, first >>>>> 24 bytes: 2024-09-30T12:38:26.880510+02:00 sonja kernel: >>>>> 00000000: ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff >>>>> ff ff ff ff ff ff ........................ >>>>> >>>>> While traversing over zbranches during the journal replay one >>>>> zbranch points to a znode, which was not written to the flash and >>>>> therefore the flash is empty. >>>> >>>> UBIFS guarantees two things for znodes: >>>> 1) all index nodes(in commit seq N) are written on flash before >>>> master nodes(for commit seq N) are written. >>>> 2) all index nodes(in commit seq N) won't be erased from flash >>>> before master nodes(for commit seq N+1) are written. >>>> So, I don't understand that why znodes not exist during journal >>>> replaying? >>> >>> In addition to that, is just one znode missing or are larger parts >>> of the flash empty? > > Some background: > The system producing the broken image runs on an older kernel version > (4.9.286-rt189). Attaching UBI performs without any error. It seems, > that there is only one znode missing. There are empty parts on the > flash, but as far as I can see all PEBs have UBI header Information. > The missing znode is somewhere in th middle of an PEB and other znodes > are in that PEB. > >> After reviewing the TNC related code, I cannot find out any illegal >> behaviors to make znode lost or valid index LEB erased. > OK. Is it possible, that this behaviour is caused by a already fixed > bug? This happend on a system running 4.9.286-rt189. I found one patch [1], which fixes a data lost problem while writing failed. [1] https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?h=v6.12-rc2&id=31a149d5c13c4cbcf97de3435817263a2d8c9d6e > >> Hi Benedikt, could you provide a corrupted UBIFS image? Let me try >> getting more information. > I need to clarify this. > > Regards > Benedikt Spranger > . >
diff --git a/fs/ubifs/io.c b/fs/ubifs/io.c index 01d8eb170382..0bbb426f9006 100644 --- a/fs/ubifs/io.c +++ b/fs/ubifs/io.c @@ -1110,6 +1110,22 @@ int ubifs_read_node(const struct ubifs_info *c, void *buf, int type, int len, return err; if (type != ch->node_type) { + /* + * While recovering, we may face lost data i.e. empty flash. + * Give callsites a hint by returning -ENODATA. + */ + if (c->replaying) { + u8 *b = buf; + + for (l = 0; l < len; l++) { + if (b[l] != 0xff) + break; + } + if (l == len) { + ubifs_errc(c, "no node, but empty flash"); + return -ENODATA; + } + } ubifs_errc(c, "bad node type (%d but expected %d)", ch->node_type, type); goto out; diff --git a/fs/ubifs/tnc_misc.c b/fs/ubifs/tnc_misc.c index d3f8a6aa1f49..4d085fc1300f 100644 --- a/fs/ubifs/tnc_misc.c +++ b/fs/ubifs/tnc_misc.c @@ -300,7 +300,11 @@ static int read_znode(struct ubifs_info *c, struct ubifs_zbranch *zzbr, err = ubifs_read_node(c, idx, UBIFS_IDX_NODE, len, lnum, offs); if (err < 0) { kfree(idx); - return err; + /* + * While recovering we may face a non written znode. + * Inject an empty znode in this case. + */ + return (err == -ENODATA) ? 0 : err; } err = ubifs_node_check_hash(c, idx, zzbr->hash);