Issue 445: UBIFS is instable

Reported by Nikolaus Schaller, May 16, 2013

e.g.


[ 1259.263183] UBIFS error (pid 3561): ubifs_read_node: bad node 
type (224 but expected 9)
[ 1259.278778] UBIFS error (pid 3561): ubifs_read_node: bad node at 
LEB 2049:7600, LEB mapping status 1
[ 1259.302215] Not a node, first 24 bytes:
[ 1259.306030] 00000000: e0 ff e0 ff e0 ff f0 ff e0 ff e0 ff e0 ff 
e0 ff e0 ff e0 ff e0 ff e0 ff                          
........................
[ 1259.333557] [<c0011c50>] (unwind_backtrace+0x0/0xe0) from 
[<c016c010>] (ubifs_read_node+0x25c/0x27c)
[ 1259.356903] [<c016c010>] (ubifs_read_node+0x25c/0x27c) from 
[<c0186b60>] (read_znode+0x4c/0x45c)
[ 1259.372558] [<c0186b60>] (read_znode+0x4c/0x45c) from 
[<c01875f8>] (ubifs_load_znode+0x74/0xe8)
[ 1259.385864] UBI: scrubbed PEB 717 (LEB 0:2049), data moved to PEB 
2590
[ 1259.392883] [<c01875f8>] (ubifs_load_znode+0x74/0xe8) from 
[<c016f100>] (ubifs_lookup_level0+0x100/0x1c4)
[ 1259.404296] [<c016f100>] (ubifs_lookup_level0+0x100/0x1c4) 
from [<c016f1e8>] (is_leaf_node_in_tnc+0x24/0x170)
[ 1259.414764] [<c016f1e8>] (is_leaf_node_in_tnc+0x24/0x170) 
from [<c0170740>] (ubifs_tnc_has_node+0x80/0xa4)
[ 1259.424987] [<c0170740>] (ubifs_tnc_has_node+0x80/0xa4) 
from [<c0175ff4>] (sort_nodes+0xd0/0x1c0)
[ 1259.434356] [<c0175ff4>] (sort_nodes+0xd0/0x1c0) from 
[<c0176280>] (move_nodes+0x48/0x18c)
[ 1259.443176] [<c0176280>] (move_nodes+0x48/0x18c) from 
[<c01766c8>] (ubifs_garbage_collect_leb+0x304/0x3c0)
[ 1259.453460] [<c01766c8>] 
(ubifs_garbage_collect_leb+0x304/0x3c0) from [<c0176918>] 
(ubifs_garbage_collect+0x194/0x380)
[ 1259.464843] [<c0176918>] 
(ubifs_garbage_collect+0x194/0x380) from [<c015ea04>] 
(reserve_space+0xfc/0x27c)
[ 1259.475006] [<c015ea04>] (reserve_space+0xfc/0x27c) from 
[<c015ebc4>] (make_reservation+0x40/0x154)
[ 1259.484649] [<c015ebc4>] (make_reservation+0x40/0x154) from 
[<c015f9cc>] (ubifs_jnl_write_data+0x16c/0x22c)
[ 1259.494964] [<c015f9cc>] (ubifs_jnl_write_data+0x16c/0x22c) 
from [<c0162238>] (do_writepage+0x98/0x1c0)
[ 1259.505126] [<c0162238>] (do_writepage+0x98/0x1c0) from 
[<c0083fa8>] (__writepage+0x14/0x50)
[ 1259.514190] [<c0083fa8>] (__writepage+0x14/0x50) from 
[<c00846d0>] (write_cache_pages+0x238/0x338)
[ 1259.523681] [<c00846d0>] (write_cache_pages+0x238/0x338) 
from [<c0084810>] (generic_writepages+0x40/0x5c)
[ 1259.533782] [<c0084810>] (generic_writepages+0x40/0x5c) 
from [<c00cab84>] (__writeback_single_inode+0x34/0x118)
[ 1259.544433] [<c00cab84>] 
(__writeback_single_inode+0x34/0x118) from [<c00cb4fc>] 
(writeback_sb_inodes+0x21c/0x2fc)
[ 1259.555358] [<c00cb4fc>] (writeback_sb_inodes+0x21c/0x2fc) 
from [<c00cb7c8>] (wb_writeback+0x148/0x218)
[ 1259.565277] [<c00cb7c8>] (wb_writeback+0x148/0x218) from 
[<c00cbec8>] (wb_do_writeback+0x5c/0x140)
[ 1259.574737] [<c00cbec8>] (wb_do_writeback+0x5c/0x140) from 
[<c00cc008>] (bdi_writeback_thread+0x5c/0x138)
[ 1259.584869] [<c00cc008>] (bdi_writeback_thread+0x5c/0x138) 
from [<c0046144>] (kthread+0xa0/0xac)
[ 1259.594146] [<c0046144>] (kthread+0xa0/0xac) from 
[<c000d998>] (ret_from_fork+0x14/0x3c)
[ 1259.603576] UBIFS warning (pid 3561): ubifs_ro_mode: switched to 
read-only mode, error -22
[ 1259.612396] [<c0011c50>] (unwind_backtrace+0x0/0xe0) from 
[<c0176ab8>] (ubifs_garbage_collect+0x334/0x380)
[ 1259.622680] [<c0176ab8>] 
(ubifs_garbage_collect+0x334/0x380) from [<c015ea04>] 
(reserve_space+0xfc/0x27c)
[ 1259.632843] [<c015ea04>] (reserve_space+0xfc/0x27c) from 
[<c015ebc4>] (make_reservation+0x40/0x154)
[ 1259.642486] [<c015ebc4>] (make_reservation+0x40/0x154) from 
[<c015f9cc>] (ubifs_jnl_write_data+0x16c/0x22c)
[ 1259.652832] [<c015f9cc>] (ubifs_jnl_write_data+0x16c/0x22c) 
from [<c0162238>] (do_writepage+0x98/0x1c0)
[ 1259.662811] [<c0162238>] (do_writepage+0x98/0x1c0) from 
[<c0083fa8>] (__writepage+0x14/0x50)
[ 1259.671783] [<c0083fa8>] (__writepage+0x14/0x50) from 
[<c00846d0>] (write_cache_pages+0x238/0x338)
[ 1259.681304] [<c00846d0>] (write_cache_pages+0x238/0x338) 
from [<c0084810>] (generic_writepages+0x40/0x5c)
[ 1259.691467] [<c0084810>] (generic_writepages+0x40/0x5c) 
from [<c00cab84>] (__writeback_single_inode+0x34/0x118)
[ 1259.702178] [<c00cab84>] 
(__writeback_single_inode+0x34/0x118) from [<c00cb4fc>] 
(writeback_sb_inodes+0x21c/0x2fc)
[ 1259.713073] [<c00cb4fc>] (writeback_sb_inodes+0x21c/0x2fc) 
from [<c00cb7c8>] (wb_writeback+0x148/0x218)
[ 1259.722991] [<c00cb7c8>] (wb_writeback+0x148/0x218) from 
[<c00cbec8>] (wb_do_writeback+0x5c/0x140)
[ 1259.732482] [<c00cbec8>] (wb_do_writeback+0x5c/0x140) from 
[<c00cc008>] (bdi_writeback_thread+0x5c/0x138)
[ 1259.742584] [<c00cc008>] (bdi_writeback_thread+0x5c/0x138) 
from [<c0046144>] (kthread+0xa0/0xac)
[ 1259.751892] [<c0046144>] (kthread+0xa0/0xac) from 
[<c000d998>] (ret_from_fork+0x14/0x3c)
[ 1259.760467] UBIFS error (pid 3561): make_reservation: cannot 
reserve 3998 bytes in jhead 2, error -22
[ 1259.770202] UBIFS error (pid 3561): do_writepage: cannot write 
page 1586 of inode 56951, error -22
[ 1289.783630] UBIFS error (pid 3561): make_reservation: cannot 
reserve 3998 bytes in jhead 2, error -30
[ 1289.795623] UBIFS error (pid 3561): do_writepage: cannot write 
page 1587 of inode 56951, error -30
[ 1289.807250] UBIFS error (pid 3561): make_reservation: cannot 
reserve 160 bytes in jhead 1, error -30
[ 1289.819000] UBIFS error (pid 3561): ubifs_write_inode: can't 
write inode 56951, error -30
[ 1293.101898] UBI: fixable bit-flip detected at PEB 1199
[ 1293.111022] UBI: schedule PEB 1199 for scrubbing
[ 1293.134521] UBIFS error (pid 3560): ubifs_read_node: bad node 
type (224 but expected 2)
[ 1293.150146] UBIFS error (pid 3560): ubifs_read_node: bad node at 
LEB 730:24392, LEB mapping status 1
[ 1293.173583] Not a node, first 24 bytes:
[ 1293.177398] 00000000: e0 ff e0 ff e0 ff e0 ff e0 ff e0 df e0 ff 
e0 ff e0 ff e0 ff e0 ff e0 ff                          
........................
[ 1293.204864] [<c0011c50>] (unwind_backtrace+0x0/0xe0) from 
[<c016c010>] (ubifs_read_node+0x25c/0x27c)
[ 1293.228393] [<c016c010>] (ubifs_read_node+0x25c/0x27c) from 
[<c01876cc>] (ubifs_tnc_read_node+0x60/0x124)
[ 1293.251739] [<c01876cc>] (ubifs_tnc_read_node+0x60/0x124) 
from [<c016d8d8>] (tnc_read_node_nm+0xb4/0xd8)
[ 1293.263671] UBI: scrubbed PEB 1199 (LEB 0:730), data moved to PEB 
2290
[ 1293.271026] [<c016d8d8>] (tnc_read_node_nm+0xb4/0xd8) from 
[<c016f4c4>] (ubifs_tnc_locate+0x9c/0x180)
[ 1293.282348] [<c016f4c4>] (ubifs_tnc_locate+0x9c/0x180) from 
[<c016f9d4>] (ubifs_tnc_lookup_nm+0x20/0x7c)
[ 1293.292663] [<c016f9d4>] (ubifs_tnc_lookup_nm+0x20/0x7c) 
from [<c016453c>] (ubifs_lookup+0xac/0x1d0)
[ 1293.302673] [<c016453c>] (ubifs_lookup+0xac/0x1d0) from 
[<c00b2740>] (lookup_real+0x30/0x4c)
[ 1293.311859] [<c00b2740>] (lookup_real+0x30/0x4c) from 
[<c00b2f68>] (__lookup_hash+0x30/0x3c)
[ 1293.320983] [<c00b2f68>] (__lookup_hash+0x30/0x3c) from 
[<c00b2fb4>] (lookup_slow+0x40/0xa0)
[ 1293.330139] [<c00b2fb4>] (lookup_slow+0x40/0xa0) from 
[<c00b4d58>] (path_lookupat+0x104/0x75c)
[ 1293.339538] [<c00b4d58>] (path_lookupat+0x104/0x75c) from 
[<c00b53d0>] (filename_lookup+0x20/0x5c)
[ 1293.349365] [<c00b53d0>] (filename_lookup+0x20/0x5c) from 
[<c00b6ec8>] (user_path_at_empty+0x54/0x7c)
[ 1293.359527] [<c00b6ec8>] (user_path_at_empty+0x54/0x7c) 
from [<c00b6f00>] (user_path_at+0x10/0x14)
[ 1293.369445] [<c00b6f00>] (user_path_at+0x10/0x14) from 
[<c00ae500>] (vfs_fstatat+0x30/0x58)
[ 1293.378570] [<c00ae500>] (vfs_fstatat+0x30/0x58) from 
[<c00ae798>] (sys_fstatat64+0x14/0x30)
[ 1293.387725] [<c00ae798>] (sys_fstatat64+0x14/0x30) from 
[<c000d900>] (ret_fast_syscall+0x0/0x30)
[ 1294.822845] UBIFS error (pid 3561): make_reservation: cannot 
reserve 3864 bytes in jhead 2, error -30
[ 1294.854400] UBIFS error (pid 3561): do_writepage: cannot write 
page 1588 of inode 56951, error -30
[ 1295.281829] UBI: fixable bit-flip detected at PEB 1435
[ 1295.287414] UBI: schedule PEB 1435 for scrubbing
[ 1297.000030] UBIFS error (pid 3561): make_reservation: cannot 
reserve 3864 bytes in jhead 2, error -30
[ 1297.010314] UBIFS error (pid 3561): do_writepage: cannot write 
page 1589 of inode 56951, error -30

Comment 1 by Nikolaus Schaller, May 16, 2013

Other boards and systems are seeing similar issues:

http://lists.infradead.org/pipermail/linux-mtd/2012-December/045308.h
tml

Comment 2 by Nikolaus Schaller, Jun 17, 2013

On one device I tried:


umount ubi0:gta04-rootfs
ubidetach /dev/ubi_ctrl -m 4
flash_eraseall -j /dev/mtd4

and the errors apparently were gone (at least for the limited 
testing time). So it might be that the mtd driver has problems with 
an uninitialized NAND and injects these errors into the UBI layer.

Important note: using flash_eraseall resets the wear level counters!

Comment 3 by Nikolaus Schaller, Oct 1, 2013

Well, we can just observe this.

Status: WontFix

Letux Kernel