Hacker News Reader

Comment by cm3 | Hacker News Reader

I wish they would finally find a solution that actually fixes USB storage sync lockups. Whatever USB storage you use and whether USB2 or USB3, it's very easy to lock up a machine when data is flushed and is slow to do so. Given all the asynchronicity in the kernel, it's surprising there's such a Windows-like global wait-until-I'm-finished unresponsiveness bug.

View on HN

I've never experienced the entire machine locking up before (going back 10+ years), only IO becoming severely locked up when writeback can't keep up with new IO, and always happened on non-USB media too.

Xorg becomes totally unresponsive and I cannot switch to a VT, so I assume it's a global lock somewhere.

while sleep 10 do ; cat /proc/meminfo | grep Dirty ; done

Watch the dirty cache data when this is happening, if you want to know for sure. Put a couple "sync;sync;sync"'s &&'ed on that cat, if you want to watch stimulating behaviour..

(More often than not, its the device. Linux is dealing with it fine.)

As I wrote above, the device is taking long to finish the flush operation, and that's nothing special, but making the machine not react to input and actually block everything else like, say, the network stack and thereby interrupt downloads, is not what I perceive as fine.

Which scheduler are you using on the USB device? Did you try deadline or BFQ?

There is a long standing (CFQ-related) kernel issue causing system lockups in high I/O situations:

https://bugzilla.kernel.org/show_bug.cgi?id=12309

That second to last comment from Tashnikov is highly interesting, as i have noticed Firefox triggering a virtual lockup on Android when started.

Impossible to understand why BFQ isn't available as an option, when there are competing options for many things in the upstream kernel already.

The most likely reason is a clash of personalities.

Linus manages the kernel and should decide to include it as BFQ is just another option in a pluggable subsytem. BFQ is arguably more important than many things because it fixes bad performance and unresponsive behavior. I hope it's not that Jens Axboe has too much influence because he's the block layer maintainer.

http://algo.ing.unimo.it/people/paolo/disk_sched/

https://lwn.net/Articles/601799/

CFQ as it's the default since 2006. I'll try deadline on USB storage.

Deadline so far in limited testing doesn't seem to cause lockups as CFQ does and it seems to more realistically reflect the workload in the form of lower MB/s during such phases. Thanks for the tip.

np ;)

With blk-mq + nvme and later lightnvm that whole IO-scheduler mess will hopefully go away over the course of the next couple years (at least for personal computers).

I recall seeing something like that when input devices were living on the same hub as the storage device getting IO at the time. But that was years ago, and the USB stack has seen considerable work since.

Edit: never mind, Johnp_'s comment and bugzilla link is highly interesting.

Reply on news.ycombinator.com