**** BEGIN LOGGING AT Tue Jan 17 03:00:02 2017 Jan 17 03:39:50 ~obsolete Jan 17 03:40:05 ~battery Jan 17 03:40:06 extra, extra, read all about it, polarcell is http://www.ebay.co.uk/itm/390402445382 BL-5J Replacement Jan 17 03:40:46 ~deprecated Jan 17 03:40:47 i heard deprecated is a term in computer software that indicates a feature is no longer officially supported even if it still exists and may eventually be removed altogether. Deprecated is often misspelled 'depreciated' which is a financial term. Jan 17 03:43:56 ~broken package Jan 17 03:44:17 ~pkg Jan 17 03:44:18 somebody said pkg was http://maemo.org/packages/ Jan 17 04:08:17 Found it :) http://wiki.maemo.org/Community_SSU/known_broken_packages Jan 17 04:15:56 ~listvalues known_broken_packages Jan 17 04:15:59 Factoid search of 'known_broken_packages' by value returned no results. Jan 17 04:18:54 freemangordon: sorry, to open a ticket (aka 'create an issue') on github, I'd need an account which I don't have and don't want to start a browser compatible with the capchas and stuff they have there Jan 17 08:01:47 DocScrutinizer05: hmm i can't seem to find dbus-scripts from apmanager nor apt and besides all that looked a bit over my head atm. Thx for ur investigation i hope it'll make editting mce.ini to work as expected :) Jan 17 08:48:56 freemangordon: have you ever tried libgoo/gstgoo (ti/omx api abstraction)? Jan 17 09:12:12 Just had a bug (affecting only one channel in telepathy-idle, as far as I know) where my local_name ( and in outcoming messages, remote_uid ) was affected. Null at times, or wrong value at other times. No idea why. Cleaned it up is SQL database with two update commands, hoping all is well. Jan 17 10:02:46 DocScrutinizer05: i just realised that probably mediaplayer is overdriving mce.ini setting in that matter. Jan 17 10:03:28 because it needs to be playing something for volume keys to work Jan 17 10:04:10 while locked that is. I found dbus-script-settings and am trying with it on my test device Jan 17 10:15:22 Vajb: most probably, yes Jan 17 10:15:34 I actually wanted to implement that behavior in other players :) Jan 17 10:16:07 hmm maybe while on it u can change it ;) Jan 17 10:16:20 because I find not being able to change volume when device is locked quite frustrating Jan 17 10:16:35 or maybe in next version of omp one can choose wether to disable or enable volume keys Jan 17 10:16:44 or even choose their behavior Jan 17 10:16:49 since unlocking it with systemui/powerbutton will stop playback for a second Jan 17 10:17:10 and same goes for devlock unlock UI Jan 17 10:17:18 yup, but with slider it is not stopped Jan 17 10:17:23 slider? Jan 17 10:17:26 lock switch? Jan 17 10:17:31 yes Jan 17 10:17:46 same with keyboard slider, if device doesn't need to enter code Jan 17 10:18:02 I'd like to fix that as well actually Jan 17 10:18:13 hmm for me playback is not interrupted with keybslider Jan 17 10:19:18 it is only if I need to enter code iirc Jan 17 10:19:23 (not sure about that one, though) Jan 17 10:24:31 ah with code it stops briefly Jan 17 10:26:50 yeah Jan 17 11:12:25 ok tried with dbus-script-settings and it did nothing :) Jan 17 11:12:39 guess i lack the know how Jan 17 11:12:59 easiest for me would be to disable it with epoxy Jan 17 12:05:36 vajb: or maybe.. use one of the other players available? *hinthint* Jan 17 12:08:22 oh a fresh aproach :) Jan 17 12:16:04 :) Jan 17 12:16:45 physical abuse to poor n900 because some software doesnt have a proper option seems excessive Jan 17 12:19:31 well i didn't say i will do it, but that it would be easiest solution ;) Jan 17 12:20:32 i think best would be option in player to choose wether to disable volume or to use it for some other thing Jan 17 12:21:01 yeah, post to the thread then Jan 17 12:27:24 oh such a 2016 solution Jan 17 12:27:58 but maybe i will when i have some spare moment and can recall the problem. Jan 17 12:43:36 ok, record the video of your life-damaging issue, post on yt and every possible social site and expect problem to resolve itself. Jan 17 12:43:40 and give you pony. Jan 17 12:43:52 ;) Jan 17 12:44:12 2017 enough? Jan 17 13:14:09 :D Jan 17 13:14:54 now i begin to wonder if it is possible to upload video to yt from n900 Jan 17 13:20:56 KotCzarny: btw, laptop os working very well. With my typical use i haven't been able to make it stuck (which is good) so thx a lot for the hint! Jan 17 13:21:02 is* Jan 17 13:21:16 :) Jan 17 13:43:08 Vajb: wanna dev a youtube-sharing-plugin? :> Jan 17 13:44:57 bancoh: authoring ;) Jan 17 13:50:29 KotCzarny: ? Jan 17 13:51:56 shoot, edit and upload own material = authoring Jan 17 13:54:11 ah, right Jan 17 14:07:52 TMO: The maemo.org - Talk database has encountered a problem. =( Jan 17 16:18:54 KotCzarny: http://talk.maemo.org/showpost.php?p=1522133&postcount=1715 I did it and whats more i used microb. Failed with Opera first. Jan 17 17:13:32 Vajb: ((realised that probably mediaplayer is overdriving mce.ini)) good point. Needs further checking Jan 17 17:19:19 doc: just made feature request of the matter in omp thread in tmo. Jan 17 17:23:05 this is a architetural decision issue. Jan 17 17:28:55 any app *usually* receives input events via X11. However X11 usually doesn't send events to the app when it doesn't have focus (you see that with maemo medi aplayer which only works vol+/- when in foreground, which is basically correct behaviour). Now when you lock screen, X11 is supposed to also obey mce.ini settings regarding keypress events. However the concept of "X11 focus" makes no sense when screen locked - so an app like media Jan 17 17:28:56 player interested in keypress events when screen locked should probably open /dev/input itself and take *all* events it's interested in (here: vol+/-), and same app might make sure the /dev/input* delivers events so app may want to bind keypad kernel driver when device locked, to override what mce (right now doesn't) do Jan 17 17:31:57 this is a typical issue for getting handled by middleware managing resource allocations. mce does a very poor job on this, freeesmartphone.org has a more comprehensive and coherent concept for this, alas FSO isn't used in maemo Jan 17 17:34:54 see http://wiki.openmoko.org/wiki/FSO_Resources fsoraw (my invention ;-S ) Jan 17 17:35:02 ;-D even Jan 17 17:46:40 http://wiki.openmoko.org/wiki/FSO_Resources#Automatic_way particularly Jan 17 17:47:05 raw == resource allocation wrapper Jan 17 18:02:47 actually locking device shall put a screensaver (here: lockscreen) on top of all other windows and give X11 focus to it, even when immediately after the screen is shut off physically. So any keypress when device (screen) locked would only create X11 events delivered to the lockscreen process. Vol+/- should be a side channel (only while device locked?) via keypad->kerneldriver->/dev/input*->(hulda or )mce->dbus-signal to which an app like Jan 17 18:02:48 mediaplayer could listen Jan 17 18:05:01 that's a great approach Jan 17 18:08:31 alas architecture, and arch is hard to change once a system got established Jan 17 18:10:55 vajb: good job! also tmo is one of the few sites actually checked to be usable on n900 ;) Jan 17 18:19:46 hi DocScrutinizer05 Jan 17 18:19:56 if you had a linux (md) raid array and were having trouble w/ it, what channel would you use to get help? :/ Jan 17 18:20:23 ugh Jan 17 18:20:33 try alis Jan 17 18:20:50 im sure theres a channel about raid somewhere Jan 17 18:20:56 #debian ? even #devuan maybe, or is there #linux-admin ? Jan 17 18:20:59 try ##linux first timeless Jan 17 18:21:48 timeless: I'd probably first check stackoverflow if some answers there Jan 17 18:21:59 oh yes, searching too ^ Jan 17 18:24:31 timeless: https://www.ixquick.com/do/search?q=stackoverflow+md+raid Jan 17 18:24:51 http://stackoverflow.com/questions/41525921/ubuntu-mdadm-failed-array-cannot-get-replacement-drives-into-the-array Jan 17 18:24:57 "Questions on professional server- or networking-related infrastructure administration are off-topic for Stack Overflow unless they directly involve programming or programming tools. You may be able to get help on Server Fault." – Mureinik, Mark Rotteveel Jan 17 18:26:22 ;) Jan 17 18:26:53 http://serverfault.com/questions/180138/software-raid-mdadm-not-adding-spare Jan 17 18:26:59 seems like a good match to my problem Jan 17 18:28:15 https://www.ixquick.com/do/search?q=stackexchange+md+raid Jan 17 18:28:31 oh boy. Had a good idea to putting telegram downloads folder to tracker, but since it not in MyDocs i couldn't reach it from tracker-cfg. Then i tried to make symlink in mydocs to point that folder, but that sems to be impossible too.. Jan 17 18:29:41 no symlinks on VFAT ;-P Jan 17 18:30:19 DocScrutinizer05: none of those are helpful Jan 17 18:30:22 but I'm pretty sure you can index whatever you like, via tracker.cfg Jan 17 18:30:56 timeless: sorry then, good luck anyway Jan 17 18:34:57 timeless: #devuan is "de* Veteran Unix Admins n". maybe you find some veterans there that have a good advice Jan 17 18:35:13 thx Jan 17 18:36:08 take care to not go too offensively off topic though. Not a general problem there but don't expect to be on topic with such question Jan 17 18:36:14 yeah learnt that :) Jan 17 18:36:49 timeless: also, reply time is within hours raher than minutes Jan 17 18:38:04 timeless: considered pinging warfare and/or xes, our maemo sysops? Jan 17 18:42:08 is nick highlight (what you did) enough? Jan 17 18:44:11 warfare: ^^^ Jan 17 18:44:15 xes: ^^^ Jan 17 18:45:10 warfare: xes: md raid issue, maybe you can help? Jan 17 18:45:43 timeless: you probably should post a terse explanation resp question Jan 17 18:45:56 can someone suggest a place to get help w/ an md raid array? I had a raid1 array (one partition mirrored on two drives), and one of the drive's cables came out -- i've tried to mdadm --manage /dev/md1 --add /dev/sdf7 the drive, and it rebuilds, but once it finishes rebuilding, it's converted into a spare instead of left as a normal drive Jan 17 18:45:59 ^^ Jan 17 18:47:49 you looked into https://raid.wiki.kernel.org/index.php/RAID_setup ? Jan 17 18:49:02 https://raid.wiki.kernel.org/index.php/A_guide_to_mdadm Jan 17 18:49:54 sorry I'm no mdadm expert myself, just used those pages when setting up mine Jan 17 18:52:09 I *guess* you first need to switch the defective drive 'offline' somehow, the possibly format or rebuild or whatever, then simply online it again. *adding* a new (third) driver to a raid1 will probably result in exactly what you see, and needs reconfiguration of the raid setup to make the drive an active part of the raid Jan 17 18:53:42 don't bash me when that's nonsense, I just uttered wild guessing Jan 17 18:55:28 http://wstaw.org/m/2017/01/17/plasma-desktopU17764.png Jan 17 18:57:37 http://www.linuxquestions.org/questions/linux-server-73/mdadm-re-added-disk-treated-as-spare-750739/#post4936897 Jan 17 18:57:41 doesnt seem optimistic Jan 17 18:57:47 maybe some bug in mdadm Jan 17 19:00:35 maybe first disk also has a problem? Jan 17 19:01:05 KotCzarny: yeah, i'm pretty sure i read that a while ago Jan 17 19:01:15 and yeah, i was sorta nodding "yes, i'd go w/ zfs too" Jan 17 19:01:24 KotCzarny: the first drive is failing, yes Jan 17 19:01:48 https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=767243#15 Jan 17 19:01:52 sda is failing https://www.irccloud.com/pastebin/j8byIHuF/ Jan 17 19:01:54 seems to also confirm this behavior Jan 17 19:02:33 corrected by brute force ;) Jan 17 19:02:45 Actual RAID is silly. Jan 17 19:02:52 but probably some data will be lost Jan 17 19:02:59 KotCzarny: that bts url looks promising Jan 17 19:03:12 make a backup Jan 17 19:03:13 ;) Jan 17 19:03:19 Maxdamantus: yeah... i'm a fan of zfs. i'm not a fan of raid Jan 17 19:03:29 it's more or less broken 7 ways to sunday, by design Jan 17 19:03:35 but,... i inheritted this system Jan 17 19:03:44 KotCzarny: thankfully (or not), this is the backup Jan 17 19:03:48 :) Jan 17 19:03:48 You basically need a journal (preferably using really fast NVRAM) for it to be consistent. Jan 17 19:03:58 literally, peach's only job in life is to hold backups of everything else Jan 17 19:03:59 otherwise you have write holes in pretty much any configuration. Jan 17 19:07:03 timeless: ummm https://www.irccloud.com/pastebin/j8byIHuF/ looks like a hw defect on sda Jan 17 19:07:17 * timeless nods Jan 17 19:08:24 so AIUI you should simply swap sda for a working HDD, assuming that you have intact data on e.g. sdb in a [sda sdb] raid1 Jan 17 19:09:37 make sure you (or rather md) don't overwite data on sdb with crap from fresh sda Jan 17 19:09:38 so, is the argument that sda is bad and i was unlucky in that sdf fell out of the raid Jan 17 19:09:45 :) Jan 17 19:09:49 there already are sdb..sde Jan 17 19:10:03 in a raid1? Jan 17 19:10:29 so you have a raid1 with 3 spares? Jan 17 19:10:36 my raid hell https://www.irccloud.com/pastebin/jvbwumkt/ Jan 17 19:10:54 b / e is a pair Jan 17 19:11:02 c / d is a pair Jan 17 19:11:07 OH shit Jan 17 19:11:11 a / f is an unhappy pair Jan 17 19:11:21 because the stupid data cable fell out of f Jan 17 19:12:13 so you have FOUR raid1 on sda? Jan 17 19:12:20 on partitons Jan 17 19:13:25 i think this is mostly "because linux" + "no zfs" Jan 17 19:13:25 w/ zfs, in a pair, if one drive fails, your replace it w/ a bigger drive, then you let it mirror, then you replace the smaller drive, let it mirror, then you grow the pool Jan 17 19:13:25 yes Jan 17 19:13:25 but... i think "because linux" + "no zfs", it instead went "oh well, let's just create new partitions for each grow operation" Jan 17 19:13:32 that's pretty weird since your kernel logs hw IO error on sda, not sdf Jan 17 19:13:58 sdf was just unlucky, loose cable Jan 17 19:14:04 sda is probably a much older drive Jan 17 19:14:10 if i were to guess Jan 17 19:14:17 smartmon sda Jan 17 19:14:18 timeless, your nick seems familiar, but i cant remember what project Jan 17 19:14:25 Qt i think Jan 17 19:15:05 based on my memory Jan 17 19:15:15 but it's nearly 6 years ago now :/ Jan 17 19:15:23 :) Jan 17 19:15:27 i touched pretty much everything Jan 17 19:15:34 microB Jan 17 19:15:38 yeah that Jan 17 19:15:51 plus ux + localization + bug help + community Jan 17 19:15:59 yeah, translations Jan 17 19:16:55 the still fuckedup "tana_fi_*" subject ;-P Jan 17 19:17:05 * timeless looks for smartmon Jan 17 19:17:15 SMART Jan 17 19:17:21 you know SMART, do you? Jan 17 19:17:58 yeah, i get email from smart every day complaining about sda dying on peach :( Jan 17 19:17:58 can you get that failing pair offline ? Jan 17 19:18:08 Self Monitoruing And Reporting Foo Jan 17 19:19:15 I *guess* md4 is SOL Jan 17 19:19:32 DocScrutinizer05: installing smartmontools didn't give me a `smartmon` Jan 17 19:19:40 o.O Jan 17 19:19:41 i still have `smartctl` and `smartd` (which i had before) Jan 17 19:19:51 aah yeah smartctl Jan 17 19:19:55 smart-notifier - graphical hard disk health status notifier Jan 17 19:19:59 maybe this one? Jan 17 19:20:09 though i dont know what doc wanted to do Jan 17 19:20:09 ;) Jan 17 19:20:12 smartctl https://www.irccloud.com/pastebin/CxNAOOMm/ Jan 17 19:20:18 usually smartctl/hdparm is enough Jan 17 19:20:19 smartctl -a /dev/sda Jan 17 19:21:26 ^ DocScrutinizer05 Jan 17 19:21:35 reading Jan 17 19:22:23 * timeless runs badblocks on sda6 and sdf7 Jan 17 19:23:04 wd black went boo boo? oh well, nothing is safe anymore! :) Jan 17 19:23:12 Num Test_Description Status Remaining LifeTime(hours) LBA_of_first_error Jan 17 19:23:14 # 1 Short offline Completed: read failure 90% 27923 953187909 Jan 17 19:25:03 you got a bad sector at 953187909 Jan 17 19:25:07 I'd guess Jan 17 19:27:28 FWIW: (maybe just reference) http://paste.opensuse.org/95982284 Jan 17 19:28:08 https://bugs.maemo.org/show_bug.cgi?id=2368 Jan 17 19:28:10 :> Jan 17 19:28:10 04Bug 2368: -, timeless isn't nice Jan 17 19:28:25 resolved fixed ;) Jan 17 19:28:27 * timeless nods Jan 17 19:28:52 timeless: md issue fixed? (I don't want to read the whole backlog, I'm just lazy) Jan 17 19:29:16 warfare, raid1 doesnt rebuild with badblocks on only drive left Jan 17 19:29:20 just a quick summary Jan 17 19:29:31 unless you know how to ignore read errors Jan 17 19:30:00 oO Jan 17 19:30:58 and i bet he has more lurking around on other drives too Jan 17 19:31:17 your damn /proc/mdstat puzzles hell outa me, by the indices [2] [3] etc Jan 17 19:31:46 https://www.irccloud.com/pastebin/jvbwumkt/ Jan 17 19:32:51 I always thought a raid1 had just [1],[0] (in that sequence) with [0] being 'master' Jan 17 19:33:16 I never tried to really learn what it actually is all about Jan 17 19:34:33 anyway one thing's for sure: you should do a dd if=/dev/sda of=whateverishugeenough; and then swap sda for a drive without smart errors Jan 17 19:35:43 warfare: sdf data cable fell off, seems md4 never recovers, prolly because of sda has IO errors since ages already Jan 17 19:35:44 ^ bad suggestion, dd_rescue is better, it wont break on read errors Jan 17 19:36:11 (I won't start to question the use of partions for raid devices) Jan 17 19:36:24 * DocScrutinizer05 neither :-P Jan 17 19:37:01 DocScrutinizer05: not md4, md1 Jan 17 19:37:13 warfare: inheritted crap Jan 17 19:37:27 timeless: Ugh, good luck. Jan 17 19:37:36 i'm pretty sure it's because the drives were replaced by bigger ones and the way the person grew the lv was by adding paired partitions Jan 17 19:38:36 Can I recommend putting everything on 2 physical disks, put lvm on top and let lvm mirror the blocks? Jan 17 19:39:32 timeless: But I think the data on md1 is gone. Jan 17 19:40:00 timeless: ((md1) makes sense with your smartmon result "IO error at 10%" Jan 17 19:40:31 10% (the checked 10 remaining 90) is most likely in md1 Jan 17 19:40:32 warfare: if i'm going to rebuild this stupid machine, there's no way i'm using md Jan 17 19:40:47 my general preference is zfs/zpool Jan 17 19:41:03 i know of 0 good arguments in favor of md or lvm Jan 17 19:41:06 timeless: Please use freebsd for that :) Jan 17 19:41:11 warfare: fine w/ me Jan 17 19:41:24 i'm really agnostic Jan 17 19:41:32 veteran meeting ;-D Jan 17 19:41:42 * timeless has used osx, osol, nexenta for zfs Jan 17 19:42:14 * timeless installs devscripts Jan 17 19:42:16 * DocScrutinizer05 waits for GeneralAntilles chiming in Jan 17 19:43:08 warfare: what do yiu use for offsite backup? I'm using duplicity right now, it sucks donkey balls Jan 17 19:43:28 DocScrutinizer05: I just rsync the backuppc directory over. Jan 17 19:43:56 aaah ok, that requires a vull double up of HDD space Jan 17 19:44:02 full* Jan 17 19:45:10 And everything else which is backup worthy is in a few dmg's on my home server which get copied to a hetzner host every week. Jan 17 19:45:23 BackupPC is nice but conceptually runs on backup target, while duplicity runs on source Jan 17 19:46:53 I guess you could run BackupPC on, and beackup, localhost. and possibly even to a sftp target, but... Jan 17 19:48:50 * DocScrutinizer05 sometimes feels like knowing enough of IT and computers to eventually find a job as bricklayer Jan 17 19:49:13 fwiw https://www.irccloud.com/pastebin/MjFaRZFC/badblocks-sda6-sdf7 Jan 17 19:49:24 it's frustrating when you know enough of a topic to start hating it Jan 17 19:49:38 so sdf7 is actually ok? Jan 17 19:49:43 yeah Jan 17 19:49:50 sdf was just unlucky wrt the data cable Jan 17 19:49:57 it's sda that's failing Jan 17 19:50:07 maybe you can get lucky if you move those bad sectors to sda? Jan 17 19:50:16 ? Jan 17 19:50:20 so it can't rebuild the logically broken sdf Jan 17 19:50:34 from physically broken sda Jan 17 19:50:36 hdparm --write-sector would force badblock reallock Jan 17 19:50:45 KotCzarny: got a specific command Jan 17 19:50:49 (otherwise i'll read the man page) Jan 17 19:50:56 its in that bts ticket Jan 17 19:51:02 ah yeah :) Jan 17 19:51:30 but dont confuse sda sector with relative sda6 rector Jan 17 19:51:35 I seen same "cannot reallocate bad blocks" on my sdb (now nuked) Jan 17 19:51:56 KotCzarny: err Jan 17 19:52:11 yeah, https://www.irccloud.com/pastebin/MjFaRZFC/badblocks-sda6-sdf7 usn't in line with LBA on smartctl log Jan 17 19:52:28 so, since the sectors were reported by badblocks for sda6 Jan 17 19:52:48 you should write to sda6 Jan 17 19:52:56 you can also try to find out what file occupies those sectors Jan 17 19:53:08 KotCzarny: ooh, i like that idea Jan 17 19:53:09 and maybe redo the specific backup only Jan 17 19:53:09 how? :) Jan 17 19:53:17 what fs? extX ? Jan 17 19:53:24 i think the answer is "lvm" Jan 17 19:53:28 nah Jan 17 19:53:45 oh, raw backup? Jan 17 19:53:54 this is what the world looks like https://www.irccloud.com/pastebin/axU4eK7j/ Jan 17 19:55:01 basically, /dev/md1 is one of the things backing VG peach-raid1 which in turn backs LV peach-sys-backups which in turn backs Jan 17 19:55:13 o.O Jan 17 19:55:13 blkid Jan 17 19:55:18 ext3 /backups Jan 17 19:55:37 a maze of twisty hell... because someone couldn't be bothered to use ZFS... "because linux" Jan 17 19:56:12 **make a full backup of sda first!** Jan 17 19:56:15 blkid https://www.irccloud.com/pastebin/k4sfXJMV/ Jan 17 19:56:27 that's helpful Jan 17 19:56:34 well, you can try hdparm --read-block 104846128 and GUESS Jan 17 19:57:54 KotCzarny: got a syntax for that? Jan 17 19:58:05 all i see in hdparm is --read-sector Jan 17 19:58:25 read man hdparm! it's a very dangerous tool Jan 17 19:58:26 hdparm --read-block 104846128 /dev/sda6 Jan 17 19:58:39 it will dump the sector on the screen Jan 17 19:58:57 no. https://www.irccloud.com/pastebin/K24oqE17/ Jan 17 19:59:13 read-sector then Jan 17 19:59:14 ((blkid)) full list (no parameters) would be even more helpful ;-) Jan 17 19:59:15 :) Jan 17 19:59:37 cute https://www.irccloud.com/pastebin/Clsp7XFG/ Jan 17 20:00:07 * timeless goes to try to figure out what the hex says https://www.irccloud.com/pastebin/K9rTDp1q/ Jan 17 20:00:31 ((cute)) HAHA indeed ;-P Jan 17 20:01:06 mysterious Jan 17 20:01:27 honestly, go make a dd_rescure backup of sda6! Jan 17 20:01:50 doc, its not that easy, its a part of live structure Jan 17 20:02:50 umount is your friend Jan 17 20:03:30 and even without, how high is the risk to backup totally harbled data Jan 17 20:03:38 garbled Jan 17 20:03:54 prolly zilch as long as you remount ro Jan 17 20:04:12 or simply don't write (noatime!) Jan 17 20:05:18 you even can do a simple tar backup on logical files of md1 Jan 17 20:05:37 you can see the errors pop up in syslog then Jan 17 20:06:10 and with a tad of luck you can identify the badblock-riddled files Jan 17 20:07:31 funny thing is that, when backup fails or is taken down for maintenance it's probably the time when backup restore would be needed Jan 17 20:07:35 actually the latter is roughly what I did when my sdb blew chunks, got me a raid1 and copied all files over Jan 17 20:07:44 can you duplicate backups to another source? Jan 17 20:08:02 s/source/destination/ Jan 17 20:08:02 KotCzarny meant: can you duplicate backups to another destination? Jan 17 20:10:46 so, the problem is that this is a fragment of a VG which is part of a LV Jan 17 20:11:01 i think i could remove sda6 and try adding sdf7 Jan 17 20:11:20 ??? Jan 17 20:11:36 timeless: this could actually work. Jan 17 20:12:08 aren't both _already_ added to md1? Jan 17 20:12:22 sdf7 isnt active Jan 17 20:12:36 got a command? Jan 17 20:12:39 ... https://www.irccloud.com/pastebin/FhMubmPl/ Jan 17 20:12:49 device busy https://www.irccloud.com/pastebin/DeGtG6uK/ Jan 17 20:13:06 KotCzarny: hrm, would it help if i unmounted the LV? Jan 17 20:13:30 maybe Jan 17 20:13:45 timeless: why do you want to remove drives from a md? Jan 17 20:13:53 or add? Jan 17 20:13:58 mdadm --manage /dev/md1 --remove /dev/sdf7 Jan 17 20:14:06 but question is, can you take whole lv offline? Jan 17 20:14:47 More interesting might be if linux recognizes the pv signature on the drive. Jan 17 20:14:56 aiui a md would still try to function as good as possible even when one drive blows chunks Jan 17 20:15:28 warfare: that worked Jan 17 20:15:35 but it left me w/ the sda6 which is the failed drive Jan 17 20:15:39 can i mark it as failed? :) Jan 17 20:15:52 actually, maybe that's what i should do Jan 17 20:15:57 a totally uneducated guess: I'd try to assemble md1 like it originally was, and do a backup of it Jan 17 20:16:09 mdadm --manage /dev/md1 --fail /dev/sda6 Jan 17 20:16:22 timeless, mount |grep /dev/md1 ? Jan 17 20:16:22 mdadm: set device faulty failed for /dev/sda6: Device or resource busy Jan 17 20:16:26 *now* we're talking :-) Jan 17 20:16:53 KotCzarny: /dev/md1 might be a pv, being part of a vg with parts of a lv on it.. Jan 17 20:17:07 KotCzarny: as noted, [sda6+sdf7] contribute to md1 which contributes to an PG which contributes to an LV which is ext3 Jan 17 20:17:19 something like what warfare said Jan 17 20:17:23 all "because linux" Jan 17 20:17:36 then if you want to take it offline whole thing would need to go offline? Jan 17 20:17:37 no, because ENOBRAIN Jan 17 20:17:39 * timeless so wants to shoot this system and replace it w/ fbsd + zfs in the cloud Jan 17 20:17:56 and that's why i asked if it can go offline Jan 17 20:18:23 KotCzarny: oh, "yes, there's nothing wrong w/ taking the whole thing offline" Jan 17 20:18:27 then you can try zeroing the sectors, maybe copying them from sdf drive and all will be peachy Jan 17 20:18:43 timeless: why not start with genuine md1, untweked and then see what it reports? Jan 17 20:18:49 or you can just say the whole thing failed and experiment on it ;) Jan 17 20:19:03 DocScrutinizer05: eh?? Jan 17 20:19:40 you --remove'd and --add'ed drives to md1. Maybe try original productive assembly first? Jan 17 20:19:45 if that makes any sense Jan 17 20:20:37 ok, i'm wrong Jan 17 20:20:37 or doesn't it 'come up' anymore? Jan 17 20:20:42 i can't take this thing offline Jan 17 20:20:58 there are two LVs (root and backup) but they share a VG Jan 17 20:21:13 OUCH!! Jan 17 20:21:44 * timeless really really really really wants to shoot this configuration Jan 17 20:21:48 don't shoot the system, shoot the admin who committed that crime, possibly using the system as projectile Jan 17 20:22:06 you know what they say about horse with broken leg Jan 17 20:22:13 ;) Jan 17 20:22:30 if you have time you can check for files that fail to read Jan 17 20:22:32 timeless: what's the total storage size? Jan 17 20:22:41 ~5TB ? Jan 17 20:22:53 timeless: can't you simply get a few HDD and copy the whole shite to sth sane? Jan 17 20:22:55 err, maybe ~4.5TB? Jan 17 20:22:58 more resource consuming but otherwise you would have to solve the lv puzzle Jan 17 20:23:08 sure, why not, you are going to bed anytime soon? Jan 17 20:23:23 it's 3pm Jan 17 20:23:35 i don't have any 5TB drives handy Jan 17 20:23:45 then don't rush this Jan 17 20:23:57 find . -type f -exec cat {} >/dev/null \; Jan 17 20:23:57 not worth the headache Jan 17 20:23:58 ;) Jan 17 20:24:14 sth like that, yes Jan 17 20:24:17 maybe do some error redirect to catch the failing file Jan 17 20:24:56 i wonder how that raid would react on badblock during read though Jan 17 20:25:25 isn't that what a raid genuinely is about, to handle it Jan 17 20:25:49 it will try to read same block from sdf7 Jan 17 20:26:09 that's why I suggested to start with productive assembly of md1 Jan 17 20:27:22 when BOTH sdf7 and sda6 fail with read error (or 'block not synced on mirror') on same block, then I suppose md driver will throw normal read error to userland Jan 17 20:27:35 DocScrutinizer05: sorry, you've lost me Jan 17 20:27:44 what command would i use to assemble this stupid thing in a way that works Jan 17 20:28:00 i'm pretty sure the goal is to have sdf7 sda6 instead of sda6 sdf7 Jan 17 20:28:03 what does system use during boot? Jan 17 20:28:05 although i have no idea if i can even do that Jan 17 20:28:36 i can't remember, it probably went into maintenance mode Jan 17 20:28:57 that's relevant info Jan 17 20:29:01 timeless, hdparm --write-sector on sda6 would zero the badblocks (and data) and will allow for rebuild probably Jan 17 20:29:17 or kill more blocks Jan 17 20:29:27 when you failed to guess right block Jan 17 20:29:34 DocScrutinizer05: is there an easy way for me to check dmesg/syslog to confirm? Jan 17 20:29:42 * timeless ponders Jan 17 20:29:45 15:10:47 up 26 days, 22:00, 3 users, load average: 0.08, 0.04, 0.11 Jan 17 20:30:01 i don't have syslog going back that far Jan 17 20:30:05 hmm, err. less /var/log/messages? Jan 17 20:30:15 cat /etc/fstab? Jan 17 20:30:20 I'm not sure Jan 17 20:30:32 https://www.irccloud.com/pastebin/rYywcTJq/fstab Jan 17 20:30:44 i'm pretty sure /backups came up as readonly Jan 17 20:30:54 and / and /boot were fine Jan 17 20:31:05 https://raid.wiki.kernel.org/index.php/A_guide_to_mdadm#Assembling_your_arrays Jan 17 20:31:07 ok, i have /var/log/messages Jan 17 20:31:20