**** BEGIN LOGGING AT Fri Nov 27 02:59:56 2020 Nov 27 03:49:37 stintel: can you help me debug this? Not entirely sure it's opkg Nov 27 03:49:53 What are you trying to build Nov 27 03:52:19 I'm trying to bisect boot freeze on apu2 Nov 27 03:52:30 during that bisect I am running into that shit Nov 27 03:53:11 currently I am here: https://bpa.st/7V7Q Nov 27 03:54:18 almost 6AM here and still bisecting this shit Nov 27 03:55:28 am supposed to work in ~4h but I took the day off as I've been bisecting this for hours Nov 27 04:19:52 I am going to bet this is the problem: Nov 27 04:19:53 - config restore - Nov 27 04:19:53 /etc/preinit: line 11: can't open /rom/etc/passwd: no such file Nov 27 04:19:53 /etc/preinit: line 11: can't open /rom/etc/group: no such file Nov 27 04:19:53 /etc/preinit: line 11: can't open /rom/etc/shadow: no such file Nov 27 04:19:53 [ 9.617724] urandom-seed: Seed file not found (/etc/urandom.seed) Nov 27 04:20:06 aka the problem I warned people about Nov 27 04:20:17 probably only tested on squashfs Nov 27 04:21:19 5 hours of bisecting for nothing Nov 27 04:22:47 stintel: which commit introduced the problem? Nov 27 04:23:28 I was unable to find out with bisect Nov 27 04:23:51 because I wrote image to USB stick and booted from that Nov 27 04:23:57 which doesn't test sysupgrade Nov 27 04:24:17 so 99% sure it's related to the "switch core services to another user than root" Nov 27 04:25:01 oh I see Nov 27 04:55:55 stintel: what is the pro for ext4 vs squsahfs? Isn't squash better anyway because ext4 doesn't really do sysupgrades? Nov 27 04:58:25 I'm using an APU2 with 16GB SSD, a real filesystem seems more suitable Nov 27 04:58:36 nonetheless, we support ext4, so we should break it Nov 27 04:59:55 ok so booting the device with OpenWrt on USB, then mounting the /boot of the SSD and runngin "tar xcf /boot/sysupgrade.tgz /etc/passwd" and then rebooting fixed it Nov 27 05:00:13 definitely related to switching ubusd from root to ubusd Nov 27 05:00:26 whoever the fuck caused it, please test it and fucking fix it Nov 27 05:00:45 lost a day of pay because of your fucking untested commit Nov 27 05:10:34 first time running btrfs scrub. I'm surprised nothing has blown up Nov 27 05:23:47 btrfs problems are often caused by buggy drive Nov 27 05:24:15 using btrfs as main fs on all but embedded for almost 10 years Nov 27 05:24:28 it's much more stable than openwrt, even though that's a weird comparison Nov 27 05:30:36 stintel: I'm running it on a device that has had probably over 100 kernel freezes Nov 27 05:30:58 mangix: probably unrelated to btrfs Nov 27 05:31:02 because of someone insistance on carrying a buggy patchset Nov 27 05:31:09 *insisting Nov 27 05:31:33 no I mean I expect btrfs errors after all these kernel freezes Nov 27 05:31:36 none so far Nov 27 05:31:45 it's 2020, for anything !embedded, there is btrfs, bcachefs or zfs, in order of preference Nov 27 05:31:50 anything else makes no sense Nov 27 05:32:12 silent corruption is real. filesystems that don't detect it are obsolete Nov 27 05:32:31 well, the device I'm talking about is an mvebu device. It's somewhat embeddewd. Nov 27 05:32:58 different story. I'd love to see btrfs becoming a first class citizen in openwrt, though Nov 27 05:33:22 imagine sysupgrade to a new subvolume, boot fails, revert to previous subvolume Nov 27 05:33:34 "unbreakableWrt" Nov 27 05:33:36 wasn't there a proposal to replace squashfs + overlay to btrfs on the mailing list? Nov 27 05:33:53 I suggested it once, somewhere Nov 27 05:34:01 there's one really really big problem with that though Nov 27 05:34:02 not relevant for small flash devices though Nov 27 05:34:16 btrfs testing on big endian is a joke Nov 27 05:34:31 mangix: we could help with that Nov 27 05:34:55 last I tested 4.19 on OpenWrt, I created a volume with mkfs.btrfs and it got corrupted within 8 seconds Nov 27 05:35:03 I remember submitting a patch for bird, and upstream being amazed about me running it on mips64 Nov 27 05:35:15 we just need to help eachother out Nov 27 05:35:29 I know it's within 8 seconds as it immediately remounts ro Nov 27 05:36:14 anyway, I tried reporting upstream. Didn't get too far. I needed to give more info. I wasn't familiar enough with btrfs at the time. Nov 27 05:37:13 stintel: you have a mips64 device? Nov 27 05:37:16 mangix: ERL Nov 27 05:37:39 ah is that that ubiquiti device? Nov 27 05:37:40 I have 2, one is in my "testing" vlan Nov 27 05:37:44 yes, edgerouter ltie Nov 27 05:37:47 lite* Nov 27 05:38:05 apparently I should have gotten an extra apu2 too Nov 27 05:38:21 could have saved all of us from my rant earlier Nov 27 05:39:07 sounds like work is needed to get btrfs in here Nov 27 05:39:14 but the fact that I warned about the hackiness of the patch to add new users/groups during sysupgrade, and the fact it still broke ... Nov 27 05:39:18 that way you could just revert a subvolume Nov 27 05:39:33 the turris people do this actually Nov 27 05:39:45 yes, no boot within ~180s? revert to previous subvol/snapshot and be fucking done with it Nov 27 05:40:25 the turris omnia is actually more flexible than that. the reset button actually has 6 modes Nov 27 05:40:41 boot to last snapshot, boot to first snapshot, etc... Nov 27 05:40:44 could be but I'm not a turris fan Nov 27 05:41:00 they should aim to contribute more instead of maintaining their own fork Nov 27 05:41:22 well, they don't anymore for newer releases Nov 27 05:41:35 they still maintain their fork for their legacy branch Nov 27 05:41:43 but stable is based on 18.06 Nov 27 05:41:51 it's just stupid Nov 27 05:42:05 get your shit upstream (with us) and we'll maintain it for ya (more or less) Nov 27 05:42:17 yeah, they've been doing that Nov 27 05:42:45 that's not my experience Nov 27 05:42:53 I think for the DSA patch they want it upstream at kernel.org first Nov 27 05:44:33 but it's easier to maintain a "fork" with shitty code and/or history and don't contribute back, than to address comments and fix it so it can be accepted Nov 27 05:46:57 oh great, all my IoT stuff is offline because of the fucking outage Nov 27 05:52:13 de7ca7dafadfd650d031e0379ce0c002868d5936 Nov 27 05:52:25 read the github comments Nov 27 05:56:08 well then Nov 27 05:56:41 that's actually very amusing Nov 27 05:59:00 sigh Nov 27 06:02:03 aparcar[m]: PR that badly needs to be merged: https://github.com/openwrt/openwrt/pull/2693 . Add my Tested-by if you want. I have done so. It works with my old Omnia Nov 27 06:02:45 don't know what I find worse, that someone reported my issue 14d ago, or that a "lead" dev decided to degrade ext4 images to "development tool" status Nov 27 06:03:09 the latter Nov 27 06:03:34 my comment agrees Nov 27 06:04:17 we need to fix this somehow Nov 27 06:04:50 Amusingly I've used ext4 on an Archer C7v2 before Nov 27 06:04:58 can we somehow hardcode users that are read during boot, so every service is able to start\ Nov 27 06:05:04 I want to say it booted faster. I don't remember Nov 27 06:05:20 and in case those users don't exist in the "real system" we keep running with the old UID Nov 27 06:05:24 and inform the user Nov 27 06:05:39 it's not ideal but 10000000x better than leaving system unbootable Nov 27 06:07:23 alright, kicked garbage cpufreq patches to the curb Nov 27 06:09:19 anyone good at implementing automated tests? I will pay someone to prevent the issue I just had from happening Nov 27 06:10:07 I rely on OpenWrt for my day to day tasks Nov 27 06:19:58 stintel: git history shows the turris people having 15 commits Nov 27 06:20:06 at least it's something :\ Nov 27 06:20:19 yeah right Nov 27 06:20:38 sorry, if you're not trying to upstream, I don't like you Nov 27 06:20:44 that was obtained with git log --pretty=%h»¦«%ae»¦«%s»¦«%aD | grep -c nic.cz Nov 27 06:22:44 ah looks like that's not completely accurate. some of them use their gmail addresses Nov 27 06:22:48 mangix: compare that to commits they have in their fork Nov 27 06:22:54 it's just not acceptable Nov 27 06:23:27 why not? it's FOSS Nov 27 06:23:30 * ynezz hides Nov 27 06:23:33 doesn't even matter. omnia support wasnt really pushed by them iirc, more a community effort Nov 27 06:23:44 true Nov 27 06:23:58 Turris MOX support is also non-existent Nov 27 06:24:04 ynezz: maintaining a fork without PR to upstream causes extra effort for both parties Nov 27 06:24:09 it should be discouraged Nov 27 06:26:44 it's hard if your managers don't understand upstream first Nov 27 06:27:06 found 17 more email addresses from the turris people Nov 27 06:27:07 they Nov 27 06:27:10 so 32 commits Nov 27 06:27:19 *17 more commits Nov 27 06:27:20 they'll get it eventually if everyone resigns Nov 27 06:29:03 for me it's a goverment like entity then a company Nov 27 06:29:13 stintel: new patch on mailing list. looks related to your troubles Nov 27 06:34:11 ping rmilecki Nov 27 06:34:55 whoops, I have that backwards Nov 27 07:39:04 wait so how do we get those turris people to contribute more? Nov 27 07:39:12 isn't bkpepe part of them? Nov 27 07:39:47 he is Nov 27 07:40:31 while you're leveraging, send round a hit squad to deal with NXP :) Nov 27 07:41:57 s/NXP/any vendor/ Nov 27 07:42:14 valid point :) Nov 27 07:42:29 and in my eayes NXP is at least trying Nov 27 07:42:52 GL.iNet might be good on that front Nov 27 07:43:06 like the number of contributions? Nov 27 07:43:16 wel NXP need to try a lot harder with the 88W8864 Nov 27 07:43:31 that's already abandonware Nov 27 07:43:45 yeah that's totally avabdoned Nov 27 07:43:54 *abandoned Nov 27 07:44:06 if they've abandoned it, the least they could do is open source the firmware blob. costs them nothing. Nov 27 07:44:30 as always, there are probably some IP strings attached Nov 27 07:44:36 mangix: plenty of trustworthy people tested this omnia thing, I'd merge it Nov 27 07:45:39 ynezz: what's the scope here, involve $vendors more into helping out at openwrt? Nov 27 07:45:46 as I understand it, Marvell originally intended to upstream the firmware to Linux, but it never got very far. :| Nov 27 07:45:56 mangix: pong Nov 27 07:47:14 rmilecki: what is this libroxml thing?> Nov 27 07:47:27 ? XML parser Nov 27 07:47:54 is it used by anything? Nov 27 07:48:10 it used to be used by ucwmp Nov 27 07:48:16 by nbd Nov 27 07:49:06 aparcar[m]: no scope, just generic morning rants Nov 27 07:49:16 :) Nov 27 07:49:25 rmilecki: so it's safe to remove? Nov 27 07:49:36 move, pls Nov 27 07:49:53 removing is too harsh and not acceptable I would say Nov 27 07:49:56 i'd rather move it to packages, I have some software using i Nov 27 07:50:04 ok Nov 27 07:55:37 what's gonna be moved? Nov 27 07:58:14 https://github.com/openwrt/packages/pull/14036 Nov 27 08:01:53 terrible PR title Nov 27 08:02:42 mangix: sorry, got disconnected, in case you need anything more Nov 27 08:03:41 ynezz: what is the policy for moving packages, I guess I better wait for nbd to merge it? Nov 27 08:03:46 ++ Nov 27 08:03:52 ignore that Nov 27 08:04:25 rmilecki: i no longer own any broadcom hardware so no :P Nov 27 08:04:30 ;) Nov 27 08:04:33 mangix: talking to me? Nov 27 08:04:46 mangix: get yourself some Northstar ;) Nov 27 08:05:26 aparcar[m]: the only broadcom hardware i've ever had is the MIPS stuff Nov 27 08:05:50 mangix: I'm talking about the libroxml PR Nov 27 08:05:58 whoops Nov 27 08:06:02 i meant rmilecki Nov 27 08:06:31 aparcar[m]: PR title was generated by github. I'll change it Nov 27 08:06:47 What happened to the indention here? https://patchwork.ozlabs.org/project/openwrt/patch/CAA21KT-qhPSQzjkOKyK2rdwn0UgVgMZWq+Z_R8gbAj4raj3pOQ@mail.gmail.com/ Nov 27 08:09:37 rsalvaterra: Does this improve anything without manual configuration https://patchwork.ozlabs.org/project/openwrt/patch/20201125230348.536613-1-rsalvaterra@gmail.com/ Nov 27 08:14:18 ynezz: can I merge https://github.com/openwrt/openwrt/pull/3582 ? works for me Nov 27 08:15:37 aparcar[m]: No, airtime policies must be manually configured, of course. Nov 27 08:15:43 'morning! Nov 27 08:15:50 rsalvaterra: evening Nov 27 08:16:05 Almost night, I wager. ;) Nov 27 08:16:10 rsalvaterra: I hoped there would be some automatic magic Nov 27 08:16:38 Sorry, no magic here, only tohojo's heavy science. :) Nov 27 08:21:06 sounds like we want some luci magic on setting up a second wifi Nov 27 08:25:20 Possibly, but I don't use LuCI at all. Nov 27 08:25:26 new ksmbd version released Nov 27 08:25:43 wonder why they added kerberos Nov 27 08:28:39 I'm more worried about the last two mvebu commits… I hope they didn't break sysupgrade on my CZ11NIC13. (I'm still playing with the Redmi AC2100, so I won't use the Omnia for the next few days/weeks.) Nov 27 08:36:26 aparcar[m]: Since you're on a patchwork merging spree, this patch (which is marked as not applicable) fixes a real bug and should be merged, even if not in its current form: https://patchwork.ozlabs.org/project/openwrt/patch/541cbfbd-76f2-59b3-a867-47b6f0fc7da9@gmail.com/ Nov 27 08:38:18 At the time we supported Linux 4.19, but we've since moved all targets to 5.4, I believe. Nov 27 08:40:03 yep Nov 27 08:41:43 In that case, it could be merged as-is. Nov 27 08:42:39 Ah, no, we need to drop the @ge5.1. Nov 27 08:45:47 aparcar[m]: well, if it works and doesn't break things it's fine with me Nov 27 08:46:20 I didn't reviewed the implementation Nov 27 08:52:33 rsalvaterra: I'm getting mixed messages here. What is your course of action? Nov 27 08:53:15 that first change is questionable Nov 27 08:53:25 the ge@5.1 part can be removed Nov 27 08:54:36 What first change? Nov 27 08:54:47 on your linked patch Nov 27 08:55:29 You mean adding the lzo-rle.ko? Nov 27 08:55:49 yes. there's a ge@5.1 part Nov 27 08:56:14 it's not necessary anymore Nov 27 08:56:36 I can explain the questionable change. Nov 27 08:57:03 I know what it is, but since there's no kernel below 5.4, it's not necessary Nov 27 08:57:55 please send a proper v$(n +1) patch explaining it Nov 27 08:58:05 Ah, I thought you were talking about putting lzo and lzo-rle in the same bag (which is due to an upstream silliness). Nov 27 08:59:06 aparcar[m]: Should I? It's not my patch. :/ Nov 27 08:59:15 no. all these @KERNEL_X.X lines tend to get removed. there's also no point matching various kernel versions Nov 27 08:59:26 unless you're backporting to 19.07 or something Nov 27 09:00:24 huh, no more 3 series kernels are supported upstream. BrainSlayer must be sad. Nov 27 09:01:19 Linux 3.x? What year is this? O_o Nov 27 09:01:41 * rsalvaterra notices Linux 3.18 running on his phone… Nov 27 09:01:48 hahahaha Nov 27 09:02:32 I remember he was pissed when 3.10 got dropped Nov 27 09:02:56 because broadcom devices had some pci issue that he couldn't fix Nov 27 09:03:47 The biggest problem of Broadcom devices is Broadcom itself. :P Nov 27 09:03:58 last supported kernel is 4.4. that was a really good kernel. Nov 27 09:04:34 ramips hackery worked just fine with it. Nov 27 09:05:00 4.4 has my first patch ever! :D Nov 27 09:06:15 Which, by the way, was to fix an issue I noticed on my WDR3600. The lz4 algorithm was broken on big-endian and I really wanted to fix it. Nov 27 09:06:45 I even got an old Power Mac G5, so I could test more easily. :) Nov 27 09:13:24 * mangix hates big endian Nov 27 09:13:54 * rsalvaterra <3 big-endian <3 Nov 27 09:14:21 mostly because nobody tests for it Nov 27 09:14:37 mangix: nobody should be replaced by ci Nov 27 09:14:41 I love it exactly because of that. Nov 27 09:15:52 btrfs last I tested was totally broken on big endian. ksmbd took hundreds of commits to get it working on big endian. I had to generate lots of tcpdump output. f2fs-tools was broken on big endian for a while Nov 27 09:15:56 what a bug that was Nov 27 09:16:12 That's a problem of kernel developers who assume too much. Nov 27 09:16:44 the hillarious part about that is that they claim to test big endian (btrfs people when I reported the issue) Nov 27 09:17:19 The biggest problem with btrfs isn't big-endian support, but that's another story. :P Nov 27 09:17:33 i disagree Nov 27 09:17:43 * rsalvaterra is looking forward to bcachefs. Nov 27 09:18:28 Btrfs ate my data too many times (aka once). Nov 27 09:18:48 I have the opposite experience Nov 27 09:19:01 When did you start using btrfs? Nov 27 09:19:21 btrfs has survived multiple kernel freezes due to a totally broken mvebu patchset Nov 27 09:19:31 a year ago Nov 27 09:19:48 I'm running btrfs scrub. No errors Nov 27 09:19:51 simply amazing Nov 27 09:20:12 Surviving kernel freezes is not exactly a feat for a filesystem worth using. Nov 27 09:20:31 Any decent journaling filesystem does that. Nov 27 09:20:59 Well, I *stopped* using btrfs around 2014… :P Nov 27 09:21:09 add to that that i'm running my array in RAID5 mode (aka totally dangerous even for btrfs) Nov 27 09:22:06 mangix: I see you like living on the edge. Try disabling barriers too. :P Nov 27 09:22:14 no :) Nov 27 09:22:28 Aww… XD Nov 27 09:24:10 the kernel freeze was from a totally broken mvebu patchset that implements dynamic frequency scaling Nov 27 09:24:40 the problem is that I've had over 100 freezes with various different kernels Nov 27 09:24:44 from 4.19 to 5.9 Nov 27 09:25:09 Ah, I remember actually having DFS on the Omnia, with the original TurrisOS (Linux 4.4). I enabled it manually (changed the governor to ondemand) and had no issues at all… Nov 27 09:25:34 you would have if you tried running btrfs scrub Nov 27 09:25:42 it only freezes on high I/O load Nov 27 09:26:13 All kinds of I/O, or only block device I/O? Nov 27 09:26:39 (My connection at the time was 100 Mb/s.) Nov 27 09:26:44 definitely the latter. I've experienced it when running btrfs scrub and qbittorrent Nov 27 09:27:03 connected to a btrfs array of 4 drives Nov 27 09:27:49 Wait… you have a RAID-5… of spinning rust…? O_o Nov 27 09:28:00 Tell me they're SSDs. Nov 27 09:28:18 on mvebu? you have to be kidding me Nov 27 09:28:31 SSDs would be faster than the interface Nov 27 09:28:48 Speed is not the point… Nov 27 09:29:19 and what would that be? Nov 27 09:29:27 … it's data safety. Nov 27 09:29:53 I don't know that SSDs are safer than spinning rust Nov 27 09:29:59 The rebuild of a RAID-5 is extremely stressful for the whole matrix, especially if you have the bare minimum amount of drives (four). Nov 27 09:30:11 bare minimum is 3 Nov 27 09:30:15 mangix: who is gonna review 2000 lines of codechane? https://github.com/openwrt/openwrt/pull/3004 Nov 27 09:30:19 It's quite common for a second drive to fail during a rebuild. Nov 27 09:30:46 rsalvaterra: turns out, the drives have different ages Nov 27 09:30:48 mangix: Right, three for RAID-5, sorry. Nov 27 09:30:50 so I'm not too worried Nov 27 09:31:20 aparcar[m]: which commit? Nov 27 09:31:29 all combined Nov 27 09:32:05 the point of that PR is to provide people with a way to build musl 1.2.1, not to merge as is. Nov 27 09:32:16 I've posted those commits separately on the mailing list Nov 27 09:32:50 except maybe 2 or 3 Nov 27 09:33:19 ack Nov 27 09:33:51 once those other ones get merged, I will rebase and remove them gradually Nov 27 09:34:02 ack Nov 27 09:34:19 there are also GCC10 fixes there mixed in Nov 27 09:36:42 aparcar[m]: very simple one: http://patchwork.ozlabs.org/project/openwrt/patch/20201015061027.7614-1-rosenp@gmail.com/ Nov 27 09:37:18 oh actually this one's important: http://patchwork.ozlabs.org/project/openwrt/patch/20201029005257.101772-1-rosenp@gmail.com/ philipp64 was asking about it earlier Nov 27 09:38:11 for both: how do I test these, what target etc Nov 27 09:39:01 the first is a basic compilation fix. you need musl 1.2.1 to see a failure. for the second, it fails on CI with several packages Nov 27 09:39:05 let me find an example Nov 27 09:39:40 https://github.com/openwrt/packages/pull/14028 Nov 27 09:40:17 https://github.com/openwrt/packages/pull/13971 too Nov 27 09:49:58 mangix: mhh I need to find a way for reasonable CI testing this Nov 27 10:07:31 aparcar[m]: ping Nov 27 10:09:47 noltari: ping Nov 27 10:12:49 * enyc meows Nov 27 10:13:15 mangix: hrrm curious now, are things happening so OpenWRT 20 may actually use gcc-10 ? Nov 27 10:15:13 f00b4r0: pong Nov 27 10:18:51 mangix: if you just move libroxml to package, and not just delete it, I would merge the patch to master Nov 27 10:19:09 like we did yesterday, so please add it and I will push to master later Nov 27 10:19:35 aparcar[m]: I just replied to you on GH, I had given the gist of that reply to adrian earlier (it more colorful terms), but now it's logged for future reference ;) Nov 27 10:19:56 tl;dr: kernel2minor is devil's work, and shouldn't be touched with a 10ft pole. Nov 27 10:19:57 great thank you Nov 27 10:20:35 f00b4r0: that what they told about the openwrt build system and look at me know, cruising Nov 27 10:20:46 hehe Nov 27 10:20:58 enyc: Don't count on it… GCC 10.x enables a much more aggressive inliner, which increases code size significantly. We'd need to evaluate if it's actually beneficial in terms of performace. Nov 27 10:21:03 well, if you can parse code commented in cyrillic, be my guest ;) Nov 27 10:22:43 f00b4r0: okay that's good (and sad) feedback, thanks for taking the time Nov 27 10:23:24 aparcar[m]: while I don't like to be ad-hominem, I've come to the conclusion that any piece of code submitted directly or indirectly by "adron-s" should be seriously scrutinized Nov 27 10:24:03 aparcar[m]: he was also responsible for packing a near full copy of u-boot, laden with his own copyright, as an "intermediary loader" for ipq Mikrotik devices Nov 27 10:24:24 aparcar[m]: thankfully that nonsense has been removed thanks to the hard work of John Thompson (john-tho) Nov 27 10:25:06 rsalvaterra: i see, can those inline-options simply be turned-off ? Nov 27 10:26:00 enyc: gcc 10 is not on the roadmap for 20.xx, I think the work will timely begin once the new release branch is merged (ideally mid december) Nov 27 10:26:45 enyc: Yes, they can be turned off, fortunately. Nov 27 10:28:34 enyc: what does the acronym meows stands for ? :D Nov 27 10:29:32 f00b4r0: I don't know anything about this but I understand your concerns Nov 27 10:29:49 aparcar[m]: np, that's the point of the explanation :) Nov 27 10:29:50 however https://github.com/adron-s/kernel2minor/compare/1e5a52c7941945f6d64807ebca4a5923ba5466bd...062eed9f47e8f47a5aeb659558ad5548697dc8e2 looks actually quite understandable... doesn't it? Nov 27 10:30:02 mrkiko: On the internet nobody knows you're a cat. :) Nov 27 10:33:02 is that version define even used anywhere Nov 27 10:33:15 or is it just for vanity's sake Nov 27 10:33:54 Namidairo: talking about kernel2minor? https://github.com/adron-s/kernel2minor/blob/062eed9f47e8f47a5aeb659558ad5548697dc8e2/kernel2minor.c#L88 Nov 27 10:38:03 aparcar[m]: am on phone Nov 27 10:38:07 sizeof() taken on pointer Nov 27 10:38:11 your call Nov 27 10:38:14 adrianschmutzler: already merged Nov 27 10:39:08 rsalvaterra: :D Nov 27 10:40:34 f00b4r0: my favorite cyrillic is DОИДЛД ТЯЦМР Nov 27 10:40:42 ;P Nov 27 10:41:02 reads like doidld tyatsmr Nov 27 10:42:43 mrkiko: othing Nov 27 10:43:58 aparcar[m]: wheres the current roadmap? Nov 27 10:45:11 enyc: https://openwrt.org/docs/guide-developer/releases/goals/20.xx Nov 27 10:46:25 f00b4r0: no I'm not that good at C 🙂 I'm just saying it doesn't look terribly complicated, I guess you have a better understanding Nov 27 10:46:52 aparcar[m]: what is 'new release branch is merged' ? -- I expcet this to be a 'fork' ? Nov 27 10:47:06 aparcar[m]: bugs don't always look terribly complicated ;) Nov 27 10:50:47 enyc: sorry I meant "branched". so it'd be openwrt-20.12 Nov 27 10:50:56 f00b4r0: ;/ Nov 27 10:52:37 i'm off, bye and thanks for all the patches Nov 27 10:53:30 aparcar[m]: Bye! :) Nov 27 10:55:47 mangix: You should be punished for that cyrillic usage. :P Nov 27 10:57:06 I wonder when 20.x testing will be of help to the project Nov 27 10:58:03 enyc: Just try master. For now, master is effectively 20.x, until branch time. Nov 27 10:58:17 rsalvaterra: there's an entire part of the internet dedicated to it: https://reddit.com/r/fauxcyrillic Nov 27 10:59:07 mangix: Having a Russian ex-wife, the cringe is strong in that one. :P Nov 27 11:01:14 Also, what variant? There's Russian cyrillic, Ukrainian cyrillic, Bulgarian cyrillic, etc.… :P Nov 27 11:04:17 All of them Nov 27 11:04:49 and yes, you have a stroke reading aome of them. Nov 27 11:06:48 Oh, well… I just sent the link to my ex-wife and I'm bracing for impact. :P Nov 27 11:18:51 I'm having a strange issue on some systems (mainly the Omnia and this Redmi AC2100, which are totally unrelated)… Sometimes, on sysupgrade, the machine just reboots, without upgrading. I have to repeat the process several times, until eventually it effectively does the upgrade. Has anyone noticed this? Nov 27 11:22:50 rsalvaterra: I only have stuff like that on 4/32 devices, where I attributed it to insufficient RAM (without ever verifying ...) Nov 27 11:23:39 adrianschmutzler: I wouldn't say the Omnia is RAM limited… :P Nov 27 11:24:34 I also saw the patch dropping the caches before sysupgrade… I can't say I like it, it shouldn't be needed… Nov 27 11:24:53 that's why I never submitted it here Nov 27 11:25:12 but I'm using it very extensively downstream Nov 27 11:26:33 but I still cannot believe that it really stopped 100 % of the soft-bricks, just this little change. Nov 27 11:26:50 actually, that would be even more reason to look for the actual cause ;-) Nov 27 11:27:53 That must be fixed by people who have forgotten more about memory management than I'll ever know in my entire life. :P Nov 27 11:28:30 I also remember somebody telling me that the change would be completely useless and it's not expected to help at all ... Nov 27 11:29:06 The mm subsystem is *extremely* conservative, since it impacts basically everything where Linux runs. Nov 27 11:30:12 adrianschmutzler: I would have told you that at the time, if you asked me. Nov 27 11:30:33 It doesn't make sense. But if it works, it works. Nov 27 11:30:57 Anyway, I believe it's safe to assume the issue I'm seeing is unrelated. Nov 27 11:31:58 I'm glad I tried it anyway ... :-) Nov 27 11:32:14 it's not so fun if you have 3-5 % soft-brick on upgrades ... Nov 27 11:32:54 I wonder if playing with swappiness and overcommit settings will have any effect… Nov 27 11:33:09 I have lots of update failures if I don't reboot first. Nov 27 11:33:25 normally it's "try upgrade, reboots, but reboots into same image as before, try again, works fine" Nov 27 11:33:42 karlp: Exactly what I see! Nov 27 11:33:51 I completely understand that it _shouldn't_ be useful, but sometthing's clearly not working properly somewhere Nov 27 11:34:12 karlp: I still have this kind with 4/32 Nov 27 11:34:13 I've never tried this dropping caches thing, but *sahrugs* Nov 27 11:34:18 But sometimes it just reboots more than once. Yesterday it was like 3-4 times, on the Redmi(!). Nov 27 11:34:21 but it's just booting the old and not breaking anymore Nov 27 11:34:25 well, I'm, on 16/64, and it still happens... Nov 27 11:34:39 before the cache "fix", part of these never came back up Nov 27 11:34:55 and actually what most people did was rebooting every device before any upgrade Nov 27 11:34:58 I've never had that at least, but I do carry a watchdog patch on ath79. Nov 27 11:35:36 that was downstream, actually probably already at 18.06 (so ar71xx) Nov 27 11:35:47 where I originally dealt with the cache thing Nov 27 11:36:05 but still, I don't think this is really connected to rsalvaterra's current issues Nov 27 11:41:23 I think, that this explicit dropping of caches works because there is some race somewhere on unhandled error return (retry one more time after error) Nov 27 11:48:37 ynezz: I also suspect my issue is caused by some sort of race condition. Nov 27 11:51:50 The problem is, I'm not exactly using the serial console all the time, to see what's happening at sysupgrade… :P Nov 27 12:00:43 ther'e s nothing useful in my experience Nov 27 12:01:09 sometimes you see more with console Nov 27 12:03:13 but indeed, it would be nice to have all the output in the syslog Nov 27 12:03:28 so the serial console wouldn't be necessary Nov 27 12:03:41 My configuration is not exactly vanilla, of course… but I'm sure all developers have their own patches/hacks haven't been/can't be upstreamed. Nov 27 13:11:42 adrianschmutzler: may or may not be related, it seems ash is unreliable (probably on the piping side): https://github.com/openwrt/openwrt/pull/3037#issuecomment-734288696 Nov 27 13:23:54 i have some devices where i have to explicitly stop snmpd before a sysupgrade and maybe a few other processes to avoid OOM'ing Nov 27 13:25:18 i wonder if procd is restarting them after sysupgrade tries to TERM and KILL them Nov 27 13:27:10 russell--: And maybe rmmod(ing) all non-vital modules isn't entirely unreasonable… Nov 27 13:28:41 russell--: the TERM/KILL loop runs in stage2 at which point procd should've exec'd itself into upgraded Nov 27 13:29:22 however I was talking with blogic the other day that we need to implement a proper "init 1" equivalent in procd (that is, gracefully shutting down all servies as if /etc/init.d/$service stop has been called, respecting settings such as term grace period etc.) Nov 27 13:29:56 afair he has took a look at a possible implementation and it is only a few dozen lines or so Nov 27 13:30:06 Ooh, runlevels for procd, that's so sysvinit! :D Nov 27 13:30:25 once this is done we can get rid of the TERM/KILL shell loop Nov 27 13:30:40 next todo item would be invoking /sbin/block to handle umounts Nov 27 13:30:47 swapoff etc. Nov 27 13:31:28 jow: I'm trying to remove swapon/swapoff from block… :| Nov 27 13:31:42 (Only the applets, not the hotplug functions.) Nov 27 13:32:35 The BusyBox implementations are more featured. Nov 27 13:32:55 talking about hotplug-call: it's kinda be nice to have hotplug calls trigger though ubus, so we can have acl for them (as ntpd and probably other services are using them and making that work non-root is tricky otherwise) Nov 27 13:33:46 (By "trying" I mean "I have patches pending review".) Nov 27 13:37:15 dangole: had a discussion with champtar about that a while back: https://pastebin.com/h82H7fvN Nov 27 13:40:39 rsalvaterra: why is it so brittle? Nov 27 13:40:58 rsalvaterra: wasn't expecting it to be Nov 27 13:42:05