**** BEGIN LOGGING AT Wed Mar 31 02:59:56 2021 Mar 31 09:10:16 Hm… Is there an easy way to do a treewide refresh of all kernel configs? Mar 31 09:44:47 * ldir likes rsalvaterra's question - I was also going to ping Hauke so he could explain what I had done wrong (and how to do it right) with my missing kernel symbol patch Mar 31 09:46:03 Yeah… I'd basically like to decrazify the ubifs compression configuration… but I'm too lazy to do it by hand. :) Mar 31 09:47:11 I need to fix the generic target and let it propagate to all other targets. Mar 31 09:48:16 is there a way you can take the 'update_kernel' script as a start? Mar 31 09:49:08 I thought about something like that… haven't looked at the script, though. Mar 31 09:49:50 Hmm… surprisingly concise. Mar 31 09:52:59 I could probably get by with a couple of sed-fu chops… Mar 31 09:54:57 https://openwrt.org/docs/guide-developer/build-system/use-buildsystem#kernel_configuration_optional just confuses me Mar 31 09:55:12 the whole target vs subtarget etc Mar 31 09:55:42 ldir: I believe that part I grok. What do you want to do? :) Mar 31 09:58:17 ok, let's say I've just done a kernel bump to 5.10.27 and I want to make sure that I'm not going to get any 'missing symbol' prompts. I'm building for my apu2 so x86_64 Mar 31 09:59:16 and as a harder problem - can you explain how a 3 human seater sofa has been completely occupied by 1 supposedly small spaniel! Mar 31 10:02:06 ldir: As a spaniel, small or not, demands and receives respect and admiration. Simple But I'm sure it would share if you asked nicely and provided a small treat ;p Mar 31 10:04:00 ldir: Well, x86 has both target and subtarget. After updating the kernel, you should probably start by making kernel_oldconfig CONFIG_TARGET=target, followed by make kernel_oldconfig CONFIG_TARGET=subtarget. That should take care of the common symbols. Mar 31 10:05:47 As for the spaniel issue, just be thankful it's a dog, and not a cat… :P Mar 31 10:06:48 rsalvaterra: don't think there's Mar 31 10:07:09 guidosarducci: Yeah, I thought so… :) Mar 31 10:07:24 rsalvaterra: .. a CONFIG_TARGET=target (oops) Mar 31 10:07:24 Grommish: oh it knows all about small treats :-) Currently on its back, legs & tail outstretched, totally comfy and relaxed, no stress here at all - I'm relegated to sofa2... I know my place! Mar 31 10:07:51 ldir: stress is for humans! Mar 31 10:08:02 guidosarducci: Ah, that I'm sure there is. ;) Mar 31 10:09:18 guidosarducci: It's actually in the link ldir pasted. Mar 31 10:10:32 ldir: Sofa, so good. :P Mar 31 10:10:33 rsalvaterra: wiki =/= source code though.. Mar 31 10:11:05 rsalvaterra: if you find CONFIG_TARGET=target let me know then :^) Mar 31 10:12:38 guidosarducci: Hmm… maybe it just ignores CONFIG_TARGET when it's set to target and refreshes the target by default? In any case, it does what's expected. :) Mar 31 10:13:12 waits for his build to finish and then will have a play. Mar 31 10:13:50 how does the generic 'target/config' relate to all this Mar 31 10:14:13 It's a hierarchy, I guess. Mar 31 10:14:41 Targets inherit the generic configuration and change what they need, subtargets inherit the target configuration and change what they need. Mar 31 10:15:38 nbd: russell-- : I tripped over some NLS fixes for packages you maintain. Appreciate if you could have a look at https://github.com/openwrt/openwrt/pull/4038 when you can.Thanks! Mar 31 10:17:51 guidosarducci: LGTM Mar 31 10:18:49 guidosarducci: thanks for your thoughts on the dnsmasq thing Mar 31 10:19:38 nbd: so you don't think the ICONV needed to be there either? And testing glibc + musl is enough? No weird corner cases? Mar 31 10:20:23 ldir: I'll send Simon a mail tomorrow, pretty sure I know the right fix. Mar 31 10:20:37 guidosarducci: as I've said the OOM thing for me is an 'interesting' distraction Mar 31 10:21:22 guidosarducci: For the TCP DoS? Mar 31 10:21:32 You guys found the root cause? Mar 31 10:21:55 ldir: 'annoying' is more the word I was thinking of, became less of an issue once stared using banIP and ipsets. Mar 31 10:22:45 guidosarducci: I'm more concerned by the 20 simultaneous TCP connections limit and the implied DoS Mar 31 10:22:54 rsalvaterra: yes, I think so, it's an old problem and I'm guessing others have also tried to beat it up. Mar 31 10:24:22 ldir: you mean someone holding the connections open? I assume there's timeout... connection exhaustion would still be an issue with a threaded dnsmasq too, right? Have to pick *some* limits... Mar 31 10:25:30 guidosarducci: I suggested implementing a proper event loop, based on libevent or libv. :) Mar 31 10:27:41 I am not a proper programmer - a threaded implementation would avoid the OOM 'false' positive - and yes there has to be some limit. Best I can come up with is a limit per query source IP address... you can DoS yourself and hopefully not break TCP DNS for others. Mar 31 10:29:16 This is an old article, but quite explanatory: http://www.kegel.com/c10k.html Mar 31 10:29:21 threading would likely be a big architectural change. Mar 31 10:29:30 ldir: that's the essence of "DNS fairness" :) Mar 31 10:30:29 guidosarducci: i think if iconv was needed, it likely would have complained in your build tests Mar 31 10:30:35 ldir: and yes, I believe that's why Simon has stayed away from suggestions to do threading in the past. Mar 31 10:31:19 nbd: yeah, me too. Just wanted to check since I didn't know what linking errors you had when making your previous fix. Mar 31 10:31:32 It's probably high time for dnsmasq to be dragged to the 21st century… Mar 31 10:35:47 Also, it would be great if dnsmasq had a way to do per pool/host DNS forwarding… having two dnsmasq instances running feels a lot like a big hammer. Mar 31 10:36:04 nbd: thanks, just need russell-- to have a looksee too. Will head off and try again later. Mar 31 11:28:38 >KGB-0< https://tests.reproducible-builds.org/openwrt/openwrt_sunxi.html has been updated. (0% images and 98.1% packages reproducible in our current test framework.) Mar 31 12:22:41 rsalvaterra: why? so much simpler, processes aren't that big a deal. you're asking for all sorts of extra software instead. it's not like you need a 10000 processes vs 10 processes Mar 31 12:23:56 karlp: On a limited system, 20 process can be too much. Mar 31 12:25:02 What I'm still not clear on are the implications on reported free memory on process fork. Mar 31 12:26:04 karlp: https://forum.openwrt.org/t/opening-taxi-app-oom-reaper-kills-dnsmasq/91052 is the context Mar 31 12:26:06 rsalvaterra: perhaps you've dreamt up a networking solution that requires bigger hardware, no matter what? Mar 31 12:27:23 that thread doesn't seem related to "having two instances feels like a hammer" ? Mar 31 12:28:31 I didn't say it was related… ;) Mar 31 12:28:46 in that forum thread we can see that 'free -m' is reporting free memory reducing by 'dnsmasq' size for each fork of dnsmasq (dnsmasq forks to handle TCP based dns queries) Mar 31 12:29:53 simple question, is it normal/expected to see free memory reduce by parent process size for each child if each child is simply a fork of the parent and hasn't changed any pages yet. Mar 31 12:30:48 I'm assuming yes, and I'm assuming that's because the OS is being pessimistic and assuming each child could turn into a completely different process. Mar 31 12:46:32 guidosarducci, regarding NLS changes - there might be some case missing with C++ software - CXXFLAGS isnt set in include/nls.mk but CXXFLAGS is used in cmake.mk Mar 31 12:48:31 does dnsmasq only fork itself for tcp requests? not for udp? Mar 31 12:51:02 karlp: correct Mar 31 12:52:32 there's a compile time limit of 20 simultaneous TCP handling children Mar 31 12:52:52 explains why this has ~never been an issue before... Mar 31 12:53:33 turning the process limit down is going to just result in, "internet is slow, appears dns resoluion perforamnce is terrible" pretty quickly I'd imagine... Mar 31 12:54:00 DoH doesn't touch this, so what's driving all these tcp dns requests? Mar 31 12:54:11 some domain results getting too big out of the box now or what? Mar 31 12:54:50 IDK Mar 31 12:55:03 brb Mar 31 13:09:34 dangole: Hi! You don't know a way to treewide-refresh all the kernel configurations, do you? :P Mar 31 13:09:57 I could *really* use something like that now… :) Mar 31 13:31:54 rate limiting as such on its own is not the solution for a denial of service protection let alone OOM - best I can think of is limit number of concurrent queries by source IP address. Mar 31 14:19:13 rsalvaterra: I would just adjusted https://git.openwrt.org/?p=maintainer-tools.git;a=blob_plain;f=update_kernel.sh;hb=HEAD Mar 31 14:20:39 ynezz: Me too, if I were more shell-script-literate. :) Mar 31 14:22:01 Thing is, I don't know how to refresh the kernel config for a specific target (I mean, ignoring what's present in the .config file). Mar 31 14:22:25 If I can refresh one target, I can refresh all of the, obviously. Mar 31 14:22:46 *all of them Mar 31 14:26:12 is anyone else having trouble building on x86_64? This is just a default config: https://pastebin.com/AT75R3sd Mar 31 14:43:38 if I disable button hotplug, it builds OK but then I select ipt-ipmark from xtables-addons: https://pastebin.com/MrCGgiBF Mar 31 14:47:09 netprince_, maybe try to update xtables-addons to 3.18 and see if error is gone then Mar 31 15:31:35 Hello, Mar 31 15:31:35 I have a Verizon G3100 Wifi 6 gateway made by Arcadyn Technologies that is just collecting dust. Mar 31 15:31:35 Can I donate it to a developer? Mar 31 15:31:36 It's a state-of-the-art gateway with powerful antennas. Mar 31 15:31:37 Here is a teardown of the router. Mar 31 15:31:37 https://www.joshualowcock.com/reviews/review-new-verizon-g3100-wifi-6-mesh-router/ Mar 31 15:31:38 Thanks, Mar 31 15:31:38 Gabriel Fair Mar 31 15:31:39 Signal: +1-864-354-5307 Mar 31 15:31:39 https://Keybase.io/gabefair Mar 31 16:43:46 I found an actual teardown: https://fccid.io/RAXG3100/Internal-Photos/Internal-Photos-4330446 Mar 31 16:48:02 SoC: BROADCOM BCM43684KRFBG (MIPS64 ARM64v8) Mar 31 20:02:54 rsalvaterra: how do you test the kernel bumps? Mar 31 20:03:22 I'd create a CI which builds all kernels for all targets once, or do you perform complex runtime tests? Mar 31 20:04:39 aparcar[m]: I usually do the bump, run-test on my current device. Mar 31 20:04:48 ok Mar 31 20:05:05 mhh, so qemu could test that too> Mar 31 20:05:14 But I was talking about kernel config refreshes, not version bumps. Mar 31 20:05:52 rsalvaterra: k nevermind Mar 31 20:08:46 rsalvaterra: ldir: I am doing the generic kernel configuration stuff normally manually Mar 31 20:08:55 except for sorting Mar 31 20:09:08 For example, I'd like to treewide enable zstd (and disable zlib and lzo) for ubifs. This would be incredibly easier if I could refresh the kernel configs in one fell swoop. Mar 31 20:09:57 many target configurations have manuall changes or do not refresh cleanly Mar 31 20:10:03 this is always risky Mar 31 20:10:28 Hauke: How do you specify the target/subtarget you want to refresh, when it's different from the one in the current .config? Mar 31 20:11:37 I change the .config Mar 31 20:12:02 Ugh… right. :P Mar 31 20:12:08 I would move the options for CONFIG_UBIFS_FS to the generic kernel config and then only have CONFIG_UBIFS_FS=y in the target config Mar 31 20:12:10 if needed Mar 31 20:12:27 then we can easily edit it Mar 31 20:12:39 Hauke: Yes, I already done that. ;) Mar 31 20:12:45 rsalvaterra: nice Mar 31 20:12:45 *did Mar 31 20:13:40 Ok, I can do the changes for the targets/subtargets I own, but can't vouch for the others… :/ Mar 31 20:13:52 Would that be acceptable? Mar 31 20:19:24 Hauke: How do you do the sort? I'm doing insertion eye-sorting. :P Mar 31 20:20:06 if you only touch the UBI_FS settings, you can add the generic settings and then remove the specific settinsg fro all target Mar 31 20:20:18 then list on which boards you tested it Mar 31 20:20:22 even if it is not all of them Mar 31 20:20:48 when the UBI_FS settings are now common a refresh of the configuration should not be neede, it should be at least not worse then before Mar 31 20:21:02 Ok, that's what I'm doing, but I wanted to make sure. Mar 31 20:21:27 rsalvaterra: for sorting see this commit: https://git.openwrt.org/0b5a67aa03f04c8f6b3bfd94a78bd31a266716df Mar 31 20:22:30 Hah! So that's what kconfig.pl is for, nice. Mar 31 20:32:20 mangix: do you have a comment for https://github.com/openwrt/packages/pull/15302? Mar 31 20:49:58 aparcar[m]: Hi Paul, wondering if you had a chance to look at buildbot config after the libelf-dev/libdw-dev discussion? Anything else I can do/test? Thanks. Mar 31 21:02:07 guidosarducci: care to elucidate further on the dnsmasq OOM thing? I suggested a sysctl that I thought would help but alas no :-( Mar 31 21:04:02 plntyk: none of the packages my PR updates with NLS fixes use C++ or cmake. Different problem? Mar 31 21:05:55 ldir: where was this? I didn't see anything posted. Mar 31 21:06:31 it was in the openwrt forum thread. Mar 31 21:07:00 I fear I don't understand the problem. Mar 31 21:12:04 the behaviour suggests that the child process really ARE as big as the parent thus you need 20*n where n is the size of the parent process to be able to satisfy dnsmasq's memory potential. Mar 31 21:12:40 that doesn't quite square with how I thought fork worked. Mar 31 21:14:31 ldir: ahh, OK, wasn't following that, just the dnsmasq ML. Let me finish the mail and I'll cc you. It's painful on IRC, and need to step away. Cheers, bbl. Mar 31 21:14:43 that's 240MB on my system which is fine on my 4GB equipped APU Mar 31 21:15:01 guidosarducci: thank you Mar 31 21:38:30 What a rabbit hole… Mar 31 21:54:00 Hauke: ping Mar 31 21:54:24 ah, *that* rabbit hole Mar 31 21:54:41 This is what I have at the moment… https://github.com/rsalvaterra/openwrt/commit/02372b6a6d0eef78d05a4b070d8959b4af350765 Mar 31 21:55:41 … and I've come to realise all targets/subtargets are in desperate need of a kernel config refresh. Mar 31 21:56:24 * ldir rsalvaterra: don't forget to rinse & repeat as each target migrates to 5.10 Mar 31 21:56:30 * ldir runs away Mar 31 21:56:58 I know, right? :P Mar 31 21:57:04 probably in a hail of bullets - nice party! Mar 31 21:58:04 There are kconfigs which haven't been refreshed ever since nbd's config filter landed, and that was quite a while ago. Mar 31 21:58:50 * rsalvaterra sighs Mar 31 21:59:09 I've never understood the kernel config symbol refresh process. Mar 31 21:59:45 so if you can make me understand you're in with a chance! Mar 31 22:00:14 I understand it, but I don't like it. :P Mar 31 22:01:38 well hold on to that state and see if you can translate it to idiot :-) Catch you tomorrow Mar 31 22:01:54 Haha! Mar 31 22:02:10 Cheers! ;) Mar 31 22:02:36 * ldir goes home Mar 31 22:44:49 rsalvaterra: pong Mar 31 22:45:32 Hauke: So, this is what I have… https://github.com/rsalvaterra/openwrt/commit/02372b6a6d0eef78d05a4b070d8959b4af350765 Mar 31 22:46:38 … but all kconfigs need to be refreshed. :/ Mar 31 22:47:06 rsalvaterra: could you split this into two commits, one moving the changes to the generic kernel config and then a 2. one doing the actual change Mar 31 22:47:40 I would not refresh the target configurations, that would introduce too many changes and probably new problems Mar 31 22:47:52 sometimes there are manuall changes in there Mar 31 22:48:10 Sure, that's an easy one. :) Mar 31 22:48:11 or the result of the refresh depends on the sub target which is seletced Mar 31 22:50:06 Hi! Can someone please help explain the difference between NDP-Proxy relay and hybrid modes? What is the subtlety which makes the two distinct modes necessary? Trying to document them..... Mar 31 23:05:59 where do the /etc/config files get installed? Mar 31 23:06:06 what packages? Mar 31 23:09:18 Hauke: The top two commits: https://github.com/rsalvaterra/openwrt/commits/ubifs. I can open a pull request, and make it available to the wider audience, of course. Mar 31 23:14:01 "opkg search /etc/config/*" only turns up collectd as owning /etc/config/collectd ... am I doing something wrong? Mar 31 23:24:21 philipp64: it depends a lot on the individual files, some are shipped inside the package as conffiles, some are created on firstboot or hotplug events (and not technically part of a package) Apr 01 01:15:58 philipp64: look at the root-/usr/lib/opkg/info/*.conffiles? Apr 01 01:18:24 swalker: trying to remember the provenance of the /etc/config/ipsec files on my machines... they've been through so many revisions I can't remember what I started with. Apr 01 01:22:54 compare the hash from /usr/lib/opkg/status if the conffile is listed? Apr 01 02:23:54 it's not... which makes sense, I guess. VPN is too complicated to include sample config files. **** ENDING LOGGING AT Thu Apr 01 02:59:56 2021