**** BEGIN LOGGING AT Thu Aug 22 02:59:56 2019 Aug 22 03:22:23 here's my latest version: https://dpaste.de/8Apy Aug 22 06:37:42 any netlink expert around? **** BEGIN LOGGING AT Thu Aug 22 06:44:40 2019 Aug 22 08:04:30 jow: depends Aug 22 08:05:22 blogic: this code: https://github.com/jow-/nlbwmon/blob/master/nfnetlink.c#L457 occasionally ends up in a tight recvmsg() / EAGAIN loop Aug 22 08:05:44 this usage pattern is widely used and other software apparently does not suffer from it Aug 22 08:06:13 the only way I could see it ending up in an endless loop is that the kernel simply does not reply to my request Aug 22 08:06:32 so the subsequent recvmsg() attempts on the nonblocking sockets immediately return Aug 22 08:06:53 errm Aug 22 08:06:58 that for construct will be endless Aug 22 08:07:16 no, it won't. The err variable is changed by the callbacks Aug 22 08:07:30 see line 450-452 Aug 22 08:07:31 line 378 ? Aug 22 08:08:18 ahh Aug 22 08:08:21 of course Aug 22 08:08:58 hmmm Aug 22 08:09:16 so the only way it can end up looping forever is if there's no reply from the kernel or if there is a reply that somehow happens to not trigger any of the ack, finish or error callbacks Aug 22 08:09:40 I wonder if netlink allows for such a situation that the kernel simply does not reply to a request Aug 22 08:09:59 other software like iw, hostapd etc. uses the very same usage pattern Aug 22 08:11:29 (calling both the int variable and the jump label "err" is not that clever either, but thats another matter) Aug 22 08:12:40 so in case of finish or ack, err will end up being "0" after nl_recvmsgs(), in case of an error callback, it will be negative Aug 22 08:13:04 it can only stay "1" when nl_recvmsgs() returns without invoking any of the callbacks Aug 22 08:13:29 according to the documentation, nl_recvmsgs() may immediately return with "0" if the given socket is nonblocking and there's nothing to read Aug 22 08:14:03 which is fine, hence the loop - since we eventually expect something to arrive due to our previously sent request Aug 22 08:15:09 but apparently there are occasional situations where an reply never arrives Aug 22 08:15:29 thinking about protecting that reply loop with a poll() call now Aug 22 08:15:36 but that feels wrong Aug 22 08:15:48 *retry loop Aug 22 08:18:11 really weird Aug 22 08:18:49 have you tried to printf the handle_* calls ? Aug 22 08:19:04 see if it does call any of them ? Aug 22 08:29:35 not yet, so far I've been unable to reproduce it myself, just gathering user reports Aug 22 08:50:24 ahhh Aug 22 09:00:46 Hi. I am working on converting firewall3 to using "net,iface" ipsets instead of matching on interfaces (first step to firewall4) for zones. I have something more or less ready, but there is one case I am not 100% sure on how to handle Aug 22 09:01:11 blogic: ping Aug 22 09:04:59 In "print_interface_rule_set()", the raw chain for the helper rules is controlled by looking at the subnet and device. Currently, my code just checks if loopback interface is part of set/address part of subnets and then adds a match on the set containing loopback. This seems to work fine, but I am not sure if there arent any side effects Aug 22 09:05:18 Any ideas? Or would it be better if I finish what I have & submit an RFC? Aug 22 09:05:43 Motivation for changing to sets (when ipset is available), is to get rid of the firewall reload when interfaces come and go Aug 22 09:06:23 kristrev: hm, there is no print_interface_rule_set() - typo or maybe already refactored code from you? Aug 22 09:06:41 I guess you mean print_interface_rule() Aug 22 09:07:17 yes, sorry about that. print_interface_rule_set() is a new function from me Aug 22 09:08:47 okay, and your worry is that you made the loopback check less explicit? Aug 22 09:09:01 instead of specifically probing for IFF_LOOPBACK you soimply rely on 127.0.0.0/8 ? Aug 22 09:09:49 blogic: could you please comment here regardingt the subtarget? If it's a bad approach I'd check other ways to solve it Aug 22 09:11:24 Right now, my creates a set (IPv4 as example) with 0.0.0.0/0,. I have then added some special code for checking if lo is part of set and if so, I create a rule in the OUTPUT chain matching on the entire set Aug 22 09:11:43 ah Aug 22 09:12:09 However, I see now that with the existing code, dev is NULL and all packets passing through raw, output will hit the chain with the helper rules so then I guess matching on the set is fine Aug 22 09:12:43 I still lack a clear picture so I don't fully understand your concern Aug 22 09:12:55 I could be making problems out of nothing :) Aug 22 09:13:19 jow: speaking of the firewall, did you ever look at https://patchwork.ozlabs.org/project/openwrt/list/?series=108356 ? Aug 22 09:13:34 Perhaps it is easier if I finish up what I have and submit an RFC. There are probably other cases I have missed as well Aug 22 09:14:57 kristrev: so the invariant you need to cover is that when the "dev" you're looping over right now is {a/the} loopback interface, the -i of the emitted rule must be omitted (or in the set case, the src interface must not be matched against a set) and the chain must be changed from PREROUTING to OUTPUT Aug 22 09:15:58 yes, that sounds about right Aug 22 09:16:45 the "normal" path now just loops over subnets, as the devices are allready added to set the set Aug 22 09:16:51 aparcar[m]: ah yes, I did look at it and thought it is fine, then forgot about it Aug 22 09:18:21 oh I have more fine things then: https://patchwork.ozlabs.org/patch/1115478/ duck Aug 22 09:18:21 aparcar[m]: well push it to firewall.git Aug 22 09:18:58 *ill Aug 22 09:19:01 **will Aug 22 09:19:15 howere, since I see that there is no in or out interface for the rule created in output, raw, perhaps it would be fine to always create a chain containg the helper rules? Aug 22 09:20:28 or, sorry, I forgot about sub Aug 22 09:20:37 kristrev: hardcoded to ::1/128 / 127.0.0.0/8 ? Sounds fine to me, but you still need to specially treat loopback while iterating then (to skip it) Aug 22 09:21:06 so it probably won't gain you much, you'll just adding a secondary largely redundant code path I guess Aug 22 09:25:06 you mean that the address of the set member would be hardcoded to 127.0.0.0/8? Aug 22 09:25:55 I don't want to suggest anything, I think you should keep the simple approach you've described and not "always create a chain containg the helper rules" Aug 22 09:26:30 basically keep the old "loop over every device" and "check if the current device is loopback" logic Aug 22 09:27:03 to decide whether the source match must be null of iface/net bound and whether the target chain must be OUTPUT or prerouting Aug 22 09:27:33 Ok. I will try to keep things simple, finish up, submit and then we will see what needs to be changed :) Aug 22 09:27:39 Thanks for the help Aug 22 09:27:57 yw Aug 22 09:40:21 does compiling linux-cryptodev-linux fails when ran as root? https://paste2.org/h6Mz5J3z Aug 22 09:40:49 no, its problem with objtool or something being broken Aug 22 09:41:32 the SDK is broken there Aug 22 09:42:31 uhm okay, is that a current snapshot problem or more likely a local issue? Aug 22 09:43:19 The attempted fix for this was https://git.openwrt.org/?p=openwrt/openwrt.git;a=commitdiff;h=a9f6fceb4255ba847415cf82035f5fde3dc0ce2a Aug 22 09:43:26 but it is wrong/incomplete Aug 22 09:44:00 the objtool executable must be copied with bundle-libraries otherwise it may fail to run on systems with another glibc version or library abi Aug 22 09:44:50 try executing /openwrt/build_dir/target-x86_64_musl/linux-x86_64/linux-*/tools/objtool manually and see if it works or crashes Aug 22 09:46:14 jow: it's late here in hawaii, I'll try it tomorrow Aug 22 11:25:07 blogic: PLEASE provide me bootlog from any target/linux/mediatek/ device Aug 22 11:25:17 rmilecki: :-) Aug 22 11:25:24 let me do it right now Aug 22 11:25:36 :) Aug 22 11:26:43 rmilecki: just te mtd part ? Aug 22 11:26:59 yes Aug 22 11:27:22 https://pastebin.com/C3HFF5sh Aug 22 11:27:48 thanks! Aug 22 11:29:07 blogic: which device is that? so I can look at its .dts Aug 22 11:29:54 oh, that's unexpected: [ 11.205400] 6 cmdlinepart partitions found on MTD device eMMC Aug 22 11:29:57 bpi64-r2 Aug 22 11:30:12 i only have a emmc unit here Aug 22 11:30:34 cmdlinepart is unexpected, I expected partitions to be coming from .dts Aug 22 11:30:53 not for emmc Aug 22 11:34:26 I would have expected partitions to come from the mbr/gpt ... Aug 22 11:35:41 I've had issues with some cursed lantiq device (ARV7510PW22) in which the partitioning seems to be hardcoded into the kernel somewhere Aug 22 11:37:59 and since there's three blocks that get writeprotected for no reason in the middle of nowhere, using openwrt with that won't work (openwrt falls within those blocks and won't work) Aug 22 13:10:36 jow: i'm trying to implement notify_check_result(), so I can have something like Aug 22 13:10:38 "notify_check_result asus_device_match 1; notify_check_result asus_checksum 1; notify_check_result asus_version_check 0" Aug 22 13:10:39 ideally i'd have notify_check_result() store an array of pairs (name + value) but ash doesn't support arrays at all Aug 22 13:10:41 jow: any hint? Aug 22 13:10:58 jow: should I store them in variable with some magic "for" and "shift"? Aug 22 13:11:13 export NOTIFY_RESULT_${name}=${value} Aug 22 13:11:30 check_results="asus_device_match 1 asus_checksum 1 asus_version_check 0" Aug 22 13:11:43 export NOTIFY_RESULTS="${NOTIFY_RESULTS:+$NOTIFY_RESULTS:+ }${name}" Aug 22 13:11:47 and later Aug 22 13:12:22 ok, I see that Aug 22 13:12:29 for name in $NOTIFY_RESULTS; do eval "value=\$$NOTIFY_RESULT_${name}"; echo "Result of ${name} is ${value}"; done Aug 22 13:12:30 I can handle that now, thank Aug 22 13:12:48 i thought it's possible without eval... Aug 22 13:12:49 use the same loop construct to clear the data Aug 22 13:12:52 hm, maybe not in ash Aug 22 13:13:06 unset "NOTIFY_RESULT_${name}"; and finally unset NOTIFY_RESULTS Aug 22 13:13:08 ${!foo} Aug 22 13:13:23 you could let jshn do the heavy lifting for you Aug 22 13:16:04 jow: i could do: https://pastebin.com/AbXy26gG but that doesn't allow using jshn in platform_checks() so I'm not sure if that's a good idea Aug 22 13:44:16 rmilecki: why does it not allow it? Aug 22 13:44:46 jow: I don't think jshn.sh is "thread" safe Aug 22 13:44:59 rmilecki: you mean you can't use jshn for other purposes then? That should be easily solvable with switching namespaces, moment Aug 22 13:45:04 jow: check_image.sh is in a middle of creating a JSON Aug 22 13:45:16 which I think means that platform.sh can't work on *another* JSON Aug 22 13:46:26 or can I use JSON_PREFIX to handle that? Aug 22 13:46:45 https://pastebin.com/yb7nEGwh Aug 22 13:47:15 oh, we even have a helper for that Aug 22 13:47:18 amazing! Aug 22 13:48:13 the fun part is then somehow merging the TEST_RESULTS ns data into the main ns later on Aug 22 13:48:22 I have currently no idea how to do that Aug 22 13:48:43 ah Aug 22 13:49:27 wait, i have an idea :) Aug 22 13:49:58 I think I can basically build JSON in check_image.sh using custom namespace Aug 22 13:50:12 and switch out of custom namespace when calling platform_checks Aug 22 13:50:21 then use custom namespace again in the notify_check_result() Aug 22 13:50:26 I think it should wokr Aug 22 13:55:19 i'm taking some time off, i'll keep working on that later Aug 22 15:58:59 Is there a work-around for the broken ip-tiny/ip-full package? Aug 22 16:01:31 Which I guess is due to the broken build for libcap Aug 22 18:33:55 jow: https://paste2.org/k2wpfc1J Aug 22 18:34:07 I don't see it failing, but maybe I'm just executing something wrong? **** BEGIN LOGGING AT Thu Aug 22 19:10:56 2019 **** BEGIN LOGGING AT Thu Aug 22 19:35:42 2019 **** ENDING LOGGING AT Fri Aug 23 03:00:40 2019