**** BEGIN LOGGING AT Fri Mar 08 02:59:56 2019 Mar 08 05:00:52 how can i defer module loading? Mar 08 05:01:11 AutoLoad without anything loads "my" modules too early for device nodes in /dev being created Mar 08 06:29:24 mirko: right now you can't really Mar 08 10:03:02 Hello. 26 days ago I submitted a pull request, and it got zero attention so far: https://github.com/openwrt/packages/pull/8174 Mar 08 10:03:19 The pull request is about the Tor package (namely, providing a size-reduced variant of tor-geoip), and Tor itself seems to be maintained in OpenWRT (it got a version update two weeks ago), so I would expect at least a reply Mar 08 10:03:45 Did I do anything wrong regarding the procedure? Mar 08 10:55:27 Hauke: ^^ Mar 08 10:58:57 hmmm Mar 08 10:59:06 big upload over VDSL line. Mar 08 10:59:18 ISP stops seeing LCP echo-requests and kills the line Mar 08 10:59:33 qdisc fq_codel 0: dev dsl0 root refcnt 2 limit 10240p flows 1024 quantum 1522 target 5.0ms interval 100.0ms memory_limit 4Mb ecn Mar 08 10:59:33 qdisc noqueue 0: dev dsl0.101 root refcnt 2 Mar 08 10:59:33 qdisc fq_codel 0: dev pppoe-vdsl root refcnt 2 limit 10240p flows 1024 quantum 1526 target 5.0ms interval 100.0ms memory_limit 4Mb ecn Mar 08 10:59:51 should I have codel on the dsl0.101 vlan interface too ? Which queue do packets actually land on? Mar 08 11:01:20 I think it should be either dsl0 or pppoe-vdsl, not vlan Mar 08 11:01:33 it's on bother of those Mar 08 11:01:50 tcpdump on my side and tcpdump at the ISP disagree about when the LCP echo replies were sent. Mar 08 11:02:02 it's almost as if something in the middle is playing silly buggers with the LCP Mar 08 11:03:55 I think the mistake here is that nothing is setting the bandwidth Mar 08 11:04:35 yeah, just looking up the limits for codel Mar 08 11:04:41 here is what it looks like on adsl: https://hastebin.com/azutodisex.txt Mar 08 11:05:17 codel isn't supposed to *need* explicit limits, is it? Mar 08 11:05:45 relies on BQL Mar 08 11:05:53 now... is BQL implemented for the lantiq VDSL ... Mar 08 11:05:58 yes, and on adsl, there is no BQL Mar 08 11:06:20 I thought I fixed that for PPP over ATM. Mar 08 11:06:41 and on LTE it's even worse (bandwidth allocation depends on the number of packets queued in hardware) Mar 08 11:07:45 regarding the fix for ppp over atm - how do you know the queue size for atm? Mar 08 11:08:33 and regardless, at least for Rostelecom in Russia and Globe in the Philippines, the bad queue is not actually in the modem, the ISP shapes you, badly Mar 08 11:08:33 You don't necessarily know but what you do is track the packets until the moment they leave the box. Mar 08 11:08:54 "until the moment they leave the box" is the hard part Mar 08 11:09:07 I mean, the moment they leave the OS-controlled queue Mar 08 11:09:12 i.e. submitted to the ATM device. Mar 08 11:09:28 if there's really queuing in hardware, that's out of scope. Mar 08 11:10:08 yes, that's what I mean Mar 08 11:10:50 but as long as there is a significant queue in the place you don't control, you need explicit limits for fq_codel to work Mar 08 11:11:13 yeah Mar 08 11:11:27 does codel even support a limit or do I have to chain it to something that does? Mar 08 11:12:49 you don't need to, just use cake? Mar 08 11:19:46 # CONFIG_PACKAGE_kmod-sched-cake is not set Mar 08 11:19:58 dammit :) Mar 08 11:23:59 patrakov: so there's BQL on VDSL but not ADSL? Mar 08 11:24:13 I have no idea about VDSL Mar 08 11:24:41 the lantiq vdsl drivers are missing BQL Mar 08 11:25:14 as I thought neither have BQL - bu**er Mar 08 11:25:29 I can't remember if I did actually fix the PPPoE code to keep tracking the skb even after it's passed over to the parent device. Mar 08 11:25:32 s'been a while... Mar 08 11:28:33 my not-so-informed opinion is: I would actually be surprised if this fix gets accepted upstream, as it differs from what all other tunnel-like drivers (e.g. sit or vlan) do Mar 08 11:29:26 i.e. you have to somehow have an answer to "just shape the dsl0 interface and use noqueue on the pppoe" objection Mar 08 11:29:29 that's OK. We can fix them too. Mar 08 11:30:34 The idea is that the local TCP stack *knows* when its packet has left the box Mar 08 11:30:51 if there's an artificial horizon when it enters the tunnel, it can't see. Mar 08 11:31:11 ok, it makes sense Mar 08 11:31:16 shaping the pppoe works, *if* the packets aren't then magically considered as immediately "sent". Mar 08 11:31:40 but only partly, because you still get the problem I have here that LCP &c on the dsl0 interface aren't getting through Mar 08 11:31:43 we need to shape the dsl0 Mar 08 11:32:58 root@lantiq:~# sysupgrade /tmp/openwrt-lantiq-xrx200-bt_homehub-v5a-squashfs-sysupgrade.bin Mar 08 11:33:00 wish me luck :) Mar 08 11:33:13 good luck :) Mar 08 11:36:25 root@lantiq:~# tc qdisc replace dev dsl0 root cake pppoe-ptm bandwidth 20Mbit Mar 08 11:36:25 root@lantiq:~# Mar 08 11:36:25 better :) Mar 08 11:37:40 * karlp is glad his kids aren't big enough to bandwidth hogs, and the uplnk is fast enough to not have to care about any of this stuff for just the wife and I Mar 08 11:37:59 the problem I'm looking at is that when I upload something large, the line disconnects Mar 08 11:38:08 because the ISP stops seeing my LCP echo responses. Mar 08 11:42:03 unless saying this jinxes it.... limiting the egress bandwidth with cake does seem to have fixed it. Mar 08 11:42:18 BQL on the ltq_ptm device would have made it work out of the box though, I think Mar 08 11:42:25 Hauke? Mar 08 11:48:22 hey, i've a problem with software offloading, i'm using OpenWrt master bcm53xx with kernel 4.19 Mar 08 11:48:30 i tried enabling sw offloading with: uci set firewall.@defaults[0].flow_offloading=1; uci commit firewall; /etc/init.d/firewall restart Mar 08 11:48:48 it doesn't change NAT performance for me :( Mar 08 11:48:50 before: [ 3] 0.0-30.0 sec 1.19 GBytes 341 Mbits/sec Mar 08 11:48:58 after: [ 3] 0.0-30.0 sec 1.20 GBytes 344 Mbits/sec Mar 08 11:49:35 my iperf2 client is x86_64 connected to the "lan" network, my iperf2 server is running on x86_64 connected to the "wan" network Mar 08 11:49:48 jow: nbd: Hauke: help? Mar 08 11:52:20 network + firewall configs: https://pastebin.com/PbVZDyr2 Mar 08 11:54:17 last time I tried I still had spontaneous reboots with flow offloading on x86/64 Mar 08 11:54:29 with no kernel backtrace :( Mar 08 11:55:17 ok, what's the "correct" way in OpenWRT to use cake, other than manually setting up a hotplug script to run tc qdisc replace dev dsl0 root cake pppoe-ptm bandwidth 20Mbit Mar 08 12:01:39 FWIW I see FLOWOFFLOAD in the iptables -L: https://pastebin.com/RwiZ6jmh Mar 08 12:01:40 FLOWOFFLOAD all -- anywhere anywhere /* !fw3: Traffic offloading */ ctstate RELATED,ESTABLISHED FLOWOFFLOAD Mar 08 12:06:33 rmilecki: please use "iptables-save" not "iptables -L" Mar 08 12:07:13 dwmw2_gone: luci-app-sqm Mar 08 12:08:27 iptables-save: https://pastebin.com/nG9JJKPj Mar 08 12:08:47 dwmw2_gone: it has a select box where you specify the interface Mar 08 12:08:52 patrakov: thanks Mar 08 12:10:18 rmilecki: cat /proc/net/nf_conntrack | grep OFFLOAD, does it show anything? Mar 08 12:10:29 zero Mar 08 12:10:32 no output for that Mar 08 12:10:59 oh, wait, now I started iperf again, I see something Mar 08 12:11:06 ipv4 2 tcp 6 src=192.168.1.2 dst=192.168.13.1 sport=51322 dport=5001 packets=23727 bytes=739567600 src=192.168.13.1 dst=192.168.13.10 sport=5001 dport=51322 packets=59870 bytes=3117880 [OFFLOAD] mark=0 zone=0 use=3 Mar 08 12:11:51 hm, it doesn't show cake as being available Mar 08 12:11:54 despite cake being in use Mar 08 12:12:20 screenshot please? Mar 08 12:12:32 dwmw2_gone: your is issue the the same as in https://bugs.openwrt.org/index.php?do=details&task_id=854, isn't it? Mar 08 12:12:48 patrakov: a screenshot of what? did i miss sth? Mar 08 12:13:21 mkresin: it's the ISP doing the kill because of failed echoes, but similar Mar 08 12:13:36 192.168.1.2 is my x86_64 with iperf2 client connected to the "lan" Mar 08 12:13:37 192.168.13.1 is my x86_64 with iper2 server connected to the "wan" Mar 08 12:13:39 rmilecki: sorry Mar 08 12:15:31 Mar 8 12:15:16 lantiq.infradead.org SQM: Starting SQM script: piece_of_cake.qos on dsl0, in: 0 Kbps, out: 20000 Kbps Mar 08 12:15:32 Mar 8 12:15:16 lantiq.infradead.org SQM: ERROR: QDISC cake is NOT useable. Mar 08 12:15:42 qdisc cake 8001: dev dsl0 root refcnt 2 bandwidth 20Mbit diffserv3 triple-isolate nonat nowash no-ack-filter split-gso rtt 100.0ms ptm overhead 30 Mar 08 12:15:48 dwmw2_gone: screenshot please, here is what it says here: https://imgur.com/a/O97Jfet Mar 08 12:15:49 apart from being currently in use, that is... Mar 08 12:17:12 does it say "NOT useable" if you put it on the pppoe interface? Mar 08 12:17:41 ah, let me give it ifb kmod etc. Mar 08 12:32:35 that looks better Mar 08 12:32:39 qdisc cake 8004: dev dsl0 root refcnt 2 bandwidth 20Mbit besteffort triple-isolate nonat nowash no-ack-filter split-gso rtt 100.0ms raw overhead 0 Mar 08 12:35:06 i've switched to the 18.06 with kernel 4.14.104 and now I see a "gain" from the sw offload Mar 08 12:35:07 * dwmw2_gone looks at how to teach it the overhead Mar 08 12:36:02 without offloading (30 seconds tests): 530 Mbits/sec , 530 Mbits/sec , 531 Mbits/sec Mar 08 12:36:03 with offloading (30 seconds tests): 557 Mbits/sec , 560 Mbits/sec , 557 Mbits/sec Mar 08 12:36:47 5% gain... is was hoping for MORE Mar 08 12:37:02 nbd: any idea why my gain is so low? Mar 08 12:44:04 hm, luci-app-sqm does have link layer config but not ptm Mar 08 12:44:04 'noatm overhead 30' still sees disconnects Mar 08 12:44:04 'ptm overhead 30' does not. Mar 08 12:49:08 dwmw2_gone: does it actually work? http://www.dslreports.com/speedtest Mar 08 12:49:18 yes Mar 08 12:49:35 limited to 20Mb/s, with 'ptm overhead 30' Mar 08 12:49:48 does it get bufferbloat below 10 ms? Mar 08 12:50:00 my test here is that the line stays up while I upload something large. Mar 08 12:50:26 anything else is a secondary concern. I don't actually have performance issues. I *am* using VoIP while I do this and haven't noticed a problem. Mar 08 12:51:19 20Mb/s with 'noatm overhead 30' and the line still bounces. I have to bypass luci-app-sqm's link layer config and manually set 'pppoe-ptm' in the dangerous extra options Mar 08 12:51:31 20Mb/s with 'ptm overhead 30' works. Mar 08 12:53:34 where did you get 20 Mb/s from? Is it the speed you pay for, or what your modem connects at? Mar 08 12:55:21 both Mar 08 12:56:41 I see. I asked because both in Russia and in the Philippines, on ADSL connections that I had there, these speeds differ Mar 08 12:57:41 e.g. in PH, Globe will refuse to sell you a 15 Mbps connection if the modem cannot connect at 18 Mbps (but they still shape at 15 Mbps, badly, and the cake ATM overhead calculation doesn't apply because the modem is not the bottleneck) Mar 08 13:00:35 the recommendation is actually to set the cake bandwidth to something like 5% less than what you get without shaping, and increase until you get bad results on dslreports Mar 08 13:04:13 This is 80/20. The downlink is indeed shaped with 5% overhead, IIRC. But that's done at the ISP side not here. Mar 08 13:04:19 I'm only shaping the uplink here. Mar 08 13:06:37 dwmw2_gone: https://pastebin.com/n4s35gmG is sorta thing I use Mar 08 13:07:07 right. I just have Mar 08 13:07:09 option eqdisc_opts 'pppoe-ptm' Mar 08 13:07:14 option linklayer 'none' Mar 08 13:08:58 but it's moderately suboptimal that we have to set Mar 08 13:08:59 option qdisc_really_really_advanced '1' Mar 08 13:09:14 just to work around the fact that we can't select 'linklayer ptm' Mar 08 13:10:38 oh lordy you are heading for a real bike shed zone with that and I advise you to turn back ;-) Mar 08 13:15:29 hah. why? Mar 08 13:17:27 rmilecki: maybe offload isn't pickinhg Mar 08 13:17:34 up the packets Mar 08 13:17:54 in the ingress hook Mar 08 13:17:56 nbd: any simple way of checking that? Mar 08 13:18:15 add some printk :) Mar 08 13:18:22 also there's a big regression between 18.06 + 4.14 and master + 4.14, I'm going to bisect that soon Mar 08 13:18:25 nbd: ok Mar 08 13:18:35 rmilecki: use iperf3 instead of iperf2, and measure speed both ways? Mar 08 13:18:50 what kind of regression? Mar 08 13:19:20 patrakov: i stick to iperf2 as later I'm going to use many streams to check speed when using both CPU cores Mar 08 13:19:22 nbd: NAT speed Mar 08 13:19:25 without offloading Mar 08 13:20:29 With 18.06 branch and kernel 4.14.104: 530 Mb/s Mar 08 13:20:30 With master branch and kernel 4.14.104: 341 Mb/s Mar 08 13:20:32 With master branch and kernel 4.19.25: 351 Mb/s Mar 08 13:36:05 please check the netifd rps/xps change Mar 08 13:36:18 it might be related Mar 08 13:38:03 i'm recompiling without KERNEL_SLABINFO ... i forgot to disable that after debugging some memory problem few weeks ago Mar 08 13:46:37 nbd: Has there been any progress with the MT7628 4addr issue, or is it an MTK bug we're not likely to see fixed? Mar 08 13:47:12 i haven't done any testing on it in a while Mar 08 13:47:26 i didn't have time to look into it yet Mar 08 13:49:40 Okay, long as it's still somewhere on the ever-growing list :) Mar 08 13:58:35 nbd: disabling KERNEL_SLABINFO has helped a lot... sorry for the noise Mar 08 14:09:44 how are patches in target/linux/generic/backport-4.9 managed? I have a commit downstream that adds one and one hunk doesn't apply so I need to refresh it but I'm not sure how. Mar 08 14:10:11 make target/linux/refresh V=s Mar 08 14:10:17 assuming it's only a refresh Mar 08 14:10:22 and it doesn't need a manual fix Mar 08 14:10:34 hexa-: otherwise: https://oldwiki.archive.openwrt.org/doc/devel/patches Mar 08 14:10:57 ok, I assume it's beyond refreshing and needs a manual fix Mar 08 14:11:03 thx Mar 08 14:11:42 hexa-: so QUILT=1, then enter build_dir, then "quilt push -a", then (after failure) "quilt push -f", then fix files, then "quilt refresh" Mar 08 14:11:58 it's a short summary what you need Mar 08 14:12:00 that wiki page should help Mar 08 14:14:32 hexa-: you may find https://raphaelhertzog.com/2012/08/08/how-to-use-quilt-to-manage-patches-in-debian-packages/ useful too Mar 08 14:22:02 that was easy, thanks alot Mar 08 17:29:23 has anyone strated working on v5.0 support ? Mar 08 17:36:19 blogic: a bit. Currently working with 5.1-rc0 + net-next for mt7621 ethernet and mt7530 part. Mar 08 18:54:07 blogic: I used 5.0 on lantiq VR9 Mar 08 18:54:12 but without the OpenWrt patches Mar 08 18:55:13 mirko: aren't these virtual box tools open source? Mar 08 18:56:27 patrakov: I will have a look at your pull request, sorry I am not following packages sdo closely Mar 08 18:56:37 thanks! Mar 08 19:01:45 dwmw2_gone: I also saw such problems and I limited my 10 MBit/s upload to 9 MBit/s, but I haven't investigated this problem further Mar 08 19:01:53 I hostly saw it with big git push ;-) Mar 08 19:02:19 I can have a look if I find something regarding LCP echo respone missing Mar 08 19:02:46 the problem is that we use in OpenWrt a more or less recent PHY FW and a very old datapath driver and firmware Mar 08 19:02:57 and the normal SDK does much more directly in HW Mar 08 19:26:34 Monkeh: ping Mar 08 19:30:37 nbd: pong Mar 08 19:31:37 i can't reproduce the wds issue with the latest version anymore Mar 08 19:31:44 not sure if it's my setup or if something got fixed along the way Mar 08 19:31:48 Let me dig out my test rig and see Mar 08 19:31:49 mind giving it a quick try? Mar 08 19:31:57 cool Mar 08 19:35:21 Give me a few minutes to rebase Mar 08 19:41:55 Hauke: i figured the additions are proprietary except the kernel modules Mar 08 19:42:07 Hauke: but apparently you're right ;) Mar 08 19:43:21 nbd: Plus a little while to build.. I miss using my 24-thread Xeon box for this. Mar 08 20:08:43 Monkeh: nbd: there was a bug in mac80211 which broke 4addr mode with ath10k, this was fixed in openwrt some weeks ago Mar 08 20:09:03 could be that some other drivers were also affected Mar 08 20:09:06 Hauke: This is a rather particular explosion with mt76 - with no ath10k involved Mar 08 20:09:27 Hauke: i'm pretty sure that's unrelated Mar 08 20:09:47 Hokay, finally compiled. Mar 08 20:10:32 ok Mar 08 20:12:43 Er, hm Mar 08 20:12:49 nbd: Slight delay, mild bricking. Mar 08 20:14:28 Very my fault. Mar 08 20:32:22 nbd: Does not appear fixed. Mar 08 20:34:53 nbd: http://dpaste.com/31T3DGV Mar 08 20:51:58 nbd: It survived about 60 seconds with 32 RX PSE stuck resets before finally dying just now. Mar 08 21:36:22 oh DAAAMN! I just figured out a cool way to test stuff in my openwrt build. echo ':qemu-mipsel:M::\x7fELF\x01\x01\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x02\x00\x08\x00:\xff\xff\xff\xff\xff\xff\xff\x00\xfe\xff\xff\xff\xff\xff\xff\xff\xfe\xff\xff\xff:/usr/bin/qemu-mipsel:'"${QEMU_BINFMT_FLAGS}" >/proc/sys/fs/binfmt_misc/register; mount bind rootfs-ramips dev and proc directory and then chroot in! :) Mar 08 21:36:56 That's from the Gentoo qemu ebuild, except that I had to modify it Mar 08 21:46:26 the above presumes you have qemu-mipsel Mar 08 21:46:33 or whatever your arch is Mar 09 00:18:51 dansan: ubuntu has a binfmt-support package, which has caused issues for me in the past during openwrt build ;) i think it was an openssl test where it'd hang due to it automatically running via binfmt Mar 09 00:18:54 had to disable it ... **** ENDING LOGGING AT Sat Mar 09 03:00:34 2019