**** BEGIN LOGGING AT Mon Mar 25 03:00:06 2019 Mar 25 08:40:28 whom should I ping about the IRC buildbot? Mar 25 08:48:22 ynezz: what about it? Mar 25 08:49:08 I don't remember seeing any build failure, yet the builds are broken Mar 25 08:49:20 ynezz: #openwrt-devel-build Mar 25 08:49:37 ah, nice, wasn't aware about that, thanks Mar 25 08:56:10 'lo Mar 25 08:58:40 who is this stranger 'blogic'? Mar 25 08:59:42 no clue Mar 25 09:00:02 i think he once rebooted a project to make it work without his interaction Mar 25 09:00:12 looking around, we seem to have taken the correct decisions Mar 25 09:01:20 recent OpenWrt/kernel network performance looks shit :( Mar 25 09:01:31 i spent the last 9 days debugging that Mar 25 09:01:34 stuff is moving still that's for sure :-) Mar 25 09:02:38 rmilecki: have you found a smoking gun? Mar 25 09:03:27 i wish it was about a single patch/change Mar 25 09:04:01 it involved kernel changes, OpenWrt patches, queue manager Mar 25 09:04:11 git reset --hard v3.18 Mar 25 09:04:18 it'll take me few more days to narrow all o fthat Mar 25 09:04:42 ynezz: for me 3.18 and 4.1 give the same performance Mar 25 09:04:51 things to wrong with 4.2 Mar 25 09:04:59 and later kernels don't work well with OpenWrt patches Mar 25 09:05:05 *things go Mar 25 09:05:28 but I can do that... I'll squeeze everything of that hardware I got ;) Mar 25 09:05:47 well, I've just taken some random version from history :) Mar 25 09:07:37 rmilecki: I wish you all the best in your investigation Mar 25 09:08:30 thx Mar 25 09:11:43 rmilecki: out of interest what are the symptoms you see? Is it a uniform slowdown or is 'bursty' in nature? Mar 25 09:12:23 performance of routing betwen LAN and WAN dropped from 800-900 Mb/s down to 400-550 Mb/s Mar 25 09:12:26 testing with iperf2 Mar 25 09:20:24 rmilecki: which kernel? Mar 25 09:20:33 4.14 and 4.19 Mar 25 09:20:45 performance went to shit from 4.14 to 4.19 Mar 25 09:21:19 i'd recommend using perf along with the FlameGraph scripts Mar 25 09:21:26 it was most definitely because of upstream's net struct reduction Mar 25 09:21:26 it's really nice for tracking down what's causing the slowdown Mar 25 09:21:46 i backported that to 4.14 once. slow down. Mar 25 09:24:46 nbd: i'll finish my bisecting the way I started, but that tool looks interesting, may be worth using once I get a list of breaking commits/patches, thanks Mar 25 09:25:08 or you could just use it on the latest version Mar 25 09:25:13 and see which part takes up the most time Mar 25 09:56:12 rmilecki: lots of mitigations have also been enabled (spectre etc) .. I can imagine these will also contribute to performance hits .. Mar 25 09:56:18 ynezz: if wanted, I can move the buildbot irc bot back here, just wanted to avoid spamming Mar 25 09:57:21 jow: would it be possible to have best of both worlds? making a translation from real-name to nick? this would allow to be triggered from another channel, while avoiding the spamming Mar 25 09:57:45 meaning: adding the nick iso replacing it Mar 25 09:57:57 uhm, not sure if this is possible Mar 25 09:58:09 we would need to maintain some database for that somewhere Mar 25 09:58:19 contact page from openwrt? Mar 25 09:59:07 jow: I think openwrt-devel-build just needs a bit more publicity that's all Mar 25 10:19:59 well, if you're not interested, there's just /ignore Mar 25 10:37:59 [Mon 2019-03-25 02:57:20 AM PDT] jow: would it be possible to have best of both worlds? making a translation from real-name to nick? this would allow to be triggered from another channel, while avoiding the spamming <------ you can set this in your irc client, no need to do it server-side Mar 25 10:39:08 but the real problem with a separate #openwrt-devel-build channel is that people have to actually join it Mar 25 10:41:20 and if you're going to make the bot translate to irc nick anyway, might as well make it use memoserv instead of just highlighting people Mar 25 10:42:30 it's easy to turn on e-mail notifications of memoserv msgs Mar 25 10:44:55 isn't it overkill? I would just leave the bot here, if you don't like the "spam", then just /ignore it? Mar 25 10:45:04 seems a bit easier solution to me :) Mar 25 10:46:32 A bot that needs even speculation of "use ignore" sounds generally bad idea Mar 25 10:46:56 not that I've read too far what you are trying to achieve, but in general :P Mar 25 10:47:24 I personally also like the broken buildstuff being posted here. It really rings a bell loud and clear to get build breakages fixed asap. Mar 25 10:48:57 DonkeyHotei: thanks for the hint! Mar 25 10:49:26 fyi, new kernel bumps pushed to staging Mar 25 10:49:39 mm... a bot that has something real to say is generally good.. butalso needs smart way of doing it.. instead of, for example, 1000 lines insame second that every imaginable platform buildfailed ;) Mar 25 10:50:15 also a good point Mar 25 10:51:17 I'm just a bit afraid that a seperate channel will cause it to be neglected a bit Mar 25 10:51:41 yes Mar 25 10:51:59 well.. in irc context most likely that would happend... Mar 25 10:58:43 i'm also ok with bot posting here Mar 25 11:02:18 olmari: this is #openwrt-devel and build failures are natural part of the development, so I don't see a point to have another channel for the build failures Mar 25 11:02:41 I didn't suggest another chanel :P Mar 25 11:04:10 ok, then I've misread your comment "A bot that needs even speculation of "use ignore" sounds generally bad idea" :) Mar 25 11:05:13 ok, will move it back here then when I touch the cluster again Mar 25 11:09:46 jow: in general maybe need to think what is the things we really want to see here.. For example I kind of think something like "build success" is at least something that is expected and not needed to report separately :) Mar 25 11:13:56 olmari: only state changes are reported, so build success is only posted if the previous build of the same target was broken Mar 25 11:14:17 if a target is always broken or always ok, nothing will get reported at all Mar 25 11:14:40 mm Mar 25 11:15:04 sounds reasonable, assuming a ok message is even beneficial in public :) Mar 25 11:16:44 ..ogcourser, as pointed out by someone, this IS devel channel, so this type of messages ought to be expected Mar 25 12:12:41 does anyone know where sysupgrade fetches the info to know the partition it should flash to? Mar 25 12:21:15 I'm currently adding support for Mikrotik RB922 boards, but sysupgrade fails with "Could not open mtd device: firmware" I tried on a RB2011 which has fully identical storage layout and naming, and over there it works nicely (no mtd partition is called "firmware" here too) Mar 25 12:32:38 from /proc/mtd I would guess Mar 25 12:33:44 hm, but this error seems to come from something else then mtd Mar 25 12:34:07 no, it's in mtd as well Mar 25 12:36:35 target/linux/ath79/base-files/lib/upgrade/platform.sh:PART_NAME=firmware Mar 25 12:37:29 package/base-files/files/lib/upgrade/common.sh: get_image "$1" "$2" | mtd $MTD_ARGS write - "${PART_NAME:-image}" Mar 25 12:43:26 blocktrron: https://patchwork.ozlabs.org/patch/1060214/ is it going to work with `m25p80@0` -> `flash@0` change ? Mar 25 12:46:42 the bootloader has no fdt support, so yes Mar 25 12:47:29 ynezz: ^ Mar 25 12:51:19 ynezz: i found the error. thanks for the hints! Mar 25 13:19:39 It seemed a full rebuild was needed for platform.sh to be updated inside the images .. Mar 25 13:21:20 rsyncing /home of my workstation/buildbox to a new SSD. {build,staging}_dir is taking ages, it's probably faster to skip it and rebuild :P Mar 25 13:23:56 indeed Mar 25 13:37:58 and get a cleanup of old directories for free :) Mar 25 14:01:57 blocktrron: can you check this pls if its fine with you https://git.openwrt.org/?p=openwrt/staging/ynezz.git;a=commit;h=dcd2e1803838c4c1dc267d34396b3c752ce50ac6 ? Mar 25 14:04:47 stintel: just make sure you keep the dl directory :) Mar 25 14:19:36 stintel, 0/ Mar 25 14:24:37 karlp: yeah Mar 25 14:24:39 nitroshift: o/ Mar 25 14:28:17 ynezz: lgtm Mar 25 14:29:18 Regarding the LED trigger definition: I've had a discussion about this topic a few months back on IRC. The standpoint then was to match the definition of the target. Mar 25 14:29:51 heh, found 2 WD RE4 2TB drives in my workstation. not connected, just sitting there gathering dust Mar 25 14:32:06 I'm fine with it either way, however as the userspace definition is only applied on initial bootup, it would be great to have this correct in the first go. As ramips seems to use userspace definitions exclusively (except for one box) i would prefer to keep this in line (in case this standpoint hasn't changed) Mar 25 14:33:06 Otherwise we might end up with an inconsistent mess Mar 25 14:47:48 blocktrron: I still don't follow, isn't the UCI approach doing the same but later then the DTS approach? Mar 25 14:49:25 and from my point of view that DTS approach should be preffered as it's more upstream ready, isn't it? Mar 25 14:57:06 ynezz: in case the trigger definition is moved from the DTS to the userspace, the LED entries won't be created on a sysupgrade preserving the configuration. While the DTS approach is more upstream ready, the userspace is the way it's currently done for the ramips target as a whole and i would stick with that. Mar 25 14:58:35 However, i'm not sure if there is any consensus on this topic or if it's just a relict of the rt28xx driver not exposing led trigger functionality. Mar 25 14:59:05 well, we're requesting this DTS approach in every new PR on GitHub Mar 25 15:02:13 git grep linux,default-trigger target/linux/ramips/ | wc -l yields number 70 Mar 25 15:02:45 as i said, if there was any consensus made on this topic in the meantime i'm fine with putting it in the device-tree. I was told to use the userspace back when i was switching the mt76 based boards away from the netdev triggers. Mar 25 15:04:46 these are all USB ports (except for one CreativeBox), not radios. Mar 25 15:05:16 well, I can see that mkresin has commited that only board with radios, so he either overlooked it or is fine with this DTS approach as well (or at least isn't against it) Mar 25 15:05:16 Maybe this is already inconsistent mess depending from where you take a look at it Mar 25 15:05:51 Hauke: just noticed the 3.18 removal stuff in your staging. I'll stop bumping 3.18 from now on :) Mar 25 15:06:22 blocktrron: indeed, we should probably do some cleanup Mar 25 15:08:07 blocktrron: so can I push it as it is in my staging repo or you would prefer to wait for someone else who might be fine with this uci based LED trigger approach? :) Mar 25 15:08:54 xback: all the targets on kernel 3.18 are looking unmainated Mar 25 15:09:28 ynezz: go ahead. i was just curious Mar 25 15:10:04 I'm now curious as well :) Mar 25 15:10:14 mkresin: ping Mar 25 15:10:29 hauke: fully agree on the removal. I kept it updated to lower efforts if someone wanted to upgrade them .. but it looks dead indeed Mar 25 15:37:58 xback: will you also update kernel 4.19? Mar 25 15:38:38 xback: when I saw some problems with kernel 3.18 in most cases I just deactivated this fetaure for kernel 3.18 in the last year Mar 25 15:41:30 Hauke: i might pick up ppc44x when i have some itme Mar 25 15:41:38 but agree with the removal for now Mar 25 15:48:47 wigyori: ok that would be nice Mar 25 15:50:20 Hauke: i'll update it later this week. Mar 25 15:50:42 xback: ok thanks Mar 25 15:51:04 Hauke: there are some changes regarding flow-offload which need proper testing, and also some people dropped patches for 4.19. So i'm giving it some time to avoid them having to rebase all the time Mar 25 15:51:27 ok Mar 25 16:00:18 Hauke: yeah - ppc44x makes some sense these days, ppc40x probably doesn't Mar 25 16:47:52 Anybody using wil6210 driver? Mar 25 16:49:16 I am mostly done with Mikrotiks LHGG60-ad, but ping times and performance on wil6210 is not good at all. Ping fluctuates really from 20ms to 40++ms. Mikrotik does have a custom board file and use newer firmware than available in linux-firmware but upstream wil6210 will not load them as they lack headers Mar 25 17:04:09 400+ ms Mar 25 17:16:37 help Mar 25 17:44:16 If it wasn't you, I'd be saying "Perhaps you shouldn't be buying Ethernet cables at 10 for a dollar" Mar 25 18:28:46 https://ae01.alicdn.com/kf/HTB1Ys.QSFXXXXb1aXXXq6xXFXXXl/CAT-6-10cm-50cm-0-1m-0-5m-CAT5e-rj45-cable-CAT6e-UTP-Ethernet-Network-Cable.jpg_640x640.jpg Mar 25 18:46:58 greearb_: ping Mar 25 18:48:00 greearb_: I just finished my support for Mikrotik rb922 which allows me to test Wave1 (AR988x) on a proper scale (> 10 boards) Mar 25 18:48:24 greearb_: On first boot, setting the interface into IBSS mode shows a dump: https://pastebin.com/RHg9SJRT Mar 25 18:48:52 ath10k_ct is used, together with the -htt flavour. repro rate is 100% Mar 25 18:52:36 xback, what git commit of ath10k-ct driver are you using? Mar 25 18:53:18 greearb_: latest state in OpenWrt. not including the updates you sent an hour ago Mar 25 18:53:25 openwrt master Mar 25 18:53:41 xback, let me know what that is so I can check exactly what code you are using? Mar 25 18:54:51 and do you know what backports kernel you are using? If it is 4.20 or higher, maybe it is worth moving the ath10k-ct driver to the 4.20 tree since that is my main testing target Mar 25 18:55:20 greearb_: 9360f389234a58f742c6cb3f8eea5a419c7523f6 Mar 25 18:56:41 please try latest ath10k-ct driver, I think that will help, and if not, at least we'd be debugging recent code Mar 25 18:56:48 2e917efb607fdc807a8ec6e9ad8de176fc51d329 Mar 25 18:57:02 will do Mar 25 19:07:24 The very latest ath10k-ct driver and FW work fine on my 988x, but I'm just using AP mode and not IBSS. Mar 25 19:09:07 just build it :) flashing to board atm Mar 25 19:12:45 greearb_: still the same on the latest commit: https://pastebin.com/FrVyB8gQ Mar 25 19:14:46 maybe I screwed up the 4.19 backport of my 4.20 patches Mar 25 19:15:19 I can test that tomorrow, and if you have time, you could try using the 4.20 version of my driver instead of 4.19 that is currently being picked. Mar 25 19:15:37 ath10k-ct-2019-03-25-2e917efb/ath10k-4.19/mac.c:6455 Mar 25 19:15:46 [ 46.733528] [<8708bbc0>] ath10k_mac_vif_beacon_free+0xb8c/0x1008 [ath10k_core] Mar 25 19:16:03 greearb_: will do :) Mar 25 19:16:45 I didn't test that beacon thing in ibss mode either, so possibly that is the difference Mar 25 19:17:33 AP and IBSS *should* be treated the same for beacon deletion though I would think Mar 25 19:26:09 greearb_: after 15 minutes, a lot of these come up: [ 871.592347] ath10k_pci 0000:01:00.0: SWBA overrun on vdev 0, skipped old beacon Mar 25 19:38:11 xback, but otherwise moving to 4.20 fixed the warning splat at least? Mar 25 19:40:53 greearb_: this is on the latest 4.19. will try 4.20 tomorrow Mar 25 19:41:32 so it works even after the splat then? Mar 25 19:41:58 currently trying to setup a 2nd board before I leave, to check if they connect up Mar 25 19:42:06 nod Mar 25 19:43:46 greearb_: both boards connect through IBSS Mar 25 19:45:13 greearb_: but the speed is dramatically. just a few kbit/s running iperf Mar 25 19:45:26 that's it for today. will test further tomorrow Mar 25 19:45:29 nod Mar 25 21:09:44 For a 4.19 build of ath79, from a mechanics point of view, changing target/linux/ath79/Makefile KERNEL_PATCHVER is sufficient, yes? Mar 25 21:13:21 jeffsf: yes Mar 25 21:13:41 Thanks ldir -- steam coming out of my build box now Mar 25 21:14:15 Will see how it works on Archer C7 with 802.11s and BATMAN soon Mar 25 21:14:45 If you hear my wife screaming from there, you'll know it wasn't as successful as I hoped ;) Mar 25 21:15:08 good luck - my 4.19 build failed due to some local patches that I've still not fixed up Mar 25 21:15:21 Thanks -- actually have a couple spares Mar 25 22:03:00 mangix: do you've somewhere dmesg from gb-pc2 running some latest snapshot? Mar 25 22:16:58 Is there a make target that builds all the host components (so I can parallelize that, then parallelize the target components)? Mar 25 22:24:16 jeffsf: make tools/install toolchain/install target/compile ? Mar 25 22:24:21 Thanks Mar 25 22:24:52 Some strange timing thing that my normal `make -j12 clean download world` is tripping over Mar 25 22:29:05 No, problem compiling batman-adv -- looking into it now Mar 25 22:29:22 (for ath79 Linux 4.19) Mar 25 22:31:18 `compat-hacks.h` -- LOL, heavy on the hack, light on the compat, it seems! Mar 25 22:42:01 ynezz: eh not exactly. i have one from my branch Mar 25 22:44:39 i have the upstream spi driver as well as the spi speed increase Mar 25 23:07:49 4.19 built and flashed okay on a tplink wdr3600 Mar 25 23:13:05 Looks like the problem is *not* 4.19, as a 4.9 build fails at the same point (batman-adv) Mar 26 00:08:49 batman-adv was building yesterday afaik Mar 26 00:09:15 Yeah, strange, "phase of the moon" -- been trying to bisect, now is working everywhere I've tried Mar 26 00:20:57 after some initial poking, it's always worth a "make dirclean world" to avoid too many goose chases Mar 26 00:21:34 Yeah, going to sleep on it, since it's breaking again. Need to walk through things more carefully Mar 26 01:36:19 jow: know why this was rejected? https://patchwork.ozlabs.org/patch/1058103/ **** ENDING LOGGING AT Tue Mar 26 02:59:56 2019