**** BEGIN LOGGING AT Thu Dec 19 02:59:59 2013 Dec 19 03:44:27 is it normal that with trunk from a few days ago I lose internet connectivity as soon as I run sysupgrade ? Dec 19 03:44:57 I know dns stops working, but active ssh session blocks immediately, that didn't happen before afair Dec 19 03:55:53 also still having high ksoftirqd/0 when doing a speedtest on WAN with qos-scripts enabled. with qos-scripts enabled, peaks @ 115Mbps, qos-scripts disabled @ 141Mbps. it's an online speedtest site at my ISP, and it's consistent - every time I disable qos-scripts speed goes up. Dec 19 03:56:35 on rspro with trunk that is. reverted kernel back to 3.10.13 and still seeing it Dec 19 04:02:16 also revert changes to TX and RX buffer size in ag71xx driver, didn't help either. now tested 3.10.12, still an issue :/ Dec 19 04:13:39 without qos-scripts @ ~140Mbps I see 3% CPU on ksoftirqd Dec 19 04:15:21 with qos-scripts enabled @ ~106Mbps I see up to 50% CPU on ksoftirqd Dec 19 04:16:05 so that's quite a difference. if anyone has a suggestion what I can try to revert .. Dec 19 04:58:54 build #424 of ar7 is complete: Failure [failed shell_14] Build details are at http://buildbot.openwrt.org:8010/builders/ar7/builds/424 Dec 19 07:23:07 build #349 of ep93xx is complete: Failure [failed shell_14] Build details are at http://buildbot.openwrt.org:8010/builders/ep93xx/builds/349 Dec 19 09:30:27 nbd: [florian]: hi guys, could you take a look at my e-mail thread "Updating vlan interface link on switch port link change?" please? Dec 19 09:33:12 zajec: its tricky Dec 19 09:33:27 zajec: first of all there is no real conenction between a switch and a netdev Dec 19 09:34:03 zajec: then you need to mask out the cpu port Dec 19 09:34:24 and only consider a vlan iface to be down if all contained ports (except the cpu port) are down Dec 19 09:34:41 and why should a port on the local switch be different than a port on a remote switch breaking out the same vlan.... ;) Dec 19 09:34:59 the biggest problem I see is that there is no programamtic way to map a switch to eth0, eth1 etc. Dec 19 09:35:47 also link state doesnt say much about the state of whats behind... what do you connect there usually? a dsl modem or a cable modem? these dont change the eth link state, regardless if they are sync, training or dont even have a line pulse Dec 19 09:36:55 roh: well people expect that openwrt magically renews its ip address (or gets ifdown, ifup events) if they pull the ethernet cable Dec 19 09:37:25 so your link state should rather be tied to the pppoe or whatever makes the wan work Dec 19 09:37:36 jow_laptop: well.. that doesnt work for a speedport either ;) Dec 19 09:37:46 I know Dec 19 09:38:29 * zajec reading Dec 19 09:38:47 zajec: what do you do on the wan port? (protocol) pppoe? dhcp? Dec 19 09:38:53 dhcp Dec 19 09:39:03 i just want IP to be updated as I switch between ISPs Dec 19 09:39:14 then set the dhcp refresh time ridicilous low, and see if that fixes your problem Dec 19 09:39:15 "ip" (gateway, mask, etc. too ofc) Dec 19 09:39:37 switch isp? Dec 19 09:40:11 zajec: the only way I see that does not involve a lot of cross-layer hackery is creating a custom daemon (maybe a modified variant of lsm) which watches switch port state Dec 19 09:40:21 Let's say I have WAN connected to ISP#1 and i want (for whatever reason) switch to my ISP#2 Dec 19 09:41:00 and then issues an ifup if it sees an intermediate link down on all (sans cpu) ports of a vlan Dec 19 09:41:30 the next practical hurdle is that not all switch drivers support port link state Dec 19 09:41:44 so its not a feature you can universally rely on Dec 19 09:42:22 oh Dec 19 09:42:27 that's becoming complex :| Dec 19 09:42:35 why replug cables? just ifdown wan1 and ifup wan2 Dec 19 09:42:36 thats why I said its tricky Dec 19 09:42:48 wan is just a label for some vlan with some config on an interface Dec 19 09:42:49 at a first glance it appears simple, but its extremely difficult Dec 19 09:42:53 roh: i want olny one port for be used as WAN Dec 19 09:43:12 roh: i don't want to loose another switch port Dec 19 09:43:28 roh: I can somewhat understand the reasoning, I too had the requirement in the past to update wan dhcp without ssh'ing to the box and without powercycling it Dec 19 09:43:45 jow_laptop: exactly ;) Dec 19 09:43:49 jow_laptop: did you solve it somehow? Dec 19 09:44:06 zajec: no, in the end we just told the users to reboot the box if they switch cables Dec 19 09:44:21 jow_laptop: well.. as long as the isp doesnt filter/block on more frequent dhcp lease updates... do them every 30seconds or so ;) Dec 19 09:44:45 roh: there seems to be at least one cable isp in the states which is extremly picky about too frequent dhcp requests Dec 19 09:44:48 i just dont see if that can be changed without changing source Dec 19 09:44:52 roh: yeah, it's kind of workaround that could work... just a bit more exepnsive Dec 19 09:45:28 roh: do you mean OpenWrt source? Dec 19 09:46:21 well.. whats source.. config, patches, upstream source... dunno. something which needs recompiling is what i meant Dec 19 09:46:46 zajec: a workaround is something like while true; do if swconfig dev switch0 port 0 get link | grep -q down; then ifup wan; fi; sleep 5; done Dec 19 09:46:49 roh: oh come on, it's not a problem to change a source... it's open source after all Dec 19 09:47:03 or something similar Dec 19 09:47:09 jow_laptop: i get your point, thanks Dec 19 09:47:09 of course it needs hysteresis etc. Dec 19 09:47:14 so best done in C Dec 19 09:47:48 jow_laptop: well.. in the end it _would_ be nice to get events from the switch.. and if we had that, one _could_ hook custom scripts on these ;) Dec 19 09:48:23 roh: that requires continuous polling of the switch registers, so it can be as well be made from a daemon Dec 19 09:48:28 * roh would be rather interrested in logging link changes or similar stuff from switches... the port counters etc... Dec 19 09:48:28 jow_laptop: new word for today: "hysteresis" :p Dec 19 09:48:54 jow_laptop: jep.. but i guess that would need some switch support in netifd, right? Dec 19 09:49:08 and since the swconfig <-> kernel communication is netlink it also fairly easy to bypass swconfig cli and do it in an efficient manner Dec 19 09:49:11 jow_laptop: normally PHY subsystem also does polling... so that's not an issue Dec 19 09:49:13 and why polling? from what i read most switches can generate interrupts on the mii link? Dec 19 09:49:24 jow_laptop: but I would got the simples way first, so I'll just write a deamon for my tests Dec 19 09:49:35 atleast for some events, like error counters rising, link state changing, autoneg completing etc. Dec 19 09:49:48 roh: ok didn't know Dec 19 09:49:59 dunno about the low-end stoneage brcm stuff, but atleast ar8327 can do most of those Dec 19 09:50:01 but then I foresee it being only implemented for one or two switches Dec 19 09:50:07 roh: i had no idea about MII interrupts Dec 19 09:51:08 jow_laptop: well.. one needs to start somewhere... and maybe we can use openwrt for real switches at some time ;) Dec 19 09:51:16 not just boring plastic routers Dec 19 09:51:43 ;) Dec 19 09:53:14 zajec: thinking further about it... you must also not issue an ifup if the vlan is part of a bridge together with other ifaces Dec 19 09:53:27 zajec: and neither if the proto of the iface on top is static or none Dec 19 09:53:40 it does not make much sense to "reconnect" a static or unmanaged iface Dec 19 09:54:21 :| Dec 19 09:56:09 but if you make the switch port link state monitoring configureable its easy to only watch the wan port and configure an ifup wan trigger for that Dec 19 09:56:13 jow_laptop: ping Dec 19 09:56:17 oh hey :) Dec 19 09:56:20 'lo Dec 19 09:57:02 oh someone commented on my ticket already Dec 19 09:57:07 wondering if you saw https://dev.openwrt.org/ticket/14405#comment:9 Dec 19 09:57:51 DJJeff: "same here" ? Dec 19 09:58:33 jow_laptop: what do you mean by making monitoring configureable? Dec 19 09:58:44 DJJeff: at least I see nothing of interest in comment 9 Dec 19 09:59:03 a temp solution could be to take the DHCP address and put it in as a static IP Dec 19 09:59:09 so that the firewall does not reload Dec 19 09:59:13 zajec: supposing there is a daemon watching port link state, one could configure it to tell which ports to monitor Dec 19 09:59:27 also it does ignore /etc/firewall.user Dec 19 09:59:41 when it gets reloaded due to ifup Dec 19 09:59:43 DJJeff: firewall reloads ignore firewall.user for a reason Dec 19 09:59:56 a firewall re*load* only clears internal uci chains and rebuilds them Dec 19 10:00:06 it leaves user rules and custom stuff in INPUT, OUTPUT etc. alone Dec 19 10:00:32 if it'd re-run firewall.user on realods it would accumulate duplicate user rules very quickly Dec 19 10:01:00 I put some custom IP's in firewall.user to DROP them Dec 19 10:01:12 but you likely moved those rules into an internal chain Dec 19 10:01:13 when it was getting re*load* Dec 19 10:01:24 the got lost Dec 19 10:01:29 show me the rules Dec 19 10:01:33 ok just sec Dec 19 10:02:49 iptables -I zone_wan_forward -s $ip -j DROP Dec 19 10:02:49 iptables -I zone_lan_forward -d $ip -j DROP Dec 19 10:02:57 yes, thats an internal chain Dec 19 10:03:19 and it was getting removed on reload Dec 19 10:03:27 use forwarding_wan_rule Dec 19 10:03:52 http://wiki.openwrt.org/doc/uci/firewall#packet.flow Dec 19 10:04:02 you can use chains of type system or user Dec 19 10:04:10 internal chains are deleted and recreated on reloads Dec 19 10:04:37 reloading resets the counters too which was annoying Dec 19 10:04:44 but I can deal with that... sorta Dec 19 10:05:22 reloading does only clears the coutners of internal chains because they're recreated Dec 19 10:05:26 again Dec 19 10:05:32 all rules outside of internal chains are not touched Dec 19 10:06:15 ah forwarding_wan_rule is rule #1 of zone_wan_forward Dec 19 10:06:20 I get it now... Dec 19 10:06:59 http://pastebin.com/z59mN0W1 Dec 19 10:06:59 and lan_rule for lan_forward Dec 19 10:07:51 *_rule chains are save Dec 19 10:08:06 in the sense that they're not destroyed and rebuilt on reloads Dec 19 10:08:21 so whatever you add there stays until you isse a firewall stop or restart Dec 19 10:08:58 running /etc/init.d/firewall restart is not the same as a reload? Dec 19 10:09:13 no Dec 19 10:09:33 a restart is a complete flush followed by a complete rebuild of the ruleset Dec 19 10:09:48 a reload just updates internal rules dealing with interfaces and their ips Dec 19 10:11:09 awesome thanks! Dec 19 10:13:55 jow_laptop: i think i still don't understand 2 things... Dec 19 10:13:57 1) What's wrong with calling "ifup" for static or none interface? It won't help ofc, but is it going to hurt anyone? Dec 19 10:13:58 2) Even if my (let's say) eth0.2 is bridged with eth0.4, what's wrong with calling "ifup eth0.2" after port going down? Dec 19 10:20:35 build #378 of au1000 is complete: Success [build successful] Build details are at http://buildbot.openwrt.org:8010/builders/au1000/builds/378 Dec 19 10:21:27 zajec: it triggers an hotplug event which in turn might cause random services to restart Dec 19 10:21:38 so imagine a lan vlan bridged with wifi access points Dec 19 10:22:05 if some switch port goes down (maybe because a client pc was turned off) a lot of services will suddenly restart, potentially breaking sessions of wifi users Dec 19 10:22:33 ouch Dec 19 10:22:34 ;) Dec 19 11:35:59 jogo r39136 packages/net/ znc/patches/002-Uclibcpp_build_fix.patch znc/patches/001-move_rootcheck_after_config.patch znc/patches/003-Reduce_rebuild_time.patch znc/Makefile * packages: znc: update to 1.2 Dec 19 12:55:54 nbd: is there a way to debug why openvpn doesnt come up? i have no output in logread and the daemon doesnt come up Dec 19 13:32:52 nbd r39137 packages/libs/libzstream/Makefile * libzstream: fix build failure by removing pedantic errors Dec 19 14:06:22 nbd: ping Dec 19 15:09:16 tripolar turn up debugging verbosity Dec 19 15:24:29 i think i found an bug in netifd - when i start the device - it starts some bridges and openvpn tunnel - the device isnt reachable anymore at the br-lan, flashing iptables or restarting the firewall works and i can reach the device again - i think one fw restart is missing while the device comes up Dec 19 15:31:48 this is my fw config Dec 19 15:31:48 http://paste.debian.net/71786/ Dec 19 15:32:28 i drop everything except ssh Dec 19 15:56:30 stintel: pong Dec 19 15:58:26 nbd: could you have quick log at what I posted here last night? http://pastebin.com/wHfQ6DA1 since you work both on qos-scripts and ag71xx driver I thought you might have an idea Dec 19 15:58:35 if not, sorry for disturbing :) Dec 19 15:59:58 it's normal that qos adds quite a bit of overhead on the routing path Dec 19 16:00:25 and due to the extra latency, it's also normal for the reported speed in speedtest to go down Dec 19 16:06:28 hmmm, but I'm quite sure that speedtest gave me better results before, with qos enabled Dec 19 16:13:47 stintel: when was 'before'? Dec 19 16:15:36 nbd: well, that's difficult to say. at parent's house we had 150Mbps down, but I moved out 30/06/13. my connection in appartment was *only* 120Mbps down until 24/09, then it was also upgraded to 150. not sure if before was here, or still at parents. Dec 19 16:16:06 I'll do some testing on my 2nd rspro Dec 19 16:16:14 start with AA and test it there Dec 19 16:27:50 stintel: if use webbased speedtest thingies, i dont think it makes sense wasting anybodys time debugging. Dec 19 16:35:00 roh: it's hosted at my ISP, and the difference in result is consistent Dec 19 16:35:14 with qos-enabled it's always 25-40Mbps slower Dec 19 16:35:34 qos doesnt mean 'faster' Dec 19 16:35:43 I know that Dec 19 16:35:47 it means 'more reliable' or more solid latency Dec 19 16:35:56 but it shouldn't limit bandwidth by that much Dec 19 16:36:41 speedtest does not measure 'bandwidth' Dec 19 16:36:59 iirc it measures the throughput of a TCP connection Dec 19 16:37:07 that's not the same thing at all Dec 19 16:37:11 TCP is latency sensitive Dec 19 16:37:20 also... latency a bit, but rather bad Dec 19 16:37:50 quite optimistic 'schaetzeisen' .. good to impress simple minded customers who want so see some 'meter in the red' Dec 19 16:38:15 it's comparable to a signal strength display that uses '%' as unit ;) Dec 19 16:38:21 hrrr Dec 19 16:39:19 bah, implementing multi-channel support in ath9k is making my head spin... Dec 19 16:41:00 well, it shouldn't limit throughput of my TCP connections by that much then. all I know is something changed, and the speedtest result is consistent with that Dec 19 16:47:43 testing with iperf now, put a device on the wan side of my router, same result Dec 19 16:50:29 qos off now peaks at 166Mbps as it's not limited by my wan connection. so if my connection is upgraded again I'll need a new router anyway. hmmm Dec 19 17:20:26 nbd r39138 trunk/package/kernel/mac80211/patches/ (17 files) * ath9k: merge another round of upstream (or -pending) fixes and cleanups Dec 19 17:20:31 nbd r39139 trunk/target/linux/ar71xx/patches-3.10/612-MIPS-ath79-set-buffalo-txgain.patch * ar71xx: add missing tx gain table flag for buffalo wzr-hp-g450h Dec 19 18:03:28 build #443 of uml is complete: Failure [failed compile_5] Build details are at http://buildbot.openwrt.org:8010/builders/uml/builds/443 Dec 19 18:06:11 build #432 of rb532 is complete: Failure [failed shell_12] Build details are at http://buildbot.openwrt.org:8010/builders/rb532/builds/432 Dec 19 18:08:44 build #432 of ppc44x is complete: Failure [failed shell_12] Build details are at http://buildbot.openwrt.org:8010/builders/ppc44x/builds/432 Dec 19 19:08:50 nbd: just fyi, with my 2nd rspro, same image as the "problematic" one, but with wiped settings, just qos enabled with same speeds as my primary router, I get ~141Mbps, so must be something in the settings of the primary router Dec 19 19:09:23 so it's probably some setting instead Dec 19 19:10:11 ksoftirqd goes even higher on this one Dec 19 19:23:05 nbd: I think I found it, restored backup from primary router on 2nd rspro, same lower speed. then I removed about all firewall rules, and it's going over 130Mbps now Dec 19 19:23:14 so yes, I need a faster router :) Dec 19 19:23:24 :) Dec 19 19:23:34 next year i will rewrite qos-scripts Dec 19 19:23:49 and the rewrite will use netfilter as little as possible Dec 19 19:23:56 that should also make it faster Dec 19 19:24:07 good to know Dec 19 19:24:17 what about the new nftables in kernel 3.13 Dec 19 19:24:21 is that on some roadmap ? :) Dec 19 19:24:32 I wonder if it will be more efficient Dec 19 19:24:37 i'm looking forward to giving that a try Dec 19 19:24:49 once 3.13 is done Dec 19 19:26:04 this looks like a nice device: http://www.gateworks.com/product/item/ventana-gw5410-network-processor Dec 19 19:26:54 also, very expensive Dec 19 19:27:09 but yes, nice device Dec 19 19:27:12 probably, I only see development kit, and last I remember from previous gateworks devkits Dec 19 19:27:19 iirc > 1000USD or so Dec 19 19:28:08 oh well, I'll start with cleaning up some old firewall rules. maybe it will get better again Dec 19 19:28:39 I guess 6 zones don't help either :? Dec 19 19:29:30 at least I'm glad to know it's not a regression Dec 19 19:55:22 build #469 of orion is complete: Failure [failed compile_8] Build details are at http://buildbot.openwrt.org:8010/builders/orion/builds/469 Dec 19 20:41:59 mhei r39140 packages/lang/php5/Makefile * [packages] php5: upgrade to 5.4.23 Dec 19 21:38:46 build #457 of cobalt is complete: Failure [failed shell_14] Build details are at http://buildbot.openwrt.org:8010/builders/cobalt/builds/457 Dec 19 21:50:58 build #459 of lantiq is complete: Failure [failed compile_5] Build details are at http://buildbot.openwrt.org:8010/builders/lantiq/builds/459 Dec 19 21:58:22 build #402 of sibyte is complete: Failure [failed shell_12] Build details are at http://buildbot.openwrt.org:8010/builders/sibyte/builds/402 Dec 20 00:00:35 build #32 of mxs is complete: Failure [failed compile_5] Build details are at http://buildbot.openwrt.org:8010/builders/mxs/builds/32 Dec 20 00:01:31 build #459 of ramips is complete: Success [build successful] Build details are at http://buildbot.openwrt.org:8010/builders/ramips/builds/459 Dec 20 00:22:44 build #475 of brcm47xx is complete: Success [build successful] Build details are at http://buildbot.openwrt.org:8010/builders/brcm47xx/builds/475 Dec 20 00:39:43 build #412 of avr32 is complete: Failure [failed compile_5] Build details are at http://buildbot.openwrt.org:8010/builders/avr32/builds/412 Dec 20 00:46:47 build #350 of iop32x is complete: Failure [failed shell_12] Build details are at http://buildbot.openwrt.org:8010/builders/iop32x/builds/350 Dec 20 01:17:31 build #467 of atheros is complete: Success [build successful] Build details are at http://buildbot.openwrt.org:8010/builders/atheros/builds/467 Dec 20 02:18:41 build #33 of mvebu is complete: Failure [failed compile_4] Build details are at http://buildbot.openwrt.org:8010/builders/mvebu/builds/33 **** ENDING LOGGING AT Fri Dec 20 03:00:00 2013