**** BEGIN LOGGING AT Fri Aug 16 03:05:29 2019 Aug 16 07:13:33 can someone explain that code to me, please? https://git.openwrt.org/?p=project/procd.git;a=blob;f=upgraded/upgraded.c;hb=HEAD Aug 16 07:13:40 uloop + process + reboot is obvious to me Aug 16 07:13:46 why do we need chroot + fchdir though? Aug 16 07:20:51 rmilecki: most likely to avoid the workdir being on the old rootfs Aug 16 07:21:26 jow: it seems to be handled by sysupgrade_exec_upgraded() which is /sbin/upgraded caller Aug 16 07:21:31 rmilecki: rememeber, the entire purpose of the two stage approach (make procd/init exec itself into upgraded) is to allow to free up the rootfs entirely fro umounting later Aug 16 07:21:37 https://lxr.openwrt.org/source/procd/sysupgrade.c#L25 Aug 16 07:22:15 rmilecki: won't help as at this point the process is still procd launched on boot, still having a reference on the old rootfs Aug 16 07:23:25 I /think/ that the extra chroot(".") in upgraded is meant as a sanity check Aug 16 07:23:33 to see if the current work directory still exists Aug 16 07:23:44 ok... Aug 16 07:24:09 jow: and that fchdir()? any idea about it? Aug 16 07:25:26 i'm wondering if that upgrade process could be slightly simplified Aug 16 07:25:27 could we get rid of /sbin/upgraded and just make sysupgrade_exec_upgraded() exec /lib/upgrade/stage2 directly? Aug 16 07:28:49 blogic: ^^ Aug 16 07:28:56 this won't work Aug 16 07:29:07 or at least it would complicate stage2 cnsiderably Aug 16 07:29:48 because whatever process ends up being the new pid 1 needs to ensure that it stays alive during the entire upgrade process (otherwise kernel panic), it needs to keep poking the watchdog Aug 16 07:29:56 jow: in theory there should be some more buildinfo files now, but i only see the config.buildinfo Aug 16 07:29:59 it needs to handle child signals, reap thm etc. Aug 16 07:30:50 https://github.com/openwrt/openwrt/commit/454021581f630d5d04afeb8ff6581c1bda295c87 Aug 16 07:30:59 rmilecki: so stage2 would need to somehow fork into two processes, one while true loop poking the watchdog fd and doing wait(pid) Aug 16 07:31:12 rmilecki: and the other doing the upgrade tasks Aug 16 07:33:04 jow: i'm sorry, I'm not so exeprienced with this Aug 16 07:33:08 so I can see that procd's sysupgrade_exec_upgraded() uses execvp("/sbin/upgraded"", argv); Aug 16 07:33:09 does it mean that /sbin/upgraded is becoming a new PID 1? Aug 16 07:33:15 rmilecki: yes Aug 16 07:33:50 jow: so what is so magic about /sbin/upgraded then? it seems to be just running uloop Aug 16 07:34:11 rmilecki: handling the watchdog fd Aug 16 07:34:23 and waitpid() the tasks to avoid zombies Aug 16 07:34:27 I admit I didn't look at watchdog yet Aug 16 07:34:33 and being alive so the kernel does not panic while stage2 runs Aug 16 07:35:34 jow: why can't we have procd handle watchdog, run uloop & fork to exec stage2? Aug 16 07:35:40 and ensuring that a reboot is triggered even when stage2 aborts Aug 16 07:36:00 rmilecki: because procd runs off the old rootfs, keeping an open handle to it Aug 16 07:36:04 this would prevent unmounting the old rootfs Aug 16 07:36:22 ahh, so basically /sbin/upgraded is there to let procd go away? Aug 16 07:36:26 yes Aug 16 07:36:32 that makes sense now, thank you! Aug 16 07:37:30 on a systemd desktop distro one would use "systemctl daemon-reexec" for the same purpose Aug 16 07:38:14 not that I know systemd ;) Aug 16 07:38:22 but everything makes more sense now Aug 16 07:38:53 and on sysvinit this can be done through inittab, by adding "::sysinit:/new/process" to /etc/inittab, then sending a special kill signal to pid 1 Aug 16 07:39:18 ok, nice to know Aug 16 07:39:42 so tl;dr - it is done to start a new pid 1 off another, non-rootfs fs (ramdisk, new mount, ...) to free up the old rootfs for umounting Aug 16 08:39:24 jow: i'd like to extend sysupgrade interface Aug 16 08:39:25 right now stage2 determines if config should be preserved by manually checking for the /tmp/sysupgrade.tgz existence Aug 16 08:39:27 i'd like that (and other potential options) to be specified explicitly Aug 16 08:39:29 I can add support for something like: Aug 16 08:39:30 ubus call system sysupgrade "{ ..., \"save_config\": 1 }" Aug 16 08:39:32 problem #1: adding any new option will require updating ubus method Aug 16 08:39:33 problem #2: passing many options will make code ugly in path: procd ubus handler -> sysupgrade_exec_upgraded() -> upgraded -> /lib/upgrade/stage2 Aug 16 08:39:35 another solution would be to use something like: Aug 16 08:39:36 ubus call system sysupgrade "{ ..., \"options\": { \"save_config\": 1 } }" Aug 16 08:39:38 then I could easily pass all options as a string Aug 16 08:39:39 disadvantage: we cannot list all possible options using ubus method policy Aug 16 08:47:10 hello guys Aug 16 08:47:16 rmilecki: ping Aug 16 08:47:22 rmilecki: pong Aug 16 08:48:52 rmilecki: :) :) ; I have a BCM device for which the manifacturer claims 512MB RAM memory, but the OEM CFE reports 64MB, and forcing the kernel to use the state memory results in crashes after some days. From where may I start my investigation? Infact, the DRAM chip tells the manifacturer sais the truth, but ... Aug 16 08:49:21 mrkiko: not sure if I can help you Aug 16 08:49:23 is that MIPS or ARM? Aug 16 08:49:29 MIPS Aug 16 08:49:50 rmilecki: well, my problem is to understand from where I may start analysing things ... Aug 16 08:50:22 mkresin: how did you focre kernel to use more RAM? Aug 16 08:50:29 rmilecki: I was trying to compare bootlogs from other devices I actually don't have, like RT AC66U Aug 16 08:50:52 * rmilecki is going afk for few minutes Aug 16 08:51:01 rmilecki: (e.g.: mem=128M) Aug 16 08:51:10 rmilecki: for procd ubus handler -> sysupgrade_exec_upgraded() I'd simply forward the blob Aug 16 08:51:30 rmilecki: for sysupgrade_exec_upgraded() -> upgraded I'd export the options in this blob message as env variables Aug 16 08:52:05 jow: does forwarding happen via ubus in C? Aug 16 08:52:09 e.g. { "foo": true, "bar": false } -> setenv("UPGRADE_OPT_FOO", "1"); setenv("UPGRADE_OPT_BAR", "0"); Aug 16 08:52:54 mrkiko: I'm afarid I do not understand your question Aug 16 08:53:48 jow: oh sorry, wasn't posed correctly. I was wondering how blobmsg messages are forwarded, what mechanisms are available. I know only very little about libblobmsg / blobjson Aug 16 08:54:39 mrkiko: well 64 megabytes is 512 megabits.. so.. (: Aug 16 08:58:16 mrkiko: besically they're not forwarded. With "forward" above I meant either passing the pointer to the static blob buffer along or duplicating the blob message into heap-allocated memory and pass the pointer to that to another function Aug 16 08:58:42 the usual design for ubus enabled services in openwrt is to have one global static blob buffer which holds the received message Aug 16 08:59:11 then, after the method finished executing the static blob buffer is reused for the next method call Aug 16 08:59:56 in case methods run concurrently, you need to duplicate the blob buffer into heap memory to keep using it privately since it could be overwritten any time Aug 16 09:05:02 jow: i'll try that, thanks! Aug 16 09:06:00 mrkiko: I don't think that doing mem=128M will make kernel automatically use 128 MB of RAM if it's only aware of 32 M or 64 M Aug 16 09:06:14 mrkiko: kernel has to know how to access RAM you request it to use Aug 16 09:06:31 mrkiko: i think mem= use may be limited to reducing amount of used RAM Aug 16 09:06:43 mrkiko: pastebin your dmesg Aug 16 09:07:03 rmilecki: it did, even tought infact effects where disastrous :D Aug 16 09:08:30 olmari: thanks, infact I'll look at this thing Aug 16 09:12:46 rmilecki: http://www.gstorm.eu/dmesg.txt Aug 16 09:14:02 olmari: thank you, I'll look at it.... Aug 16 09:14:52 mrkiko: Linux also detects 64 M of RAM, so it may be what's really there Aug 16 09:15:04 mrkiko: Linux can't support more than 128 MiB Aug 16 09:15:15 but 128 MiB can be supported for sure Aug 16 09:15:31 if you have more than 128 MiB, kernel will print a warning it cannot support that Aug 16 09:19:11 why this 128 MB Limit? Aug 16 09:22:07 rubberduck: higher memory has to be addressed differently Aug 16 09:22:12 i did'nt have to to find out how Aug 16 09:22:40 there are very few bcm47xx MIPS models with more than 128 MiB RAM Aug 16 09:22:46 I saw one or two Aug 16 09:23:41 rubberduck: feel free to send a patch for arch/mips/bcm47xx/prom.c Aug 16 09:26:59 jow: so - just for sure - are we talking about something like Aug 16 09:27:01 { "prefix": "...", "path": "...", "command": "...", "options": { "foo": true, "bar": false } } Aug 16 09:36:42 * russell-- guesses mrkiko is confusing bits for bytes Aug 16 09:37:26 rmilecki: thank you very very much Aug 16 09:37:35 russell--: yeah, I think so as well, as olmari righly suggested Aug 16 09:37:59 I don't have direct access to informations printed on chip, I am blind BTW, and I missed the knowledge that rmilecki gave :D Aug 16 09:38:21 rmilecki: even tought I guess he already instructed me on something like that when I faced the same issue in this channel, so sorry for the noise guys Aug 16 09:38:32 no worries Aug 16 09:38:35 no prob :0 Aug 16 09:38:37 :) Aug 16 09:41:04 rmilecki: yes Aug 16 09:41:30 jow: do you expect translation of options into env variable to be handled manually or automatically? Aug 16 09:41:47 erm automatically Aug 16 09:41:58 let's see what I can do :) Aug 16 09:45:23 char *p = namebuf + snprintf(namebuf, sizeof(namebuf), "UPGRADE_OPT_"); char *name = blobmsg_name(cur); while (*name) *p++ = toupper(*name++); setenv(namebuf, blobmsg_get_string(cur), 1); Aug 16 09:45:27 something along these lines Aug 16 09:46:29 (for each blob attr in "options": { ... }) Aug 16 09:47:36 however - you should probably restrict this to bool and numeric blob attr types Aug 16 09:48:05 if you allow free formed strings there's a certain potential to inject malicious code which might or might not be properly handled by the shell code Aug 16 09:48:28 e.g. '{ ..., "options": { "foo": "$(rm -rf /)" } } Aug 16 09:49:58 ok Aug 16 09:50:23 * rmilecki thinks about adding enum value BLOBMSG_TYPE_OBJECT = BLOBMSG_TYPE_TABLE for years now Aug 16 09:50:33 i can never remember BLOBMSG_TYPE_TABLE vs. BLOBMSG_TYPE_ARRAY difference Aug 16 09:50:41 i have to check it every time ;) Aug 16 09:55:15 mrkiko: are you sure about the 512 MB RAM being Megabyte, and not Megabit? 512 Mb == 64 MiB Aug 16 09:56:06 KanjiMonster: yeah infact, I guess they're megabits as I said Aug 16 09:56:19 KanjiMonster: thank you man Aug 16 09:57:03 ah, indeed. scanned twice over the backlog and didn't see olmari already suggesting that Aug 16 09:58:47 KanjiMonster: :) happens to me as well!! ... yeah, I think it's an mb vs MB confusion Aug 16 09:59:14 mb? mini bits? ;) Aug 16 09:59:27 *milibits Aug 16 10:24:05 dedeckeh: ping Aug 16 10:30:40 Marketing departments favourite conversion Aug 16 10:33:15 screen readers are case-insensitive. he meant MB vs. Mb Aug 16 10:33:28 jow:pong Aug 16 10:33:55 dedeckeh: I am currently porting luci protocol handlers and was stumblink over the "weakif" setting in the ds-lite proto backend Aug 16 10:34:01 what is the relation of weakif to tunlink` Aug 16 10:34:03 ? Aug 16 10:36:26 jow:weakif has no link with tunlink Aug 16 10:36:58 jow:weakif allows to specify an interface to be used as source IPv6 address Aug 16 10:38:42 ah okay Aug 16 10:39:01 and about encaplimit - is option encaplimit "ignore" the same as omitting it? Aug 16 10:39:13 russell--, olmari, rmilecki, KanjiMonster ... it turns out we where not wrong, it's 512 MB. Aug 16 10:39:29 mhm Aug 16 10:41:37 it seems so to me, at least when fo,llowing the logic in https://lxr.openwrt.org/source/netifd/system-linux.c#L2674 Aug 16 10:42:15 an encap limit netlink attribute is only added when the value is defined, not equal to "ignore" and a valid number 0-255 Aug 16 10:42:43 jow:correct Aug 16 10:47:22 jow: right - kernel decided to enable it by default (actually was a bug that it was missing IIRC), so we added the option with 'ignore' to disable it, then found out the header breaks some commercial ds-lite/lw4o6 setups, so we changed it to be disabled by default (but since some might have already added ignore to work around it, still support that value) Aug 16 10:48:42 *disables images for 16 mib ram devices in master for bcm63xx* Aug 16 10:50:29 rmilecki: I see there's beensome work to support highmem; apart from the fact its little bit old now, what where the reasons for the work to not continue? Aug 16 10:52:08 e.g.: https://gitlab.labs.nic.cz/turris/openwrt/blob/4475136d4250a9a53c602c7014d43f2c206a5fdf/target/linux/brcm47xx/patches-3.14/153-MIPS-BCM47XX-Detect-more-then-128-MiB-of-RAM-HIGHMEM.patch Aug 16 11:20:28 mrkiko: IIRC random panics without an obvious reason, like https://www.linux-mips.org/archives/linux-mips/2014-07/msg00498.html Aug 16 11:26:39 jow: blobmsg_name() returns const char, so I think I shouldn't modify (uppercase) it Aug 16 11:36:24 * rmilecki away Aug 16 11:39:48 russell--, olmari, rmilecki, KanjiMonster no I was wrong :D :D it's mb Aug 16 11:39:56 ... :D Aug 16 12:16:37 Hauke: fyi, I updated ath10k-ct in my staging tree. It's now possible to compile the 5.2 flavour Aug 16 13:44:35 oooh, musl does ppc64 now, interesting :) Aug 16 15:01:40 rmilecki: how about that for response time :-) Aug 16 15:01:59 rmilecki: too late, patch sent ;) Aug 16 15:02:14 blogic: i'll be stil happy to get target/linux/mediatek/ device bootlog though! :) Aug 16 15:06:00 blogic: thanks for review Aug 16 15:06:24 blogic: that asprintf() is cool! I didn't know that Aug 16 15:30:33 Hello, i may found a bug where OpenWRT is forwarding DNS from a device that is full Blocked in my Firewall. Aug 16 15:31:36 is this maybe alrdy known? Or should i just write it down at the issue tracker? Aug 16 15:32:41 Define full block.. generally if there is any allows before block, allow goes through.. Aug 16 15:33:29 I set the rule on Top of every other FW Rule, its like Device with IP or Mac from lan to lan reject everything just for testing Aug 16 15:34:35 With L3 (IP) layer firewsll you can't block devices on same subnet Aug 16 15:34:44 Its also working, i cannot reach the device at this point, but openwrt is still forwarding DNS at lan Aug 16 15:35:32 You could paste(bin) relevant configs Aug 16 15:35:51 Can't guarantee anything, can look Aug 16 15:44:01 No idea how i could just get the relevant configs, i just can you tell that i set on Luci Lan interface -> DHCP-Server -> Advanced Settings -> DHCP Options : 6,192.168.10.5 (My DNS Server) And first FW Traffic Rule, TCP UDP IPv4From IP 192.168.10.130 in lanTo IP 192.168.10.1 in lan Aug 16 15:45:15 Cant reach the 130 Device after restart of FW, but DNS is still getting DNS querys from 130 forwarded from 1 Aug 16 15:53:13 Like said, you really can't block ip traffic with L3 firewall that resides on same network... Aug 16 15:55:08 Ah okay, so its not a Bug Aug 16 15:55:44 Any ideas how i could Block the DNS forward just from this device? Aug 16 15:57:49 L2 firewalling (no idea openwrt suppoert on that) Aug 16 15:58:14 But if both devices are on same lan and mot on router directly; no wylay Aug 16 15:58:20 Way" Aug 16 15:58:31 VLAN? Aug 16 16:00:34 openwrt does package ebtables iirc Aug 16 16:02:07 Separate networks (vlan) and so can be made to work too... Aug 16 16:02:41 Hence they would not be on same network anymore --> fw would work Aug 16 16:03:01 yeah, ebtables would be overkill for this Aug 16 16:03:03 But you should really have blocks on the actual router/gateway point Aug 16 16:03:06 For sane config Aug 16 16:03:22 Blocks? Aug 16 16:04:17 Ah i got it... Well im gonna check it out. Thanks for the insight didnt knew that L3 FW isnt working on same Subnet! Aug 16 16:06:38 That is because they're on same (IP) network... the openwrt device being there or not does not have any matter in L3 sense Aug 16 16:09:07 L2 layer (ethernet) FW would work, if both computers/devices needing filter is connected directly to openwrt.. but like DonkeyHotei said, that is kinda overkill setup for just that Aug 16 16:12:22 Well i didnt understood it in the detail, since the 130 Device is not reachable from other devices in the same Subnet with the L3 FW Rule active so it is kind of working but not for the DNS Forward Aug 16 16:13:35 Well, some packages are not send, i still can ping it as i just found out... Aug 16 16:21:59 As said, thanks for your time and info, im gonna figure something out! Aug 16 16:28:05 ClubMateOverdose: Depending from your exact network setup... block said devices traffic on router, have DNS-server on own subnet, etc... variations are plenty Aug 16 16:29:24 ClubMateOverdose: olmari: DonkeyHotei: you can actually do L3 filtering with iptables on a bridge with net.bridge.bridge-nf-call-iptables=1, but of course you can only filter the traffic if it goes through the bridge Aug 16 16:30:39 but that means all traffic go through the small CPU of the router, so expect poor performance Aug 16 16:31:31 mmhm... I don't know how it compares to ebtables filtering, but I'd expect similar kind of performance roughly (ie worse by a factor) Aug 16 16:33:35 olmari: with some work one could use (hw) flow offload and gain back the performance Aug 16 16:34:50 Could be true... tough first I'd made perhaps saner network setup alltogether (again, without knowing details really I can't tell) Aug 16 16:36:17 Hmm, net.bridge.bridge-nf-call-iptables=1 combined with flow offload might finally be a solution to my multi-WAP-client-isolation problem. I tried a while back with ebtables (after painstakingly hacking that in to an OpenWRT build, since it isn't available by default), but it was too slow. I should try that. Aug 16 16:39:25 Sounds interesting, im gonna paint my Setup in a minute. Maybe you have a better Idea on the first look! Aug 16 16:44:59 olmari: totaly agree, it's way easier to have multiple vlans, and it's also easier to reason about. Having L3 firewalling inside a subnet is going to be confusing for many people, and performance will likely be poor. My point was just that it's possible ;) Aug 16 16:46:15 Is this helpfull in any means? http://i.imgur.com/4LxcD3a.png Aug 16 16:47:24 ClubMateOverdose: do you really need a separate DNS server ? Aug 16 16:47:36 champtar: is there a reason DNS is separately on lan instead letting router handle that part too? (perfecly legal setup, but often overkill) Aug 16 16:47:48 ClubMateOverdose: I mean... Aug 16 16:48:30 Well, i have pihole there, i know i can adblock on OpenWRT but the Web interface from PiHole does give more information about single devices and the current DNS logs Aug 16 16:49:01 ClubMateOverdose: In that case you can just block 192.168.10.130 on the dns server ? Aug 16 16:49:22 Did that too, same result sadly Aug 16 16:49:47 Ah no i cant actualy, i did the other thing arround Block 130 on the DNS Aug 16 16:50:14 The 130 Device doesnt give me the opportunity to block Aug 16 16:50:48 Anyways, you can block .130 traffic from internet and so on, but not (easily) on same LAN.. also you need then to block DNS queries from .130 i nthe pihole...aand then what I might do if nothing else can change, put the DNS-server to it's own VLAN and subnet, allow zone forwards between LAN<->DNS and then you can use the openwrt FW to block stuff... and/or other way round, put the .130 on own VLAN/subnet and deal with Aug 16 16:50:49 that Aug 16 16:51:17 ClubMateOverdose: is there a switch missing in your drawing ? Aug 16 16:52:32 No, and olmari, the block to wan works perfectly, but the DNS is spamming my pihole, i can block it sure but it is spamming the logs. Its not a bit deal but it bothers me kind of! Aug 16 16:52:51 Router and modem are two Devices Aug 16 16:53:18 the LAN interface includes WLAN Aug 16 16:53:40 I did say VLAN, not WLAN (: Aug 16 16:54:13 Yes i guess i have to try out the VLAN way Aug 16 16:54:54 I can just repeat myself thanks for your help guys! Aug 16 16:55:27 if .130 needs to be able to be blocked from everyone else, then setting "hostile" VLAN for that would be mine way... if cutting ethernet and slapping face with large trout isn't option Aug 16 16:56:23 .130 needs to get blocked just from DNS the .5 and WAN all other Devices need access to it Aug 16 16:56:48 ClubMateOverdose: how much traffic is there between 192.168.10.130 and 192.168.10.X Aug 16 16:57:21 Not much, its just a private home setup Aug 16 16:58:07 so it's ok if you only have 200M instead of 1G for example ? Aug 16 16:58:19 Sure Aug 16 16:59:05 In that case I agree with olmari, create a separate vlan for 192.168.10.130 (with a different subnet so ip will change) Aug 16 17:00:00 Im gonna try that out, thanks for your help! Aug 16 17:37:37 xback: does ath10k-ct also run? mac80211 complained for me becasue of some ad-hoc mode settings Aug 16 17:40:34 rmilecki: good work on the sysupgrade env var thing... one thing I wondered Aug 16 17:41:02 rmilecki: does upgraded depend on libblobmsg-json now or are all the blob_attr accessor functions inline ones? Aug 16 18:57:31 rmilecki: i didn't add anything to upgraded Aug 16 18:57:44 rmilecki: it's procd that parses blobmsg and calls setenv Aug 16 19:22:32 where is the tmp/.config-target.in generated? Aug 16 22:04:52 jow: I keep getting these error reports of filesize missmatches for snapshots. Is the ImageBuilder updated at a different time than the packages? https://github.com/aparcar/attendedsysupgrade-server/issues/172 Aug 16 22:05:02 Or are the packages already synced but the Package index outdated? Aug 16 22:28:49 aparcar[m]: IB is built in phase1 along the images and non-shared packages Aug 17 00:32:58 KanjiMonster: as expected the problem was on my side, sorry! Aug 17 01:55:24 https://www.phoronix.com/scan.php?page=news_item&px=Linux-5.4-Dropping-Intel-IOP-3 Aug 17 01:56:25 Oh, no difference for OpenWrt **** ENDING LOGGING AT Sat Aug 17 03:00:04 2019