**** BEGIN LOGGING AT Thu Aug 06 02:59:57 2020 **** BEGIN LOGGING AT Thu Aug 06 04:09:43 2020 Aug 06 09:28:16 is there a person here who uses the usoc with much load and frequency? We are investigating allocations of memory and sometimes the allocations are huge and have to be just garbage. Aug 06 09:30:06 e.g. it tried to alloc 1,650,795,048 bytes Aug 06 09:31:28 https://github.com/berlin-open-wireless-lab/DAWN/blob/master/src/network/tcpsocket.c#L81 Aug 06 10:21:20 a sample of aforementioned garbage https://paste.ubuntu.com/p/qckKRS77q5/ Aug 06 10:41:06 nick[m]3: that code looks really dangerous Aug 06 10:41:46 nick[m]3: your're essentially taking untrusted network input, cast it to an unsigned integer, substract that from another unsigned integer and realloc the result Aug 06 10:43:05 nick[m]3: any received length < 4 will result in an integer underflow for example Aug 06 10:44:22 jow: but is there a working example how to use that callback correctly? Aug 06 10:45:17 nick[m]3: the usage is fine Aug 06 10:45:21 jow: thanks for the hints Aug 06 10:45:22 but what you do is wonky Aug 06 10:46:35 jow: how would u do it? Aug 06 10:47:19 wouldn't line 91 bail it out though Aug 06 10:47:40 line 91 just checks that you can read 4 byte Aug 06 10:48:04 oh right that's why there's multiple Aug 06 10:48:09 but if these 4 byte happen to be garbage, you're allocating that much grabage Aug 06 10:48:43 or if they happen to be \0\0\0\1 you're allocating UINT_MAX - 3 byte Aug 06 10:49:28 I'm even allocating garbage if there is no underflow Aug 06 10:49:56 to me it seems as if your message processing gets out of sync and that you're processing the wrong offsets in your binary data stream as message size Aug 06 10:51:56 jow: but that is why we check for the 4 bytes without processing them Aug 06 10:53:08 yeah but consider for example this situation: Aug 06 10:53:11 i think the implication is that they might not be the *right* 4 bytes Aug 06 10:53:20 you get a read callback that something is readable Aug 06 10:53:35 ustream happens to buffer 5 bytes. 4 bytes of length plus first byte of payload Aug 06 10:53:48 the first part succeeds, length is read Aug 06 10:54:03 length states payload is 100 byte Aug 06 10:54:11 ah shit you are right Aug 06 10:54:20 you notice the buffer only has one remaining byte buffered after reading the length Aug 06 10:54:24 you back out of the callback Aug 06 10:54:28 ustream receives more data Aug 06 10:54:31 callback is invoked Aug 06 10:54:46 you interprete the 2nd to 5th byte of payload as length Aug 06 10:54:54 message processing is out of sync Aug 06 10:54:59 and will not recover Aug 06 10:55:28 you will need to implement some state machine logic Aug 06 10:55:32 so I have to buffer my the length and if I read that? Aug 06 10:55:44 have a state reading_length Aug 06 10:56:02 if that suceeds store the expected length, transition to a state reading_body Aug 06 10:56:27 with each further callback, read a min(1024, expected_length) chunk Aug 06 10:56:37 substract the amount of read bytes from expected_length Aug 06 10:56:53 if expected_length reached zero, you've read the whole message, process it Aug 06 10:56:59 and transition state back to reading_length Aug 06 10:57:35 depending on how your message processing works further down the road you also might need to buffer and concat the several chunks your read Aug 06 10:57:58 and while you're at it, you can also add sanitiy checks Aug 06 10:58:08 I guess no message can legitimately be 4GB in size **** BEGIN LOGGING AT Thu Aug 06 11:00:15 2020 Aug 06 11:00:31 jow: thanks a lot! Aug 06 11:40:47 build #561 of ath25/generic is complete: Failure [failed pkgbuild] Build details are at http://buildbot.openwrt.org/master/images/builders/ath25%2Fgeneric/builds/561 blamelist: Adrian Schmutzler , Felix Fietkau , Jo-Philipp Wich Aug 06 11:50:54 Package kmod-fs-exfat is missing dependencies for the following libraries: nls_base.ko Aug 06 11:54:35 adrianschmutzler: https://patchwork.ozlabs.org/project/openwrt/patch/20200801182102.715059-1-rosenp@gmail.com/ Aug 06 11:55:06 nick[m]3: do you send multiple messages over the same tcp connections? Aug 06 11:55:18 nick[m]3: or just one message per conenction? Aug 06 11:55:45 ynezz: thanks. I wonder why this only happens now and the same buildbot built fine yesterday ... Aug 06 11:55:53 multiple messages Aug 06 12:44:05 jow: why did u ask? Aug 06 12:49:10 nick[m]3: was wondering if UDP datagrams would fit better to your use case Aug 06 12:49:52 jow: I already have a multicast version Aug 06 12:49:58 jow: reliable transport would be nice Aug 06 13:20:52 mqtt? Aug 06 13:21:06 no need to worry with this pesky raw sockets. **** BEGIN LOGGING AT Thu Aug 06 13:27:57 2020 Aug 06 13:44:33 Hi all, is there an easy way to select a kernel config and build the firmware? I just want to set CONFIG_DEBUG_KMEMLEAK=y Aug 06 13:48:56 Is it ok just to add CONFIG_KERNEL_DEBUG_KMEMLEAK=y into main .config? Aug 06 14:15:22 Neighbor11111116: make kernel_menuconfig Aug 06 14:18:42 Thanks Paul, I thought is frowned upon to do that Aug 06 14:21:35 PaulFertser Aug 06 14:21:56 What does that actually do? I changed it and compared /.config with the prior one and nothing has changed. Aug 06 14:22:48 jow: Is there no pushback on the stream to rewinf to zero? Aug 06 14:23:41 Neighbor11111116: it changes target/linux/.../config . You can see it with "git diff" Aug 06 14:30:25 nick[m]3: no Aug 06 14:33:30 build #411 of rb532/generic is complete: Failure [failed pkgbuild] Build details are at http://buildbot.openwrt.org/master/images/builders/rb532%2Fgeneric/builds/411 blamelist: Tobias Welz , Adrian Schmutzler , Felix Fietkau , Hans Dedecker , Jo-Philipp Wich Aug 06 14:35:19 nick[m]3: unrelated note, I feel like you might want to wrap all those fprintf's and printf's if you want more logging levels later Aug 06 14:43:07 Namidairo: yeah. :) But first I fix tcpscoket ^^ Aug 06 14:44:58 PaulFertser, thanks! **** BEGIN LOGGING AT Thu Aug 06 15:06:44 2020 Aug 06 15:37:31 build #423 of ramips/rt305x is complete: Success [build successful] Build details are at http://buildbot.openwrt.org/master/images/builders/ramips%2Frt305x/builds/423 Aug 06 15:38:46 build #426 of mpc85xx/p1020 is complete: Success [build successful] Build details are at http://buildbot.openwrt.org/master/images/builders/mpc85xx%2Fp1020/builds/426 **** BEGIN LOGGING AT Thu Aug 06 17:12:45 2020 Aug 06 17:27:56 meh, I should hand out my git credentials, pushing too much crap lately Aug 06 17:35:13 jow: well.. if you're feeling in the mood.. https://github.com/openwrt/openwrt/pull/3241 :p Aug 06 17:51:49 build #500 of ramips/rt3883 is complete: Failure [failed pkgbuild] Build details are at http://buildbot.openwrt.org/master/images/builders/ramips%2Frt3883/builds/500 blamelist: Davy Hollevoet , Daniel Golle , Shibajee Roy , Adrian Schmutzler , Hans Dedecker Aug 06 17:51:49 , Tobias Welz , Felix Fietkau , Jo-Philipp Wich Aug 06 18:26:13 blocktrron: is this also fixing the 6GHz mesh warning: https://git.kernel.org/linus/65ad3ef9fced4062dfd74e2f89443fb5ce184321 Aug 06 18:26:53 * blogic read there was a new ifi exploit hittin 1bn devices, I only had to read the headline to know that it was brcm fmac related ... boy was I not disappointed reading the rest of the story Aug 06 18:27:07 brokencom delivered as expected Aug 06 18:29:28 blogic: The broadcom wifi firmware is the most intresting firmeware out there to look into because it is deploy in so many devices like the iPhone Aug 06 18:29:54 I do not think that the rest is better, just no one looked at it Aug 06 18:30:03 Its a mess Aug 06 18:30:18 its really horrific Aug 06 18:30:53 its a head 2 head race between intel and brcm, the crowd holds its breath, who will hit rock bottom first ... :-) Aug 06 18:31:17 all the FWs are horrific to be honest Aug 06 18:31:59 is the broadcom really worse than the rest? Aug 06 18:32:08 MTK enetrs the stage and starts opening its FW src to trusted devs Aug 06 18:32:11 ok it does a lot that is a problem Aug 06 18:32:24 intresting Aug 06 18:32:24 it is really bad Aug 06 18:32:40 Hauke: dwmw2_gone can tell you more ;) Aug 06 18:33:35 Hauke: i mailed the SFC paper today, sorry about the delay Aug 06 18:33:43 blogic: I saw it thanks Aug 06 18:33:53 I run linux and it failed as expected on my new HP multi prinetr Aug 06 18:34:16 so i had to drive 45km to my parents home to use a win$ scanner ;) Aug 06 18:34:25 my printer supports postscript Aug 06 18:34:34 mine aswell Aug 06 18:34:40 but ubuntu was being a pain Aug 06 18:34:54 and my scanner is about 15 year old Aug 06 18:34:57 It should work, but then iapparently not Aug 06 18:53:50 Should factory images be reproducible? (same commit, same result on any build machine) Aug 06 18:54:19 Because I found localtime() is used sometimes when generating timestamps Aug 06 18:55:34 Hauke: I'll have a look at this as soon as I'm back home Aug 06 18:56:04 Most likely this fixes the warning we're seeing - AFAIR I've first came up with a similar solution Aug 06 18:58:20 svanheule[m]: that would be a bug, then Aug 06 18:59:41 jow: Okay, small patch incoming then :-) Aug 06 20:00:22 woah a lot of reverts on the master branch Aug 06 20:18:00 aparcar: I forgot to mention the change in my v2 patch. Is it worth the extra noise on the ML just to mention I bumped PKG_RELEASE? Aug 06 20:20:47 svanheule[m]: no people likely see my email, too :) Aug 06 20:21:19 aparcar: I investigated that pacakge reproducibility issue a little bit Aug 06 20:21:30 the problem is twofold Aug 06 20:22:11 1) ipkg-bild uses SOURCE_DATE_EPOCH which - in the case of SDK - is hardcoded to the SDK build time Aug 06 20:22:47 2) depending on toolchain version etc. building the same package sources might generate different binaries (this seems to be the lesser problem atm) Aug 06 20:23:49 I do have gathered two examples of wireguard-tools .ipk, once built on july 7th, once on july 10th where the "wg" executable differs in random sections of the file, I currently cannot explain that Aug 06 20:24:22 at first I suspected the MIPS16 activation might play a role but the ipk's I got are both mvebu ones Aug 06 20:26:56 also thought a little bit about incremental builds and see a lot of detail problems Aug 06 20:27:38 what are the problems? I really like the idea Aug 06 20:28:32 - we don't have the concept of a toolchain version, master might change the gcc or binutils version at any time, causing the same sources to yield different compilation results, it might also affect C++ ABIs Aug 06 20:29:26 I'm not familiar with ABIs, does that mean different toolchain version result in incompatible bin files? Aug 06 20:29:32 - we don't have comprehensive ABI version coverage for libraries so we cannot ensure that all affected packages are rebuilt (and version bumped!) when a library is updated Aug 06 20:29:46 I'd understand that kmods need rebuilds, but regular packages? Aug 06 20:30:09 - we have no concept on how to handle changes caused by header-only libraries Aug 06 20:31:05 it is unclear how to handle changes that would impact the global toolchain (e.g. hard/softfloat, mtunes etc.) Aug 06 20:31:32 - opkg currently has troubles selecting the right packages in case multiple variants are available Aug 06 20:31:32 jow: I assume that includes static libraries? Aug 06 20:31:35 what about we create a inverted dependency tree to know what packages need a rebuild after a library update? Aug 06 20:31:54 aparcar: sure we could do that, nd then we'd rebuild this packages using the same version Aug 06 20:31:57 opkg won't notice Aug 06 20:32:22 what about a 4 char toolchain hash in the package versions? Aug 06 20:33:39 is not monotonically inceasing, depending on the hash result, opkg would see the version as downgrade then Aug 06 20:33:50 i'm specifically talking about BUILDONLY packages Aug 06 20:34:14 mangix: yes, static libraries are another vector introducing non-deterministic compilation results Aug 06 20:35:12 and you can't include the versions of all related linbraries resources etc. in the pckage version (I discussed the hash idea a while back with Hauke here already and we didn't came to a satisfying solution) Aug 06 20:35:30 In the packages feed, I currently have gerbera link to BUILDONLY static libraries for size reasons. Aug 06 20:35:50 main point is: I don't think we can fix that with automation or clever scripting, this is something that can only be solved with a lot of repository discipline Aug 06 20:36:15 the Packages index don't contain a date, what if opkg would have a date attached to the package index and thereby knew what's the newest toolchain version? Aug 06 20:36:17 *a lot* of care is needed when bumping library packages Aug 06 20:36:49 how would that help? Aug 06 20:37:06 opkg would know which hash is newer Aug 06 20:37:32 or we use git log --oneline | wc -l on the tools/toolchain folder to have a numeric value Aug 06 20:37:42 (hacky) Aug 06 20:37:50 doesn't help Aug 06 20:38:00 toolchain results are influenced by cflags etc. as well Aug 06 20:38:05 ok Aug 06 20:38:47 but then again, how does it work right now at all? We rebuild everything, fine, but how can opkg know the difference? Aug 06 20:38:55 it does not Aug 06 20:39:41 but at least we sure that everything fits together Aug 06 20:39:44 *can be sure Aug 06 20:39:45 fair Aug 06 20:39:55 well. how does other distros do it? Aug 06 20:40:04 debian only builds once afaik Aug 06 20:40:33 yes, but they have strict library versioning requirements and extra tooling to track exported shared object symbols etc. Aug 06 20:41:13 the also keep multiple versions of libraries in the binary deb repos afair Aug 06 20:41:59 Hello, I would like to know who can help me with the problemhttps://github.com/openwrt/mt76/issues/431 Aug 06 20:42:48 build #489 of ramips/rt288x is complete: Failure [failed pkgbuild] Build details are at http://buildbot.openwrt.org/master/images/builders/ramips%2Frt288x/builds/489 blamelist: Davy Hollevoet , Daniel Golle , Shibajee Roy , Adrian Schmutzler , Hans Dedecker Aug 06 20:42:48 , Tobias Welz , Felix Fietkau , Jo-Philipp Wich Aug 06 20:44:17 aparcar: so what we need a kind of baseline profile defining the general ABI compatibility. I think on a traditional Linux distro this was done by versioning the GNU libc version Aug 06 20:44:48 e.g. on Debian it was libc5 long ago and eventually became libc6 Aug 06 20:44:48 what if we stop rebuild packages that have no dependencies to shared libs? Aug 06 20:45:32 we'd get a repo that is half old and half randomyl rebuilt Aug 06 20:45:45 not sure what that'd buy us, apart from further potential inconsistencies Aug 06 20:46:11 okay I clearly need to read up on ABI stuff Aug 06 20:46:21 I think what we can agree on at this point is the following: there must be no ipk uploads / refreshes whatever with at least one source commit since the last build Aug 06 20:46:31 *without at least Aug 06 20:47:17 rsalvaterra: pong Aug 06 20:47:24 which also means that if for example a non-abi-tracked library is updated, maintainers need to at least bump the version on all packages using this library in order to trigger a rebuild and reupload Aug 06 20:47:34 jow: but then again we need to figure out what affects what, right? Aug 06 20:47:55 aparcar: nasty latency, heh? ;) Aug 06 20:48:25 jow: what about we use a but that automatically creates PKG_RELEASE bumps based on the inverted dep tree? Aug 06 20:48:25 if a header-only or a static library is updated (e.g. boost), *all* user packages must get their versions bumped Aug 06 20:49:06 rsalvaterra: I'm now on IRC without matrix, no mobile responses anymore :P Aug 06 20:49:15 aparcar: sorry about the emails not hitting the mailing list, I replied from my phone and the Gmail app only sends HTML. :( Aug 06 20:49:51 jow: s/but/bot Aug 06 20:50:08 aparcar: could be a way, but it boils down to repository organization again Aug 06 20:51:28 we can create a script like bump_lib.sh which increases the PKG release of all packages affected. This script is run by whatever maintainer handles the lib Aug 06 20:51:39 aparcar: I suggest to study https://www.debian.org/doc/debian-policy/ch-sharedlibs.html Aug 06 20:52:00 this should give a glimpse on the difficulties involved Aug 06 20:52:54 so without having read it yet, the PKG_RELEASE bumps don't just cut it? Aug 06 20:53:19 it would be a crude solution Aug 06 20:53:49 the proper way is ABI version tracking (which we already have, comparable to debian's shlibs) Aug 06 20:54:06 so what's the missing bit? Aug 06 20:54:18 rsalvaterra: I don't know which email sorry Aug 06 20:55:37 putting *all* repository libraries under ABI tracking, find a solution for static linking, a solution for header-only libraries and a way to express toolchain ABI baseline dependencies (could be done by versioning the implicit "libc" dependency and by us increasing the "libc" version whenever we do incompatible changes to the toolchain) Aug 06 20:56:43 aparcar: the one about the BusyBox ip patch, in which I suggest making the relevant packages depending on the BusyBox features, at config time. Aug 06 20:57:54 the end result will involve a mixture of: better repository discipline (way stricter dependency constraints), better automation (auto-bump header-only or static library package users on updates), better build logic (figure out build dependency trees from changeset), better repository building logic (index and sign repo metadata serverside and only queue bare .ipk file uploads fro mthe builders) Aug 06 20:58:25 I believe it's safer, we don't want to give the (power) user loaded guns. :) Aug 06 21:00:06 and we need better buildroot side instrumentation (e.g. inspecting ELF DT_NEEDED sections) to figure out *actual* and *effective* library dependencies Aug 06 21:01:53 setting https://openwrt.org/docs/guide-developer/package-policies#abi_version to all library packages and implementing server-side .ipk upload queueing & indexing will get us halfway there Aug 06 21:02:10 it will roughly be equivalent to the current state of the repository Aug 06 21:04:01 auto-bumping of PKG_RELEASE would be the next step, but I am not sure if this belongs into the repository at all Aug 06 21:04:10 I mean into the Git repository Aug 06 21:04:27 this sounds like something that should be handled in the binary repo itself Aug 06 21:14:11 jow: I'll look into this, sounds like a fun topic where I can learn some stuff Aug 06 21:15:06 I kind of want to finish some other projects before jumping into yet another adventure Aug 06 21:15:27 jow: do you have an opinion on fakeroot for ipkg builds? Aug 06 21:17:11 fakeroot sounds good Aug 06 21:17:59 we also need a way to specify desired permissions and ownerships in .ipk archives Aug 06 21:19:27 jow: yes we could extend the $(INSTALL_DATA) stuff to work with modes Aug 06 21:19:28 something like PKG_FILE_MODES:=/etc/foobar.conf:nobody:nobody:0600 /usr/sbin/suidexecutable:root:nobody:4755 Aug 06 21:19:52 could be then reused the same way for FILES=foo/ in IB Aug 06 21:20:18 you want to store file modes in a variable rather then the install steps? Aug 06 21:20:37 Can someone explain why calling make package/feeds/packages/rustup/host/install tells me make[1]: *** No rule to make target 'package/feeds/packages/rustup/host/install'. Stop. Makefile: https://hastebin.com/ehesekicib.makefile Aug 06 21:21:57 aparcar: I think having a central list somewhere in the Makefile head area is clearer than file ownerships scattered throughout the entire code, but a matter of taste I guess Aug 06 21:22:18 Grommish: make package/rustup/host/install instead? Aug 06 21:22:32 is that how it works? Aug 06 21:22:56 never used host stuff, but normally, it's jsut maek "package/name/{clean,compile}" regardless of what fee it's in Aug 06 21:23:31 there's examples on https://openwrt.org/docs/guide-developer/packages Aug 06 21:23:31 Oh, it's where its supposed to be.. and calling make package/feeds/packages/rustup/host/{configure,install} does the configure Aug 06 21:23:46 it then acts like I don't have a Host/Install in the Makefile Aug 06 21:23:52 make[1]: *** No rule to make target 'package/feeds/packages/rustup/host/install'. Stop. Aug 06 21:25:05 jow: that would make the building certainly easier. I'd extend ipkg-build script to run the permissions withn a fakeroot and then store it in data.tar.gz Aug 06 21:25:09 https://hastebin.com/oxotahaker.coffeescript Aug 06 21:25:59 jow: good once that's done I only have to establish the CI and then fix the ABI handling - easy Aug 06 21:36:55 jow: any idea for having some authentication? I would do it like telecommunication providers and just say my backbone is not accessed by other people... ^^ tcpsocket implementation seems to run nice! :) still need to think about sanity checks Aug 06 21:39:36 Namidairo: tcpsocket is fixed. do u have any good logging example available u could give me? Aug 06 21:51:39 xdarklight: I found multiple bugs in the lantiq mac driver and fixed, them I am not sure if I found the root cause of the problem you reported, I will send them upstream soon and add you in cc Aug 06 22:35:54 Hello, anyone know how to flash rootdisk images to banana pi r64? Aug 06 22:35:58 Package binutils is missing dependencies for the following libraries: Aug 06 22:35:59 libctf-nobfd.so.0 Aug 06 22:36:02 what is this? Aug 06 22:38:48 guess this broke it: https://github.com/openwrt/openwrt/commit/60f595daab45f1ffab2f422ba493a09b41e51ef4 **** BEGIN LOGGING AT Thu Aug 06 22:42:42 2020 Aug 06 22:49:57 bpi_bananapi-r64-rootdisk-squashfs-sysupgrade-emmc.bin.gz how to write this to emmc from u-boot? Aug 06 22:50:21 magic write x emmc Aug 06 22:51:44 turris omnia firmware uses a miniman initramfs to flash the emmc Aug 06 22:52:00 *minimal Aug 06 22:54:33 Thanks, I'll try to look into that. It seems some kind of repartitioning has to be done to the emmc on r64 **** BEGIN LOGGING AT Thu Aug 06 23:34:16 2020 Aug 06 23:56:24 I'm getting ethernet errors on an mt7688 board i'm working on, same hardware worked fine on 4.14, been seeing this since the switch to 5.4: Aug 06 23:56:33 [ 334.148074] WARNING: CPU: 0 PID: 0 at net/sched/sch_generic.c:448 dev_watchdog+0x26c/0x274 Aug 06 23:56:36 [ 334.164454] NETDEV WATCHDOG: eth0 (mtk_soc_eth): transmit queue 0 timed out Aug 07 01:46:31 any reason not to create /var files in packages? Aug 07 01:58:55 aparcar: it's a tmpfs Aug 07 02:10:05 DonkeyHotei: If you a chance, would you provide your knowledge on bringing up a package? https://github.com/openwrt/packages/pull/13040/files I'm doing something wrong with it, so I'm outreaching to the community ;p Aug 07 02:12:09 line 42 is definitely bad Aug 07 02:13:12 Well, it'll be replaced for the specific arch down the line Aug 07 02:13:18 for now, it's the correct I need for testing Aug 07 02:13:38 I'll eventually have to do something similar to what golang did Aug 07 02:15:24 That update was the result of helpful suggestions from swalker. Host/Configure will run, if i call it directly (make package/feeds/packages/rustup/host/configure), but it doesn't get called during a full build Aug 07 02:16:13 Once Host/Configure is run, just the path needs to reflect $(BUILD_DIR_HOST)/.cargo/bin Aug 07 02:16:55 but, I can't get it to work ;x when I build with Suricata, which uses rustc/cargo, it can't find it, so it fails.. even though I have a call for the rustup/host *shrug* Dunno Aug 07 02:17:04 It's why I'm looking for what I did wrong :D Aug 07 02:17:40 https://hastebin.com/oxotahaker.coffeescript <- example **** ENDING LOGGING AT Fri Aug 07 02:59:57 2020