**** BEGIN LOGGING AT Thu May 06 02:59:56 2021 May 06 08:04:18 Hi May 06 08:04:29 Any devtool experts here? May 06 08:36:02 gah, can someone explain what the difference between "bitbake --runall=fetch foo" and "bitbake --runonly=fetch foo" is? does any of them run the task only for those recipes which need to be recompiled and where sstate is not uptodate? May 06 08:36:14 abelloni: Looks like a "fun" build for SWAT :(. I triaged a couple of bits May 06 08:37:57 mcfrisk: One will look at every recipe in the taskgraph and then run the task whether it would have run in the original command or not May 06 08:38:13 mcfrisk: the other will run the specified task on if it was in the original task graph May 06 08:38:30 s/on/only/ May 06 08:39:07 mcfrisk: so I think you'd want --runonly May 06 08:41:59 RP: Thanks! darn, I'm running --runonly=fetch and it seems to run fetch for all tasks even if they don't need to be compiled, so ignoring the sstate. I'm trying to switch to offline builds and need to hack the real bitbake run to work without any networking inside the build container to be sure. And need to run a fetch task before that in online mode and that is too slow atm.. May 06 08:42:54 mcfrisk: oh, sorry, I misunderstood. It will run them all since there is no sstate associated with the task May 06 08:44:15 mcfrisk: we don't have a way to do what you describe, that isn't an easy thing to express/discover :/ May 06 08:47:58 RP: hmm, the problem for me is that BB_NO_NETWORK is not enough. I really need to break all networking in the build environment to make sure that recipe build scripts don't access random Internet/intranet servers etc. Hence build must be split to an online fetch step to update local caches, followed by an offline build. May 06 08:51:59 mcfrisk: This is what --runonly=fetch was designed for but it will run all the fetch tasks as it can't tell if the cache is up to date or not May 06 08:52:18 and when I do this, build time for full sstate one is doubled from 30 to 60 minutes. the fetch'ing of all recipes takes too long. Should only be done incrementally somehow, only if recipe needs to be recompiled May 06 08:52:46 mcfrisk: I'm surprised as the fetch tasks should see the sources already present and run quckly May 06 08:54:03 mcfrisk: we do that on the autobuilders for example https://autobuilder.yoctoproject.org/typhoon/#/builders/83/builds/2117 step 14 which took about 60s May 06 08:57:55 abelloni: I triaged one of the a-full builds, there is a second which probably can have some of the entries copied over May 06 08:57:57 RP: does that have a local download pre-mirror? May 06 08:58:15 mcfrisk: yes, it isn't from scratch May 06 09:01:59 RP: I'll check in detail what's wrong with my setup. I also should have a full local DL_DIR and nothing needs to be fetched again. Thanks a lot! May 06 09:07:32 RP: ah right, I was looking at the qemux86-64-ptest failure of that build May 06 09:20:31 abelloni: there are a ton of issues there, I've tried to help a bit... May 06 09:20:49 yes, I'm very late triage, I'm on it right now May 06 09:20:55 triaging May 06 09:27:05 abelloni: I was meaning in that build. Would be good to be up to date before triage today though May 06 09:44:15 "Hello world !" [cit.] May 06 09:44:35 Doeas anyone ever worked with the "stateless-rootfs" FEATURE ? May 06 10:03:00 mcfrisk: runall=fetch will *definitely* only fetch what is missing from DL_DIR, so you've got problems with that persisting between builds May 06 10:07:20 zeddii: should meta-virt work with gcc11? May 06 10:30:17 rburton RP: yep, I confirmed with some instrumentation that running "bitbake -k --runonly=fetch foo" takes only 140 seconds which is decent for a fully uptodate DL_DIR and large project. there is something else making my builds slower. May 06 10:31:01 a lot of do_fetch taks do get executed and that put me off to wrong path. will check the timings instead. May 06 10:31:24 iirc they run but do nothing as the files exist May 06 10:32:02 yep, and seem to be fast. I can even increase parallel options as fetching isn't memory limited like real compilation May 06 10:42:23 abelloni: looks like there is a git config setup issue on fedora33 May 06 11:54:16 guys, If I need to port systemd vXXXX >243 to thud which has v239, Which can be a good strategy ? May 06 11:59:42 stop using thud May 06 12:00:01 to backport, copy the recipes and verify the output carefully May 06 12:02:08 I really can't at the moment May 06 12:02:55 I've already tried to copy the recipes but it's no so straightforward May 06 12:02:59 No, it's not May 06 12:03:09 yap, I've noticed May 06 12:04:02 You're upgrading fundamental parts of the stack on a release which has been unmaintained for 18 months. It's not going to be easy at all. May 06 12:04:17 damn May 06 12:06:47 If you're using thud for new development I hope you have a support contract, so maybe ask whoever provides that if they have updated systemd recipes May 06 12:10:35 I am the support May 06 12:10:39 the developer May 06 12:10:44 and the tester May 06 12:11:00 the repo mainter and the issue tracker May 06 12:11:43 I maybe be a little short of workforce at the moment May 06 12:14:36 kanavin: I started picking off bits of your patchset. I assume you'll be working on a second iteration so I should probably wait on that before going too much further? May 06 13:13:52 RP: so far, there's actually nothing that would warrant a second iteration :) May 06 13:16:58 rburton: I'm working on gcc11 support in meta-virt now. uprevs and such to fix issues. May 06 13:45:06 Hi, we want ot use an sstate dir shared between multiple users, but somehow every once in a while we get messages like this one May 06 13:45:08 Exception: bb.process.ExecutionError: Execution of '/workdir/build-xavier-nx/tmp/work/armv8a_tegra-sorama-linux/cuda-nvdisasm/10.2.89-1-r0/temp/run.sstate_create_package.10212' failed with exit code 1: May 06 13:45:09 mktemp: failed to create file via template ‘/workdir/build-xavier-nx/../files/bitbake/sstate-cache/bb/a5/sstate:cuda-nvdisasm:armv8a_tegra-sorama-linux:10.2.89-1:r0:armv8a_tegra:3:bba5d5db21e3f24d4d85064b9bd2354bb67833210254ca47f8fe29c1133be669_package_write_deb.tgz.XXXXXXXX’: Permission denied May 06 13:46:11 The root cause is that the file already exists and is created by a different user, but why would a 2nd user want to overwrite this file? May 06 13:46:22 the sstate dir is on the same system May 06 13:47:37 eFfeM1: sstate dir shared between multiple users doesn't work AFAIK May 06 13:47:41 kanavin: I saw quite a bit of discussion. I'm not sure I agree with just removing that expat change for example :/ May 06 13:49:48 mckoan: it should May 06 13:50:21 eFfeM1: It shouldn't be using XXXXXXX directly but replacing that with a tmp file mask :/ May 06 13:50:31 mckoan I found references that it should, even with NFS on an NFS dir May 06 13:50:44 RP, ahhh May 06 13:50:48 the autobuilder does exactly that May 06 13:51:27 RP: I looked into expat, and wrote a followup May 06 13:51:28 still struggling with another issue at the moment but that at least gives a hint on where to look May 06 13:51:47 RP: expat ptests were never enabled in the first place May 06 13:51:49 zeddii: I had wondered if that was what you'd done :) May 06 13:52:01 zeddii: I do hate it when your own sanity tests complain about your sanity :/ May 06 13:52:30 eFfeM1: it's dangerous because if one user does a cleansstate, it's gone for everybody. And I assume Yocto isn't really built to handle sstate-cache that disappears after parsing everything? May 06 13:52:46 RP: the rest was Khem indulging in bike-shedding to be honest which he does sometimes :-/ May 06 13:53:05 if that is somehow working, I guess having two users recreating the sstate-cache from scratch at the same time could result in both users trying to install the same package May 06 13:53:46 qschulz yes cleansstate would be a pain May 06 13:54:26 eFfeM1: what you can do if you want to share the sstate-cache, is to have them all (or a main server) have a webserver exposing the sstate-cache directory and set up SSTATE_MIRRORS in local.conf to point to those webservers May 06 13:54:34 kanavin: well, there were questions which needed answering in some areas May 06 13:54:52 qschulz: we use atomic renames to avoid that May 06 13:55:52 RP: I guess you should hold off the items that prompted a discussion, but I still don't see what I could do about them realistically May 06 13:57:07 RP: cmake with qt needs a clear test that would confirm that nothing breaks with patch removal, and there isn't, as no one anymore remembers what the original issue really was May 06 13:57:38 kanavin: I was going to look at just pulling some and that started worrying about patch inter-dependencies May 06 13:58:14 qschulz yes but putting it on a server would require that someone every once in a while updates the server cache (and I fear that someone then is going to be me :-) May 06 13:58:39 meanwhile I peeked into sstate_create_package and it has May 06 13:58:41         TFILE=`mktemp ${SSTATE_PKG}.XXXXXXXX` May 06 13:59:05 eFfeM1: but mktemp is supposed to replace that, its a pattern May 06 13:59:09 and the system has a good mktemp, so the filename is a indeed odd May 06 13:59:12 RP, I know May 06 13:59:21 so this is very strange May 06 13:59:23 eFfeM1: I wonder if pseudo is breaking things somehow May 06 14:00:07 pff, that is an area I did not venture in before May 06 14:00:43 but I can try to add some debugging code; after my current job finishes May 06 14:01:03 RP: right, the gi-docgen thing is tightly coupled with pango/gdk-pixbuf update. meson update is followed by a couple patches that fix things. Same for gnu-efi update. dnf/libdnf updates are best taken together. The rest should be self contained. May 06 14:02:45 eFfeM1: except if all your users have a webserver, in which case, they all sync their sstate-cache when a sstate-cache entry is needed May 06 14:02:49 * alephan < https://matrix.org/_matrix/media/r0/download/matrix.org/RVjoYWaGxlzCYbGoyKegFJhl/message.txt > May 06 14:07:00 qschulz that is an interesting idea May 06 14:07:22 for now I found another clash, this one does not have the XXXXX pattern in the name May 06 14:08:40 ./c8/97/sstate:qtwebsockets:aarch64-sorama-linux:5.15.3+gitAUTOINC+43f9a2dbf9:r0:aarch64:3:c897b6b269a45a33e78a40993b40e8e5902a7af6df075e7189fad549a8e5ae26_deploy_source_date_epoch.tgz May 06 14:12:10 RP: I can write better commit messages to those items which caused discussions, but there's nothing to actually change in those patches May 06 14:19:18 alephan: your message is sent to the IRC channel as a link, not as actual message. If you can just make sure we get the text directly, you'll have a much bigger chance to have an answer May 06 14:19:29 (I think it's usually because of formatting or messages being too long May 06 14:19:56 qschulz: exactly I'll repost May 06 14:19:59 Thanks a lot May 06 14:23:45 There seems to be a bug when using `runqemu` in builds done from sstate mirrors. In this case, STAGING_BINDIR_NATIVE will not be populated for `qemu-helper-native` so the `runqemu` script will fail with: May 06 14:23:48 runqemu - ERROR - Native sysroot directory [...]/build/tmp-newlib/work/x86_64-linux/qemu-helper-native/1.0-r1/recipe-sysroot-native/usr/bin doesn't exist May 06 14:23:51 See https://git.yoctoproject.org/cgit/cgit.cgi/poky/tree/scripts/runqemu#n1548 May 06 14:23:56 I'm trying to figure out if this is a known issue or somehow specific to `dunfell`. Is anyone aware of this? May 06 14:29:51 qschulz: did this ^ do the trick? May 06 14:29:57 kanavin: if you could tweak the messages I think that would help May 06 14:32:20 kanavin: can we figure out whether there are ptests not listed in fast/slow ? May 06 14:33:28 alejandrohs: do you ever use WSL2, or just an observer on the state of things ? May 06 14:34:27 kanavin: I did notice the cruft in puzzles recipe does actually do something but I'm going to ignore it May 06 14:34:48 kanavin: I never agreed with it in the first place, rburton probably would argue for it :) May 06 14:35:11 zeddii did a build under WSL2 work for you? I tried it a few times but somewhere halfway I ended up in a total hangup of my system May 06 14:36:09 suspect it may be better if you can use ext2 filesystems under WSL2; that is in the insider program, but I did not want to put a non-mainstream windows version on my system May 06 14:38:18 heh. yah. I don't want to do builds, but was looking for the better I/O on WSL2 versus the original, when I was poking at repos. May 06 14:38:36 but they've messed up the networking so badly, that I was looking to throw scorn at someone :D May 06 14:48:30 alephan: it did, fingers crossed it'll get people to asnwer it now :) May 06 14:54:01 Cheers! May 06 15:16:03 alephan: I think the key question is what you're running before calling runqemu May 06 15:16:20 and whether we'd expect that to restore the helper May 06 15:17:08 RP: that is a good point. And I think that gives me the answer to the problem May 06 15:17:17 I'm building a zephyr application. May 06 15:19:43 alephan: meta/conf/machine/include/qemu.inc:EXTRA_IMAGEDEPENDS += "qemu-system-native qemu-helper-native" May 06 15:19:51 alephan: that is how we do it with core May 06 15:19:55 Exactly. May 06 15:20:45 In zephyr that is done with a `DEPENDS_append_qemuall` in the recipe May 06 15:21:17 zeddii: found fedora has a patch to fix xen-tools May 06 15:21:36 hm, fetch failing with do_unpack: Unpack failure for URL: 'git://git.yoctoproject.org/yocto-kernel-cache;type=kmeta;name=meta;branch=yocto-5.10;destsuffix=kernel-meta'. No up to date source found: clone directory not available or not up to date: /builds/persist/downloads/git2/git.yoctoproject.org.yocto-kernel-cache; shallow clone not enabled May 06 15:23:47 rburton: aha. nice. We were just going to mask it, but that works better. May 06 15:24:12 I don't think I did anything odd with the kernel-cache repo that would have caused that 2nd thing though. May 06 15:24:31 RP: I'm not sure how this actually works. Because when the sstate mirror is used, do_populate_sysroot_setscene won't populate the native recipe sysroot. May 06 15:25:05 So even if the dependency is fine, it won't bring in the bindir of native sysroot for the wrapper's recipe May 06 15:25:37 There is another dependency that makes that explicit somewhere May 06 15:25:56 alephan: meta/classes/testimage.bbclass:TESTIMAGEDEPENDS_append_qemuall = " qemu-native:do_populate_sysroot qemu-helper-native:do_populate_sysroot qemu-helper-native:do_addto_recipe_sysroot" if you're using testimage May 06 15:26:06 That's it! May 06 15:26:30 But that is only for a testimage May 06 15:26:43 How come it doesn't fail on normal images in the core? May 06 15:26:43 zeddii: https://src.fedoraproject.org/rpms/xen/blob/rawhide/f/xen.gcc11.fixes.patch is the fedora one. no idea if its 'right' or being upstreamed. May 06 15:27:58 alephan: I'm not quite sure, I'd have assumed it was the EXTRA_IMAGEDEPENDS May 06 15:28:27 I'll dig a bit. Thanks for the hints. May 06 15:29:29 dl9pf, smurray: https://autobuilder.yoctoproject.org/typhoon/#/builders/121/builds/41/steps/11/logs/stdio ? :/ May 06 15:29:34 new lua dependency? May 06 15:31:16 RP: yeah, I rebased the meta-agl next branch which picked up the new wireplumber recipe, it now has a Lua dependency May 06 15:32:26 RP: my apologies, I forgot that all layers are getting tested in meta-agl now, was only testing meta-agl-core May 06 15:43:04 ERROR: Task (/home/drofty/yocto/poky/meta/recipes-core/glibc/glibc_2.31.bb:do_unpack) failed with exit code '1' May 06 15:43:04 HOW TO SOLVE THIS PLS HELP May 06 15:46:31 moto-timo: thanks for your suggestion regarding python3-protobuf-distutils-native. However that is for building a custom recipe with custom do_install. Wouldn't it be better to extend the setuptools.bbclass so that we can build|install with setup.py as usual? (setuptools extension really just add new variables to their setup.py) May 06 15:49:05 drofty: you have a path to a log file a few lines above, read it to start debugging or post here the important content (where the error comes from) May 06 15:49:51 s/add/expect/ May 06 15:50:33 RP: meta-oe is added as a dependency, any ideas on why that's not enough to get lua? May 06 15:53:00 hey qschulz https://pastebin.com/BA63gvjZ here is the log .. should i delete this and run bitbake again ? /home/drofty/yocto/poky/build/downloads/git2/sourceware.org.git.glibc.git May 06 15:53:00 this file May 06 15:54:02 i have 100mbps internet still files are getting downloaded at 250KBps ._. any way to increase the speed? May 06 16:20:50 Hi May 06 16:20:52 PACKAGECONFIG[gtk] = "--with-gtk3,--without-gtk3,gtk+3" May 06 16:21:01 what does this line mean May 06 16:21:19 qschulz May 06 16:22:37 Nithesh: https://docs.yoctoproject.org/ref-manual/variables.html?#term-PACKAGECONFIG May 06 16:23:26 JPEW i couldn't understand there May 06 16:23:49 how is gtk disabled here May 06 16:24:12 Nithesh: Bascially, if "gtk" is in "PACAKGECONFIG" it's enabled, otherwise it is disabled May 06 16:25:03 then how  --without-gtk3 work May 06 16:25:28 If "gtk3" is *not* in PACAKGECONFIG, that gets added to the configure arguments May 06 16:26:02 ok thanks May 06 16:27:00 oh, conflicting packageconfig support got merged? May 06 16:29:00 rburton: yup a few releases ago May 06 16:29:08 somehow missed that! May 06 16:30:17 docs.yoctoproject.org is super slow for me, same for some of you? May 06 16:35:53 Any advice on getting the Yocto Eclipse plugin working? It seems that the repository is empty/broken? http://downloads.yoctoproject.org/releases/eclipse-plugin/2.6.1/neon/, https://imgur.com/a/1pu9KmT May 06 16:36:14 On that note, is that project deprecated/not in use? Hasn't been updated since 2.6.1 May 06 16:36:40 rburton: documented since dunfell, might predates that even? May 06 16:37:59 smurray: if you changed the dependencies that would be the issue May 06 16:38:02 nate100: https://docs.yoctoproject.org/ref-manual/migration-2.7.html#eclipse-support-removed May 06 16:38:12 smurray: we had to turn off autodetection :( May 06 16:38:28 RP: I've got a fix to the meta-pipewire layer.conf in hand, will push shortly May 06 16:39:19 smurray: let me know if I need to change the autobuilder config May 06 16:39:37 smurray: with fixes pending for OE-Core we may be able to renable autodetection again May 06 16:42:03 kanavin: I hacked expat-ptest into fast and it all passes: http://autobuilder.yocto.io/pub/non-release/20210506-11/testresults/qemux86-64-ptest-fast/ May 06 16:42:17 kanavin: I'll send a patch to add that May 06 16:55:26 RP: I believe it should be good now May 06 16:56:21 rburton: Saur: ping2 https://lists.yoctoproject.org/g/yocto/message/53345 May 06 16:57:44 smurray: thanks May 06 16:59:01 how to set sstate mirrors can i use it for free? m building dunfell version May 06 17:03:09 what is MACHINE_GUI_CLASS? May 06 17:03:58 Hi, bitbake does not re-generate ipk files here. The timestamp is the old one, and removing the old ones does not generate new ipk files May 06 17:04:23 at least not in tmp/deploy/ipk/ May 06 17:06:09 Scorpi: if nothing changed in the recipe or dependencies of this recipe, then it's not rebuilt May 06 17:06:20 Scorpi: and it's bitbake BTW :) May 06 17:06:27 qschulz: I changed the recipe May 06 17:06:45 qschulz: yeas, it's the same in this case (busybox) May 06 17:07:17 qschulz: I made a change to the defconfig, and now reverted that change and created a custom recipe with a .bbappend with the config change May 06 17:07:44 qschulz: so I guess it should be rebuilt May 06 17:08:27 Not necessarily May 06 17:08:49 if the bbappend applied to the bb is identical to the modified bb you had before, it won't be rebuilt May 06 17:09:22 qschulz: I won't, as it comes with a config snipset that was not present before May 06 17:10:28 qschulz: I also saw that the patch, configure, compile steps etc. where done May 06 17:10:52 qschulz: then I noticed that the ipk files had old timestamps, which should not be the case I think May 06 17:12:29 qschulz: in the log file, I can see that the configure step invokes merge_config with my config snipset, so it seems to work as intended May 06 17:15:45 Scorpi: which release are you on? May 06 17:20:16 qschulz: 3.1.3 May 06 17:21:53 I'm not entirely sure but I think that release is using the hash equivalence server May 06 17:22:34 which means if a task with a different hash ends up producing the same output than another task, the tasks that follow are taken from the sstate cache May 06 17:24:29 okay, makes sense May 06 17:25:50 can I wipe that specific files from the sstate cache using bitbake? May 06 17:26:33 bitbake -c cleansstate but... why? May 06 17:27:27 To get the ipk files in tmp/deploy, as I deleted them May 06 17:30:53 I thought that there were the build result, which is regenerated when it is missing when I invoke bitbake again May 06 17:32:12 I'm figuring out wic right now. I've got a WKS file that describes the partitions needed by my target platform's boot ROM (the typical fat32 partition up front.) The boot ROM requires the presense of a first-stage boot loader with a specific filename. This file does not need to appear anywhere on the booted target's rootfs, only on the FAT partition. May 06 17:32:38 How can I prepare a second filesystem for `wic' to copy to my fat partition? May 06 17:33:55 Scorpi: or remove your whole TMPDIR, that should make Yocto install them again May 06 17:34:20 just don't remove the sstate-cache and downloads directories, the rest is safe to delete May 06 17:34:43 (well maybe there's some hashserv files but I'm not on 3.1+ so can't say for sure) May 06 17:35:06 qschulz: -c cleansstate works May 06 17:44:31 Scorpi: yes but only for that package, if you remove more, the others still won't be available May 06 17:53:53 Hi i'm trying to reduce the size of the image. I see python3 is really big. Is there a way to reduce the size of it? Not seeing anything specific in the recipes May 06 17:55:09 qschulz RP wrt my sstate problem: I found that even if I start my bitbake with a umask 0000 it still creates dirs in sstate with mode 755 and other users then cannot write into that directory May 06 17:55:22 haven't found a decnt way to fix it May 06 17:57:48 actaually just discovered an os.umask(0o022) after a fork in bitbake/lib/pyinotify.py May 06 17:57:55 no idea if that is realted May 06 17:57:58 related May 06 18:05:20 eFfeM1: the idea is the sstate code should be using a different umask but I can't remember what/where May 06 18:22:45 RP suspected that but haven't found it yet, I've been peeking into sstate.bbclass, that one uses bb.utils.mkdirhier which in turn calls some os.makedirs function or so (forgot the exact name) May 06 18:25:28 eFfeM: meta/classes/sstate.bbclass: bb.note("Using umask 0o002 (not %0o) for sstate packaging" % omask) May 06 18:25:39 eFfeM: but I didn't look in detail May 06 18:26:36 RP ah ok didn't find that one, still I noticed a dir with mode 755 May 06 18:26:55 RP anyway, that will help me to proceed May 06 18:27:43 thanks a lot! May 06 18:32:17 i've specified version 5.8.13 of the kernel in my image and am seeing linux-yocto/5.8.13+gitAUTOINC+b976de4f41_3c5d210805-r0 under the tmp/work directory. does this come from a repo? May 06 18:35:04 there is a kernel-meta folder under there with an "arch" subfolder that looks somewhat like the arch folder in the linux kernel tree, but it's missing quite a few subfoldres May 06 18:36:37 can someone help me understand where yocto is getting this from? May 06 18:38:02 and what are these .scc files there? May 06 18:44:50 I believe kernel meta comes from here http://git.yoctoproject.org/cgit/cgit.cgi/yocto-kernel-cache/ May 06 18:45:08 I'm not sure how the arch subfolder is populated, I ran into a similar issue May 06 19:00:15 Does anybody in Yocto do this kind of analysis? https://r13y.com/ May 06 19:01:45 matthewcroughan: https://www.yoctoproject.org/reproducible-build-results/?branch=master May 06 19:03:03 Interesting, so do you have the same issues with GCC that Nix does? May 06 19:03:26 And so is there a test framework that you use for images? May 06 19:03:44 The raw test results is in json, I'm wondering how the QEMU image is ran, for example. May 06 19:04:30 kanavin: I dont mean to indulge in wasting time. it has become very hard to upgrade to newer major release is what we hear from lot of users and I am also witness myself one of the reason is we break them all the time. so I am trying to pay attention to some packages that impact us here and . we can not cover unknown cases, but where we know we should be considerate, unfortunately powertop is used in some RDK profile, which I May 06 19:04:30 have now punted out, so that change does not bother RDK, but probably we want projects to use our stuff and not carve our way to insulate themselves, there are field issues which we have spend 4 months to find our that someone thought it was ok to disable certain option in curl. so if someone is reviewing stuff, pay attention and try to reason on usability grounds too, there is an effort to upgrade to dunfell, and year plus, its May 06 19:04:30 still not done due to these kind of issues, I am being asked to not upgrade yocto anymore, and most probably I have to give in this time. May 06 19:07:39 sakoman: How do I find out what it was that "wasn't" reproducible.? May 06 19:07:43 The excluded packages. May 06 19:20:17 To get `wic' will interpret a .wks file with the line 'part / --source rootfs' to mean that it should create a partition, the partition should be mounted at /, and yocto's root filesystem should be copied there. May 06 19:20:49 But if I have a fat32 filesystem that shouldn't be mounted, how do I copy files to it? May 06 19:20:56 Can I generate a second rootfs? May 06 19:21:25 How do I tell e.g. the uboot recipe that u-boot.bin belongs in the second partition? May 06 19:29:43 do something like this for a second rootfs part --source rootfs --ondisk mmcblk0 --fstype=ext4 --label root2 --align 4096 --fixed-size=2G May 06 19:29:47 yo dudX May 06 19:30:22 that being said I can't seem to generate a proper third rootfs May 06 19:31:06 renegade: wouldn't a line like that would generate a second partition that would get a second copy of the same rootfs? May 06 19:31:26 I was looking to make a different rootfs (perhaps called bootromfs) May 06 19:31:43 ah my bad, yes that will be a copy May 06 19:32:19 ecdhe: sounds like you want a build-in-build. yay multiconfig! May 06 19:33:08 hmm, yeah, you might need a second image recipe for that, then have wic pull in that image binary.. May 06 19:33:55 RP: right, then we'd need to see if it still passes with expat version update, and whether dropping the patch affects the output May 06 19:35:00 he could use rawcopy as a source to copy that image? May 06 19:35:02 khem, that's why here I have insisted on setting up rolling master builds, precisely so we avoid big bang updates where a million things break anyway, regardless of how careful we are. I don't think being nitpicky about powertop and friends is going to fundamentally change that. May 06 19:36:24 kanavin: right, those are the next logical steps. I would like to have the testing if we can May 06 19:36:43 khem, here we're almost done with pushing those builds all the way to 3.3 - in addition to dunfell builds which have been going almost since dunfell was tagged May 06 19:37:45 since we are talking wic. Why is an extended partition tacked on the end when I make more than 4 partitions? May 06 19:38:39 I have 1 boot, 2 rootfs, 1 bootenv, 3rd rootfs. If I open the image in disks the 3rd rootsfs seems to be pushed after an extended partition May 06 19:38:49 RP: I'll try to compile a list of other ptests that are not listed, I guess it could be a simple grep and tweaking/sorting the output May 06 19:39:01 renegade: don't you need an extended partition table for more than 4 entries? May 06 19:39:03 renegade: that doesn't have anything to do with wic, but the limitations of a dos style partition table. presumably if you used gpt it wouldn't have that issue May 06 19:39:07 but not 100% sure on that.. May 06 19:39:24 https://imgur.com/a/3UaUpDi May 06 19:39:29 kergoth: matches some recollection I have too :) May 06 19:39:56 kanavin: I'm wondering if we could somehow detect ptests that weren't listed, add some kind of sanity check May 06 19:41:12 hmm, i should probably suggest rolling master CI builds at work too, fixing things when we switch to the next release is always a pain... May 06 19:41:19 how would i use gpt? All i'm doing is changing a .wks file right now and hoping for the best May 06 19:42:22 https://pastebin.com/8ZNi6u1S May 06 19:43:02 RP: it'd be extra tricky because we also have ptests that exist, but are commented out due to being broken in various ways May 06 19:44:46 sorry if i missed the answer to my earlier question - had to reboot May 06 19:45:28 is the channel logged? May 06 19:47:01 kanavin: we could create a list of ones we know to ignore? May 06 19:47:26 re: 5.8.13+gitAUTOINC+b976de4f41_3c5d210805-r0 May 06 19:48:40 khem: do you have any thoughts? May 06 19:50:53 RP: I wonder if ptest class could include ptest-packagelists.inc directly and simply check the presence of ${PN}-ptest in there May 06 19:51:49 the list would need to be accessible to the class though, which probably means it needs itself to become a class May 06 19:53:51 RP: https://twitter.com/TheYoctoJester/status/1390379236876247043 May 06 19:56:04 has anyone noticed that converting task-depends.dot files to pdf using dot can take an extraordinary amount of time? May 06 19:56:31 yates, don't. It's a text file and totally human readable. May 06 19:56:37 dot task-depends.dot -Tpdf -o task-depends.pdf May 06 19:56:37 yates: yup, i gave up after hours and hours. May 06 19:56:44 yates: sures. anyone == everybody who ever treid, in that context. May 06 19:57:11 yates, if you have specific questions, it's better to quickly hack together a python script that massages the dot into the answer May 06 19:58:02 ah. well it appears i'm not alone.. May 06 19:58:44 yates: i tried some of the suggestions on https://stackoverflow.com/questions/10766100/graphviz-dot-very-long-duration-of-generation May 06 19:58:57 oe-depends-dot in oe-core is a better bet. or just grep. :) May 06 19:59:01 than rendering, that is May 06 19:59:23 kergoth: ag FTW May 06 19:59:31 or rg, yeah :) May 06 20:01:17 kergoth: ripgrep? May 06 20:01:22 yeah May 06 20:04:53 because of a specific reason? just curious? May 06 20:06:05 because rust, of course ;-P May 06 20:06:40 rfs613: thats the answer i expect(ed) in many cases, but not by kergoth :) May 06 20:08:50 I originally switched for performance reasons, along with some of the options grep has, like -x May 06 20:09:11 but my dotfiles fall back to ag or even ack depending on what i've installed, they all get the job done May 06 20:09:25 kergoth: ah interesting. May 06 20:09:40 https://beyondgrep.com/feature-comparison/ is pretty thorough May 06 20:10:01 any ideas on how i can use gpt instead of mbr? May 06 20:10:09 i still like ack as a fallback as it can be built into a single portable script, nothing to install, so i keep a copy in the dotfiles May 06 20:10:37 kergoth: yeah May 06 20:11:31 hi, what is proper way to modify a script that is part of a lower layer recipe "files://" that i can't touch ? May 06 20:11:38 yates, you might also be interested in 'bitbake -g --ui=taskexp TARGET' May 06 20:20:58 LetoThe2nd: maybe I do, maybe I don't May 06 20:21:20 LetoThe2nd: do you have some prior art for this kind of thing that I can poke through now that I understand better what I'm looking for? May 06 20:22:53 ecdhe: https://youtu.be/YvdkvxAIIGQ May 06 20:28:44 LetoThe2nd: watching now May 06 20:29:25 Seems like build in a build is a weird way to describe this... if you build "grep" and you build an image that includes grep, then you have a build in a build May 06 20:30:11 building a filesystem is building an object that comprises many other built objects May 06 20:31:01 ecdhe: *shrug* you are free to call it whatever you deem more fitting. i will stick with build in build. just for the sake of the "yo dawg" meme, ofc May 06 20:31:26 is linux-yocto/5.8.13+gitAUTOINC+b976de4f41_3c5d210805-r0 some sort of a code for dynamaic fetches? May 06 20:31:35 inception builds May 06 20:32:24 * LetoThe2nd must resist urge to make contraception pun May 06 20:32:39 baby builds May 06 20:33:07 (motivated by George Carlin's "Baby-Maybe" skit in the 70s) May 06 20:33:20 * c4t3l_home giggles and spills coffee May 06 20:34:14 Papa-Stopper, "Embyr-NO" ... May 06 20:34:29 ok, i'll shutup now. May 06 20:35:09 well we could call it "evil builds" and "mini me"s, for non-mc and mc builds? May 06 20:36:08 folks, i'm truly confused. i mean, more than usual. May 06 20:37:08 mission: accomplished. May 06 20:37:55 the motivating problem: http://paste.ubuntu.com/p/smYx75j3mX/ May 06 20:38:06 LetoThe2nd: you are cruel May 06 20:39:03 so i traced things as far as the line in run.do_kernel_metadata: bsp_definition=$(spp ${includes} --find -DKMACHINE=qemucsky -DKTYPE=standard) May 06 20:39:43 i found ${includes} is "-I/edg/build-csky-toolchain/tmp/work/qemucsky-poky-linux/linux-yocto/5.8.13+gitAUTOINC+b976de4f41_3c5d210805-r0/kernel-meta" May 06 20:40:12 i further found there are not ..kernel-meta/bsp folders for my board: qemucsky May 06 20:40:35 what i can't figure out is: where is this folder coming from? May 06 20:41:09 i thought it may be from here, but it looks different: https://git.yoctoproject.org/cgit/cgit.cgi/linux-yocto/tag/?h=v5.8.13 May 06 20:41:26 yates: yoctoized kernels are.... special. i've never used those, i personally favor vanilla ones. you might want to read the kernel-dev-manual May 06 20:41:26 like, a LOT different. different structgure. May 06 20:42:00 matthewcroughan: see line 30 here: https://git.openembedded.org/openembedded-core/tree/meta/lib/oeqa/selftest/cases/reproducible.py May 06 20:43:03 LetoThe2nd: are you saying the linux-yocto kernel is not based directly on a mainline kernel? pfft!!!! May 06 20:43:32 ok, i'm a dumbass... May 06 20:43:57 yates: based on? yes. equivalent? no. May 06 20:44:06 'er no, that's wrong. May 06 20:44:42 but unfortunately, my day is done. no time to discuss. will be back tomorrow. May 06 20:45:01 zeddii: i'll pay your overtime! May 06 20:45:23 name your price. (in Raymond Reddington style...) May 06 20:45:24 I'm wrong? Film at 11 ;-) May 06 20:46:51 how cool is it that Daniel, from the original Stargate movie, is our Blacklist FBI CI? May 06 20:48:08 "more confused than usual" May 06 20:49:53 * LetoThe2nd also calls it a day. gn'8 May 06 20:49:58 l8r May 06 20:56:10 looks like this will let me use gpt bootloader --ptable gpt May 06 21:57:45 sakoman: interesting.. so Yocto considers GCC reproducible? May 06 21:58:10 yates: what was question you had for me ? May 06 21:58:42 matthewcroughan: most of package in oe-core are reprodcible May 06 21:59:13 And GCC is in oe-core, yet Nix can't call it reproducible. I wonder what the difference is. May 06 21:59:38 Does Yocto consider "reproducible" to mean something different than what Nix means? May 06 22:00:30 matthewcroughan: do you mean gcc package itself or the output of it ? May 06 22:00:39 The GCC binary. May 06 22:00:46 The output of the recipe. May 06 22:01:18 Nix can't reproduce it, because when GCC is compiled, it for some reason changes the entrypoint memory address in the ELF header. And therefore a few other things. May 06 22:01:28 yes, its maked as reproducible, why does nix mark it not so May 06 22:01:39 https://r13y.com/diff/9849af16e34e39b94dffede2bedb4fe45992133b4be5fe4d26c2c3820379122b-177b2699c9f6b676ac4f4ec85438dda0f93a00fd01a0e6fe3d9111bf2420f72d.html May 06 22:02:24 There is supposed to be no diff, but there is. May 06 22:02:35 "Each build is run twice, at different times, on different hardware running different kernels." May 06 22:06:15 Does the reproducible-build-results stuff run under that same scrutiny? May 06 22:06:32 2 systems, 2 times, 2 kernels? May 06 22:18:34 hi khem May 06 22:20:32 matthewcroughan: yes May 06 22:23:38 the motivating problem: http://paste.ubuntu.com/p/smYx75j3mX/ May 06 22:23:54 so i traced things as far as the line in run.do_kernel_metadata: bsp_definition=$(spp ${includes} --find -DKMACHINE=qemucsky -DKTYPE=standard) May 06 22:24:07 i found ${includes} is "-I/edg/build-csky-toolchain/tmp/work/qemucsky-poky-linux/linux-yocto/5.8.13+gitAUTOINC+b976de4f41_3c5d210805-r0/kernel-meta" May 06 22:24:16 rburton: how do I reproduce the results? May 06 22:24:28 what i can't figure out is: where is this folder coming from? May 06 22:24:33 matthewcroughan: oe-selftest -r reproducible May 06 22:25:10 khem: where is this folder coming from? May 06 22:25:49 i thought it may be from here, but it looks different: https://git.yoctoproject.org/cgit/cgit.cgi/linux-yocto/tag/?h=v5.8.13 May 06 22:28:24 my guess is that there is no qemucsky folder under kernel-meta/bsp, hence the bitbake error. May 06 22:28:54 i don't understand where the bsp folders that are there are coming from. May 06 22:31:38 what does the encoding "5.8.13+gitAUTOINC+b976de4f41_3c5d210805-r0" mean? May 06 22:38:35 khem: ? **** BEGIN LOGGING AT Thu May 06 23:07:26 2021 **** BEGIN LOGGING AT Thu May 06 23:39:43 2021 May 07 00:30:57 matthewcroughan: does nix have --enable-initfini-array May 07 00:33:04 matthewcroughan: we have all those combinations we even use differerent distros on build host which have more differences beyond just kernel like default shell interpreter etc. May 07 00:33:32 khem: It looks as if the answer to that is no. What would the implication be? May 07 00:36:00 ah let me look at your diffoscope output May 07 00:38:07 khem: might help :) https://github.com/NixOS/nixpkgs/issues/108475 May 07 00:38:10 using either --disable-initfini-array or --enable-initfini-array leaves the possibility of letting system decide for you weather to enable them or not May 07 00:38:42 so its better to specify that for predictability May 07 00:40:08 What does that option do exactly and why would it change? What parameters would decide it one way or the other? May 07 00:40:55 It's unlikely that it would be the case, because Nix sets up a pure environment for these builds, where the host verifiably doesn't interact with the environment for the build, same way as bitbake does. May 07 00:46:34 it uses .init_array instead of .init for contstuctors May 07 00:47:18 this may not be the problem though May 07 00:47:27 for the diffs you posted May 07 00:53:05 real difference is that .text size 0x10 bytes difference everything else is displacements, but there is no dump of .text section to see what is different May 07 00:53:15 so perhaps dump that and it will have clues May 07 00:58:26 yates: linux-yocto tooling has a seprate metadata repo called kernel-cache see https://git.yoctoproject.org/cgit/cgit.cgi/yocto-kernel-cache/ where it provides various bits for machine configs May 07 00:59:23 yates: you can extend it also to include your machines see https://github.com/akuster/meta-odroid/tree/master/recipes-kernel/linux/linux-yocto-dev/odroid-kmeta **** ENDING LOGGING AT Fri May 07 02:59:56 2021