**** BEGIN LOGGING AT Mon Feb 15 02:59:58 2021 Feb 15 08:41:30 yo dudX Feb 15 08:41:41 hey LetoThe2nd Feb 15 09:13:02 Hello, is there a way to use ptest and extract richer logs from the test VM to the host running the build / test Feb 15 09:13:21 in my case I'd like to extract some junit xml compatible logs to display in gitlab Feb 15 09:18:09 mornin' folks Feb 15 09:26:22 Hey all. If you wanted to have a binary recipe vs a source based recipe of the same PN, how could you cheat the state cache into believing they are the same and not cause rebuilds? Feb 15 09:26:29 when they are interchanged? Feb 15 09:32:47 ptsneves: two different recipes, preferred version to change between them? alternatively, virtual package and preferred provider? Feb 15 09:33:47 I would not cheat via sstate and would make this a per user thing, e.g. if user has access to repo, prefer the source one, else binary one. Feb 15 09:35:18 mcfrisk yes the preferred provider would be the mechanism i would use. The issue is they would cause rebuild. Meaning if i have a PN which is depended by many other recipes i will have to publish a state cache and build with both providers Feb 15 09:35:46 so if i do not cheat i need to deliver 2 builds if i want users to have sstate cache acceleration on both options Feb 15 09:37:22 ptsneves: I would not cheat sstate, if the hashes match exactly then sstate re-use is automatic, right? Feb 15 09:37:23 this cheating seems similar to SIGGEN_EXCLUDERECIPES_ABISAFE but not quite Feb 15 09:38:44 mcfrisk the hashes would never match exactly because hashes contain metadata which is always going to be different. It would be a different matter if bitbake would only decide on propagating the rebuild based on output differences. Feb 15 09:39:30 exactly, that's why I would not cheat sstate. If you can make sstate hashes equal, then things would already work.. Feb 15 09:39:50 and I do see the same problem, but no easy and efficient way to solve this.. Feb 15 09:41:36 i will explore the SIGGEN_EXCLUDERECIPES_ABISAFE and how it works internally and if nothing is promising i will just not allow the source build at all in the main branch :( Feb 15 09:41:50 ptsneves: if the source/binary decision is within a single recipe, then hashes would match. If the output of binary and source matches, then output hashes would also match. Then things would work. Feb 15 09:43:12 mcfrisk....hmmm i was going to answer that the metadata is different, but indeed i can vardepvalueexclude the src_uri for example Feb 15 09:43:15 this would force source build to also check the binary build and keep them in sync. In our solution/hack for same problem we always have the binary recipe out of sync and some people are not able to do local builds until issue gets fixed.. Feb 15 09:43:27 can I pass cmake targets as part of PACKAGECONFIG? Feb 15 09:43:57 ptsneves: I would not exclude SRC_URI, but force it to be in sync with both source and binary download URL. Feb 15 09:44:33 developers of the recipe need to know this and solve the chicken and egg situation before bitbake build passes though.. Feb 15 09:45:52 mcfrisk I think if we enforce the CI to always build from source(the situation is where somes devs are not allowed to see certain code) then we will have it enforced in the build Feb 15 09:46:01 in the CI gating i mean. Feb 15 09:46:43 mcfrisk regardless i think you gave me an exploration venue :) Feb 15 09:47:22 ptsneves: this is what we do, CI has access but some developers don't. problem is keeping the two recipes in sync. with single recipe that would work. sharing sstate to developers is tricky and we don't do that. have not validated what's in there... Feb 15 09:49:15 haven't read the whole thing, but what about hashserv? Feb 15 09:49:40 it is supposed to handle this if the output of some tasks is identical? Feb 15 09:50:26 Also you probably need the recipes to be in two different layers if you want the exact same name and PV Feb 15 09:51:07 Hello, how do I make 2 different kernel defconfigs for 2 different images with the same machine? the goal is to have a debug image and a classic image Feb 15 09:53:56 vygu2: have the debug distro modify the kernel config selection Feb 15 09:54:13 I want the same distro Feb 15 09:54:40 vygu2: well then you want some that you won't get Feb 15 09:55:45 ok it is not possible Feb 15 09:56:18 vygu2: its yocto chant #1 again Feb 15 09:56:42 mcfrisk  so my challenge is the same except i do not want to sacrifice state cache. What i will explore is having a single recipe with an SRC_URI that picks binary by default unless some conf is set to use source. To not thrash the state cache i make the conf variable that toggles the binary<->source as vardepexclude and vardepvalueexclude the Feb 15 09:56:42 SRC_URI part of the binary or source repo. In the task then i have a conditional that runs the app's build system if source mode is enabled. My concern would be that potentially the binary part would be out of sync, but we can make sure that any actual change in the source repo will lead to an automatic binary release to be used by devs. Feb 15 09:57:24 ptsneves: I think you need to build both source and binary all the time and make sure they don't differ in output Feb 15 09:57:54 qschulz where can i read about the harhserv. It sounds like a very big improvement on state cache handling, if it is able to prune the dep tree propagation when it finds the change in metadata did not produce a change in output. Feb 15 09:58:42 thanks Feb 15 09:58:44 mcfrisk another great tip! On the CI i can just compare the source build with what is stored in the binary. If it is not, break the release Feb 15 09:58:59 ptsneves: exactly, that forces the sync Feb 15 09:59:12 mcfrisk when i mean another great tip i mean that you gave me :) Feb 15 09:59:17 also when developers update the source, they don't forget to update the bninary Feb 15 09:59:39 mcfrisk yep. Ok sounds like i can try a PoC on my side. Feb 15 09:59:50 mcfrisk why did you not enfoce this checking on your side? Feb 15 10:00:16 ptsneves: we started our hacky solution with yocto 2.0 and still use it :) Feb 15 10:00:44 and don't dare to share sstate from CI with developers Feb 15 10:00:53 also due to IT infra... Feb 15 10:01:49 mcfrisk what if i told you we are in 2.1 :) On the process of integrating gatesgarth Feb 15 10:01:49 even download cache is problematic as on CI side that contains all source, also the ones where access should be restricted... Feb 15 10:02:03 ptsneves: lol! Feb 15 10:02:31 mcfrisk: there is nothing to say you can't have two DL_DIRs switched between by recipes Feb 15 10:04:09 ptsneves: hashserv is enabled by default since zeus IIRC Feb 15 10:05:16 * RP notes pseudo is broken with glibc 2.33 :( Feb 15 10:05:35 qschulz very very good then. I guess our multiple hacks are hindering, because I do not recall seeing some optimizations I would expect on such a clever new depedency checking. Feb 15 10:05:39 ptsneves: 3.1 actually from the changelog Feb 15 10:05:43 RP: "pseudo broken"? ;-) Feb 15 10:06:35 LetoThe2nd: no, totally ;-) Feb 15 10:06:46 RP: awwwwwwww Feb 15 10:06:47 ptsneves: couldn't find quickly where it's documented, maybe it's not yet in whcih case a bugzilla entry should be created if not already existing Feb 15 10:14:49 qschulz will talk with my colleagues about the topic. RPpseudo is a pain in the ass but is already showing us quality improvements in our path towards gatesgarth. Feb 15 10:16:06 ptsneves: hash equivalence it is also called Feb 15 10:19:55 qschulz thanks a lot. Is this "enabled" like this: http://git.yoctoproject.org/cgit/cgit.cgi/meta-yocto/commit/?id=29c3a4087c4f358ab88193fdf3787f731a4f4913 ? Feb 15 10:20:44 qschulz  so this is not by default neither for vanilla yocto, nor for our case where we have a custom local.conf.sample, which would explain why i missed this. Feb 15 10:21:29 ptsneves: https://stackoverflow.com/a/66189225 see JPEW answer on SO Feb 15 10:23:20 ptsneves: you're asking the wrong person though, I'm not using those cool kids' releases (yet?) and know nothing about hashserv except that I'm aware of its existence Feb 15 10:24:32 mcfrisk is your binary release divided per package or things are just a single bin blob for you? Asking for a friend Feb 15 10:24:34 :) Feb 15 10:30:22 qschulz will have a look. By the way we have an NFSv4 serving sstate cache for several years now and for hundreds devs with no notable issues. Recently we had to re-do poky's state cache cleanup script because the bash version was too slow for several TB cleanups. We will provide it as an RFC on the mailing list soon. Not as a patch because we are Feb 15 10:30:23 actually not implementing the removal functionality, only the collection part. The removal in our infrastructure is a bit of a critical step and needs to be done more carefully then just rm . Feb 15 10:38:44 ptsneves: our release is ipkg repo and product images for flashing, so both Feb 15 10:58:33 also, release creates a sstate mirror which is the used by all followup releases and CI builds which are based on that release Feb 15 11:00:00 Hi guys, i'm having an issue while building a yocto image. I have included meta-openembedded/meta-oe and when I try to build it I have : Could not inherit file classes/python3targetconfig.bbclass Feb 15 11:00:33 jojo28: triple check that all layers are using the same branch Feb 15 11:02:02 qschulz : okay, I will check this. I come back once it is done. Thanks for the tip. Feb 15 11:16:24 qshulz : I have three layers : Feb 15 11:16:24 meta-qt5 Feb 15 11:16:25 meta-raspberrypi Feb 15 11:16:25 meta-openembedded/meta-oe Feb 15 11:16:26 All three are on the gatesgarth branch now. But the issue is still there :/ Feb 15 11:17:32 jojo-the-real-de: you have more layers than this, or you forgot to add some because this cannot work :) Feb 15 11:17:50 you need at the very least openembedded-core layer which is available in its own git repo or in poky/meta Feb 15 12:28:54 hello.bitbake reports ENOSPC but I see there is a lot of space(df -ih). Ιt is an LVM partition 897gb.Is there something I am missing? Feb 15 12:34:07 kayterina: and what about df -h? Feb 15 12:35:24 size 883G Used 39G, Avail 800G Use 5% Feb 15 12:36:07 kayterina: then you are probably missing something somewhere. put both in a pastebin, please. Feb 15 12:36:18 ok,I'll do Feb 15 12:38:27 kayterina: plus, how did you diagnose that its ENOSPC and not something else? please add the offending log snippets also. Feb 15 12:39:22 this is the error: https://pastebin.fun/AqGqTN8wiZ Feb 15 12:40:49 kayterina: this should have nothing to do with real space, only with inotify watches, as i read it Feb 15 12:49:09 max watches says 65536. If I googled correctly I use 3723 (lsof | grep inotify | wc -l) Feb 15 12:50:48 kayterina: but probably not during a bitbake run. Feb 15 12:52:00 just for reference, on my build machine this is 512k Feb 15 12:52:36 aha.ok,thaks Feb 15 13:39:19 qschulz : You're right, sorry i wasn't exhaustive. I have : meta, meta-poky, meta-yocto-bsp, meta-raspberrypi, meta-oe . Feb 15 13:39:20 I build core-image-sato Feb 15 13:56:14 created a new distro and changed the local.conf to reflect that. disctro conf includes DISTRO_FEATURES += "opengl x11 vulkan" among other things themn bitbake core-image-minimal. despite a successful very long compilation there is no wic filesystem in poky/build/tmp/deploy/images/intel-corei7-64/ Feb 15 13:56:27 am i missing something here? Feb 15 14:04:48 intera91: well something, somewhere has to actually say that you want a wic image.... Feb 15 14:05:23 LetoThe2nd: sounds good but where would that be? Feb 15 14:06:09 intera91: a good place is usually the machine config, and setting IMAGE_FSTYPES there. Feb 15 14:10:09 seebs, fray: If you'd like nightmares, the makewrappers part of http://git.yoctoproject.org/cgit.cgi/poky-contrib/commit/?h=rpurdie/t222&id=90dd9005b484a9ddb3b6e17c3b14fc3764fc9a2c is 'fun' :/ Feb 15 14:12:05 RP: cue: https://youtu.be/aOeP4p1fjMs Feb 15 14:15:38 LetoThe2nd: I have included meta-intel in my bblayer.conf, the meta-intel layer contains a machine.conf (intel-corei7-64.conf) which already has a IMAGE_FSTYPES += "wic" line Feb 15 14:16:17 and that's the machine in my local.conf Feb 15 14:16:41 intera91: well then time to bitbake -e you image build and start looking. Feb 15 14:20:28 qschulz, qtsneves: I point I didn't make in the SO post about hash equiv: It's enabled in local.conf.sample in 3.0 and poky.conf enabled by default in 3.1 Feb 15 14:33:42 LetoThe2nd: loads of output to that command but no indication where the wic file is/should/would be created Feb 15 14:34:12 intera91: why don't you pipe it into grep and look at IMAGE_FSTYPES? Feb 15 14:34:41 intera91: i think i have demonstrations of that in almost every episode of https://www.youtube.com/playlist?list=PLD4M5FoHz-TxMfBFrDKfIS_GLY25Qsfyj Feb 15 14:36:05 intera91: the variables and assignment one even very prominently, including a lot of explanations and discussion. Feb 15 14:36:45 LetoThe2nd: watching now many thanks ;-) Feb 15 15:40:02 bandito_embedded: and i've just checked that python3target.bbclass exists since gatesgarth in meta/classes - so please go and check if its there. Feb 15 15:45:25 eww Feb 15 15:50:49 seebs: I haven't documented the problem yet but you can probably guess? Feb 15 15:51:44 seebs: I'm open to any better idea if you have one... Feb 15 15:54:26 so... if you have an empty path, and the directory you're in is a symlink, we were following symlinks and this was surprising? Feb 15 15:57:21 seebs: the problem is a call like fstatat64 (pathfd, "", &st, AT_EMPTY_PATH) to a symlink Feb 15 15:58:01 Hi. According to this https://layers.openembedded.org/layerindex/recipe/121520/, I should have i2c-tools recipe file within the oe layer but I could find it under pokey/meta-openembedded/meta-oe/recipes-devtools after cloning pokey. Shouldn't I see it there? Am I missing something here? Feb 15 15:58:02 seebs: pseudo_root_path changes the pathfd to an actual path but AT_NOFOLLOW_SYMLINK isn't set, only AT_EMPTY_PATH Feb 15 15:58:08 ohh. Feb 15 15:58:58 seebs: AT_EMPTY_PATH with path of "" implies AT_SYMLINK_NOFOLLOW Feb 15 15:59:35 yeesh Feb 15 16:07:41 Where should developers post yocto related questions? Feb 15 16:08:06 is there a yocto of forum for questions? Feb 15 16:08:27 I think it's the good place Feb 15 16:08:32 You mean poky ? it's in poky/meta/recipes-devtools/i2c-tools/ Feb 15 16:09:46 when in doubt find and grep can help, especially find . -not -path '*/\.git/*' | fzf -e -m --tiebreak=begin Feb 15 16:10:30 maybe there's a bitbake tool for that but I don't know it, I'm new :) Feb 15 16:10:55 yup it is there. Thx! I thought it would be under oe since it is part of it. Feb 15 16:11:53 moose: well, openembedded-core is named meta, but meta-oe is a separate layer https://layers.openembedded.org/layerindex/branch/master/layer/meta-oe/ Feb 15 16:12:22 Gottcha. Thx! Feb 15 16:12:39 Sorry but I am new to yocto platfrom Feb 15 16:14:35 How can I add i2c-tools to my image? I thought it would be there by default since I have meta layer in my bblayers.conf but my image doesn't have it. Feb 15 16:14:38 kyanres: bitbake-layers show-recipes i2c-tools gives all the layers in which a recipe is Feb 15 16:15:17 moose: https://www.youtube.com/playlist?list=PLD4M5FoHz-TxMfBFrDKfIS_GLY25Qsfyj Feb 15 16:15:23 this will help you get started Feb 15 16:15:48 layers are just a collection of recipes and configuration files Feb 15 16:16:12 you need to explicit which packages you want to include in the image you want to create Feb 15 16:17:35 Okay thanks! so I guess I need to specify in my conf file that I want i2c-tools recipe included somehow. I will check it in the video. Feb 15 16:31:52 JPEW: good news is we were testing reproducibility correctly on the autobuilder Feb 15 16:33:18 moose: you want a package out of the i2c-tools package recipe (that is an important difference) Feb 15 16:34:29 qschulz, thanks. I'm glad show-recipes exists, but it's "slow" Feb 15 16:34:35 With *i2c* as example, show-recipes takes 3 seconds, while find takes ~10ms. Obviously, since show-recipes has to parse everything Feb 15 16:34:43 btw I tried setting BB_SERVER_TIMEOUT="-1", but it's the same Feb 15 16:38:22 RP: Ah, that's good Feb 15 16:39:13 kyanres: it's parsing all recipes and all layers, so yeah, it's not fast Feb 15 16:39:31 but unlike find, you pretty much know if Yocto has knowledge of the recipe and can use it Feb 15 16:39:48 (I don't use it either, but you asked for a command, I shall provide you with one :p) Feb 15 16:41:41 JPEW: I was struggling to believe we found that many issues without it working at least partly right :) Feb 15 16:43:55 RP: Ya, I was worried about that too. Hopefully the logging patches will shed some light on the timing Feb 15 16:44:28 I don't doubt that diffoscope takes a long time on large diffs, but that wasn't the culprit (near as I can tell) on the vim build Feb 15 16:47:52 JPEW: its always the builds where there is a difference :/ Feb 15 16:49:05 JPEW: having this separately visible on the autobuilder should help us get to the bottom of it Feb 15 16:54:55 "I don't use it either" -> LOL, made my day ! Feb 15 17:05:02 JPEW, kanavin: There looks to be a load of low hanging issues in the remaining reproducibity list. Just wondering what to do with the lists/info... Feb 15 17:05:19 * RP wonders about opening bugs Feb 15 17:15:33 RP: Seems reasonable Feb 15 17:19:51 JPEW: half these look easier to fix with patches than file bugs! :) Feb 15 17:25:29 RP: Hmm, sounds like a reporting problem more than anything then Feb 15 18:35:43 LetoThe2nd : Thank you very much! checkout gatesgarth and it works, thanks;) Feb 15 18:55:27 whats the recommended way to cache the uninative tarballs on a mirror ? Feb 15 20:11:25 dl9pf: I remember that being trickier than ideal, I wish I remembered the details. I think it needed a slightly more interesting url map than normal Feb 15 20:43:53 @yocton I forgot to thank you for your suggestion earlier. I am trying it out now, but didn't want to come to IRC for help then immediately bail :) Feb 15 21:12:56 JPEW, kanavin: https://autobuilder.yocto.io/pub/repro-fail/2021-2-15-rp/rp/ is diffoscope on some of the regularly failing repro packages Feb 15 21:14:30 RP: I think I patched bison to take -fdebug-prefix-map arguments, which should fix the acpica difference Feb 15 21:15:44 bluelightning: its 45 mins, I think I have the wrong calendar, sorry Feb 15 21:16:22 JPEW: right, there are a few of these which look fairly obvious. watchdog looks like a varying sendmail path from the host Feb 15 21:17:03 babletrace2 looks gross. Must be using the host python version? *why*! Feb 15 21:17:14 diamondman: No problem :) I hope that'll work Feb 15 21:18:41 @yocton: It looks good. I honestly didn't know you could invoke a task in multiple recipes with a single command, so I was going after building dependency trees instead. This makes a lot of sense Feb 15 21:19:17 cool :D Feb 15 21:24:41 yeah this looks like it will work! Thanks! Feb 15 21:28:28 JPEW: might be possible just to sed that out Feb 15 21:30:09 RP: I think I've got the apcica one Feb 15 21:30:41 JPEW: cool. I'll take watchdog :) Feb 15 21:30:55 JPEW: you can see what I mean about it being easier to fix some than open bugs! :) Feb 15 21:45:09 sorry i'm late. Here now. Feb 15 21:45:17 (kidding, I've been watching the chat). Feb 15 21:45:35 and I'm in the wrong channel Feb 15 21:45:37 monday. Feb 15 21:45:43 that joke failed! Feb 15 21:45:58 * zeddii goes to the TSC channel. Feb 15 21:57:19 zeddii: I managed that too :) Feb 15 22:32:51 sudo Feb 15 22:56:59 Password: Feb 15 23:02:46 linux-yocto SUBLEVEL seems to increment upwards long past upstream sources. Is there any documentation that explains what is happening there? Feb 15 23:05:27 (and to be clear, I understand that this is probably yocto specific patching that goes beyond upstream, but I was hoping to get some understanding on how it is organized) **** ENDING LOGGING AT Tue Feb 16 03:00:33 2021