**** BEGIN LOGGING AT Thu Sep 24 09:52:28 2020 Sep 24 10:10:21 RP: glad to help a tiny bit. Btw. feel free to replace the double backslahes in the test commands with simpler r'' strings. Sep 24 10:11:27 I just saw that I didn't sign-off the second commit. Do you need me to provide another patch revision? **** BEGIN LOGGING AT Thu Sep 24 10:13:58 2020 Sep 24 10:14:14 * RP wonders if anyone knows something about SIGPIPE error handling and python Sep 24 10:14:42 @RP related to process termination? Sep 24 10:14:53 https://autobuilder.yoctoproject.org/typhoon/#/builders/87/builds/1400/steps/8/logs/step2d failed with a SIGPIPE afaict. Yet python should have a default of ignoring it :/ Sep 24 10:15:31 ptsneves: yes Sep 24 10:16:04 ah, bin/bitbake-layers: signal.signal(signal.SIGPIPE, signal.SIG_DFL) Sep 24 10:16:14 I wonder why it does that Sep 24 10:16:31 That fits since its bitbake-layers erroring Sep 24 10:16:48 * RP just spent a hour without seeing that until I ask here :) Sep 24 10:16:48 yeah...it does not make much sense. Had some issues with zombie bitbake workers but never on bitbake-layers Sep 24 10:18:05 even so from the log i see nothing about SIGPIPE... Sep 24 10:18:15 http://git.yoctoproject.org/cgit.cgi/poky/commit/bitbake/bin/bitbake-layers?id=bcab5e12c1fa9c6e2d828ddadd3659eceb4b387d Sep 24 10:18:26 ptsneves: exit code 141 is SIGPIPE Sep 24 10:18:46 ah ok :) Sep 24 10:19:51 hmm does bitbake-layers have workers? Sep 24 10:20:01 ptsneves: no Sep 24 10:20:21 RP so to what does it communicate with? Sep 24 10:20:26 * RP is going to revert that patch. The error described doesn't happen even when I do what the commit says Sep 24 10:20:33 ptsneves: it starts a bitbake server Sep 24 10:20:41 but the server never starts workers Sep 24 10:21:02 UI <---> Cooker <---> Workers Sep 24 10:21:17 bitbake-layers is a UI and starts a cooker (server) Sep 24 10:21:55 RP but if you revert the commit will you not have possibly a zombie bitbake-layers? What is the cause of the cooker closing unexpectedly? Sep 24 10:24:17 ptsneves: the server knows how to look after itself. I think its just races between the client and server and it would have gotten over the issues if the SIGPIPE killer hadn't taken out one of the processes Sep 24 10:30:26 RP sorry if my question is stupid, but wouldnt there be a problem to the bitbake-layers if the server closed the pipe and bitbake-layers did not react? Or this is taken care? Sep 24 10:30:43 I know very little of the bitbake-layers code so sorry if it makes no sense Sep 24 10:30:59 ptsneves: The broken pipe will raise a python exception rather than just killing the process Sep 24 10:31:42 ah ok, so yeah then i do not understand the goal of that commit Sep 24 10:32:25 The original commit was to avoid some error when piping the output to less. That doesn't seem to happen and we've evidence its breaking things I care a lot more about so we'll revert :) Sep 24 10:32:47 but then it may mean that the commit wanted to let the called know there was a broken pipe and hide the possibly unhandled exception Sep 24 10:33:16 ptsneves: well, I can't reproduce the error the commit mentions Sep 24 10:33:17 s|called|caller|g Sep 24 10:34:22 RP well it was reproduced in the test. Is the pipe error supposed to happen in that test? Sep 24 10:35:13 ptsneves: no, I mean in the commit above "bitbake-layers show-recipe | less" was meant to show an error. It doesn't. Something changed in the three years which makes that no longer an issue Sep 24 10:35:42 RP: *thumbs up* Sep 24 11:18:09 RP: I hope it's fine now. I eventually wrote a cover letter to describe the patch series' revisions, but it was somehow scrapped or whatever -.-' Sep 24 11:18:42 let me know if I need to provide something else. Sep 24 11:26:29 T_UNIX: Just looking at the patch again, the globbing raises some questions. Does the fact you needed to change the globbing mean we're missing more test cases? I thought we didn't support globing any more? Sep 24 11:43:18 RP: quoting the string wold break the globbing test. So I've introduced `glob.glob` to expand the filenames and put them into quotes. Sep 24 11:44:51 It's funny that you're suggesting that bitbake would not support globbing anymore. My first thought, when read the code, was: "Is there a reason it is done like this in the first place?". Because it was such a simple fix to just put quotes. Sep 24 11:45:31 in meta-poky there is this rngd.service (rng-tools/rngd.service), it contains this line 'ExecStart=@SBINDIR@/rngd -f $EXTRA_ARGS' Sep 24 11:45:38 how do i override EXTRA_ARGS from my own layer? Sep 24 11:46:50 i've tried adding a 'default' file in /recipes-support/rng-tools/rng-tools/defaults Sep 24 11:47:08 is that correct? it didn't seem to work for me Sep 24 11:48:05 hi, can i see on target board the yocto version someway ? Sep 24 11:49:38 T_UNIX: there isn't a globbing test any more and globbing support was removed? Sep 24 11:50:34 T_UNIX: http://git.yoctoproject.org/cgit.cgi/poky/commit/?id=1560a4b0cbb8b7ad12623635c64ce43d962e8dce Sep 24 11:51:25 https://github.com/openembedded/bitbake/blob/27b53186fa67d281d29b2f8e15bcff8dc2557b8a/lib/bb/tests/fetch.py#L619 Sep 24 11:51:55 T_UNIX: modified to error if the user uses a wildcard? Sep 24 11:52:29 grr.. you're right :-D Sep 24 11:53:13 I'm running an older revision. So I keep copy/pasting between the bitbake repository and my yocto instance -.-' Sep 24 11:54:00 T_UNIX: your patch is probably appropriate for dunfell but not for master in that form! Sep 24 11:54:03 sakoman: ^^^ Sep 24 11:55:44 RP: does anybody care for such a bugfix for older revisions then? Sep 24 11:56:21 I'll change the code back to the initial, simple quotes fix. Sep 24 11:56:25 T_UNIX: well, there is still the quoting problem on master and we do have a stable series which people are using Sep 24 11:56:48 T_UNIX: hence my note to sakoman to suggest he may want to think about this Sep 24 12:00:26 so update the patch series for master to use quotes (since globbing is explicitly *not* supported)? Sep 24 12:00:59 Is it possible to run bitbake selftest standalone? Or does it depend on poky/oe stuff to execute? Sep 24 12:01:12 so I donot mixup stuff anymore -.-' Sep 24 12:01:30 T_UNIX: bitbake-selftest just needs the bitbake repo, its standalone Sep 24 12:01:45 T_UNIX: and yes, for master lets use the quotes Sep 24 12:37:49 If I want a subset of the files from make install, what is the recommended way there in the bb recipe? Sep 24 12:44:48 lxc: put the files you don't want in another package? Sep 24 12:45:34 I've done that (the other package), or rm'd them in the do-install, there's a few options. Sep 24 12:45:37 this package needs to be declared in PACKAGE_BEFORE_PN (or **prepended** to PACKAGES) and then you set the correct FILES_${PN}-newpackage to match the files you don't want Sep 24 12:50:41 RP: just to be precise on this: You do not want any expansion? So it'll use `'`' instead of `"`? Sep 24 12:51:20 T_UNIX: I don't mind expansion, we just don't need/want the globbing code. Sep 24 12:51:26 sorry for being unclear :) Sep 24 12:55:35 qschulz thanks, I'll try that. Sep 24 12:56:56 RP: Does the URI parser support variables? Sep 24 12:57:19 T_UNIX it does Sep 24 12:57:37 T_UNIX bitbake ones Sep 24 12:59:55 allright :) Sep 24 13:09:36 RP: I hope I didn't miss anything in v5 :-D I ran `bitbake-selftest -v bb.tests.fetch.FetcherLocalTest` on master + patches and everything looked fine. Sep 24 13:10:30 just for the record: I've inserted double quotes (to eventually expand variables) and used python raw strings in the test's strings. Sep 24 13:14:50 T_UNIX: looks good, thanks. I'll queue and double check all is well :) Sep 24 13:21:30 thank you for your patience and for dealing with my insufficient `git send-email` "skills" ;-) Sep 24 13:29:24 RP what is the reason the project does not use gitlab or something similar? Has it been considered? Sep 24 13:30:30 I would like to include a docker image in a yocto-built image. Any ideas? Sep 24 13:30:31 One attempt would be to fetch the tarballed docker-image from the recipe, and simply install it somewhere on the image. Then on the first boot I could `docker load` it. Sep 24 13:30:31 Any ideas of how to include the image directly? Either by `docker pull`ing it from the recipe, or fetching a tarball? Sep 24 13:34:59 ptsneves: history and now switching would be a huge disruption. We are trying to evaluate it but its not something anyone has the time to try and drive Sep 24 13:35:31 @RP Sep 24 13:35:37 thanks for the explanation Sep 24 13:38:41 ptsneves: https://wiki.yoctoproject.org/wiki/Future_Directions - comes under Code Submission process Sep 24 13:39:02 manuel1985: we've talked about this a few times at some of the summits and on the meta-virt mailing list. In fact, I'm doinga bit of container storage enhancements to the OCI images for the next release, but they will only be for yocto built images. Sep 24 13:39:12 anything that is 'pulled' is a licensing and support nightmare. Sep 24 13:39:20 I see Sep 24 13:39:24 so the approaches you are thinking of will work, but they'll be up to you to implement Sep 24 13:40:08 since we can't really do support like that in the main layers. a bbclass could be created to allow it, kind of like the binary bbclass. but that has the same licensing and support issues, etc Sep 24 13:42:58 Didn't know about the binary bbclass, will take a look in there. Perheaps I can steal some ideas from there. thank you :) Sep 24 13:43:05 Number of full builds that passed in the last week: 2 failed: 14 :/ Sep 24 13:43:49 5/7 quick passed I suppose (which surprises me) Sep 24 13:44:06 How can I call docker from within a recipe? Until I have server to pull binary artifacts (like tarballed docker images) from, I would like to call `docker pull` from within the recipe. Sep 24 13:44:32 that's not easy to do. Sep 24 13:44:40 hence why it isn't a great idea. Sep 24 13:44:45 RP are the full builds failing in opposition to the normal state cache ones? Sep 24 13:44:46 you need docker as a host tool. Sep 24 13:44:51 and docker is kinda evil. Sep 24 13:45:22 most of the tools that deal with OCI images have a huge set of dependencies, hence why none of them are easy to make into host tools. Sep 24 13:46:13 you could escape from the OE/Yocto env and call it, by making it a host dependency for your build though. Sep 24 13:46:17 and then could interact with it. Sep 24 13:46:42 but you'd have to have it configured to put its container storage into your build, so you could then build it into the image. Sep 24 13:46:59 and have it there, so some on target docker could load and run it at boot. Sep 24 13:47:16 we've also used pre-populated local registries, so a local docker can pull and run on the target. Sep 24 13:47:37 but none of the solutions are particularly 'robust', hence why we don't have something fully in meta-virt yet. Sep 24 13:48:47 ptsneves: the full builds were more heavily loaded and probably exposed more race issues or were test builds Sep 24 13:48:59 zeddi: Thank you, still digesting your words... Sep 24 13:49:49 this is a long thought on topic :D Sep 24 13:50:23 Silly question, I'm not an ultimate beginner but do not yet consider myself advanced: What does OCI stand for? Sep 24 13:50:25 if the videos from the Lyon yocto summit are available, we discussed it in person then, I can't recall if we went into it in the spring ELC Sep 24 13:50:32 open containers initiative Sep 24 13:51:01 it underpins docker (having been donated and derived from docker "the companies" work). Sep 24 13:51:18 we have multiple runtimes in meta-virt capable of launching OCI containers, and meta-virt can build them directly. Sep 24 13:54:07 Do I need to explicitly list all files in the FILES variable? Or can I give directory? I know wild card seems to work, e.g. *.so Sep 24 13:55:33 lxc need to be explicit as far as i know Sep 24 14:00:03 lxc: directory is fine Sep 24 14:00:17 +1 Sep 24 14:00:46 lxc: https://git.yoctoproject.org/cgit/cgit.cgi/poky/tree/meta/conf/bitbake.conf you can see the defaults here Sep 24 14:02:32 and weirdly enough... we both have directory and directory/* for some reasons /me :thinking: Sep 24 14:03:10 qschulz thanks for the link. If there is sub-directories in the path, do I need to list parent directory only? yeah, also noticed the combo dir and dir/* Sep 24 14:03:39 lxc: honestly? try and you'll see for yourself. It takes a minute or two to test :) Sep 24 14:04:04 lxc: oe-pkgdata-util find-path to see in which package it ends up Sep 24 14:04:36 lxc: gut feeling says it includes subdirs Sep 24 14:06:04 lxc: ooooooh! maybe that's the difference! Sep 24 14:06:21 directory/* only files? directory => everything? Sep 24 14:06:23 no idea :D Sep 24 14:20:51 hello, does anyone here have experience using open-plc-utils to connect devices using the qca7000 driver? I know this is off-topic, but I don't know where else I could ask for help. Sep 24 14:23:08 eduardas i think the kernel people would be able to help you faster Sep 24 14:23:35 there are patches in the kernel from devs for that driver. Maybe trying cold contact would get you expert help Sep 24 14:36:47 anyone seen sgdisk hang and block during wic image creation? Sep 24 16:51:25 I'm getting this error: do_kernel_metadata: A KBUILD_DEFCONFIG 'my_defconfig' was specified, but not present in the source tree Sep 24 16:51:37 do_kernel_metadata: A KBUILD_DEFCONFIG 'my_defconfig' was specified, but not present in the source tree Sep 24 16:52:07 but when I find -type f -name my_defconfig, I can see that the file is in my source tree Sep 24 16:52:21 ./yocto/poky/build/tmp/work-shared/my_machine/kernel-source/arch/arm/configs/my_defconfig Sep 24 16:53:24 is it possible the build system isn't looking in arm/configs/ because it's not looking for the arm architecture? Sep 24 16:54:07 The log file was less helpful, it says "not present in the source tree" but it doesn't actually indicate where it's looking Sep 24 16:54:29 arch/$arch/configs/ Sep 24 16:54:33 but that runs before do_patch Sep 24 16:54:40 so how is that defconfig appearing in the tree ? Sep 24 16:54:59 if you are carrying such a defconfig, just carry it in the SRC_URI, the KBUILD_DEFCONFIG is a bolt on for supporting mainline defconfigs. Sep 24 16:57:05 zeddii: I can see the defconfig file when, outside of yocto, I clone the kernel repo; it exists in /arch/arm/configs/ Sep 24 16:57:18 So it's not something I'm brining along with my recipe Sep 24 16:58:12 the code only looks in one place, so whatever reason, it isn't there. without your recipe, I can't say more. Sep 24 16:58:31 but I've already tweaked the bbfatal to me more explicit, so will send that patch. Sep 24 18:30:25 wow, a green master-next Sep 24 18:51:19 * armpit green with envy Sep 24 18:57:36 green like the hulk Sep 24 19:00:21 someone finally tweaked the css? :) Sep 24 19:07:59 I mean it's really discourteous of the build system to tell me the file isn't found but not to tell me where it looked Sep 24 19:10:40 ecdhe: bitabke or something we're running underneath? We might be able to improve the former, the latter, not so much Sep 24 19:27:58 RP: the error is ERROR: linux-custom-dev-4.14.0-r0 do_kernel_metadata: A KBUILD_DEFCONFIG 'my_defconfig' was specified, but not present in the source tree Sep 24 19:28:10 I'm not sure what tool spits that out Sep 24 19:28:34 * zeddii already mentioned that I have a patch to clarify that. Sep 24 19:28:46 * RP was about to look at zeddii :) Sep 24 19:28:52 but I've already tweaked the bbfatal to me more explicit, so will send that patch. Sep 24 19:29:08 that being said, it's three years old, so only so terrible a message ;) Sep 24 19:29:16 zeddii: I think I found my immediate issue Sep 24 19:29:32 * zeddii calls it a day. Just did two presentations in the last hour. I'm baked. Sep 24 19:30:06 I had a working build for one board; I'm adapting it to a new board, so I swapped out the linux source tree, source rev, and defconfig options to make the layer build for the new board. Sep 24 19:30:35 * RP has late meetings so is playing with pseudo Sep 24 19:30:57 Turns out the defconfig for the old board was in arch/arm64/configs, but the new config was in arch/arm/configs (without the 64) Sep 24 19:31:17 I'm trying to figure out how to change the arch from arm64 to arm Sep 24 19:31:47 RP: fired off the one-liner patch. now I'll flee for a bit. Sep 24 19:32:28 ecdhe: that arch value comes from the kernel bbclasses (not the same one I just patched), and maps the bitbake arch you are building for, to a kernel one. Sep 24 19:32:49 see meta/kernel-arch.bbclass Sep 24 19:33:21 the input arch comes from however your BSP is including tune files, etc. Sep 24 20:28:35 kergoth: is there a way to ask bitbake to dump SRC_URI for all recipes going into an image ? Sep 24 20:31:36 * RP has a path filtering pseudo which doesn't segfault :) Sep 24 20:37:35 RP: cool. Sep 24 20:44:38 RP: Excellent! Sep 24 20:46:02 Its now undergoing some wider local testing. It feels faster Sep 24 20:47:26 RP woo! Sep 24 20:49:16 fray: not sure I dare show you or seebs what I did ;-) Sep 24 20:53:23 khem: unless it's in the metadata in the resulting binary package, which may be the case, probably not easily Sep 24 20:56:06 Ah well, it blew up in JPEW's hashequiv code :) Sep 24 21:01:38 lol Sep 24 21:03:04 kergoth: I was hoping if we could dump it Sep 24 21:03:14 during parse or something Sep 24 21:03:31 and its actually misconfiguration for recipes where WORKDIR == S Sep 24 21:04:04 khem: the trouble is bitbake has no idea which recipes would go into an image, it just knows what the image's dependencies are which is different Sep 24 21:04:23 khem: once you have the list of recipes in an image you could ask bitbake via tinfoil api for example Sep 24 21:05:54 RP: Ya... not surprising Sep 24 21:06:16 JPEW: our old friend Exception: KeyError: 'getpwuid(): uid not found: 1000' Sep 24 21:06:36 RP: Hmm.... ya Sep 24 21:06:43 you can use the oe python modules to get a list of installed packages, then use pkgdata to map that to recipes, then look up the recipe data with tinfoil.. Sep 24 21:06:58 hmm Sep 24 21:07:19 RP: I'm not sure what to do there.... just continue with the numeric ID (instead of the name) and let it fail later? Sep 24 21:07:47 I remember I orginally was using the numeric ID and that cause some consternation Sep 24 21:07:49 JPEW: we should just make it print a human readable error Sep 24 21:08:19 JPEW: I did 'improve' it to print WARNING: base-files-3.0.14-r89 do_package: KeyError in . Sep 24 21:08:31 I now locally have an abspath() in there which is much nicer Sep 24 21:09:09 khem: you could always add a task to the recipes to emit their SRC_URI to a common path, then use an image postprocess to map package to recipe and then grab that and write it where you need it, presumably? Sep 24 21:10:19 JPEW: using the names is definitely correct there and its a bug that my code was breaking the lookup in this case Sep 24 21:11:00 Ok Sep 24 21:11:38 JPEW: I'm slightly bemused as that code has errored multiple times during testing of this, its quite the canary Sep 24 21:11:59 and the build is now progressing again Sep 24 21:12:09 RP: Ya. I think it bascially fails whenever one of the QA checks would have failed later Sep 24 21:12:25 JPEW:right, and in some cases where it wouldn't too Sep 24 21:12:31 I have a prototype for making your local hashequiv server pull hashes from an upstream server, but it's minimum Python 3.5. Is that going to be a problem? Sep 24 21:12:51 JPEW: I think we have 3.5 as a minimum now? Sep 24 21:13:56 Looks like sanity.bbclass still says 3.4 Sep 24 21:14:06 JPEW: raise RuntimeError("Sorry, python 3.5.0 or later is required for this version of bitbake") from lib/bb/__init__.py Sep 24 21:14:26 Ah. OK, cool Sep 24 21:14:34 sounds like sanity needs fixing Sep 24 21:14:45 RP: Does it even need to check at all? Sep 24 21:15:09 JPEW: good point Sep 24 21:15:23 JPEW: I suspect it was for old bitbake with new core? Sep 24 21:15:32 Possibly Sep 24 21:16:23 hehe, although it looks like the check was a little off since it would have allow 3.4 Alpha 1 to be used, which I don't think was the intention :) Sep 24 21:17:04 JPEW: anyone using alpha 1 of 3.4 knew what they were doing Sep 24 21:51:16 kergoth: good idea Sep 24 21:53:04 If anyone wants to play, http://git.yoctoproject.org/cgit.cgi/poky-contrib/commit/?h=rpurdie/t222&id=df9c31147e093e7a588ce0789616997bca16663b is the pseudo path filtering as it stands Sep 24 21:53:18 http://git.yoctoproject.org/cgit.cgi/poky-contrib/commit/?h=rpurdie/t222&id=d40c660748b54d9d70219ecbaed52e8ebfd7b2a0 is how to enable it (and the metadata tweaks needed) Sep 24 21:54:56 fray: I'm interested on your thoughts on the first patch, I know I need to avoid the copy and paste of a block of the code in there. Right now I just wanted to check if that was all I needed. Sep 24 22:00:00 Hey guys, got a question regarding a software recipe I'm building. Target hardware, Jetson Nano Dev Kit. Software is a cmake project that utilizes libpq, basically does some communication with an energy metering IC and then inserts the data into a postgresql database table. psql is installed on the server that has yocto/bitbake. I've installed the Sep 24 22:00:00 correct version of devtools to match the psql version (12.2). When I try to run the build I get the typical cmake error `Could NOT find PostgreSQL (missing: PostgreSQL_LIBRARY Sep 24 22:00:01 the compilation to and just use an archive in the install. Anyone run inot this issue and could lend a hand? Sep 24 22:02:05 If it's easier to just offload the compilation to the dedicated nano, I'll just go that route. BUT obviously i'd like to keep as much of it in the yocto build as possible Sep 24 22:03:27 Also, open invite if anyone wants to assist with this image build I'm working on I'd be happy to send some beer money your way, haha Sep 24 22:05:38 pandemic#0001 on discord or find me by a google search of my username '=D Sep 24 23:05:05 Looks like I broke chroot support in pseudo but otherwise the build wasn't so bad Sep 25 02:46:16 RP: https://blog.ram.rachum.com/post/621791438475296768/improving-python-exception-chaining-with **** ENDING LOGGING AT Fri Sep 25 02:59:57 2020