**** BEGIN LOGGING AT Wed Oct 16 02:59:57 2019 Oct 16 08:31:45 typo of the day: "git duff" Oct 16 08:43:41 Rofl! Excellent typo! Oct 16 09:08:21 a function in a bbclass goes to exception when try to execute "if not os.path.exists(url.localpath):", this function was originally to unpack a downloaded package which now I've placed local, any idea how can we fix this Oct 16 09:09:18 the value of "url.localpath" is none when file is place locally, it have valid value when downloaded from http:// location. Oct 16 09:14:06 i thought localpath was still valid if you're driving the fetcher right Oct 16 09:14:14 sounds like you're calling into fetch2 directly? Oct 16 09:15:52 oh right Oct 16 09:15:52 # localpath is the location of a downloaded result. If not set, the file is local. Oct 16 09:15:59 if local, then just use the uri Oct 16 09:16:56 the localpath method should resolve it for you Oct 16 09:18:45 rburton, uri.locapath? Oct 16 09:28:39 hey folks, could yall help me clarify some terms - im going to try to state the terms and could you tell me if my understanding is correct? Oct 16 09:29:15 bitbake is a build tool - it defines the syntax for recipes and so on Oct 16 09:29:58 milloni: bitbake is more like a task scheduler and runner, albeit a very specialized and sophisticated one Oct 16 09:30:18 openembedded is a set of classes and recipes that constitute a build system - i.e one can use those recipes or create own's own with the use of stuff defined in openembedded Oct 16 09:30:22 milloni: buildtool or buildsystem usually rather refers to things like autotools, cmake, meson, whatever. Oct 16 09:30:52 poky is reference distribution built on openembedded Oct 16 09:30:55 nope, thats outright wrong. Oct 16 09:31:02 okay, correct me Oct 16 09:31:03 (the openembedded thing) Oct 16 09:32:50 milloni: why not look up how the things define themselves? right the first sentence on openembedded.org is "Welcome to OpenEmbedded, the build framework for embedded Linux" Oct 16 09:33:53 that's my interpretation of this - "the build framework", i.e recipes, classes, etc? Oct 16 09:33:58 no Oct 16 09:34:10 what you are interpreting is "metadata" Oct 16 09:35:27 but if you look at https://git.openembedded.org then you will see that OE actually hosts bitbake and some sets of metadata. hence, OE cannot be defined in terms of metadata only. Oct 16 09:36:11 hmm okay - so is it bitbake + metadata? Oct 16 09:36:37 isn't that what i just wrote almost verbatim? Oct 16 09:37:12 yeah im just trying to make sure i understand correctly Oct 16 09:38:03 looks like i wasnt wrong in the essence :) Oct 16 09:40:09 and then yocto encompasses openembedded, poky and some extras (whatever is pertinent to embedded linux development)? Oct 16 09:41:03 milloni: openembedded is *NOT* part of the yocto project. Oct 16 09:42:05 milloni: YP uses it, and in turn offers a lot of maintenance and development effort back. but technically, OE is free standing. Oct 16 09:43:27 ok, but poky *is* part of yocto? Oct 16 09:44:33 milloni: poky is one of the main "products" of YP, yes. Oct 16 09:46:21 what are the others? i've mostly associated yocto with openembedded and poky (and not we've made it clear openembedded is not part of yocto) Oct 16 09:48:19 milloni: the other most prominent things are probably the documentation, the autobuilder... maybe also crops, and its kernel flavours. get an idea at https://git.yoctoproject.org/ Oct 16 09:52:25 ohh that makes it much clearer Oct 16 09:57:27 there's also a very nice webpage here: https://www.yoctoproject.org/software-overview/project-components/ Oct 16 10:02:03 milloni: the oe/yocto split is largely historic and complicated so don't worry :) Oct 16 10:03:24 rburton: we are coders. we like complicated things. Oct 16 10:03:29 lol Oct 16 10:04:40 rburton: yet, if we look at the topic distinciton between oe-devel, oe-core and yocto MLs, i think it still makjes sense to bear a kind of difference in mind. Oct 16 10:11:24 LetoThe2nd, speak for yourself Oct 16 10:11:39 Crofton|work: i always do, as you know. Oct 16 10:11:42 * alessioigor waves all Oct 16 10:11:45 Is there a bitbake variable which holds the name of image that the packages are built for (e.g. core-image-minimal)? Oct 16 10:11:50 heh Oct 16 10:13:24 alessioigor: during the build of the image itself, sure. Oct 16 10:13:49 alessioigor: otherwise, no. same thing applies as usual: one recipe cannot affect another. Oct 16 10:14:28 LetoThe2nd: yes during the build. Oct 16 10:15:22 alessioigor: during the build of the *IMAGE*. don't leave that out. Oct 16 10:15:38 alessioigor: should be ${PN} Oct 16 10:16:21 LetoThe2nd: ${PN} is the name of the package which are building... Oct 16 10:16:33 yes. Oct 16 10:17:14 alessioigor: let me guess. you want a recipe to behave differently whether its being built for a production image or a dev image, right? Oct 16 10:18:36 LetoThe2nd: It sounds interesting (I would be happy to know how do it anyway) but it isn't my case. Oct 16 10:18:59 alessioigor: anyways, its close to your case. and the answer is the same as always: its not how it worls. Oct 16 10:19:03 works, even. Oct 16 10:19:51 a package has *NO* knowledge what its being built for, other than things in then .conf files and its own recipe. a recipe (even an image recipe) CANNOT affect other recipes. Oct 16 10:23:36 LetoThe2nd: Thanks anyway! Oct 16 10:30:19 alessioigor: let me guess. you want a recipe to behave differently whether its being built for a production image or a dev image, right? Oct 16 10:30:38 glad you brought that up actually, whats the recommended way to do that? Oct 16 10:30:52 i've used an override in the past Oct 16 10:31:09 milloni: depends. usually it involved either creating a dereived distro r injecting a dev layer with appends. Oct 16 10:31:41 but certainly NO imagerecipe-to-recipe magic! :D Oct 16 10:32:13 hmm what exactly is a "distro" in yocto nomenclature? Oct 16 10:32:22 a distro is a distro Oct 16 10:32:44 basically a set of .conf files? Oct 16 10:32:56 that modify how recipes behave? Oct 16 10:33:15 feel free to read it up in the manual, or what my latest live coding session where i spent about an hour explaining it. i don't feel liek repeating it now, lunch time is here :) Oct 16 10:33:54 you've got yocto live coding sessions? link? Oct 16 10:34:05 youtube yocto live coding something. Oct 16 10:34:06 i'd be interested to watch those Oct 16 10:34:20 irc-to-google frontends is 5€ per request each. Oct 16 10:34:47 this - https://www.youtube.com/watch?v=EfKLrSxA_H8 ? Oct 16 10:34:51 hah Oct 16 11:21:24 New news from stackoverflow: How to find the order of recipe files for an image in bitbake Oct 16 11:29:48 hello all Oct 16 11:30:36 it just occured to me. maybe it's better to write a tool that converts the MXE makefiles to yocto recipes. each of them would generate packages called "mxe-", like "mxe-libiconv". Oct 16 11:31:44 a major difference from MXE makefiles to yocto recipes is the absence of do_install though. the MXE build macros build *and* install their software. Oct 16 13:11:08 googling mxe brings up links to a hallucinogen Oct 16 13:12:36 litb: recipes build and install their software though Oct 16 13:15:37 rburton, lulz. I mean this: https://mxe.cc/ Oct 16 13:15:50 https://github.com/mxe/mxe Oct 16 13:16:08 looks like buildroot Oct 16 13:16:22 this is similar to trying to build the debian distribution using yocto, which various projects aim to do, sometimes by converting debian source packages to recipe files Oct 16 13:16:47 rburton, yeah, but simpler. it's not based on kconfig, but just a bunch of makefiles + "Makefile standard library" Oct 16 13:17:47 the mxe files clearly do a 'make install' just like a bitbake recipe would Oct 16 13:18:55 rburton, ah, I see. that's cool. and since it's all makefiles, the content of their BUILD macros can just be put 1:1 into do_compile_and_install. Oct 16 13:19:04 good luck with that Oct 16 13:19:22 https://github.com/mxe/mxe/blob/master/src/curl.mk#L20 Oct 16 13:20:18 rburton: hrhr nice pick Oct 16 13:21:01 to me it rather looks like another homegrown small cross-lfs thing Oct 16 13:21:08 i've written bitbake -> srpm and reviewed srpm -> bitbake. it's hell. don't even bother trying to transform source packages from one format to another Oct 16 13:21:30 litb: I suspect you might have trouble copying it straight over, but it could be a good reference for what compile options you need to get a recipe to compile/run for MinGW Oct 16 13:22:01 litb: Although, I'd stick with the OE defaults unless you *really* have to change them :) Oct 16 13:22:17 sometimes they define $(PKG)_BUILD_$(BUILD) . I think that's what defines the steps for for building the native packages. they will map to the BBCLASSEXTEND "native" (maybe mxe-native, because I will need to use the MXE compiler) Oct 16 13:23:06 rburton, hm i see. bad news Oct 16 13:23:31 if you have mxe files already that build for your setup then as JPEW says, that's a good source of information Oct 16 13:24:01 but *transforming* source representations from one domain to another isn't enjoyable, there's just too much mismatch in the models Oct 16 13:24:58 alternatively in do_compile i can just forward to the makefile of MXE, instead of trying to extract the code and put it into the recipe automatically. then I just need to transform the metadata Oct 16 13:30:57 huh, bitbake is suddenly failing to start and fails with error: NOTE: Retrying server connection (#1)... Oct 16 13:31:22 it can see the socket exists and is trying to reconnect but the server is not responding Oct 16 13:31:28 can't see anything wrong in debug or strace output. Ideas what to try? Oct 16 13:31:37 I should delete the socket file? Oct 16 13:32:23 yeah blast it away and kill any lingering bitbake Oct 16 13:33:01 ok, found a pid from bitbake.lock and killed that. no idea how that happened. maybe a disconnected screen in devshell somewhere.. Oct 16 13:44:57 mcfrisk: Do you build in a docker container? Oct 16 13:49:22 JPEW: no, native in Ubuntu 14.04 for poky master. project SW I build in project specific lxc containers with Debian 9 Oct 16 13:50:32 mcfrisk: Ok. I've had some troubles with all the container processes getting killed when the init process exits that can result in things like that, but it sounds like that's not what you are seeing. Oct 16 13:54:45 JPEW: yes, in my case there was a bitbake process somewhere in background and I suspect it was a devshell where the screen session had been exited Oct 16 14:21:53 New news from stackoverflow: bitbake build cant find sparse Oct 16 14:29:46 can I unset an override only for another override? Oct 16 14:30:32 i.e I have a machine config that contains RC_mingw32 = ...; but that conflicts with log4cpp recipe that uses RC in its SRC_URI, and sets "RC=...". so I want to say "override for mingw32, but not for pn-log4cpp" Oct 16 14:31:09 RC_pn-log4cpp = "" doesn't work, because it overrides RC with "" for log4cpp. ideally I would need unset RC_pn-log4cpp Oct 16 14:36:01 litb: rburton pointed out that is already fixed and you need to cherry-pick back the fix: bfe144b038ee49290bed7b7d851bca52d340af2e Oct 16 14:37:13 oh nice, thanks! Oct 16 14:51:58 New news from stackoverflow: bitbake build can't find sparse Oct 16 15:53:19 anyone knows whether one can use multiple overrides at once, and whether they will combine with OR and AND? i.e PACKAGECONFIG_remove_class-target_ = "..." ideally combines the class and the machine name by AND Oct 16 15:55:16 yes and yes Oct 16 16:05:45 however I found that I don't need this and I should not use class-target here. better leave it out and only match on machinename. that way, it will also match the SDK Oct 16 16:06:08 i initially thought it would also match when generating a linux nativesdk. but the worries were unjustified, because nativesdk.class sets MACHINEOVERRIDES = "" Oct 16 16:30:46 Has anyone studied and published the relationship between the number of parallel tasks one can run vs. how much RAM they need? I'm getting "worker exited unexpectedly (-9)" in container configurations that I've seen work at workstations. Oct 16 16:49:12 tgoodwin: not sure you could have hard numbers for that as it depends on exactly what stages are running Oct 16 16:49:18 ie webkit link will happily eat all the ram Oct 16 16:50:19 tgoodwin: for our use case with lots of c++ and templates, see slide 50 of https://www.yoctoproject.org/wp-content/plugins/pdf-viewer/stable/web/viewer.html?file=https://www.yoctoproject.org/wp-content/uploads/2017/10/elce-2016-mario-goulart-mikko-rapeli.pdf Oct 16 16:51:02 works well on machines with 128 gigs of ram and 42 cores (including hyper threading ones) Oct 16 16:51:05 rburton: Right I'm looking really for an upper limit. I've disabled webkit in general. But for example a workstation has 12 cores and only 16 GB RAM. It can do a world build fine, but a container with 4 and 32G cannot. I'm trying to figure out "why" Oct 16 16:53:05 mcfrisk: thanks...reading that now. Oct 16 16:54:57 tgoodwin: add some monitoring and see what runs out to cause problems? worker exiting unexpectly might be something unrelated though, are you sure the RAM is good? Oct 16 16:55:12 by monitoring i mean ram usage, processor load, open file count etc Oct 16 16:55:31 collect that and run a build, see if anything is hitting a limit at the same time as bitbake crashes Oct 16 16:55:34 rburton: I'm putting docker stats now, but it takes quite a while to hit the point where it broke. Oct 16 16:56:01 I'm trying not to use sstate in my testing because I'm collecting the duration of a build relative to the settings too. Oct 16 16:57:54 mcfrisk: this is interesting -- thanks. BTW you have an extra _ in BB_NUMBER__THREADS on slide 48. Oct 16 16:58:20 (just FYI I realize it's from 2016) Oct 16 16:59:31 I'm trying to pull my environment log from the container to see if the runner settings aren't being detected from within the container (e.g., 'free' is currently showing a lot more RAM than the limit I set) Oct 16 17:01:05 Does anyone know why our python2 and python3 multilib patches differ in behavior? python2 adds the non-arch/non-multilib libdir site-packages as well as the multilib one, but python3 does not. i.e. lib as well as lib64 Oct 16 17:04:01 seems like one would want both available for non-arch-specific modules, but i'm not sure what upstream python mandates in this regard Oct 16 17:06:46 mcfrisk: were your LXC containers basically docker? Oct 16 17:06:53 wonder what other distros do with that.. Oct 16 17:12:34 I'm upgrading my work to Warrior. I'm building core-image-minimal for an x86-64 arch. During do_rootfs, I get the error 'Could not invoke dnf... No match for argument: libssp'. It's not obvious to me where that dependency came from. It's also not obvious to me how to get libssp built. Any thoughts? Oct 16 17:14:06 mcfrisk: do you happen to know if that return code (-9) is actually related to OOM or not? Oct 16 17:19:17 LetoThe2nd: that livestream series is great - really appreciate it Oct 16 17:28:42 Is there a way to override a bbclass? Oct 16 18:10:57 milloni: :) Oct 16 18:11:07 alessioigor: not without dirty tricks Oct 16 18:37:42 * litb has just prevented an execution of an oe-core _append to his do_install by "return;" at the end Oct 16 18:37:49 i suspect that's frowned upon :p Oct 16 18:42:35 rburton, some basic libs like bzip, zlib and iconv already build! Oct 16 18:43:08 LetoThe2nd, I played your lives streaming backwards and it sounds like Metallica lyrics ; ) Oct 16 18:44:52 armpit: dang! actually they should contain recipes for x-mas bakery when enjoyed like that. guess i have to work on my performance, then. Oct 16 18:45:55 LetoThe2nd, btw, thanks for putting on that series Oct 16 18:47:41 armpit: hehe, yw. i enjoy the output of your work too, and thats something i can offer in return. Oct 16 18:48:36 armpit: i can't offer code or money to YP, but some support and community effort. its the best i can give back, so i do. Oct 16 18:49:38 LetoThe2nd, I will buy some gold stars to hand out in Lyon for people like you! Oct 16 18:50:10 Crofton|work: glitter! i want glitter! Oct 16 18:50:39 Some people are hard to please Oct 16 18:52:15 i know. and thats why i bring my own glitter smileys. Oct 16 18:54:06 LetoThe2nd, its well appreciated Oct 16 18:54:17 I am surprised Crofton|work didn't off a beer Oct 16 18:54:26 s/off/offer/ Oct 16 18:55:25 armpit: he and me have a special relation on beer offers. as i have been the only person ever voted into OE under the premise that I continue to bring along booze to OEDEMs :) Oct 16 18:55:43 ah Oct 16 18:55:59 He will hav ebetter than I can get for him Oct 16 18:58:20 armpit: :-) Oct 16 18:59:18 armpit: weren't you there in Prague? Oct 16 19:02:56 * LetoThe2nd calls it a day then and headbangs out! Oct 16 19:17:08 LetoThe2nd: well, they say you should always go out with a bang... Oct 16 19:19:27 Hi, Could someone help me and explain how to avoid conflicts in case when 2 .bb tries to install the same file? Oct 16 19:19:58 basically one .bb is in poky and adds interfaces to /etc/network Oct 16 19:20:23 second .bb in on my layer and I want to put mine Oct 16 19:20:51 should I create .bbappend for first one and remove this file Oct 16 19:21:34 or there is better way that my recipe will override already installed file Oct 16 19:45:16 rokm: either 1) append the original recipe to install the file you want instead, or 2) don't install the first package Oct 16 20:42:03 hmm. qemumips64 segs on /sbin/init on 5.4-rc4. Oct 16 20:42:16 * zeddii tries to muster enough enthusiasm to look more closely. Oct 16 21:23:02 New news from stackoverflow: The yocto-layer script is missing from my install of poky Oct 16 23:55:36 * zeddii was hoping that all the mips64 users would have noticed my comment and jumped up to help. **** BEGIN LOGGING AT Thu Oct 17 02:29:00 2019 **** ENDING LOGGING AT Thu Oct 17 02:59:57 2019