**** BEGIN LOGGING AT Wed Feb 08 03:00:03 2017 Feb 08 07:09:18 hi all, Feb 08 07:09:24 I do not want to include "qtimageformats" when compiling yocto in qt, how can I get rid of it. Feb 08 07:09:27 ? Feb 08 08:18:27 Dear All, Feb 08 08:19:01 Can somebody point me the "correct" way of managing the defconfig file for linux-custom recipe Feb 08 08:19:21 Up till now I do use the "defconfig" file (which is a simple copy of .config) Feb 08 08:19:34 it all works but is clumsy to maintain Feb 08 08:20:44 instead there are also . cfg files Feb 08 08:21:01 which looks like a better place for customization Feb 08 08:22:44 Is there a way to tell the kernel to use only the "defconfig" (output from make saveconfig) ? Feb 08 08:23:43 Seeing this error message while building gdb , ERROR: gdb was skipped: incompatible with license GPLv2 & GPLv3 & LGPLv2 & LGPLv3 Feb 08 08:25:22 or shall I use some standard defconfig (e.g. /arch/arm/configs/omap2plus_defconfig) ? Feb 08 08:29:34 good morning Feb 08 08:50:20 morning Feb 08 08:52:27 How feasible would it be to have bitbake/devtool infer BUILDDIR by look for conf/bblayers.conf in cwd or somewhere above cwd? (the way git looks for .git) Feb 08 08:55:56 RP: hi. You helped with klcc.cross. Recipe seems having survived to RSS. Could you pls. check and spot eventual changes to do? https://tinyurl.com/hhl76c7 Feb 08 08:57:30 ant_work: Why does it need eventual changes? Feb 08 08:58:18 I've spotted some sysroot preprocess patches Feb 08 08:58:27 flowing in the list Feb 08 08:58:45 ant_work: a quick glance at what you shared looks like it should be ok Feb 08 08:59:05 all seems fine, was just wondering. Thanks :) Feb 08 09:00:29 btw, the issue was that klcc-cross is built per arch but contained machine-specific paths. Feb 08 09:00:49 so we did want it to rebuild for each machine Feb 08 09:01:43 quite: at one point it did do something like that, I'm not sure when it broke Feb 08 09:01:54 it was a while ago I suspect Feb 08 09:02:40 quite: devtool should be runnable while not located in BUILDDIR though - are you seeing otherwise? Feb 08 09:03:41 ant_work: with rss it has to relocate for each recipe that uses it. Making it machine specific would not serve any purpose really Feb 08 09:04:09 ant_work: it will now contain recipe specific paths... Feb 08 09:04:13 right..maybe that mangling is not necessary anymore Feb 08 09:05:07 it started when we got machine specific sysroot Feb 08 09:05:34 ant_work: you probably will need the mangling to change it between the recipe paths Feb 08 09:05:49 ant_work: but the good news is the old mangling should work for rss just as well Feb 08 09:06:07 it seems it does indeed Feb 08 09:06:23 the build problems of 4-5 days ago are gone Feb 08 09:07:22 I'll do some tests, thanks again Feb 08 09:13:02 something really weird going on with glibc Feb 08 09:13:26 RP: http://errors.yoctoproject.org/Errors/Latest/Autobuilder/?filter=5280ab2e897dd1dfd0354cddf34553c568a8bdc9&type=commit Feb 08 09:15:38 rburton: :(. Widespread or just occasional issues? Feb 08 09:16:04 occasional Feb 08 09:16:08 but i've seen occasional things here too Feb 08 09:16:21 rburton: and you suspect my locale change? Feb 08 09:16:21 like glibc-locale changing its contents dramatically between builds Feb 08 09:21:11 rburton: taking http://errors.yoctoproject.org/Errors/Details/130265/ specifically, its odd as if you read glibc-package.inc, do_stash_locale is added after do_install and before do_package and contains a line to move the gconv files. How do the files therefore still exist at do_package? Feb 08 09:21:42 no idea Feb 08 09:34:37 anyone know if it is possible to have bitbake handle modifying a tasks depends based on the fetched source? **** BEGIN LOGGING AT Wed Feb 08 09:36:31 2017 Feb 08 09:49:33 nrossi: that would be challenging... we can't adjust task dependencies at the stage where we're actually running tasks, it has to be done beforehand Feb 08 09:50:06 nrossi: if you can determine what you need to know earlier (e.g. in anonymous python) you might be able to do such a thing, but of course that means fetching during parsing which is ugly Feb 08 10:00:35 How can one compile QT with debug symbols in Yocto ? Feb 08 10:35:08 rburton: hi, did you try to merge Kristian's exclude-path patchset? Any problems with it? Feb 08 10:39:52 nrossi: that is indeed challenging and has some determinism issues Feb 08 10:49:16 rburton: ah, there is nothing which says do_package has to run after do_stash_locale Feb 08 10:49:39 rburton: you know, this could even explain the mystery pseudo permissions in the locales Feb 08 10:51:25 rburton: well, there is actually a depends but it also depends on whether it comes from sstate now Feb 08 10:58:02 hi. wonder if someone could help me with a uboot problem: Feb 08 10:58:30 i've always been building my custom yocto distro in debian and there is no problem with booting Feb 08 10:59:14 but today i've built the distro on a fedora machine and there was no problems building it. but it fails to boot: it does not find /boot/zimage. Feb 08 10:59:46 so i entered the uboot console and listed the files on that partition, and it says "unrecognized filesystem type" Feb 08 11:01:56 the partition type is "Linux" and filesystem type is ext4. Feb 08 11:02:40 no problem reading it on my fedora workstation, and the other partition is detected correctly (w95 fat). Feb 08 11:22:14 changing to ext2 worked, but i have no idea why it worked with ext4 when built on debian.... Feb 08 11:28:52 how can I get rid of this annoying message: root@beaglebone:~# random: sshd: uninitialized urandom read Feb 08 11:29:04 Poky (Yocto Project Reference Distro) 2.1.2 beaglebone /dev/ttyO0 Feb 08 11:36:30 what bit? Feb 08 11:48:29 Hello everyone Feb 08 12:33:07 rburton: http://git.yoctoproject.org/cgit.cgi/poky-contrib/commit/?h=rpurdie/wip-rss&id=b964c6e315b60537d8cfc2abacc67c8773ef54de - I think this might help. Not sure if its everything Feb 08 12:35:28 rburton: I have a little patch in queue since 25.1. Is the queue so big? Feb 08 12:37:44 Patch ID: 136438 It keeps the same typo as the ancestor ;) Feb 08 12:39:01 ant_work: i tend to hope zedd will review kernel stuff too, but i've just tagged that for merge Feb 08 12:39:46 thanks Feb 08 12:41:18 I was before abusing of KERNEL_OUTPUT but today KERNEL_OUTPUT_DIR does not allow that Feb 08 13:27:12 how do I describe a dependency where e.g. openssl:do_cve_check() needs cve-check-tool to be in the recipe-sysroot-native, but I don't want to use DEPENDS (since configure/compile shouldn't need cve-check-tool) Feb 08 13:28:04 or is that not possible? Feb 08 13:29:08 you can add a depends to a task Feb 08 13:29:24 but you need to get the recipe sysroot populated Feb 08 13:29:28 and a depends won't do that Feb 08 13:29:46 oh, I guess all of my oe-core fu is outdated with the addition of rss :-( Feb 08 13:29:48 yeah Feb 08 13:29:49 same here Feb 08 13:29:55 RP ^ Feb 08 13:30:14 also see mariano's insane.bbclass patch from last night Feb 08 13:30:20 which has the same problem Feb 08 13:30:35 too busy to track oe-core :-( Feb 08 13:30:56 hmm, today need to prepare insane patch checking for which Feb 08 13:33:24 jku, rburton, joshuagl: do_cve_check[depends] += " cve-check-tool:do_populate_sysroot" ? Feb 08 13:33:55 RP: exactly what I was trying to suggest Feb 08 13:34:07 afaik it already does that Feb 08 13:34:21 no, it's different :) Feb 08 13:34:23 ah Feb 08 13:34:30 i should pay attention more Feb 08 13:34:38 make me doubt myself rburton, ugh! Feb 08 13:34:47 I thought I tried that though -- let me retry, it's kind of complex Feb 08 13:39:19 huh, it does populate recipe-sysroot-native -- of course I now get a different backtrace but that's something else. Thanks! Feb 08 14:12:24 ah Feb 08 14:27:05 rburton:now it fails to dlopen modules (because of course it needs to have loadable modules) from the wrong sysroot. Is there a best practice for this situation? Feb 08 14:28:38 I mean, I could add support for a env variable CVE_CHECK_MODULE_DIR ... but is there a way to have this Just Work? Feb 08 14:29:10 jku: compile a static binary ;-) Feb 08 14:29:37 jku: seriously, I think this is actually a pretty common pattern, which many analysis tools might follow. Feb 08 14:30:41 jku: there's a bug to let us mark strings in the binary as needing relocation, but it's non-trivial Feb 08 14:30:52 ah right Feb 08 14:30:56 jku: add support for an env var, then use create_wrapper Feb 08 14:31:41 yeah I see that in gdk-pixbuf-native... let's see Feb 08 14:33:52 the relocation thing is neat, but i need to finish my rewrite of the elf parser Feb 08 14:34:10 (or just copy-paste pyelf into oe core) Feb 08 14:37:57 jku: yesterday I asked about a dangling symlink in the sysroot, something with OpenSSL. RP said you had a bug open for that. Do you have the bug number for me? Feb 08 14:38:53 pohly https://bugzilla.yoctoproject.org/show_bug.cgi?id=10976 Feb 08 14:38:54 Bug 10976: normal, High, 2.3 M3, richard.purdie, ACCEPTED , Broken openssl-native symlinks Feb 08 14:39:59 jku: yes, that looks like what I ran into. Feb 08 14:41:47 pohly: http://git.yoctoproject.org/cgit.cgi/poky-contrib/commit/?h=rpurdie/wip-rss&id=8c72b30e45dd8bbe13c8c7452c3d47e0c7ae0c5d is a fix Feb 08 14:41:52 rburton: I wonder how desktop distros do the 'cve check' thing Feb 08 14:42:37 maybe they have a human that follows some mailing list, and creates a queue of tasks Feb 08 14:42:45 yes Feb 08 14:50:41 humans + community (http://oss-security.openwall.org/wiki/mailing-lists) + tools Feb 08 14:52:47 joshuagl: what are the tools? Feb 08 14:54:18 kanavin1: distro specific i.e. https://security-team.debian.org/security_tracker.html#about Feb 08 14:55:15 "Twice a day a cron job runs that pulls down the latest full CVE lists from MITRE, automatically checks that in into data/CVE/list, and also syncs that file with other lists like data/DSA/list and data/DTSA/list." Feb 08 14:55:34 which is then manually processed to evaluate what impacts debian Feb 08 14:55:47 joshuagl: more importantly, "Processing TODO entries means checking if the problem affects Debian and if so which packages, as well as evaluate their severity. This information is based on research and not just in the CVE description in order to prevent integrating false positives or incorrect data in the security tracker." Feb 08 14:56:00 research, meaning, manual work by a human Feb 08 14:56:38 indeed Feb 08 14:57:02 I put humans first for a reason Feb 08 14:57:17 perhaps we should abandon any kind of automatic check attempts, and do it similarly Feb 08 14:58:26 kanavin1: I don't think we'll escape the need for humans but we can make their work much easier Feb 08 14:59:35 we don't have as many humans as Debian, either Feb 08 14:59:44 so we need tools to help us as much as possible Feb 08 15:00:00 I wonder who pays the salary of Debian's humans. Canonical? Feb 08 15:00:36 the security team is likely partially canonical employees and partially volunteers Feb 08 15:00:46 an awful lot of DD's are volunteers, aiui Feb 08 15:01:00 rburton: any idea of numbers and split? Feb 08 15:01:07 rburton: checking CVEs sounds like a job no volunteer would want tbh :) Feb 08 15:02:25 I'm looking at the TipsAndTricks list on the Wiki. Is there a tool in yocto to view specific logs? E.g. to view do_compile from recipe xyz? I find I do a lot of "directory-dancing" to look at specific log and run files? Feb 08 15:02:31 joshuagl: a random security team minutes mail has 8 attendees Feb 08 15:02:48 sveinse: i use 'bb' but i think its still broken thanks to tinfoil2 Feb 08 15:02:57 sveinse: 'bb log xyz compile' Feb 08 15:04:04 rburton: thanks. Otherwise that would be a great contribution to yocto IMHO Feb 08 15:05:07 bb is awesome Feb 08 15:05:16 kergoth needs to fix it :) Feb 08 15:20:37 rburton: My current thought, though not yet set in stone, is to prototype a new command that's an actual bitbake UI, and implement the commands as proper internal bitbake commands, and ideally move it into the bitbake repo. Feb 08 15:20:57 been meaning for its bits to go upstream for years, this is a good excuse to do something about it.. Feb 08 15:20:57 \o/ Feb 08 15:21:03 need to start a discussion thread on the architecture list Feb 08 15:26:59 pity, bb is too new for my krogoth release it seems :( ..or, I've got a poky/bitbake/lib/bb directory. I wonder how I can access it from shell Feb 08 15:27:28 sveinse: krogoth branch in the bb repo.... Feb 08 15:27:33 not particularly hard to find Feb 08 15:27:43 lib/bb is the bb python package used by bitbake, completely different than the tool Feb 08 15:27:52 ah, sorry, yes Feb 08 15:30:54 Sorry for asking stupid questions, but how is the bb repo related to the poky repo? Because it contains bb doesn't it? Feb 08 15:34:28 sveinse: poky repo is made from several repositories put together, including bb and oe-core repos Feb 08 15:35:58 sveinse: think of poky as a gigantic example of how to make a distribution using bitbake and oe-core Feb 08 15:38:08 sveinse: kergoth's bb is entirely unrelated (kergoth/bb on github) Feb 08 15:38:33 indeed, completely serperate utility, outside of bitbake Feb 08 15:39:36 Right. Feb 08 15:40:49 BTW are there any yocto-based distros that don't rely on poky? My impression is that the SoC vendors all tag on to Poky in some form or another (and thus as a user of these SoC package, so are we) Feb 08 15:41:38 i'd say that impression is wrong Feb 08 15:42:13 poky is literally oe-core + bitbake + very little anyway Feb 08 15:42:26 quite possible Feb 08 15:43:32 rburton: I remember someone on the list saying they're not using poky (was it Philip?) Feb 08 15:44:03 Ostro OS was not using Poky. Feb 08 15:44:54 in our product's imx6 sources we have poky, meta-openembedded, meta-fsl-arm and som machine specific layers Feb 08 15:47:13 rburton: what is the purpose of meta-yocto-bsp? Feb 08 15:47:30 But as na "application product developer"-ish, the lines between yocto, poky and bitbake and the machine layers appears quite blurry. A steep learning curve. I've been doing this for a year on and off, and I still feel I have no firm overview :D Feb 08 15:48:02 rburton: I mean, why is it separate from meta-poky? Feb 08 15:49:07 distro and bsp/machine should always be maintained separately Feb 08 15:51:44 sveinse: yocto is just an umbrella project used for companies to collaborate on bitbake, oe-core, and other components used as the core of their products. poky is an example distro, and also a repository that encompasses the other projects for convenient testing. most distros should *copy* contents from poky if appropriate rather than basing upon it, though some do. Feb 08 15:51:46 But, the bsp/machine binds the distro somehow. At least for the HW architectures we use. E.g. we've wanted krogoth for a long time, but the HW vendor did not support it yet, so we were stuck on jethro. Feb 08 15:51:51 wrong Feb 08 15:52:08 distro, machine, and image are key orthogonal axes of the build. if your distro doesnt' work witha ny arbitrary machine, then your distro is broken Feb 08 15:52:09 sveinse: that's version, not distro Feb 08 15:52:17 any machine should work with any distro which should work with any image, generally Feb 08 15:52:31 see also https://db.tt/F3bHEZRe, which i wrote a long time ago, and could probably use some updating, but covers this Feb 08 15:53:21 * kergoth also isn't much of a writer Feb 08 15:53:34 generally, but it doesn't turn out that pretty all the time. I mean, we are depending on the Yocto version released by the HW vendor, because we don't have the resources to take on the detailed Yocto work Feb 08 15:54:44 i would say that's pretty well-written, kergoth. Feb 08 15:54:47 I don't see what using vendor content instead of upstream has to do with machine vs distro Feb 08 15:54:48 I visited Electronica this year and asked around HW vendors for Yocto support and for Krogoth. And many answered that they does not support every Yocto release, as Freescale (now NXP) does not release for every Yocto release Feb 08 15:54:57 they're two different things, and good vendors maintain that separation Feb 08 15:55:05 yocto version is a different question, as rburton indicates Feb 08 15:55:57 sveinse: vendors may not want to track the six monthly releases, that's fine. Feb 08 15:56:12 sucks to be a customer if you want a release they don't support, but there's only two choices then Feb 08 15:56:29 1) use their product anyway 2) don't use their product citing that as a reason Feb 08 15:57:48 rburton: I'm not a end-user thou. We're making a custom embedded product, in which we use Yocto to build the software. But we're dependent on bsp layers from other vendors Feb 08 15:58:34 every 6 months for most semi's and board vendors and even customers is too often.. which is why we've got 1 year release cadence.. Feb 08 15:58:51 but what they are telling you is correct. If the vendor is too old, tell them and go elsewhere.. it is the ONLY way to get themt o change.. Feb 08 15:59:08 (WR's release is always on the fall version) Feb 08 15:59:51 morty, jethro, dizzy, dora, danny... Feb 08 16:00:49 I'm not complaing per se, I'm just elaborating on the binding between Yocto versions and BSP availability. But as kergoth points out, I'm mixing poky and release versions. Feb 08 16:01:53 sure, there's binding between releases and what the BSP vendors support. Feb 08 16:02:00 but kergoth is 100% correct.. for a particular version.. BSP, distro and image recipes are all orthogonal.. I do often see though BSPs making distribution changes... or adding packages to an image type.. Feb 08 16:02:03 Only slightly related, but as an OSV, manually determining the delta between the public layers from the hardware vendors and the layers they shipped in their own release BSPs is rather painful Feb 08 16:02:10 these are all things that should be discouraged.. Feb 08 16:02:40 BSPs making distro changes is explicitly forbidden by the compliance guidelines Feb 08 16:02:41 kergoth -- in general, we ignore the semi vendor BSP layers as 'less then useful'. Feb 08 16:02:43 But admitedly, and being the Yocto responsible in our company, its hard to distinguish even for me working some time on this. Lots of details. Feb 08 16:03:05 rburton only recently were those guidelines updated.. and a lot of BSPs I've seen still do it Feb 08 16:04:50 rburton: Can you sent more details on glibc issue you are seeing ? Feb 08 16:05:03 rburton: an example that I could reproduce Feb 08 16:05:11 khem: wish i could reproduce :/ Feb 08 16:05:16 Our HW vendor (which is a custom design), heavily modifies poky and meta-openembedded. And yes, this should not happen in an ideal world, many often take the easy way out and do it still. We as the customer, generally don't care about the intricate detals of how things are solved in yocto. Until it does not work... And it's not easy to police Feb 08 16:05:29 khem: http://errors.yoctoproject.org/Errors/Latest/Autobuilder/?filter=5280ab2e897dd1dfd0354cddf34553c568a8bdc9&type=commit has more failures Feb 08 16:05:46 the YP compliance guidelines (if followed) can be used as a hammer in those cases.. Feb 08 16:06:14 requiring the vendor to adhere to them, and then making them fix it when they don't is a reasonable approach -- and one I often recommend even to our own customers.. Feb 08 16:06:21 indeed Feb 08 16:06:34 that reminds me, i just noticed a recipe file in our distro layer, need to move that.. Feb 08 16:07:00 we have recipes in our distro -- but they are distribution specific and can be proven to only modify the behavior of the recipes if the distro is enabled.. Feb 08 16:07:12 (unfortunately they do change checksums just by being there, but don't change behavior) Feb 08 16:07:26 [we very much limit that behavior, but sometimes it's needed] Feb 08 16:07:27 fray OSVs and OEMs are different set of needs Feb 08 16:07:54 OSVs are benefitted immensely by following the structure since they support many different machines Feb 08 16:08:03 if the guidelines are flawed, lets get them fixed.. there should be no difference in behavior between OSV and OEM (when the OEM is playign OSV) Feb 08 16:08:12 OEMs however supports only a few. there is no clear advantage for them Feb 08 16:08:17 or perceived Feb 08 16:08:19 it's worth noting that the yocto compliance guidelines aren't just to make sure people are following conventions, they make it easier for *them* to maintain Feb 08 16:08:38 just to be clear, I'm not saying an OEM needs to do everything an OSV does.. but they do need to follow the guidelines or they're going to causing trouble for their own custoemrs Feb 08 16:09:27 if they are not being followed then either this is not communicated properly or people did not find the advantage as promised Feb 08 16:09:34 (I regularly talk to different OEMs about this, and often times they don't even know what their developers are doing since their external contractors.. which makes it difficult for them to even police them.. mention the compliance guidelines and it helps them reign in their own developers Feb 08 16:09:58 in the cases I know of, "they don't care" or worse.. they see it as a strategic 'lockin' to NOT follow the rules Feb 08 16:10:19 "if I make the changes here, it will be harder for my customer to move to a different hardware platform" Feb 08 16:10:31 (and yes, I have had an OEM tell me exactly that in the last year) Feb 08 16:10:38 I think the difficulties are that we want to get away with as little efforts in Yocto as possible. Our objective is to make our products and care very little in how the lower ends (e.g. Yocto) are implemented. I mean, my resources spendage on Yocto is challenged, because that is not our core competency. Feb 08 16:10:56 that does happen, however OEMs are getting more aware and want to own the platform Feb 08 16:11:35 problem is they want to own the platform and put in minimal effort to do that.. competition in this case is good... locking in customers annoys everyone Feb 08 16:11:39 sveinse: thats right. however you still need to figure out the best use Feb 08 16:11:47 to meet your needs. Feb 08 16:12:00 If this is the general attitude, then vendors do get away with non YP compliant editions as long as the end product works Feb 08 16:12:14 smartly, otherwise it will come in the way, I guess guidelines help with keeping that barrier Feb 08 16:12:22 It not until now, very late in the project, we realize that we should have had quality statements about Yocto Feb 08 16:12:25 although they may not be perfect and meet all needs Feb 08 16:12:32 sveinse the whole point of the Yocto Project, compliance guidelines and ecosystem is so you do NOT have to be an expert, but can be a developer user Feb 08 16:12:38 Because we don't have the necessary competency Feb 08 16:13:16 sveinse: if you can enforce compliance as prerequisite e.g. then you can get some work done Feb 08 16:13:24 I think this is something the YP (org) should be promoting.. the compliance is easy to get, and why it's so damned important for everyone to follow the rules Feb 08 16:13:33 in the quality segment. but then for own layers you have to do it yourself Feb 08 16:14:02 fray: again its about communicaton Feb 08 16:14:15 you can never shove things down anyone's throat Feb 08 16:14:26 there are compelling alternatives Feb 08 16:14:38 you have to have a story they can relate their pains to Feb 08 16:14:58 khem: well you can. sort of. In contract engineering it can be set as a requirement Feb 08 16:15:06 no.. but if you tell ODMs to require their OEMs to be Yocto Project compliant.. and the ODMs to tell their OSVs to require it as well.. in the contract Feb 08 16:15:28 now the ODM has leverage to make sure everyone follows the basic rules.. (distro/bsp/image being ortogonal) Feb 08 16:15:52 without people requiring this in a contract the compliance is a nice to have and useful -- but people get burned.. thats the bit that YP (org) should be promoting Feb 08 16:18:50 Actually, when I read about distros in Yocto, I start to think that our product should be a distro of its own Feb 08 16:19:44 sveinse: yes, it should Feb 08 16:20:07 however that is step 7. I still have to fix this pesky taskhash issue thing first... Feb 08 16:20:33 rburton: I would need to be able to reproduce the glibc issue with locale Feb 08 16:20:57 rburton: if you can tell me look at x ipk, it has this content before and is empty after Feb 08 16:21:26 then I would be able to help. We need to move to release from snapshot Feb 08 16:23:00 fray: the traditional OSV/ODM/OEM model is also flattened with yocto. many put together their own distros, so there is no one chain Feb 08 16:24:00 but if the pieces all follow the rules, the problems of conflicts are significantly reduced.. Feb 08 16:24:37 (again we're all talking aobut a simple set of rules, not a complex set)... so there really should not be a burden from the individual developers who produce 'YP compliant components' Feb 08 16:24:37 sveinse: yes, unless you want to hook into pregenerated feeds e.g. say angstrom, you essentially are your own distro Feb 08 16:39:42 RP: should rpm4 use nss or beecrypt? Feb 08 16:40:02 RP: the question came up because nss-native breaks test_sstate_32_64_same_hash, badly Feb 08 16:41:01 huh Feb 08 16:41:05 well thats a bug Feb 08 16:41:27 rburton: there's some host-specific magic in nss recipe which I didn't look at in detail yet Feb 08 16:52:47 rburton: the nss recipe is doing this in do_compile: Feb 08 16:52:47 make -C ./nss CCC="${CXX} -g" \ Feb 08 16:52:47 OS_TEST=${OS_TEST} \ Feb 08 16:52:47 RPATH="${RPATH}" Feb 08 16:52:54 where OS_TEST is set from TARGET_ARCH Feb 08 17:00:53 any hints how to debug this properly? Feb 08 17:06:01 kanavin1: i must be missing something - how does the test work for any native where the target_arch will be different Feb 08 17:06:21 i'd expect natives to be different if the host is different Feb 08 17:07:09 rburton: I would expect this as well, but the test was written by RP himself :) Feb 08 17:07:49 rburton: and the comment clearly says: " The sstate checksums for both native and target should not vary whether Feb 08 17:07:50 they're built on a 32 or 64 bit system." Feb 08 17:07:53 yeah Feb 08 17:07:57 and it passes Feb 08 17:08:24 * rburton is doing confused face Feb 08 17:08:40 * kanavin1 is ready for tea+bun Feb 08 17:12:37 native checksum will depend on bitness of build host isnt it Feb 08 17:13:09 nativesdk and target may be not Feb 08 17:13:30 exactly, which is why its odd this test appears to work :) Feb 08 17:13:50 RP, where art thou Feb 08 17:13:57 the confusion is spreading Feb 08 17:15:26 rburton: I see that http://errors.yoctoproject.org/Errors/Details/130265/ report unpackaged locale files Feb 08 17:15:28 kanavin1: sorry, here now. What was the question? Feb 08 17:15:34 rburton: I wonder how thats happening Feb 08 17:15:58 RP: please read beginning from "should rpm4 use nss or beecrypt?" Feb 08 17:16:12 RP: I have to run now unfortunately, will be available in an hour or so Feb 08 17:16:46 RP: short version: how does the sstate_32_64 test work when surely native recipes will change if the build host arch changes Feb 08 17:17:09 kanavin1: so the first answer is nss, not beecrypt Feb 08 17:17:36 RP: additionally, how does one debug a recipe that fails to be same (nss-native)? Feb 08 17:17:50 rburton, kanavin1: native sstate checksums are actually BUILD_ARCH independent Feb 08 17:18:04 We account for BUILD_ARCH in the actual locations and filenames, not the checksums Feb 08 17:18:22 hm Feb 08 17:18:37 This means we have one native checksum which represents all BUILD_ARCHs and therefore target arch checksums don't change depending on BUILD_ARCH Feb 08 17:18:40 but nss uses BUILD_ARCH directly in the do_compile to pass arguments to make Feb 08 17:18:50 right Feb 08 17:19:06 so we just need to whitelist this from the checksum basically Feb 08 17:19:20 since the filename will in fact catch it Feb 08 17:19:37 we don't do this globally since we really do want to be careful about this Feb 08 17:20:08 kanavin1: debugging the difference is via bitbake-diffsigs Feb 08 17:20:31 kanavin1: which is basically what that test actually does in many ways Feb 08 17:20:49 right fun times Feb 08 17:21:08 kanavin1: If I were debugging it I'd use the test but stop it wiping the tmp stamps directories and then run bitbake-diffsigs on nss-native's two differing stamp files Feb 08 17:21:40 well do_compile just directly uses TARGET_ARCH so in native builds that's BUILD_ARCH, so i guess it just needs a whitelist Feb 08 17:22:43 rburton: BUILD_ARCH is in the base whitelist Feb 08 17:24:26 are you sure its TARGET_ARCH ? Feb 08 17:24:38 if [ "${TARGET_ARCH}" = "powerpc" ]; then Feb 08 17:24:48 also Feb 08 17:24:48 if [ "${SITEINFO_BITS}" = "64" ]; then Feb 08 17:24:52 are you sure its TARGET_ARCH breaking the test? Feb 08 17:24:58 I suspect SITEINFO Feb 08 17:49:35 kanavin1: so, I hacked the test: http://git.yoctoproject.org/cgit.cgi/poky-contrib/commit/?h=rpurdie/wip-rss&id=5cf656702f5f7fb6e7b50eee1b59758c6727bdc2, then oe-selftest -r sstatetests.SStateTests.test_sstate_32_64_same_hash, which fails as expected, then $ bitbake-diffsigs tmp-sstatesamehash*/stamps/*/nss-native/3.27.1-r0.do_compile.sigdata.* Feb 08 17:49:35 basehash changed from 944cc4554a823ba966aeda0ac3d33b79 to 2475db3659c248d81d0e4dadb3c1b4cd Feb 08 17:49:35 Variable SITEINFO_BITS value changed from '32' to '64' Feb 08 17:55:07 kanavin1: to figure out the dependency chain you can the run bitbake-diffsigs on a single hash to see that its do_compile which has the dependency and do_compile[vardepsexclude] += "SITEINFO_BITS" should fix it Feb 08 17:55:12 rburton: ^^^ Feb 08 17:58:41 it then breaks in do_install :) Feb 08 17:59:10 (which needs the same fix) Feb 08 18:00:15 after adding the do_install change, the test passes Feb 08 18:05:46 call for participation in #beagle-gsoc: http://bbb.io/gsoc Feb 08 19:05:10 kergoth: I think your document is great. It describes an overview from the system builder's POV. Gave me a few ideas. Thanks Feb 08 19:24:37 What does it imply to be YP compatible? And I'm not refering to the branding aspects, but rather the technical implications Feb 08 19:25:07 A test suite? Best pratices? Peer review? Feb 08 19:25:57 RP: thanks, will you make a patch? Feb 08 19:26:25 RP: what does do_compile[vardepsexclude] do? Feb 08 19:27:55 kanavin_home: I believe it excludes the listed variabled from being included in the sstate cache signature calculations, such as DATETIME. Feb 08 19:28:33 sveinse: not exactly obvious :-/ Feb 08 19:28:42 I'm trying to fix a problem where this probably will be the remedy Feb 08 19:31:03 kanavin_home: my explanation or the naming of the variable? Feb 08 19:32:36 sveinse: the naming of the variable :) Feb 08 19:41:39 sveinse: https://www.yoctoproject.org/ecosystem/yocto-project-branding-program, https://www.yoctoproject.org/ecosystem/participant-registration, https://www.yoctoproject.org/ecosystem/compatible-registration Feb 08 19:41:52 the compatible registration has a list of acceptance criteria Feb 08 19:42:13 i.e. "Are all your publicly accessible layers listed in the OpenEmbedded Layers index (http://layers.openembedded.org)?", "Do all layers contain a README file which details the origin of the layer, its maintainer, where to submit changes, and any dependencies or version requirements?", etc Feb 08 19:43:33 kanavin_home: vardeps is a list of dependent variables, vardepsexclude is a list of vars to exclude from the list of dependent variables Feb 08 19:43:37 seems pretty clear to me Feb 08 19:43:38 * kergoth shrugs Feb 08 19:44:04 metadata checksumming uses variable dependency tracking to do the checksum, and the checksum is used in sstate Feb 08 19:44:21 kergoth: Is that registration questionare it? Feb 08 19:44:24 admittedly it's a pretty low level variable name, but what it does *is* low level, it just affects other layers Feb 08 19:44:47 (where layer here refers to concept, not yocto layers :) Feb 08 19:52:18 kergoth: is the variable dependency tracking used somewhere else than in checksumming? Feb 08 19:52:35 No, but checksumming is used for more than just sstate. Feb 08 19:52:59 kergoth: then it could mention 'checksum' in the name Feb 08 19:53:13 kergoth: as it is, it looks as though compile step itself is affected Feb 08 19:53:45 i don't see how it's posible to misunderstand 'vardepsexclude' to mean anything other than exclusions from variable dependencies Feb 08 19:53:47 it's literally the name Feb 08 19:54:17 kergoth: it's not obvious what 'variable dependencies' are Feb 08 19:54:37 what else would a variable depend on but another variable? Feb 08 19:54:43 could be made more clear that it's really for checksumming Feb 08 19:55:18 making a flag name into a sentence really doesn't seem like a great idea for usability, to me Feb 08 19:55:22 but by all means submit a bug Feb 08 19:55:57 it's worth noting that info could well be used for more than just checksumming, it just doesn't happen to do so right now Feb 08 19:56:12 the other usage is when writing out shell scripts - we need to write out all that will be called Feb 08 19:56:15 i.e. to enhance how variable expansion works, or limit what varaibles are allowed to be accessed from it Feb 08 19:56:26 good point, forgot about that Feb 08 19:56:26 AFAIK Feb 08 19:56:32 determines what shell fucntions get emitted Feb 08 19:56:35 and vars Feb 08 19:56:43 though the vars are generally covered by exports Feb 08 19:57:36 flags should indicate precisely what they describe, not necessarily how they're used. it's declarative (ideally) metadata, it's up to bitbake what to do with that.. Feb 08 20:01:35 I tend to support kanavin_home on this. It is not plainly obvious what it does. You need to be pretty deep into the inner workings of Yocto to understand what it does. For my part this translates to: This is what you have to write to fix tasksel issues... Feb 08 20:02:03 *taskhash mismatch issues Feb 08 20:02:28 dealing with taskhash mismatch issues is definitely painful Feb 08 20:02:49 I don't think terminology is the main issue though Feb 08 20:02:51 yeah, I'm not out of the woods yet Feb 08 20:03:42 I'd really like to prototype plugging into bitbake wrappers around the data store which limit access only to the variables bitbake's dependency tracking knows about. then an attempt to access anything else would immediately fail. would at least ensure we don't miss anything. doesn't help with cases where too much is included, but .. Feb 08 20:03:48 s/bitbake/bitbake,/ Feb 08 20:04:11 kergoth: the problem is that this stuff ends up in recipes, and someone, somewhere might need to understand what it's there for, often years away from the moment this was placed into the recipe Feb 08 20:05:18 that's exactly why it shouldn't refer to specific detaisl of how bitbake uses the info. it's an argument for keeping it as is, IMO. it's info about what variables depend on what. how bitbake uses that 10 years from now might or might not be the same as today Feb 08 20:06:03 kanavin_home: arguably, that's what detailed commit messages (and code comments) are for Feb 08 20:07:00 bluelightning: more often than not, those do not happen (especially further in the history of oe) Feb 08 20:07:43 I'd also like to try revamping the checksumming to use expanded values, not unexpanded values. would allow vardep exclusions to propogate through the dependency graph, and better yet, it'd make sure only the results matter, not how we got there Feb 08 20:08:07 usually, I just throw away the entire recipe when there's too much undocumented cruft in it, and write a new, simple one, dealing with issues as they occur Feb 08 20:08:07 kergoth: that does sound useful Feb 08 20:08:14 bluelightning: the fact that FOO = "${A}{B}" with A="1" and B="2" results in a different checksum than FOO = "12", when only FOO is referenced, bugs the crap out of me Feb 08 20:09:14 I try to comment extensively in the recipe to why a statement is needed. Because in our company, Yocto knowlegde is slim, and the objective for devs to touch recipes is *only* to add support of some piece of software. Feb 08 20:10:13 sveinse: when I did gobject introspection, I commented pretty much every line and block, because it's a very delicate thing that depends on a lot of things being exactly right Feb 08 20:13:59 here's an example of how to NOT write a recipe: http://git.yoctoproject.org/cgit/cgit.cgi/poky/tree/meta/recipes-connectivity/openssl/openssl.inc Feb 08 20:15:20 kanavin_home: agreed, that recipe is a mess :( Feb 08 20:16:40 bluelightning: that's why for openssl 1.1 I'm throwing away all of that and starting from scratch Feb 08 20:16:53 the only sane way Feb 08 20:16:58 My favourite is this: https://github.com/openembedded/meta-openembedded/blob/master/meta-oe/recipes-support/uim/uim_1.8.6.bb -- it's wrond, and we need it in our application Feb 08 20:17:42 I started on fixing it, but it turned out the rabbit hole was simply too deep Feb 08 20:18:23 kanavin_home: ditch the make -e while you're at it, if you would, please :) Feb 08 20:18:30 sveinse: don't fix, write your own :) Feb 08 20:18:57 i wonder why the os/arch mapping was done in shell rather than via a bitbake function, that's odd Feb 08 20:19:09 sad part is it was probably me that wrote that.. Feb 08 20:19:17 kergoth: ah the wisdom of age Feb 08 20:19:22 kanavin_home: I made a small emergency patch, so that it works the way it needs to, but the rest is still not pretty Feb 08 20:19:31 kergoth: that's the big downside of working on a project for a decade isn't it Feb 08 20:19:35 indeed Feb 08 20:20:19 every time i look at the bitbake codebase it reminds me of how shitty i was at python when we started. thankfully it's evolved a long way since then, but there are still remnants :) Feb 08 20:20:30 kergoth: make -e? Feb 08 20:20:58 kanavin_home: see EXTRA_OEMAKE. -e makes env vars override the makefile. it's relying on the fact that vars like CFLAGS are exported into the shell environment rather than passing them in explicitly Feb 08 20:21:07 bad Feb 08 20:21:30 kergoth: like I said, openssl 1.1 will not be based on this mess at all Feb 08 20:21:35 a clean rewrite from nothing Feb 08 20:21:55 i know, just a reminder to explicitly pass the variables instead, rather than taking the easy route :) Feb 08 20:22:15 was such a mistake doing that.. at the time i just wanted to make most recipes Just Work(™) without needing to explicitly pass things, but that just made it easy for people to stay ignorant of what the underlying makefiles were doing Feb 08 20:22:19 :\ Feb 08 20:22:31 kergoth: we all have that, don't we. I found some pieces of code in a build system which I wrote 6 years ago, and I go: What? What stupid &%#.. git blame... Oh, right. Feb 08 20:22:37 hehe Feb 08 20:23:29 where can I find the build directory for a given package? Feb 08 20:23:42 I'm trying to see what the configure options for strongswan are Feb 08 20:24:09 pthomas: tmp/work/[machine tune]/strongswan/[version] Feb 08 20:26:02 yeah I see that, but I don't see any source files or source structure Feb 08 20:26:15 does this get cleaned up? Feb 08 20:26:54 pthomas: sources are only extracted if they're needed. that's how all tasks are done. Feb 08 20:26:54 pthomas: if the build outputs are available from sstate cache, then there will be no unpacked source there Feb 08 20:27:12 bitbake -c patch or -c configure or whatever if you want to get things populated Feb 08 20:28:51 do a bitbake -c clean_sstate strongswan, followed by bitbake strongswan, then you'll have the source, the binaries, the packages, and logs for every step Feb 08 20:30:11 I want to see if it's configured with --enable-eap-tls, it's not in the .bb file, but I wasn't sure if it might be default Feb 08 20:32:14 what is the difference between clean_sstate and cleanall? Feb 08 20:32:28 sveinse: cleanall additionally removes downloaded source Feb 08 20:33:40 devtool modify recipename followed by devtool configure-help recipename is one way to get the configure options Feb 08 20:33:58 (both the help and what options are currently specified) Feb 08 20:34:07 bluelightning: ping Feb 08 20:38:53 kanavin_home: fyi, cleansstate doesn't guarantee a non-sstate build if sstate mirrors are enabled. it'll just delete the fetched file and re-download it :) Feb 08 20:38:59 annoying behavior Feb 08 20:39:07 can always -C fetch to force the matter instead, though then you'd want to clean first Feb 08 20:54:29 when I use bitbake-diffsigs and get some/other/recipe.bb.do_image_complete with hash xxx changed to yyy, this implies I need to follow the change backward to find the originating difference, right? Feb 08 20:55:03 most likely, yes Feb 08 20:55:16 easier to use -S printdiff if it's workable for your case Feb 08 21:27:54 kergoth: os -S printdiff supposed to print anything? Feb 08 21:28:37 (wiping tmp and starting over...) Feb 08 21:43:14 Now I can't seem to get the taskhash failed failure at all... Feb 08 22:04:27 I noticed something when using bitbake-diffsigs, that the paths for the recipes have changed. It uses a path which is not local to this build, but belongs to another build. I've verified that the configuration has no cross links. Leakage somehow via the sstate? Feb 08 22:07:17 sveinse: is bitbake-diffsigs actually reporting that as a difference, or is it just incidental in the output? Feb 08 22:09:44 bluelightning: Its like /path/to/recipe.bb.do_image with hash xxxx changed to /antother/path/recipe.bb.do_image with hash yyyy Feb 08 22:10:53 I ran two consecutive runs of bitbake. Both successful. And still bitbake-diffsigs reports this Feb 08 22:11:32 sveinse: the function's hash should not depend on the recipe path - is that really all it's saying or is there more to the message? Feb 08 22:14:42 bluelightning: its quite extensive: https://bpaste.net/show/1339a3048ced Feb 08 22:16:39 hmm, there's some magling of the paths there I see Feb 08 22:16:43 *mangling Feb 08 22:17:03 what about the diff for do_rootfs? Feb 08 22:17:27 Yeah, and the path on line 2 does not belong to this build, line 4 is Feb 08 22:18:14 sure... I'd ignore that part for now, it seems to be indicating that it's the do_rootfs task as actually having changed Feb 08 22:23:32 bluelightning: The thing is that I am chaining images, initrd into image, image into installer image, so it very probably some DATETIME thing which leaks. Its just very hard to find. Here is a small snippet of the setup: https://bpaste.net/show/d4300772b150 Feb 08 22:25:09 In this paste, it is always and only c-image.do_rootfs_wicenv that fails iirc Feb 08 22:30:54 Hi! I ran into an issue with poky 2.2.1 when building for aarch64 with multilib Feb 08 22:31:38 using the 32bit toolchain, if I prepare a C file which includes , the kernel's sigcontext.h is pulled in , but that contains __uint128_t type which the 32bit compiler doesn't support Feb 08 22:31:42 sveinse: hmm, can't see anything immediately wrong with that... Feb 08 22:31:42 anyone saw that before ? Feb 08 22:43:34 bluelightning: just parsed and diffed the long list of packages that diffsigs claims changed, and they are equal! Feb 08 22:50:51 I wish I could inspect the temp/ (the logs) directory of the objects fetched from the sstate cache **** ENDING LOGGING AT Thu Feb 09 03:00:02 2017