**** BEGIN LOGGING AT Wed Aug 19 02:59:58 2015 Aug 19 03:32:13 Hmm, we need a lot more unit tests for the fetcher Aug 19 04:24:41 ugh, I really need to add some more git fetcher unit tests Aug 19 05:29:46 WIP updated commits for the shallow fetch support: https://github.com/openembedded/bitbake/compare/openembedded:master...kergoth:shallow-git-mirror-tarballs fixes a couple of the core issues i spotted, still doing further testing, however Aug 19 07:28:45 <_4urele_> Hi everyone! Aug 19 07:29:59 <_4urele_> I'm building a kernel module, I added "inherit module" in my recipe, and I get the module in the final image, but I can't get anything in the rpms Aug 19 07:30:24 <_4urele_> I have empty rpms... does anyone know why? Aug 19 07:33:11 <_4urele_> forget what I said, I found the rpm containing my module (it just starts with "kernel-module") Aug 19 08:31:42 morning all Aug 19 10:29:16 Hi, I'm new to Yocto trying to port a library to this meta distribution, what is the common way to do that? create a recipe so that other people can use it in is own builds? Aug 19 10:30:47 yes Aug 19 10:31:52 is therre any kind of binary packages too? Aug 19 12:29:02 ljose: building a recipe with produce binary packages as part of the build, yes Aug 19 12:29:46 ljose: we're not ourselves a distro though, so we don't provide any official package feeds that are meant to be used for anything more than testing Aug 19 13:53:16 Have a problem where it seems my speicific host build machine may be the problem. "byacc-native" fails every time in main.c and it seems to use my host machines header (fcntl.h) when this happens.instead of the poky sysroot. Anyone seens this? Aug 19 14:15:07 * paulg brews up a local preferred version of libgphoto to work around the week long gvfs wreckage in oe Aug 19 15:02:20 Hi, if I want to build an application for yocto using the prebuild toolchain, where can I find third party libraries I need like bzip2 and openssl Aug 19 15:02:42 is there a prebuild SDK that I can use? Aug 19 15:03:55 ljose: nope, because the sdk always has to match the image on the target you want to use. Aug 19 15:04:12 so what is the point of the prebuild toolchains? Aug 19 15:04:24 ljose: e.g., the one creating the target image can derive the sdk from it. and therefore it will contain the available libs then. Aug 19 15:04:55 ljose: prebuilt toolchains in terms of what? Aug 19 15:05:44 There is some toolchains to download in the yocto page Aug 19 15:06:13 I download the one for arm and want to build my library for Beagle Bone Black Aug 19 15:06:36 ljose: never used those, but i guess they are more meant for rather trivial test cases, respectively bootloader/kernel work. Aug 19 15:07:08 So I will need to build my Beagle Bone image and that will give me a toolchain SDK too? Aug 19 15:08:20 it will not automatically give you the sdk. for example "bitbake core-image-minimal" creates the image, and "bitbake -c populate_sdk core-image-minimal" creates the fitting sdk then. Aug 19 17:27:06 Hi, does core-image-minimal include a SSH server? Aug 19 17:40:28 ljose: depends on the IMAGE_FEATURES. see local.conf and local.conf.sample.extended. Aug 19 17:45:04 RP: any thoughts on the shallow support? i'm assuming you haven't found the time, since you're always swamped, so no rush, just checking :) The current version has a couple changes/fixes to the core as well - https://github.com/openembedded/bitbake/compare/openembedded:master...kergoth:shallow-git-mirror-tarballs Aug 19 17:56:17 every time khem sends some nasty toolchain patch/fix .. I think "wow, that must have hurt to find" :) Aug 19 17:57:03 heh, indeed Aug 19 18:04:12 RP: I'm theorizing that a build with PREMIRRORS and MIRRORS both unset, and a good mirror tarball in DL_DIR would result in a clone from upstream, not using the mirror tarball, since there'd be no uri to cause the check of DL_DIR for the tarball path. Am I correct in that? I think I'll open a low priority yocto bug about it, if so. Ideally it'd be able to use an existing local tarball without needing a mirror URI to do so Aug 19 18:05:17 presumably we could just add a sane fallback value for PREMIRRORS to look directly in file://${DL_DIR}/, but that seems silly :) Aug 19 18:22:12 RP: http://git.openembedded.org/openembedded-core/commit/?h=master-next&id=b722ff65093560c6c03fbc1495ce2deb85a8d42c Aug 19 18:22:25 you should use 5.% instead of 5.2% Aug 19 18:25:55 Anyone around that's familiar with pulseaudio bits? Does https://github.com/openembedded/openembedded-core/compare/openembedded:master...kergoth:pulseaudio-bits# seem reasonable? Aug 19 18:27:13 * kergoth looking through the backlog of bits-that-need-to-go-upstream from meta-mentor's layers Aug 19 18:33:49 * kergoth checks sanity Aug 19 18:48:09 So, what method are folks using to override default network configuration for bsps? override interfaces? supply a connman config and use connman? systemd-networkd? Aug 19 18:51:16 pretty much all of the above... Aug 19 19:00:33 SO MANY corner cases to worry about in the fetcher. it's madness. we don't have nearly enough unit tests yet to cover everything Aug 19 19:00:37 * kergoth adds a note to his todo Aug 19 19:12:54 hmmm, if populate_sdk_ext ships downloads, I wonder if there'd be interest in the bits mel uses for our installer generation which saves / restores headrevs, coupled with changing the bb fetch cache policy to allow BB_NO_NETWORK from the installed environment even with AUTOREV Aug 19 19:18:20 soooo Aug 19 19:18:48 I have a beautiful and encouraging result to report, also I am beginning to feel that the abyss is staring back rather more than I'd like. Aug 19 19:19:01 Total time spent in wrapper function calls with old pseudo for do_rootfs for core-image-sato: ~3,647 seconds. (Actually this includes a lot of other do_rootfs tasks, I suspect.) Aug 19 19:19:12 Total time spent in wrapper function calls with new pseudo: ~2,400 seconds. Aug 19 19:19:25 So "a bit over an hour" => "40 minutes". Sounds good! Aug 19 19:19:48 Wall clock time... Old pseudo: 48 minutes. New pseudo: 58 minutes. Aug 19 19:20:02 I cannot reproduce this behavior on smaller or more self-contained tasks yet. Aug 19 19:23:20 I do note that reported system time goes WAY up, but oddly that's not showing as happening inside the wrappers. No idea. Aug 19 19:26:06 heh, it spends less time in wrapper functions, but the wall clock time is worse? what's it doing? Aug 19 19:26:50 I have no idea. Aug 19 19:27:05 we've been talking about this.. it's moving from IPC calls to a backend DB to using lsetxattr (and related).. Aug 19 19:27:21 And what really bugs me is, the obvious thing would be that the setxattr/getxattr calls are pricier than sending IPC and not waiting for a response, but... That should show up in wrapper time! Aug 19 19:27:48 so it COULD be that pseudo is suddenly better performing and causing collisions on parallel recipes/tasks.. or could it be causing the filesystem to sync "more", causing clock time to increase as the xattrs are written to disk? Aug 19 19:27:55 Unrelated, but am I correct in thinking that the binary locales in lib/locale are the same between 32 and 64, so belongs in nonarch_libdir? Aug 19 19:28:10 well the xattr would be cached to the FS cache, then flushed periodically Aug 19 19:28:22 binary locals are endian and word size specific.. :/ Aug 19 19:28:26 I am thinking that the xattrs being flushed during fsyncs could be hurting performance on fsyncs that weren't running under pseudos. Aug 19 19:28:57 well there are the manual fsyncs and then there are the automatic ones.. Aug 19 19:29:04 I'm wondering if it's more fo aproblem w/ the automatic ones.. Aug 19 19:29:27 one way to check.. on the machine we've got with a lot of ram.. enable xattrs on the ramdisk.. and build compeltely inside the ramdisk.. Aug 19 19:29:31 might be. So I've got a couple of things to look at, but mostly what I want to try to do is get a test case where I have something I can run that exhibits the behavior and is smaller than do_rootfs. Aug 19 19:29:36 that should avoid any disk cache penalties Aug 19 19:29:48 So, funny thing from my performance testing yesterday. Aug 19 19:29:58 I can reduce time on a particular tar command from about 9 seconds to ~7ish with the new pseudo. Aug 19 19:30:12 So I wanted to test it in a less-contended environment. Laptop with SSD. Aug 19 19:30:16 Result: I can reduce time from ~2 seconds to ~1.5 seconds. Aug 19 19:30:33 So I'm sort of feeling like everything I'm doing is outperformed by hardware. Aug 19 19:30:42 seebs, I don't doubt that Aug 19 19:31:05 That said, I want to do some more benchmarking, because I think my new profiling stuff is now reasonably close to measuring the actual cost of running the pseudo client. Aug 19 19:31:23 And if I'm right, I can probably get a nice significant boost by reducing allocation in path stuff. Aug 19 19:32:26 so, should we deprecate USE_NLS, deprecate the nls distro feature, or just add logic to align the two? in mel.conf, we're adding the distro feature based on USE_NLS to address this. any thoughts on what would be accepted for oe-core? Aug 19 19:33:14 Not sure I understand.. Aug 19 19:33:41 if someone could ensure that the binary locales worked in all wordsizes in a given endian/arch.. then we could make it common.. Aug 19 19:33:58 (if it would be possible to fix the endian-ness.. then they could even be potentially 'all' arch packages) Aug 19 19:34:26 there are two pieces to the locale support though.. does the libc you have even have locale support enabled.. and then which locales are enabled.. Aug 19 19:35:16 as for USE_NLS, isn't that more fo a global feature for 'gettext' capable packages? Aug 19 19:35:36 (I'm not sure I've used USE_NLS before in any way) Aug 19 19:37:00 I'd kind of expect that if locales are disabled, then USE_NLS would be set to 'no'.. and preferably gettext is disabled.. but maybe I'm missing something there Aug 19 19:37:11 (locales as a libc option that is) Aug 19 19:37:53 the last question i asked was independent of the localedir question. but yes, USE_NLS controls the gettext autoconf arguments and whatnot, whereas the distro feature controls libc capabilities, but as you say, the former doesn't really make any sense without the latter Aug 19 19:38:00 currently the two are entirely independent Aug 19 19:38:18 My suggestion is to make USE_NLS dependent upon the libc feature.. Aug 19 19:38:26 ??= of course so it can be overridden if necessary Aug 19 19:41:26 interesting, busybox appears to be the only recipe obeying a 'nls' distro feature, that should really go away in favor of a packageconfig and/or obeying the libc feature Aug 19 19:41:51 well.. libc/distro/USE_NLS should all be in agreement.. Aug 19 19:42:14 in this case, I'd say distro feature should influence libc feature.. (or a check that they're both set/cleared/whatever..) Aug 19 19:42:21 and then the USE_NLS is set based on the distro/libc whatever is appropriate.. Aug 19 19:42:34 then the gettext and individual packages should obey the USE_NLS.. Aug 19 19:42:42 (if they use gettext that is) Aug 19 19:42:53 OR use the libc/distro (whichever -one- is used) for general nls support Aug 19 19:43:24 i'm not sure the nls distro feature is actually used or set anywhere given current oe-core, possibly a remnant. Aug 19 19:43:31 does that make more sense.. (only reason for the USE_NLS would be to disable the gettext on a per-package basis.. even then perhaps 'PACKAGECONFIG' is better then 'USE_NLS') Aug 19 19:44:40 the problem is, you can't append to the default PACKAGECONFIG. gettext.bbclass could add the packageconfig for it, but couldn't add the default based on the distro features without poking at bitbake internals Aug 19 19:45:05 (technically, you *can* append to it, but only by directly poking at internal flags) Aug 19 19:45:29 can't just make 'PACKAGECONFIG[nls]="..."' something for each package? Aug 19 19:45:39 yes, but who will enable it? Aug 19 19:45:44 I doubt we'd want to go through every gettext-based recipe and modify its PACKAGECONFIG ??= to add nls based on something Aug 19 19:46:05 I forget that PACKAGECONFIG doesn't get automatically set.. Aug 19 19:46:12 I keep thinking the default is more or less the distro flags Aug 19 19:47:55 does ??= run before or after _append (it's after right so it never gets set/) Aug 19 19:48:05 your'e thinking in the old school Aug 19 19:48:08 there's no finalization anymore Aug 19 19:48:12 _append is applied at getvar time Aug 19 19:48:14 so is ??= Aug 19 19:48:19 but which is first? Aug 19 19:48:27 hmm, good point, not sure :) Aug 19 19:48:35 i suspect an _append would apply to a ??= Aug 19 19:48:39 i think that'd be most intuitive Aug 19 19:48:42 my memory is that _append runs before ??=.. so ??= with an append does NOT get applied to the ??= Aug 19 19:48:51 (which at the time I saw that seemed backwards to me) Aug 19 19:49:08 but I could be wrong as well.. Aug 19 19:49:10 i just tested it Aug 19 19:49:15 _append is applied after ??= today Aug 19 19:49:28 point being if _append 'works', then the gettext bbclass can just add 'nls' automatically based on the distro setting Aug 19 19:49:42 the thing is, _append is applied not just to a default value, but also to a user/recipe set value. Aug 19 19:49:49 sine _remove runs after the append Aug 19 19:49:55 though i guess nowadays with _remove, you can force the matter with that Aug 19 19:49:58 * kergoth nods Aug 19 19:50:34 maybe we should think about setting a default for PACKAGECONFIG in base.bbclass of the DISTRO_FEATURES. if the names don't line up, the recipe can override that ??= with its own ?= or ??= Aug 19 19:50:51 could have an intermediate variable if they want to pull in that and add to it Aug 19 19:50:55 that just leads the question, if two ??= which is used? Aug 19 19:50:55 hmm Aug 19 19:51:16 the whole point of ??= existing is that behavior :P Aug 19 19:51:23 ya.. thats always been my problem w/ the current PACKAGECONFIG setting.. it's REALLY hard to modify if you want to add/remove an item but not just re-set it Aug 19 19:51:24 latter always wins, rather than the first as with ?= Aug 19 19:51:40 anyone got an opinion on tracking "stable" systemd (aka what RHEL ships, 219) or just tracking latest systemd release? Aug 19 19:52:04 rburton.. comes down to functionality, patches and is someone willing to maintain it (someone oe-core/yp) Aug 19 19:52:32 * kergoth adds a todo to do the default packageconfig in base.bbclass thing Aug 19 19:52:34 I'm not really sure at this point hter eis a real 'stable' release.. just various distro patched ones.. problem being they then get distro specific changes Aug 19 19:52:55 IMHO it would be nice if the distro features automatically were PACKAGECONFIG feature.. Aug 19 19:53:14 then as distro features changes packageconfig changes.. any manual changes beyodn that have to do with package specific work.. Aug 19 19:53:14 fray: yeah the goal appears to have been a consolidated long-term release but it appears to really be what rhel ships with Aug 19 19:53:37 rburton ya.. that worries me.. since we aren't Fedora based.. it might lead to some really bad assumptions in system design Aug 19 19:53:39 fray: i do like being able to name packageconfig things after what they do to the *recipe* instead of what the distro feature is called Aug 19 19:53:52 distro features are supposed to be global.. Aug 19 19:54:00 package features what they ACTUALLY do in the apckage.. Aug 19 19:54:19 i.e. the RPM case.. rpm has perl, python, etc.. in it.. I wouldn't expert perl or python to be 'distro' features.. Aug 19 19:54:25 but nls (or locale or similar) I would.. Aug 19 19:54:41 rburton: right, but the recipe could always manually adjust any naming differences if that's the case. we could just set a default that any configs whose names line up with distro features are enabled based on the latter Aug 19 19:54:45 the key with this, if distro becomes the basis of packageconfig ... someone needs to actually docuemtn the distro namespace.. :P Aug 19 19:55:15 other option (more complicated for sure) is to define name spaces as "distro:feature" Aug 19 19:55:17 heh, true, there'd be much fewer bb.utils.contains() Aug 19 19:55:30 which is basically the implicit documentation today Aug 19 19:55:42 ya.. and I've never liked it.. but at least it's usable as it is today Aug 19 19:56:20 (I've also thought not defined distro features should be something like 'x-feature' to indicate it's not an official distro feature.. but again.. idea, not really policy at this point.. Aug 19 19:56:22 * kergoth is working through mel.conf to pare it down as much as possible and push parts of it up.. tend to forget about it and focus on the recipe changes in meta-mentor-staging Aug 19 19:56:27 if we documented it, we'd be more likely to do that Aug 19 19:56:43 kergoth, you aren't alone.. I think most people focus on recipes and forget about the distro file(s) Aug 19 19:56:54 collects cruft over the years :) Aug 19 20:00:11 Hmm. I wonder if there'd be interest in changing the default value of PATCHRESOLVE from 'user' to 'noop'. Opening a devshell automatically is quite often not ideal. If that's even still useful nowadays, I'd like to see noop the default, with a local.conf.sample.extended snippet showing how to opt-in to it. course others might disagree with that, and we do have to worry about local.conf template bloat Aug 19 20:00:20 * kergoth ponders Aug 19 20:03:44 I -hate- not having noop as the default Aug 19 20:03:52 at WR we changed the default to noop Aug 19 20:04:59 poky does too fwiw Aug 19 20:05:08 in fact i wasn't aware the default was user :) Aug 19 20:05:21 kergoth: file a bug, or send a patch quikc Aug 19 20:05:41 its a user setting, really. Aug 19 20:05:49 and default should be less offensive, which is noop Aug 19 20:06:03 there was a lot of resistence in the past of changing the default.. Aug 19 20:06:12 I LIKE 'user', I just don't want it as the default Aug 19 20:06:43 what does user do? :) Aug 19 20:06:59 on failure spawns a terminal (or screen) and sets you up to resolve the conflict Aug 19 20:07:15 it has it's uses.. but most of hte time I'd rather see a failure Aug 19 20:07:34 it's occasionally useful, since the default devshell task runs after do_patch, so you can't drop into one if a patch fails Aug 19 20:07:43 yup Aug 19 20:07:44 but honestly i'd rather add a devshell_prepatch task and explicitly run that if needed Aug 19 20:07:46 :) Aug 19 20:08:20 does it help you update the patches in the layer directly too? Aug 19 20:08:36 no Aug 19 20:08:47 actually, iirc it does copy a changed patch back out Aug 19 20:08:48 it's in the current working director (${S}) so you still have to copy the fixes back Aug 19 20:08:49 kergoth: I absolutely think noop would be a better choice, because "user" is basically highly-failure-prone for a lot of cases. Aug 19 20:08:58 kergoth, Hmm.. in 1.7 it didn't.. Aug 19 20:09:24 i wrote that code to do it *years* and years ago when i was on contract with openedhand Aug 19 20:09:39 we've definitely has requests to copy it back out from people and our response is always we'll follow what the community does here.. Aug 19 20:09:41 all the patchresolve/patchtool stuff was mine (sorry, not proud of it :P) Aug 19 20:10:42 https://github.com/openembedded/openembedded-core/blob/master/meta/lib/oe/patch.py#L599-L601 - https://github.com/openembedded/openembedded-core/blob/master/meta/lib/oe/patch.py#L634-L655 Aug 19 20:11:00 could be it's broken, but it's supposed to refresh the 'remote' patch, that is, the destination of the symlinks in the layers Aug 19 20:11:35 well, I've used it in the past... fixed the patch does the quilt refresh on it.. and it put it in the quilt .pc but didn't sync it back to the layer Aug 19 20:11:39 drop into the resolver shell, fix them, quilt refresh, exit -> patches should be copied into the layers. if not, it's a bug Aug 19 20:12:04 ya, I don't think that works.. (or didn't at least time I was paying attention which was a long bit ago.. and in 1.7) Aug 19 20:18:42 hmm, am i missing something, or does nothing in image.bbclass add the vardeps for FEATURE_PACKAGE_ for each feature in IMAGE_FEATURES? it looks to me like a change to the packages belonging to a feature won't cause a re-run of do_rootfs Aug 19 20:18:55 i think thats correct, will have to submit that too Aug 19 20:20:35 hmm, should i keep the vardepvalue version, or use inline python in vardeps Aug 19 20:22:20 is there an IRC channel for Freescale Linux or is this about as close as I am going to get? Aug 19 20:29:10 Any objections to my pushing a bunch of vardepsexcludes for SRC_URI for the _MIRROR variables? what mirror we happen to be fetching from really shouldn't force a rebuild.. Aug 19 20:43:50 I agree Aug 19 20:46:28 unfortunately there's no definitive list of the mirrors, either, but should at least hit the main ones Aug 19 20:46:31 heh Aug 19 20:46:54 well.. mirror, premirror all should exempted fro the vardeps.. Aug 19 20:48:31 the MIRROR variables aren't used in MIRRORS/PREMIRRORS Aug 19 20:48:34 they're used directly in SRC_URI Aug 19 20:48:42 e.g. APACHE_MIRROR, DEBIAN_MIRROR, etc Aug 19 20:49:03 the default fetch location, not extras Aug 19 20:49:09 Ohh I understand ok Aug 19 20:49:22 ya.. I agree.. those should be excluded Aug 19 20:49:25 k Aug 19 20:49:35 since people can and will likely adjust them as necessary Aug 19 20:55:25 Hmm, does anyone know if the vardeps handling code is taking into account metadata variables referenced as python variables? Aug 19 20:55:47 * kergoth checks siggen Aug 19 20:56:01 or rather, codeparser Aug 19 20:56:27 I thought the answer was no.. it was only handlign the actual python code itself (contents of hte variable) Aug 19 20:57:58 i'm talking specifically things like ${@FOO} where FOO="bar" Aug 19 20:58:12 and yeah, looks like it's not handling that right now, it only handles calls Aug 19 20:59:04 hmm, might be tough to exclude the call parents if i add visit_Name Aug 19 23:01:50 kergoth: I'm leaning towards killing the _MIRROR things in favour of suggesting people uses PREMIRRORS Aug 19 23:02:08 how many ways of rewriting urls do we need? Aug 19 23:02:44 kergoth: I've not got to the shallow patches yet, needs more time to think about it than I've been able to find. Am trying to get there... Aug 19 23:02:55 * kergoth nods, np, figured as much Aug 19 23:03:06 kergoth: I think noop should be the default btw, user does seem broken atm :/ Aug 19 23:03:48 * RP -> Zzzz **** ENDING LOGGING AT Thu Aug 20 02:59:59 2015