**** BEGIN LOGGING AT Thu Jan 19 02:59:57 2012 Jan 19 07:56:33 good morning Jan 19 10:07:10 morning folks Jan 19 10:07:18 morning Jan 19 10:25:30 good morning Jan 19 10:26:11 JaMa: I'm still waiting the long promised klibc_2.0. I'd prefer to avoid a set of git recipes Jan 19 10:26:40 do you mean we should take some patch temporarly? Jan 19 10:44:54 as long as we're not making 3.2 kernel default we can wait imho Jan 19 12:17:27 Hello there :) I have a small question about bitbake/fetch2 Jan 19 12:17:33 why is stderr redirected to stdout? Jan 19 12:17:57 suppose you have a ssh warning message before git ls-remote completes, then it messes up everything Jan 19 12:18:14 and your project ends up with "Warning:" as a git hash Jan 19 12:18:30 (haven't checked if that was fixed in master though, will do that now) Jan 19 12:20:42 http://git.yoctoproject.org/cgit/cgit.cgi/poky/tree/bitbake/lib/bb/fetch2/__init__.py#n413 still the case in master :/ Jan 19 12:25:52 abustany: see commit where it was added Jan 19 12:26:17 that's an idea :) Jan 19 12:26:39 abustany: http://git.openembedded.org/bitbake/commit/?id=3a54dcc09a12406ec6cf22b4b1a2cc4fc203822c Jan 19 12:27:00 abustany: probably different part.. but the point is that sometimes warning/error is important to catch Jan 19 12:27:23 well I totally agree that warning is important to catch Jan 19 12:27:53 now, whether it should error properly rather than stuffing the first part of the error message in the git hash is another question :) Jan 19 12:28:34 ah I see.. so should be changed like that commit Jan 19 12:29:22 the original commit is 67d5f048 Jan 19 12:29:27 that was in fetch, not fetch2 apparently Jan 19 12:29:37 and the reason is to handle error messages as you said Jan 19 12:29:51 the catch is that there can be stuff on stderr, but the return code can still be 0 Jan 19 12:30:22 I'm far from being a python guru, but if there's a way to get stdout and stderr in two separates buffer, then we could check for "stderr is not empty" Jan 19 12:30:30 *separate buffers Jan 19 12:32:07 (your link does not seem to load here btw, might be my connection) Jan 19 12:33:45 seems like git.oe.org went down just after I've pasted it Jan 19 12:34:00 cannot open your commit too in gitweb Jan 19 12:34:03 the commit hash I gave is from yocto btw, not sure if they are the same Jan 19 12:34:15 yeah and I can't see yours using git show in my yocto tree :) Jan 19 12:34:38 http://fpaste.org/DQ5s/ is the commit Jan 19 12:36:38 ok I can see your commit now. I still think it'd be "nicer" to clearly separate stderr and stdout, but then I don't know the codebase very well. That trick of using the number of items in the split() sure works, but it's not super generic Jan 19 12:37:01 well if stderr is caputured then at least check if it contains only one line (as expected) not hash+warning Jan 19 12:37:24 agreed Jan 19 12:37:55 (then I also know it's easy to give lessons and harder to write patches :p) Jan 19 12:40:44 JaMa: if I get myself to make a patch that makes runfetchcmd use os.popen3 (to get different handles) to properly handle stderr, would that have a chance to get upstream? Jan 19 12:41:06 apparently the proper solution is to use the subprocess module but that needs python 2.4 Jan 19 12:41:11 not sure what the requirements are Jan 19 12:41:46 apparently subprocess is already used in other parts of bitbake Jan 19 12:54:22 2.4? Jan 19 12:54:30 that should be fine for sure Jan 19 15:02:02 bitbake master requires 2.6, and we already use subprocess, see the bb.process module for convenience wrappers around it Jan 19 15:02:30 as you say, we should likely be capturing both in a case like that. it's common to combine them for convenience, but obviously it's not ideal for this case Jan 19 15:02:33 * kergoth gets caffeine Jan 19 15:11:10 ok, so subprocess it'll be then Jan 19 15:11:19 now I just have to check if not reading stderr can stall the process... Jan 19 15:11:30 there is an API that does the reading for you, but it's 2.7 apparently Jan 19 15:11:49 that said I would expect the process to block if one does not read stderr... Jan 19 15:14:10 once again Jan 19 15:14:12 read bb.process Jan 19 15:14:18 which does the same thing as check_output in 2.7 Jan 19 15:14:36 ah sorry missed that line :/ Jan 19 15:14:46 just need to pass stderr=subprocess.PIPE instead of stderr=subprocess.STDOUT to run() or Popen() Jan 19 15:56:29 kergoth: what's the preferred patch submission process, mailing list? Jan 19 15:59:17 bitbake-devel list for bitbake, oe-core for oe-core and poky/meta Jan 19 15:59:58 the patch only touches bitbake/lib/bb/fetch2/__init__.py Jan 19 16:00:04 → looks like bitbake Jan 19 16:00:07 thanks, will send that there Jan 19 16:00:31 * kergoth nods Jan 19 16:00:33 np Jan 19 16:00:40 might need to join the list to send to it, don't recall Jan 19 16:09:16 Hi All. Whats the official way to generate source tarballs from the edison release ? Jan 19 16:11:50 d4ny, you mean how we generate our Poky release tarballs? Jan 19 16:12:21 Nope, I cant seem to get INHERIT += src_distribute_local to work properly Jan 19 16:12:35 d4ny: what are you trying to accomplish? Jan 19 16:13:10 d4ny: are you looking for sources for gpl things for license compliance, or something else? Jan 19 16:13:11 I'm not familiar with src_distribute_local Jan 19 16:13:14 kergoth: To generate source tarballs from included recipes after patching. Jan 19 16:13:28 see sourceipk and sourcepkg, they might do what you want Jan 19 16:13:30 * pidge pops in. Jan 19 16:13:34 Yes, among other things Jan 19 16:13:42 src_distribute doesn't give you ap atched tree, it gives you the original sources and patches Jan 19 16:14:03 what format does it given them in? Jan 19 16:14:08 copyleft_compliance is a new class specifically for the compliance piece, but it doesn't give you the patched tree either, it gives you the tarball and other sources, and patches, and a series file to apply them with Jan 19 16:14:11 kergoth: Thx, will check that out .. Jan 19 16:16:24 kergoth: Hmm, one of the things we are aiming for is easy reproducability of packages. Preferrably tarballing the source directory after patching/configuring ... Jan 19 16:17:07 without the buildsystem, i don't see how that'll help reproducibility much, but.. :) Jan 19 16:17:32 from my experience license compliance is organization specific and everyone has slightly different requirements.. what I've seen is: patches & sources, patches & sources & series, prepatched sources, prepatches and configured sources, prepatched (maybe configured) and log files showing the build Jan 19 16:17:35 d4ny: i think sourcepkg.bbclass is just what you want Jan 19 16:17:49 or at least, the closest to it Jan 19 16:18:17 Saul and I were working on a way to articulate that list, and try to unify the multiple ways of outputting packages for license compliance, but we're likely a few weeks off.. Jan 19 16:18:57 kergoth: perhaps, but sourcepkg.bbclass does not seem to work out-of-the-box in edison. Jan 19 16:19:18 ya, the stuff we were working on was for 1.2, but I don't believe it's going to make it there either Jan 19 16:19:51 my suggestion for Edison is to write you own script that captures the info you need (external of the build system).. walk the tmp/work/... directories Jan 19 16:22:21 kergoth: after configure is essential (to us at least) since opt:ing out parts might influence package dependecies and so fourth .. Jan 19 16:22:39 fray: Any way to get a sneek peek at what your doing ? Jan 19 16:23:07 d4ny: make arguments could affect that too, for non-autotools recipes. Jan 19 16:23:09 there was some code posted to the Yocto/Poky/OE mailing lists in the past week or so.. thats what is causing us to reevaluate the work.. but we don't have anything beyond that Jan 19 16:23:10 good luck :) Jan 19 16:23:29 kergoth, thats why there is that continuum of license compliance people want.. Jan 19 16:23:59 patches & sources seems to be the "I want to share".. while prepatched, configured & logs seems to be "I want to cover my ass" Jan 19 16:23:59 fray: do you happen to know the status of any bill of materials type work, specifically for packages+licenses? I'm sure I can extract the info from buildhistory+pkgdata, but figured i'd check Jan 19 16:24:18 I'm not aware of anything, but Saul may be Jan 19 16:24:52 sgw: do you happen to know? Jan 19 16:25:12 kergoth: there is the manifest work in license.bbclass. I have a patch I haven't fully tested that looks for LICENSE_{$PN} first and then LICENSE during the population. Jan 19 16:25:55 it should either use just LICENSE, or LICENSE_ for all packages, no? Jan 19 16:25:59 will take a look, thanks for the pointer Jan 19 16:26:45 kergoth: Right now it just uses LICENSE. my patch says "look for LICENSE_ first. if you find it use it. if not, fall back to LICENSE" Jan 19 16:27:34 kergoth: it's not perfect by any means, but that's more of a recipe issue than anything. Jan 19 16:28:13 gotcha Jan 19 16:28:15 * kergoth nods Jan 19 16:28:25 thanks, somehow i missed the fact that that manifest code even existed :) Jan 19 16:29:22 fray: 17:23 interesting perspective :), I want to share, but I also want to make things easier for my users, perhaps I need to reevaluate the "easier" part... Jan 19 16:30:12 would be nice to integrate the license manifest with buildhistory in some fashion. buildhistory can show you if new packages got added to an image, but not addition of a new questionably licensed package specifically, you know. would have to map between the two Jan 19 16:30:15 * kergoth ponders Jan 19 16:32:31 d4ny I just know that we have a diverse set of license requirements.. Jan 19 16:33:38 kergoth: could probably use package.manifest as the map as license_create_manifest uses it to generate the manifest. Jan 19 16:34:22 kergoth: or just go to the source and use list_installed_packages. Jan 19 16:34:23 kergoth: so, the first line of defence ought to be elsewhere... if you've excluded a license it should never make it that far Jan 19 16:35:33 kergoth: it wouldn't be hard to produce an additional installed-package-licenses.txt file in buildhistory however if we need additional safeguards Jan 19 16:35:50 the list of things installed onto the rootfs can be determined in different ways depending on the rootfs package format correct? It seems like embedding a standard way into the package/rootfs to query and report a manifest makes sense to me there Jan 19 16:36:08 bluelightning: well, in the long run I think we need to avoid the skippackage level, since we could well want to build something but only include the packages it emitted which meet our requirements, and not others Jan 19 16:36:27 fray: we have a standard way of querying the list of installed packages now at least Jan 19 16:37:03 ahh good.. Jan 19 16:37:14 kergoth: yes, I agree... I think we'd have to have runtime dependency checks at both ends Jan 19 16:37:20 fray: License matters aside, from my perspective its most important to produce reproducability. If that means sourcepkg + patches + series file, it should probably include a trace/script on how the package was built. i.e. configured, make options... Jan 19 16:37:40 kergoth: technically we only need one at the do_rootfs end but it's nicer to the user if it fails with a comprehensible error earlier than that Jan 19 16:38:27 d4ny, the only way to ensure reproducability is to store your build settings and build system in a consistent SCM.. Jan 19 16:38:59 i.e. you want to store the local.conf, bblayers, and other configuration files.. Jan 19 16:39:02 bluelightning: could always arrange to manipulate PACKAGES, drop anything that won't comply with user license requirements, then bitbake would error out if something tried to pull it in, since the runtime provider wouldn't exist Jan 19 16:39:04 * kergoth ponders Jan 19 16:39:53 kergoth: yes that sounds like it would work at least for recipes that don't do any dynamic population of PACKAGES (which is most of them) Jan 19 16:48:24 kergoth: the way I'd probably suggest doing it would be to skip creation at package creation time Jan 19 16:48:43 kergoth: that would mean that files that were destined for the package didn't somehow end up in another package Jan 19 16:48:45 fray: My english fails me, what I mean with reproducability is the ability to build a package from a source package(generated by yocto/poky) which has the same package dependencies as the prebuilt package(built by yocto/poky). Jan 19 16:53:12 RP__: good point. Jan 19 18:34:20 zedd_, poke Jan 19 18:35:22 zedd_, something is awry with the linux-yocto patch/cfg stuff still. I've updated master and am using AUTOREV - but I can't get it to stop trying to apply a patch that I have deleted and removed from the SRC_URI Jan 19 18:38:16 I'm heading to the dentist right now for the next hour. but master is working for me. so that is odd. Jan 19 18:38:23 zedd_, found it Jan 19 18:38:31 the scc file has the patch series too Jan 19 18:38:36 duplication is B A D Jan 19 18:38:39 oh yes. it's all been fixed now. Jan 19 18:38:48 s/fixed/changed/ Jan 19 18:38:51 so I can drop the scc altogether? Jan 19 18:38:57 if you want. yes. Jan 19 18:39:00 but don't have both. Jan 19 18:39:09 patches will stay in order as listed in the src_uri Jan 19 18:39:13 excellent **** ENDING LOGGING AT Fri Jan 20 02:59:57 2012