**** BEGIN LOGGING AT Mon Feb 18 02:59:56 2019 Feb 18 07:35:12 Is there any way to see arguments of nested functions (either with bash/python) without the need to recompile everything? Feb 18 07:35:52 For example the staging.bbclass Feb 18 07:36:24 we do have in it: python·do_populate_sysroot -> bb.build.exec_func("sysroot_stage_all",·d) -> sysroot_stage_dirs·${D}·${SYSROOT_DESTDIR}$ Feb 18 07:36:49 In the debug (-v) I only see the "sysroot_stage_all" executed Feb 18 07:37:00 but no functions nested Feb 18 07:37:12 If I add bb.note() then I do need to rebuild everything Feb 18 07:37:28 is there any switch which would allow me to see the arguments? Feb 18 07:37:37 (-DD neither -v is not working) Feb 18 09:19:56 Hello, Is it possible to modify splash screen more than just to change the image with psplash? Feb 18 09:20:13 I would like to turn image + load bar upside down but i have so far been unable Feb 18 09:20:50 willie_: its a very simple piece of code so I suspect it is possible but you'd have to edit it Feb 18 09:22:03 thanks RP, are you refering to the "make-image-header" script? I'll have a look at it :) Feb 18 09:24:13 kanavin: RP told me to talk to you about this. Feb 18 09:24:22 kanavin: I think I found an issue concerning .pyc files in both the 3.5.x and the new 3.7.2 recipes: The file modification timestamp in the .pyc files lags behind the file filestamp of the actual .py files and thus precompiling them is of no use. Feb 18 09:25:15 kanavin: I know you looked at this, I can't remember what the conclusion was though Feb 18 09:25:55 willie_: you'd also have to look at the code drawing the progress bar as it would need to be rotated too Feb 18 09:41:27 @RP, turns out i could simply edit the init file! Feb 18 10:12:13 tristanram_, RP: I compared the .pyc files between two multilib installations, but I did not compare the timestamps vs the original .py files, assuming that python installation would take care of them being right Feb 18 10:13:24 tristanram_, if you have a reproducer for the issue, we could take a look Feb 18 10:14:04 kanavin: could it be that during the packaging process or some late build stage the .py files are copied around and thus the mtime changed? Feb 18 10:14:43 kanavin: Also, where are the .pyc generated in the python3 recipe? For python2.7 it was in distutils.bbclass i think Feb 18 10:14:53 tristanram_, I don't know to be honest :) Feb 18 10:15:42 my goal was to make builds green and tests (including on-target ones) pass, I only looked into what is going on when there were issues :) Feb 18 10:15:55 with this, there were no visible issues and so I didn't look Feb 18 10:20:44 kanavin: RP told me on #oe that there are no tests to check those .pyc files. Feb 18 10:20:58 I don't think so. Can you write one? Feb 18 10:21:18 how did you discover the issue? Feb 18 10:22:46 kanavin: Possibly, but first I would have to find out where those .pyc files are generated, why the mtime of the .py files changed since the .pyc creation and fix that. Else you would just have a test which would always fail. Feb 18 10:23:46 kanavin: I most use a readonly rfs. I recently started python in verbose (python3 -vv -c"") to check where the long interpreter startup time might some from. Feb 18 10:24:18 kanavin: I was able to spot many messages like "bytecode is stale for 'site'" Feb 18 10:24:27 tristanram_, ah, right. Feb 18 10:25:42 Since my rfs is readonly, I have them everytime. Python3 of course works, but I think it always tries to compile py to pyc only then to find out that the rfs is readonly: '/usr/lib/python3.7/__pycache__/site.cpython-37.pyc': OSError(30, 'Read-only file system') Feb 18 10:26:28 kanavin: As my focus is on fast boot time, I think it would be better if the .pyc files where actually not stale Feb 18 10:28:03 kanavin: Can I be of any assistance with helping you to recreate this issue on your side? Feb 18 10:28:41 tristanram_: the trouble is we're rather constrained on people with time to look into things like this :( Feb 18 10:29:00 tristanram_, I cannot promise I will solve it in good time. Too much on my plate. Feb 18 10:29:30 tristanram_: We definitely should have a bug opened for it but I know we're struggling to find people with time to work on such things :( Feb 18 10:31:43 tristanram_, obviously we would greatly appreciate any help from you, but the optimal way is that you look into the issue, and devise a solution and a regression test. I can promise that anything you send will be reviewed promptly :) Feb 18 10:33:50 kanavin, RP: I can try to have a look at it as one of my sideprojects :). Should I open an issue on https://bugzilla.yoctoproject.org for this? Feb 18 10:34:33 tristanram_: yes please, we should track it Feb 18 10:37:42 kanavin, RP: Ok, I will open an issue then. If I start looking into it, I would start looking into to packaging process. I did not yet look into the packaging process. Where can I find any kind of documentation on how the build output is processed and then lands in the image at the very end? Feb 18 10:38:28 tristanram_: Not to be funny, but the manuals? Feb 18 10:39:30 tristanram_: basially do_install is run under a fakeroot environment called pseudo, do_package (package.bbclass) does lots of processing of the files including splitting into packages, then do_package_write_XXX writes out the package Feb 18 10:39:44 tristanram_: the image is then built from the packages Feb 18 10:40:18 tristanram_: so first step is to look at the contents of the packages and see if the times match inside those. That would then be a rootfs or packaging issue depending on what you fine Feb 18 10:40:21 find Feb 18 10:43:01 RP: The manuals would have been among the first thinks I checked. But I just asked because there might be some "hidden" comment-based-doc in some bbclass that you might know of by heart ;) Feb 18 10:44:00 tristanram_: Our manuals are pretty comprehensive, to the point you may struggle to find the info you want. That's partly why I gave you a set of keywords above Feb 18 11:12:42 Greetings! I've built a GTK3 app, and integrated it in an image i cloned from core-weston (which includes gtk3-demo) : it goes into segfault as soon as it is done building the template Feb 18 11:12:51 The same code works on the host Feb 18 11:15:39 RP: Ok here is the issue https://bugzilla.yoctoproject.org/show_bug.cgi?id=13186 Feb 18 11:15:40 Bug 13186: normal, Undecided, ---, paul.eggleton, NEW , Precompiled python3 bytecode is stale Feb 18 11:16:24 RP: It might also be possible to make use of the new python 3.7 hash based byte code invalidation mode Feb 18 11:17:17 tristanram_: kanavin mentioned that, we worried it meant a lot of startup overhead hashing files though Feb 18 11:18:15 RP: Yes, that's what I would be worried too. Even if the hash method is blake2, it might take quite some time to read every file. Feb 18 11:18:51 RP: But for "readonly" rfs, UNCHECKED_HASH might be the best solution Feb 18 11:19:26 tristanram_: we don't know a given python is going to end up on a readonly rootfs in advance. Can you set that somewhere at runtime? Feb 18 11:20:10 RP: I know in my case: EXTRA_IMAGE_FEATURES = "read-only-rootfs" Feb 18 11:20:43 RP: Why not just check if "read-only-rootfs" is set? Feb 18 11:20:51 tristanram_: right, but what I mean is can the rootfs code tweak the python to use UNCHECKED_HASH Feb 18 11:21:50 RP: Ah, is the EXTRA_IMAGE_FEATURES not globally available? Feb 18 11:22:07 tristanram_: we don't build a custom python for each image! :) Feb 18 11:24:17 RP: Hmm ok, I see. Maybe my world is just too small with only my one image ;). Feb 18 12:18:21 https://downloadlagu-mp3.club/ Feb 18 13:05:04 kanavin: btw, did I do the right thing in http://git.yoctoproject.org/cgit.cgi/poky/commit/?id=c29ec9bf57486018e61ea90fb9314fbb76879eac ? Not sure its quite showing up in the rrs as we'd want? Feb 18 13:06:08 RP: at least from the AUH perspective it's right, it will skip the recipe from now on Feb 18 13:06:38 kanavin: I wasn't sure if that was the right variable to use... Feb 18 13:08:13 RP: we have it in several recipes in oe-core Feb 18 13:09:21 kanavin: that is fine, just wanted to check, thanks Feb 18 13:24:01 kanavin: https://autobuilder.yoctoproject.org/typhoon/#/builders/15/builds/532 :( Feb 18 13:37:03 RP: Timestamp in "tmp-glibc/work/cortexa7hf-neon-vfpv4-oe-linux-gnueabi/python3/3.7.2-r0/image" are correct but timestamps in the tar and ubifs are newer Feb 18 13:39:49 tristanram_, next step is to check the timestamps in packages/ and package-split/ Feb 18 13:40:32 kanavin: those are also correct. Whats the next step? Feb 18 13:44:54 tristanram_, check the content of actual package files in deploy-*/ Feb 18 13:45:16 you can usually 'unpack' those from the command line, not sure what package format are you using Feb 18 13:46:01 tristanram_, or you can skip to the do_rootfs step for your image and see what gets written to the rootfs/ directory in the image's ${WORKDIR} Feb 18 13:48:02 kanavin: ok, I will take a ipk from the deploy folder and check its contents next Feb 18 13:49:07 RP: that's a centos 7 specific failure, I'll grab lunch and get back to it Feb 18 13:50:20 tristanram_: ar -x should "open" the ipk, its like a deb Feb 18 14:00:01 RP, kanavin: Ty, timestamp is altered in the extracted ipk Feb 18 14:00:54 RP, kanavin: image: ok, package: ok, package-split: ok, ipk: nok Feb 18 14:03:48 tristanram_: ok, that means its something to do with opkg-build in package_ipk.bbclass Feb 18 14:05:56 tristanram_: looking at opkg-build in the opkg-utils-native recipe, its using --mtime=@$build_date Feb 18 14:06:00 (to tar) Feb 18 14:07:44 RP: the mtime argument would explain the different timestamps Feb 18 14:08:08 tristanram_: yes. I think this is an effort to make it reproducible which has broken the python embedded timestamp Feb 18 14:11:20 tristanram_: nice thing is its a shell script so you could hack it easily Feb 18 14:15:50 RP: i could add a patch for opkg-build in my layer Feb 18 14:18:14 RP: I'm gonna try that and see if it works for me. But I'm thinking about a less hacky fix Feb 18 14:19:06 tristanram_: I don't think that --mtime option should be being applied to data.tar.gz, only the control one Feb 18 14:19:46 tristanram_: I'm hoping Alejandro, the opkg maintainer is around later. You could put the details in the bug and cc him there Feb 18 14:20:34 So, a GTK app runs perfectly with a simple scp to a fresh-minted Debian 9 VM, will segfault on Yocto in spite of any added library. Any hint on how to debug this? Feb 18 14:21:10 sk_tandt, with gdb? Feb 18 14:21:16 sk_tandt: 1) use proper toolchain 2) use proper toolchain 3) use proper toolchain :) Feb 18 14:21:24 tristanram_: it was a recent change: http://git.yoctoproject.org/cgit.cgi/opkg-utils/commit/?id=682f8c5e35b8854a9bb858b8ee1714d27e0c00db Feb 18 14:21:35 er, a year ago even :/ Feb 18 14:21:36 sk_tandt: besides that, strace, and check that you are using a proper toolchain :) Feb 18 14:21:49 RP: Oh, nice in that case it might be fixable this way. I will update the issue in bugzilla Feb 18 14:23:25 LetoThe2nd, proper toolchain? Feb 18 14:26:58 sk_tandt: -> the toolchain that matches the image on the target in use, specifically its ABI Feb 18 14:27:50 sk_tandt: thats why there are ways to create an sdk/esdk Feb 18 14:28:56 I'm afraid I'm not following: I suppose that has a purpose only for debugging, right? Feb 18 14:29:27 sk_tandt: no. it has a purpose specifically to compile for the target Feb 18 14:33:43 Mh, then I'm afraid I'm missing something. Once I've set a machine (Say, qemux86-64), added necessary layers and added packages with IMAGE_INSTALL entries, isn't everything in place? Feb 18 14:34:07 sk_tandt, how did you build the binary? Feb 18 14:34:10 for your app? Feb 18 14:34:21 sk_tandt: the point is rather: where are you compiling? using which toolchain, provided by whom? which headers for the libraries? Feb 18 14:35:21 sk_tandt: thats exactly why you can use -c populate_sdk to generate a toolchain that comes with the sysroot matching the image. Feb 18 14:35:28 kanavin, I've added DEPENDS += "gtk+3" Feb 18 14:38:02 LetoThe2nd, I was following the Yocto Quick Build guide, but I'm starting to assume that isn't viable? Feb 18 14:38:46 sk_tandt: the quick build guide is good, but you are still not telling us how you are building what, and how it relates to that debian vm you mentioned Feb 18 14:39:06 sk_tandt: so, lets start at the beginning. you have an application in source, right? Feb 18 14:41:10 Yep! Feb 18 14:41:29 sk_tandt: and how are you *BUILDING* the application? Feb 18 14:42:03 Autotools, as generated by the Yocto Eclipse plugin Feb 18 14:42:39 sk_tandt: and how are you *TRIGGERING* the autotools build? what is the very definitive, manual action that you do that starts the building process? Feb 18 14:43:20 ...should there be one? I thought adding a recipe to an image and invoking "bitbake my-image" would've done it automatically Feb 18 14:44:20 if you have added it to the image and are building the image, then this is the action, yes. but you haven't ever mentioned that so far. Feb 18 14:44:31 Sorry, my bad Feb 18 14:44:48 I did Feb 18 14:44:48 sk_tandt: so you build the image, the application gets automatically built and included, right? Feb 18 14:45:00 Yup, I can find the executable within /usr/bin Feb 18 14:45:21 And even the files that must be copied during the installation Feb 18 14:45:24 ok. so now what does that have to do with the debian vm then? Feb 18 14:47:25 I wanted to figure out was was missing in the built image for the app to work Feb 18 14:48:05 And/or reproduce the problem with the source code Feb 18 14:48:58 sk_tandt: please, stop annotating the non-given asnwers. are took the binary from the target build and copied it out, or what? Feb 18 14:52:53 i mean, we can't see what you are doing. so if you tell or explain something, be sure to include all bits and pieces. do not take any circumstances for granted. Feb 18 14:53:55 my bets are that you are taking some assumuption about the system that the application shall run on somewhere, but don't check it explicitly Feb 18 14:54:31 but starting inside gdb and waiting for a backtrace to hit might give you a hint. Feb 18 15:04:12 Sorry, got back: LetoThe2nd, you're quite right, missed the balance between dumping too much and not giving enough info Feb 18 15:04:50 The built executable was indeed scp'ed to the end vm, and not rebuilt Feb 18 15:05:06 As for setting up gdb, will come back as soon as I have something! Feb 18 15:07:45 sk_tandt, so you transferred some pre-built binary into the qemu image? don't do that, write a recipe for your code, make sure the image pulls in the package produced by the recipe. Feb 18 15:08:52 kanavin, nope: I did build a recipe for the app, but I tried to replicate its issues copying the built executable in a vm Feb 18 15:09:59 kanavin: as far as i understood it, he copied the recipe-built binary out. hence my assumption that the autotools/build process of the app makes an assumption somewhere, and does not properly chack it. Feb 18 15:10:52 s/chack/check/g Feb 18 15:13:48 Can someone point on my error in this thought process.. If i set "logo.nologo" at the end of mmcargs i can succesfully boot without tux Feb 18 15:14:21 But when i try to re-compile with "rootfstype=${rootfstype}" \ "logo.nologo\0" \ at the end of mmcargs i get kernel panic Feb 18 15:14:37 recompiling u-boot that is Feb 18 15:24:13 RP: fix for gdk-pixbuf sent Feb 18 15:26:36 RP: Just patched away the --mtime stuff and no more stale .pyc's. Only matches. Interpreter startup time on my embedded board at idle dropped from 5s to around 1s. Feb 18 15:30:40 tristanram_, you need to discuss this with opkg upstream I guess Feb 18 16:04:21 How may I know if my Eclipse is pointing to the correct toolchain? Feb 18 16:04:58 sk_tandt: just don't use it :P Feb 18 16:05:21 sk_tandt: sorry to but the eclipse plugin is mostly unmaintained and problem ridden Feb 18 16:06:49 *say. Feb 18 16:08:07 Oh, nice! I was using it to generate the autotools configuration files: is that part of the plugin bugged ? Feb 18 16:10:27 consider every part being bugged. Feb 18 16:10:41 Sigh, thanks Feb 18 16:31:17 RP: I also sent a patch for glib meson issue (core-image-full-cmdline failures) Feb 18 16:48:39 tristanram_: adelcast is the person we need to talk to Feb 18 16:49:16 adelcast: We're seeing a problem where python's cache files (pyc) are invalid which is particularly problematic in a readonly rootfs Feb 18 16:50:11 adelcast: the problem is that opkg-build changes the data.tgz mtimes of the files from http://git.yoctoproject.org/cgit.cgi/opkg-utils/commit/?id=682f8c5e35b8854a9bb858b8ee1714d27e0c00db Feb 18 17:01:20 RP: so, change the mtime makes Python think that the files are invalid? Feb 18 17:01:26 change -> changing Feb 18 17:03:04 adelcast: correct Feb 18 17:03:47 adelcast: I'm wondering if we should only set the mtime for the outer archive and the control files/archive but not data Feb 18 17:05:21 I am looking at https://bugs.python.org/issue29708 Feb 18 17:08:48 adelcast, I think --mtime=$build_date should happen only if SOURCE_DATE_EPOCH is set. If that is the case, python3 itself will switch to hash-based mechanism automatically. Feb 18 17:09:54 if SOURCE_DATE_EPOCH is not set, then mtime shouldn't be set to what the date is right now, as it does nothing for reproducibility, and breaks the cache mechanism Feb 18 17:10:53 (if I'm reading that opkg patch correctly) Feb 18 17:11:07 kanavin: I'm not sure it should happen at all for data.tar Feb 18 17:11:47 RP: right Feb 18 17:13:43 RP: if the mtime is not set on data.tar, how is the build reproducible? Feb 18 17:15:24 kanavin: that makes sense, if SOURCE_DATE_EPOCH is not set, then we should just not even try Feb 18 17:16:00 however, if SOURCE_DATE_EPOCH is set, and python < 3.6, then we will hit the original problem Feb 18 17:16:46 adelcast, oe-core master has 3.7.2 now :) Feb 18 17:17:40 sweet, then that could work Feb 18 17:18:15 adelcast, I actually agree with RP - don't rewrite mtimes of files in data tarball at all Feb 18 17:20:58 the package manager should not be concerned with what timestamps those files have - if the goal is to have a reproducible build, then the build system that actually creates those files in the first place will take care of it Feb 18 17:22:55 oh right....that does makes sense, opkg should make sure the binaries it creates are reproducible, not the binaries it consumes Feb 18 17:23:25 that should be a simple change Feb 18 17:25:13 yep, I guess dropping --mtime=$build_date from data.tar line would suffice Feb 18 17:26:31 yeah Feb 18 17:26:49 I can submit a patch shortly Feb 18 17:42:10 I would like to add pre-built u-boot images to my distribution. They should be included in the .swu files created for SWUpdate and therefore need to be copied into the deploy folder. The bin_package.class offers a nice solution when copying the file(s) into the sysroot - is there something similar for files supposed to end up in the deployment folder? Feb 18 17:42:33 adelcast: right, I was thinking the underlying files were the build system's remit, opkg should only worry about its own files Feb 18 17:43:16 adelcast: thanks for helping :) Feb 18 17:43:25 tristanram_: ^^^ Feb 18 18:26:57 RP: AUH is done :) Feb 18 18:39:30 kanavin: ah, great. I should prepare for an influx of patches then? :) Feb 18 18:40:15 in theory :) Feb 18 18:41:56 in practice, you'll probably have to grab most of them from /home/auh/build/upgrade-helper/20190217000703/all on the auh machine Feb 18 18:42:10 but let's wait a few days as usual :) Feb 18 19:39:21 adelcast: seems to be ok with removing --mtime in one line of the script "( cd $pkg_dir && tar $ogargs $tsortargs -X $tmp_dir/tarX -c $tarformat . | $compressor $compressorargs > $tmp_dir/data.tar.$cext ) || rc=$?" Feb 18 19:40:28 adelcast: I can test it tomorrow and update the issue in bugzilla accordingly Feb 18 19:41:25 tristanram_: that sounds great, do you have the bugzilla #? Feb 18 19:57:46 adelcast: 13186 Feb 18 20:20:47 thanks, just added myself to the list, once I get the notification that the test passed, I can send patches Feb 18 20:42:24 * armpit auh is done.. check Feb 18 20:47:02 so are we automatically update packages that pass? Feb 18 22:27:40 armpit: we didn't decide to do that as yet Feb 18 22:52:14 Hi, I created my own machine and kernel for a custom x86/64 tablet, it needed a lot of customization to get it to work properly and i'm also building a custom embedded application. But when I go to run bitbake -c populate_sdk core-image-sato it gives me a nice little error I was hoping one of you could help me out, it says: meta-environment-surface-go-x86_64-1.0-r8 do_package_write_deb: Fatal errors occurred in subprocesses Feb 18 22:53:41 https://pastebin.com/899drCMD Feb 18 22:55:49 so far I can tell i'm missing this: x86_64-pokysdk-linux but I do have x86_64 executable binary Feb 18 23:05:22 GeramyL: The command that fails is dpkg-deb -b $WORK/packages-split-meta-environment-surface-go-x86_64 $WORK/deploy-debs/x86_64-nativesdk Feb 18 23:05:38 Unfortunately it doesn't seem to give you the stderr output, which might help you out finding why it failed Feb 18 23:07:11 neverpanic: If this is a custom machine and kernel meta-layer what would I be missing so that the sdk builds? Do you know? Feb 18 23:12:06 I don't know. I'm telling you, packaging fails, so that's likely a problem with your packaging, not with your machine or kernel. Feb 18 23:12:26 May I suggest you try to reproduce this command manually and see if it prints helpful output? Feb 18 23:15:47 neverpanic: thats a good suggestion yeah let me do that :) Feb 18 23:52:25 neverpanic: sorry to ask such a dumb question, but how do you propose I run the command separately? Feb 18 23:55:50 dpkg-deb: error: failed to open package info file '/home/geramy/Documents/Yocto_Kiosk_Surface_Go/Surface_Go_Build/tmp/work/x86_64-nativesdk-pokysdk-linux/meta-environment-surface-go-x86_64/1.0-r8/packages-split/meta-environment-surface-go-x86_64/DEBIAN/control' for reading: No such file or directory Feb 19 00:00:24 neverpanic: any suggestions? I'm not even sure what that is, to be honest. DEBIAN/control doesn't exist and my google foo has failed looking for it. Feb 19 00:05:20 RP, ok Feb 19 00:05:35 there are issues with meta-environment and my machine meta layer Feb 19 00:06:18 I don't see any documentation though regarding custom kernels, machine meta layers and being built with populate_sdk Feb 19 00:19:34 armpit: if you do want me to pull in yours let me know... Feb 19 00:20:17 denix: thanks :) Feb 19 00:29:55 neverpanic: looks like it was an issue with DEFAULTTUNE and using intels meta layer tune with my kernel and machine. Feb 19 00:30:16 maybe something to do with multi-inheritance, i am able to use default tune. Feb 19 00:31:04 nothing really points to that issue, but it was a good guess on my end haha :P Feb 19 01:11:17 Anyone know how to include plugins from gst-plugins1.0-bad in a build? It seems that one specific plugin, audiobuffersplit, doesn't seem to be built by default Feb 19 02:09:54 * armpit woohoo clean sumo build... finally **** ENDING LOGGING AT Tue Feb 19 02:59:57 2019