**** BEGIN LOGGING AT Wed Sep 04 03:00:01 2019 Sep 04 04:30:53 I have created a small test application for the sake of learning how to create a custom recipe. The test application is using CMake. When I build this recipe in bitbake I get the error message: "do_package: QA Issue: anpr-test: Files/directories were installed but not shipped in any package:". Could this be because the install dir in the CMake description does not use the default "bin" directory, but Sep 04 04:30:59 rather a custom directory under /? Sep 04 04:32:27 iceaway, sounds likely Sep 04 04:33:02 iceaway, unusual files like that need to be explicitly listed in FILES_* Sep 04 04:33:57 iceaway, but you can use wildcards when listing Sep 04 04:34:52 Ok great, thanks. Sep 04 04:37:50 RP: just started looking at the autobuilder runs with the toolchain test patches... I realised its running everything :| aka both gcc/glibc with user and system qemu. That will take awhile :| Sep 04 04:40:35 RP: finally I have the systemtap issue reproduced on my mips64 image. it’s 12:40 am here, so I’ll pick it up in the morning. I assume that your uprev in master-next didn’t sort it out. Sep 04 04:56:13 I'm trying to use wic to create a custom partitioning scheme for my image. I have setup two partitions / and /anpr. But when I try to create the image I get an error message: "ERROR: Nothing PROVIDES '${IMAGE_ROOTFS}/anpr'". I can see that /anpr is present in the tar.gz version of the rootfs in my deploy/images directory. Any ideas? Sep 04 05:08:39 New news from stackoverflow: bitbake failed with ExpansionError Sep 04 05:38:43 New news from stackoverflow: OpenBMC with Raspberry Pi (2 or 3) and build bmcweb? Sep 04 06:48:49 Hello, I stumbled upon something strange with the bitbake build system. Sep 04 06:50:49 in my project at work, it seems that when I have two packages, and the first package has a dependency on the second package. If i update the second package, rebuilding the first package doesn't automatically update the cached dependency Sep 04 06:51:17 meaning that I might rebuild the first package, using the old version of the second package, even though the second package has been updated Sep 04 06:52:13 do you have SRCREV = "${AUTOREV}" ? Sep 04 06:52:36 no. They are two local packages Sep 04 06:52:57 as both packages are fairly small Sep 04 06:54:01 so far only way I found that solves this issue is by manually deleting the cache of the first package Sep 04 06:54:38 bitbake is not rebuilding if it is sees that the bb file is not changed Sep 04 06:55:07 ahaa... then that would be the cause, even if the compiled package is different Sep 04 06:55:38 do you have your package in a git repo ( on your local machine ) ? Sep 04 06:56:02 both packages are in the same git repo on my local machine Sep 04 06:56:58 you can do a git init an somthing like SRC_URI = git:///Data/Development/Qt/package1;protocol=file;branch=${BRANCH} Sep 04 06:57:22 SRCREV = "${AUTOREV} Sep 04 06:57:56 i feel like you are not exactly following the problem I'm describing, as this is not related to git (as far as I know). Sep 04 06:58:32 how do you store the local package? Sep 04 06:59:11 in my companys git repository, both inside the same layer Sep 04 07:00:52 how do you "update the second package" ? Sep 04 07:01:43 "bitbake -c compile second-package-name" Sep 04 07:02:03 o wait, do you have ${SRCREV} in your PV var this will trigger rebuild if the git repo is updated Sep 04 07:03:14 @MarcWe I have taken advantage of that when it's linking two different git repos together in the same layer. However, in this situation that is not the case. Sep 04 07:03:33 feilz: you've done something before that which is requiring you to run do_configure. What? Sep 04 07:03:42 do_compile* Sep 04 07:04:08 modifying the source files Sep 04 07:04:10 modified the source code? modified the recipe? what did you do? Sep 04 07:04:19 from where? Sep 04 07:04:44 I mean, where did you operate the changes for the source code? in the workdir? how do you fetch the sources? Sep 04 07:04:59 uhh... inside the folder where the second package .bb directory Sep 04 07:05:59 feilz: are you adding directories in SRC_URI by any chance? Sep 04 07:07:22 RP: FYI, plain text version of "Yocto Project Status WW36’19" mail I received is unreadable. It's missing new lines (at least). Sep 04 07:10:46 feilz you shouldn't have to run compile for the package, the dependency should be pulled because there's been a change in one the source file. So something makes Yocto think the source files were not updated (or somethng is saying to avoid rebuilding) Sep 04 07:11:21 qschulz: hmm, it was entered as html as that is what stephen normally does so something went wrong. You can also read it on the wiki Sep 04 07:11:45 sorry, crash on my end. qschulz we use SRC_URI = "file://*" to include all files Sep 04 07:13:04 feilz: that way bitbake is not seeing change to the bb file content so it not rebuilding ( if you change things in "file://*" ) Sep 04 07:14:18 if I included all files manulally such as SRC_URI = "file://files/filename1.c \ file://files/filename2.c", the change would be seen? Sep 04 07:19:19 feilz: im not shure about that. but sins * is not working i think it is not getting picked up Sep 04 07:19:51 hello Sep 04 07:24:24 I'm facing "file /lib/systemd conflicts between attempted installs of" while doing a rootfs, can't figure out why but it happens whereas i'm installing different files from different packages. Could this also apply on directories? Sep 04 07:27:40 @qschulz the browser based irc client kept crashing on me... switched to command line. probably missed any questions / messages that came after my last comment Sep 04 07:30:04 basically i know the reason why rebuilding the first package failed, as the .bb file didn't change of the dependency. Sep 04 07:30:53 one suggestion for implementing to bitbake would be an option to force rebuild dependency packages of the package im building, to avoid having to manually delete the cache Sep 04 07:31:36 would perhaps be cool as well if it would be incorporated to "bitbake -c cleanall packagename" as well Sep 04 07:31:46 feilz: i guess you rather need to look at PR/PV in depth Sep 04 07:32:23 feilz: because bitbake intentionally rebuild when it gets told that has been a change, and does not rebuild when not being told Sep 04 07:32:56 aaaa... now i remember reading about that... Sep 04 07:33:06 okay. Sep 04 07:33:50 LetoThe2nd: I also have the same issue. When using directories with SRC_URI, if you change the content of the files, Yocto does not detect the changes and do not rebuild Sep 04 07:34:23 feilz: as well as rethinking the recipe/source structure. because if you have one source repo building several packgaes, it might be helpful to actually pour it into one recipe that provides additional packages. Sep 04 07:34:47 qschulz: directories, as "something local on your disk"? Sep 04 07:35:30 LetoThe2nd: meta/recipes-foo/foo/foo.bb SRC_URI = "file://bar" and then meta/recipes-foo/foo/bar/main.cpp Sep 04 07:35:41 change main.cpp, nothing gets rebuilt Sep 04 07:36:16 LettoThe2nd: rethinking the recipe/source structure won't be necessary in this situation. The problem I describe is what qschulz now highlights (apparently much better than I managed to do) Sep 04 07:36:51 i see what you mean, but in my opinion sticking source into the layers is a super bad practise anyways, and those who do it deserve to be punished :-P Sep 04 07:37:03 if I use PV="0.1" and then bump that number, that should force a rebuild of main.cpp Sep 04 07:37:25 feilz: maybe even PR IIRC Sep 04 07:37:34 but not great Sep 04 07:38:01 LetoThe2nd: well, it does not have to be specific to code actually Sep 04 07:38:07 I can have configuration files there Sep 04 07:38:17 especially when doing some minor fixes, having to bump for every character change or minor bugfix does seem excessive Sep 04 07:38:31 qschulz: sure, i know what you mean. taking me overly serious is a bad practise too :) Sep 04 07:38:32 feilz: -c cleanall for the package you're modifying Sep 04 07:38:59 LetoThe2nd: well, I don't like it myself as well but ATM, things are how they are Sep 04 07:39:06 oh... that's how it's solved Sep 04 07:39:16 feilz: no, worked around :) Sep 04 07:39:27 true Sep 04 07:39:38 sorry, -c cleansstate is enough IIRC Sep 04 07:40:15 yeah. Sep 04 07:40:53 anyways, hope I managed to bring this issue to light... glad we managed to find a workaround. Took me a day of bughunting yesterday to notice this :) Sep 04 07:40:57 LetoThe2nd: do you have more input on why it is working that way? It seems to be a known "issue". Is there a reason for not fixing it? Sep 04 07:41:08 feilz: been there :) Sep 04 07:41:45 LetoThe2nd: which could well be "nobody took time to send a patch". But maybe there was something already posted and got rejected for a reason. Sep 04 07:42:04 qschulz: in a nutshell it is like this: "bitbake decides what do based on looking at the recipe". so if it looks at the recipe, nothing changed, then nothing is to be done Sep 04 07:42:40 qschulz: it can be forced to behave differently by applying stuff like externalsrc, but thats the main reasoning. Sep 04 07:42:51 LetoThe2nd: well... it knows when a file from SRC_URI is changed Sep 04 07:43:10 so can't we have a glob there of the content of directories? Sep 04 07:43:53 I guess there is something like the hash of the files in SRC_URI in the sstate-cache to know if it's changed/ Sep 04 07:46:25 Hello, I have a question about the ccache bbclass which has been reworked, but the mega-manual seems not to be up to date and advises against using it, is this advice still valid? Sep 04 07:46:27 no, it does not Sep 04 07:47:08 AFAIK it does *not* look at the files mentionend in SRC_URI, only at SRC_URI itself Sep 04 07:47:36 but there are others you can explain siggen way better than me. Sep 04 07:48:17 can't find what DIRFILES does in the mega manual, am I missing sth? Sep 04 07:48:44 LetoThe2nd: that's something I haven't looked at yet and I'm eager to know about it a bit more :) so if anyone has pointers... Sep 04 07:49:37 qschulz: hand over beer to RP :) Sep 04 07:50:20 i think theres also a tool like bitbake-diffsigs or such Sep 04 07:57:14 RP: thanks, I just wanted to give a headsup as previous mails (last week's was broken as well IIRC though) were readable. Nothing to worry about, I still can open the html version, but you know.. one more key press :D Sep 04 07:59:52 qschulz: FWIW a glob of files should correctly notice when they change Sep 04 08:04:44 qschulz: you can understand a bit about it by doing bitbake-dumpsig on tmp/stamps/XXX/YY.do_fetch.sigdata Sep 04 08:05:00 That will tell you what makes up the data bitbake uses on that task Sep 04 08:17:57 Im trying to do some app development for my target using qt creator but im getting the error: /usr/lib/libstdc++.so.6: version `GLIBCXX_3.4.22' not found (required by Sep 04 08:19:22 MarcWe: on target or using native tools on the host? Sep 04 08:19:38 on the target Sep 04 08:19:55 that sounds like trying to run newer binaries on a older target toolchain Sep 04 08:20:09 gcc and glibc aren't matched up Sep 04 08:20:52 yes but im shure that i loaded up the correct tool chain Sep 04 08:21:19 MarcWe: that error says otherwise, it looks like glibc is probably too old Sep 04 08:36:18 nrossi: happen to be around? Sep 04 08:39:17 New news from stackoverflow: Description of files that are in image files folder in Yocto project and the boot sequence of Yocto linux Sep 04 09:00:35 RP: i am now Sep 04 09:01:36 nrossi: I've run the series on the autobuilder and the gcc tests are going crazy. I've locally tracked it down to all the headers in gcc/include/*.h containing just the word "timestamp" Sep 04 09:02:29 nrossi: I had to add python3-pexect and python3-ptysomething to the build to make it work. We also need to add inherit nopackages to glibc-testsuite.inc Sep 04 09:02:47 nrossi: those python3 bits and the dejagnu are missing from maintainers.inc too Sep 04 09:03:07 nrossi: any idea why "timestamp" would end up in those headers? Sep 04 09:03:49 RP: for the gcc ones I thought that might be due to builddir issues due to the multiple processes running with selftest but I had not looked into it. Though you say you have reproduced it on a local build without rm_work? Sep 04 09:04:20 nrossi: right, I don't use rm_work, nor does the autobuilder Sep 04 09:04:26 nrossi: its appeared on both Sep 04 09:04:57 RP: for the python-pexpect, i saw that when looking at the autobuilder logs. I completely forgot to check those packages. They are only there to make a FAIL -> UNSUPPORTED, needs gdb to actually run the tests. I am looking at that atm Sep 04 09:05:25 RP: I will wipe away my sstate and see if i can reproduce. Sep 04 09:05:58 nrossi: this sounds like criticism, its not meant as such btw :) It is great to see this so close! Sep 04 09:07:19 RP: dont worry, I have not taken your words in that way. Getting this stuff working is lots of moving pieces and I expected issues :) Sep 04 09:08:07 nrossi: I'm struggling with too many issues which we're struggling to debug right now :/ Sep 04 09:08:33 RP: one question I did have seeing the builds running, is did you actually want to have all machines running all usermode and system emu tests? Sep 04 09:09:10 RP: I have been following, sorry if i'm taking too much of your time away from more important tasks Sep 04 09:09:28 nrossi: this is important! :) Sep 04 09:09:43 nrossi: I'm just mentioning this in case I seem a little distracted Sep 04 09:10:00 nrossi: no, we probably don't want both usermode and system Sep 04 09:10:38 nrossi: I think that was my naive way of running things Sep 04 09:11:11 RP: probably need a "manual" tag or similar to put on some of the tests. And possibly change the selftest -t to a "tags of test must match those in list of supplied tags" Sep 04 09:11:23 nrossi: not quite decided how to integrate into the autobuilder yet, this was just a quick hack to start testing Sep 04 09:12:16 RP: no problems, I was mostly focused on getting everything working with the v2 series with less emphasis on autobuilder setup Sep 04 09:13:06 nrossi: right, and I wanted to get something run so we could find any issues with that piece. I'll ponder the integration a bit more but you're right, we need to select between the user and system modes Sep 04 09:13:50 nrossi: it may be enough just to tag them differently and run machine,toolchain-system or machine,toolchain-user or something that simple Sep 04 09:14:46 RP: possibly, I was just thinking about having the system emulated glibc tests running for x86-64 by default though Sep 04 09:16:26 schulz: lettoThe2nd: one possible solution that could solve the issue we talked about is to modify file checking to take chksum of all files included, and compare that to the new version. Think it could work? Sep 04 09:16:55 nrossi: it gets tricky as you'd want to do something different on an arm host :/ Sep 04 09:19:51 RP: I will have a bit of a think about it, see if there is a decent solution Sep 04 09:20:20 nrossi: ok. We do have a bit of time pressure now to get something ready for 3.0 even if its not perfect Sep 04 09:21:00 RP: of course, i meant think as in whilst sorting out some of the current issues :) Sep 04 09:21:15 nrossi: fair enough :) Sep 04 09:21:46 RP: also should I put the maintainers as those that already mantain said recipes, e.g. binutils, glibc? Sep 04 09:22:09 nrossi: yes, I think that is khem Sep 04 09:22:27 RP: I didn't know about those. Thanks :) Sep 04 09:22:39 RP: and for dejagnu who should I put? it is not a hard recipe to maintain so I am not against putting my name if nessecary Sep 04 09:22:40 nrossi: the python ones are trickier but even better if we don't need them Sep 04 09:22:56 nrossi: volunteers are very welcome to spread the load Sep 04 09:23:20 RP: if the gdb tests don't work anyways I think its probably just easier to drop them so they appear as "FAIL:" Sep 04 09:23:22 RP: well, the issue is that SRC_URI = "file://dir" does not add the files of this directory to the "This task depends on the checksums of files:" list. SRC_URI = "file://dir/*" as well. HOWEVER, SRC_URI = "file://dir/" does. Sep 04 09:23:32 feilz: ^ Sep 04 09:23:54 (well, both RP and feilz actually) Sep 04 09:24:19 qschulz: probably time to add a new selftest to the bitbake test suite and fix it... Sep 04 09:24:39 qschulz: it sounds a bit like a bug Sep 04 09:28:52 i'm not a bitbake developer, started using bitbake about 2 months ago when i started at my current employer :) Sep 04 09:29:21 LetoThe2nd: see, there is a point in having people "misuse" Yocto, you get to find bugs :D Sep 04 09:29:41 but glad that I could help the community :) Sep 04 09:30:08 feilz: add everything in a subdir so that you can include SRC_URI = "file://subdir/" Sep 04 09:30:26 and it does seem like the "misuse" of Yocto is a little more widespread, as I know at least 2 of the client companies that this company works for does the same setup Sep 04 09:30:49 also IIRC, we didn't see the issue when it was a simple * but a *.c for example Sep 04 09:31:04 feilz: don't forget the trailing slash Sep 04 09:31:11 qschulz: certainly! Sep 04 09:31:28 nrossi: I just reproduced the header thing again. cleansstate of gcc-cross-x86_64 and gcc-runtime Sep 04 09:31:47 i'll give it a go. Let's see if this works as expected Sep 04 09:31:55 nrossi: inspected the headers in gcc-cross-x86-64's gcc stash builddir and they were fine. Build gcc-runtime, they're not Sep 04 09:32:43 nrossi: I'll dig a bit more Sep 04 09:33:29 feilz: my issue with Yocto is that the ramp up is long so it does not get much interest from people for which Yocto is the tool and not the goal. Also, it is pretty much possible to have something that works "good enough" you don't experience the issues before a long time (not writing in the sstate cache output dirs for example). Which results in people having bad Yocto layers "but it works" TM Sep 04 09:34:04 yeah... I understand Sep 04 09:34:41 qschulz: s/"but it works"/"doesn't break enough for me to care"/ Sep 04 09:35:58 nrossi: gcc-cross-x86_64:do_install destroys the files Sep 04 09:36:29 nrossi: looks like a race where "make install" messes with the hardlinked files Sep 04 09:37:05 LetoThe2nd: the number of times I see people cleaning the whole build dir/sstate cache because there is an error in the build they don't understand/want to fix and a fuild rebuild/build fromsstate cache fix the issue. Sep 04 09:37:07 RP: but do_gcc_stash_builddir is supposed to be run before do_install :| Sep 04 09:38:26 RP: oh right, you mean onces the hardlinking is done but before the sstate part of the task? Sep 04 09:39:29 New news from stackoverflow: GitLab CI Yocto Build - How to use SSTATE and DL_DIR Sep 04 09:41:08 qschulz, you should tell them to file bugs. I've not had any problems with reusing sstate-cache for ages Sep 04 09:42:06 nrossi: the hardlink is done before do_install, gcc messes up the hardlink Sep 04 09:42:44 nrossi: it will be this: Makefile:STAMP = echo timestamp > Sep 04 09:42:55 @qschulz: tested out moving it into it's own directory, and when doing that make fails, due to no makefile found. Sep 04 09:43:17 even though that makefile.am does exist in that folder. Sep 04 09:43:34 feilz: S = "${WORKDIR}/dir"? Sep 04 09:43:56 oh Sep 04 09:43:57 sec Sep 04 09:44:28 nrossi: the "move-if-change" stuff Sep 04 09:44:31 RP: ok, i'm not fully aware of how the sstate-inputdirs stuff works, but does it not store the content in sstate before the task is "complete"? or is the issue that the hardlinks are kept to populate the outputdirs immediately instead of from the sstate binary? Sep 04 09:44:55 kroon: my goal now is to get rid of as many warnings as possible so that we have a clean build. Then teach people to not ignore the warnings (except those I couldn't get rid of and explain why), either to report them or to fix them Sep 04 09:45:08 kroon: and in the end have them report anything that is not looking normal Sep 04 09:45:38 qschulz, I apologize, I think I misread your sentence Sep 04 09:46:34 kroon: didn't see you misinterepreted it >< it matched both situations :D Sep 04 09:46:52 qschulz, :-D Sep 04 09:47:04 The advise about the ccache.bbclass from the mega-manual, is this advice still valid on Warrior?" Sep 04 09:47:05 feilz: also... might be actually a good idea to have the code outside of the yocto layers and use externalsrc instead? Sep 04 09:49:22 if it's actually code, it's better to use externalsrc IMHO (if you can't/don't want to have it in a git repo) Sep 04 09:50:31 anything else, I find it okay to use in-layer files. but you know, IMO. I don't have extensive knowledge about best practices Sep 04 09:51:31 qschulz: I do agree to that, but the project is too far in already to have them modify the project structure any longer. Sep 04 09:51:54 I do agree with what you are saying, and the previous project I did in this company we had that. Was far easier to manage :) Sep 04 09:52:49 but yeah, I'm getting forward, and that problem is long gone. Seems I still have other issues, but I'll work them out 1by1 :) Sep 04 09:53:00 feilz: :) Sep 04 09:54:14 RP: another issue with dirs in SRC_URI. If you have dir1/dir2/myfile and dir1/dir3/myfile (same hash for both). If you remove one, it does not recompile. Sep 04 09:54:49 if you have dir1/dir2/myfile and you add the same file to dir1/ for example. Does not retrigger the fetch Sep 04 09:55:28 my understanding of bitbake-dumpsig is that we store the name of the file and its hash and not the path to the file Sep 04 09:58:08 sorry, even worse, any file duplicated in SRC_URI, wherever it's from won't trigger a refetch. Again, my understanding of bitbake-dumpsig, happy to be told I'm wrong Sep 04 10:01:29 qschulz: I think you may be right. Could you file bug reports please so we don't lose the info and can investigate? Sep 04 10:03:51 qschulz: hmm... actually I may have been wrong... doesn't seem like my issue got solved after all... running bitbake package-2-name passes just fine, and no issues anywhere. however bitbake package-1-name seems to still carry some extremely old version of package-2, that makes building of package-1 fail. Sep 04 10:05:39 running cleanall in between doesn't seem to change anything Sep 04 10:06:29 feilz: how's the dependency on package-1 explicited in package-2? Sep 04 10:07:37 DEPENDS = "package-1" Sep 04 10:10:51 qschulz, SRC_URI = "file://dir" seems to work here.. Sep 04 10:11:42 kroon: yeah, I noticed the same, and it seems to work for me as well. Sep 04 10:14:22 qschulz, cleansstate for recipe, rebuild recipe, check with bitbake-dumpsigs Sep 04 10:15:11 qschulz, SRC_URI = "file://dir/*" does not work, but not sure if that is a bug Sep 04 10:22:48 qschulz, found https://patchwork.openembedded.org/patch/8443/ which has some information Sep 04 10:24:19 nrossi: http://git.yoctoproject.org/cgit.cgi/poky-contrib/patch/?id=5c514650478e5590bccdf4bba5d47385f092947f Sep 04 10:26:54 RP: makes sense, I did not realize that the sstate -> sysroot-components also used hardlinks as an optimization Sep 04 10:27:39 RP: even the sysroot-components -> builddir extraction might be problematic Sep 04 10:28:33 nrossi: we use a lot of hardlinks although most of our edit scripts do know to break them Sep 04 10:30:11 nrossi: queued http://git.yoctoproject.org/cgit.cgi/poky-contrib/patch/?id=e2de3b769020f58d443139d65d58f877b7fd8a31 as well Sep 04 10:31:22 nrossi: also, we may need to think about the exit code of the toolchain tests. "fail" isn't going to help the autobuilder Sep 04 10:31:48 nrossi: I think we mark ptest as exepected fail or something :/ Sep 04 10:31:50 RP: yer, also the logging of the failed tests is not partically helpful when 150000 of them fail ;) Sep 04 10:33:00 nrossi: I'll rerun with those two fixes and see how that looks Sep 04 10:37:56 nrossi: output looks much better locally now! Sep 04 10:39:45 New news from stackoverflow: How should the sstate-cache directory be deleted in Yocto? Sep 04 10:43:59 kroon: define "SRC_URI = "file://dir" seems to work here.." ? Sep 04 10:45:19 <__angelo> can i build another recipe form a function added by addtask ? Sep 04 10:45:22 RP: ok cool. For the pexpect/gdb stuff, the tests do pass with gdb on target. But it requires configuring gdb with python enabled which is not default in oe. So ignoring the tests would probably be fine. Any opinions? Sep 04 10:46:12 nrossi: enabling python for target gdb probably isn't that bad of an idea Sep 04 10:46:33 RP: by default or just in the glibc test config? Sep 04 10:46:41 RP: glibc selftest* Sep 04 10:47:13 nrossi: maybe by default? Sep 04 10:47:23 rburton: any opinion ? Sep 04 10:48:38 __angelo: first, what are you trying to do? It seems wrong at first sight. Sep 04 10:49:02 <__angelo> qschulz, i am trying to have an initramfs image together with a rootfs image Sep 04 10:49:15 <__angelo> *build together Sep 04 10:50:01 __angelo: I think the solution you are looking for is 'require otherRecipe' Sep 04 10:50:31 not inside an addtask, but just inside the image recipe Sep 04 10:51:53 qschulz: we talked about it right? it didn't work what I suggested? Sep 04 10:52:09 <__angelo> feilz, thanks, but at least the only "require" is not triggering the image to build Sep 04 10:52:48 <__angelo> qschulz, no didn't worked Sep 04 10:54:35 ahh, that's right Sep 04 10:54:45 require adds everything from that other file into the location where you have require Sep 04 10:54:48 but doesnt run it Sep 04 10:55:10 hence you have to make a call somewhere that activates the build sequence for the other image Sep 04 10:55:11 qschulz, bitbake recipe, touch new-file-under-dir, bitbake recipe again, bitbake-dumpsigs says fetch depends on newly touched file Sep 04 10:55:50 or solve it some other way Sep 04 10:58:18 nrossi RP: struggling to have a strong opinion. the tests should definitely skip if python isn't enabled. if we're doing special builds for the gcc tests then its trivial enough to turn py on in that setup. Sep 04 10:58:44 (skipping if py isn't present should be an upstreamable change) Sep 04 10:59:48 RP: are we dropping gcc8.3 from master? Sep 04 11:00:52 rburton: very tempted to, nobody's objected yet Sep 04 11:18:35 rburton: currently the tests fail if python or pexpect is not available, but mark unsupported if gdb is not present or if its missing python support in gdb. So its a query as to whether the addition of python-pexpect in oe-core and having to configure gdb with python is worth the test result. Sep 04 11:20:03 RP: also, should I make these various changes as additional patches or v3 of the previous series? Sep 04 11:30:26 After trying to solve my wic issue, i read on the internet that in order to use the ${IMAGE_ROOTFS} variable in the kickstarter file, it may need to end in .wks.in. If I rename my file to .wks.in, wic can no longer find it as an image type. Sep 04 11:30:38 i.e. with wic list images Sep 04 11:32:27 nrossi: i'd be tempted to say lets not do that *now* but after the rest is merged. Sep 04 11:35:35 rburton: as in don't bring in the gdb/python-pexpect? and let the (its only 6 tests) fail Sep 04 11:35:55 right Sep 04 11:38:52 RP: this fixes the issue with PRserv for me, hopefully doesn't break HashServe https://github.com/shr-distribution/bitbake/commit/57e41b66f5b28067f09973b028542f53c9d37712 Sep 04 11:58:26 JaMa: that doesn't surprise me but not sure its the right fix Sep 04 11:59:08 nrossi: maybe additional patches at this point? Sep 04 11:59:19 nrossi: I can always squash Sep 04 11:59:48 RP: ok, looks like the gcc/includes/* is not the only directory touched by stamping Sep 04 12:00:31 nrossi: I saw the cs-*.h, s-* and stmp-* files but only the include ones seemed to change Sep 04 12:01:51 RP: how can i tell what commit of oe-core the autobuilder is running against? Sep 04 12:02:52 nrossi: the build properties should show you, or look at the poky commit and it will show the oe-core hash Sep 04 12:03:06 RP: I'm not sure as well :/ Sep 04 12:04:00 hi folks, i want to use selinux with a yocto build Sep 04 12:04:12 it seems that yocto adds selinux labels on first boot Sep 04 12:04:41 what do i do in a case where the filesystem is meant to be read-only? Sep 04 12:09:39 kroon: ok, I'm building thud right now. so maybe this was fixed since then. ALSO, I've discovered this file://dir is actually a symlink to some other places (I hate it, but here it is atm). So maybe a corner case for symlinks... Sep 04 12:18:13 iceaway: did you change WKS_FILE to point to the .wks.in file? Sep 04 12:20:55 iceaway: hmm, I guess it should be listed under "wic list images" anyway though Sep 04 12:25:05 erbo: is that required? I thougt the .wks-file was found by name in the wic create -e Sep 04 12:26:45 I don't quite understand under which circumstances the WKS_FILE is required. Is it so that a wic image will be generated automatically with bitbake ? Sep 04 12:29:47 I think so, if I have IMAGE_FSTYPES including "wic" for an image, and I do "bitbake that-image", then WKS_FILE need to point out the file to use. Sep 04 12:33:10 OKay, understood. Where would one put the WK_FILE? IN the distro/image .conf file? Sep 04 12:43:01 RP: do these changes look fine? https://github.com/nathanrossi/openembedded-core/compare/4fed8b1ca1a14ae3c88c30d0b1be09e7e49e0ca9...nrossi/gnu-test Sep 04 12:44:54 RP: also, im not sure what is causing the current failures with gcc tests on the autobuilder. Unfortunately I have not been able to reproduce the issues with do_install modifying hardlinks. I suspect it is a case of more of those files being modified Sep 04 12:54:47 RP: also i think i have a reasonable solution for the autobuilder config, qemu sys/usr execution. Sep 04 12:56:56 RP: gcc sys is tagged as machine and release, glibc sys is tagged as machine and qemu-slow. When running qemu* it is "selftest -t machine", when running qemux86-64 it is "selftest -t machine qemu-slow". When doing "release" testing it is always "selftest -t machine qemu-slow release". This means glibc tests are run with kvm, but otherwise not run unless release testing. Sep 04 13:03:21 kroon: ok, could reproduce with a symlink but not with dirs. So you're right SRC_URI = "file://dir" and SRC_URI = "file://dir/" work for dirs but not symlinks (only the later then). Sep 04 13:03:57 qschulz, ahh.. was just about to test on thud Sep 04 13:04:01 iceaway: I guess WKS_FILE is usually set as part of machine configs Sep 04 13:05:12 kroon: now I'm interested by regex in SRC_URI, so going to test that :) Sep 04 13:05:17 erbo: Got it working. Apparantly using += on the IMAGE_FSTYPES variable did not work, I had to use _append. Sep 04 13:05:59 iceaway: bitbake -e and you can see who's setting IMAGE_FSTYPES and how Sep 04 13:06:08 I guess it was loosely set (?= or ??=) Sep 04 13:06:15 and when you use += you override those Sep 04 13:06:26 (IIRC) Sep 04 13:06:37 how to add dependent layers with bitbake-layers add-layer ? Sep 04 13:07:00 qschulz, symlink - that was sneaky.. should definitely be a bug reported, either bitbake should handle it as expected or at least warn about it IMO Sep 04 13:07:51 nrossi: yes, they look good Sep 04 13:08:43 nrossi: how do we make the autobuilder do that though :/ Its to complex for it to handle :( Sep 04 13:09:50 RP: you mean for the "selftest" template? Sep 04 13:10:16 New news from stackoverflow: Difference between devshell environment and bitbake task environment? || Yocto change psplash image without rebuild the system Sep 04 13:10:24 nrossi: well, its currently in the arch-qemu template Sep 04 13:10:45 nrossi: obviously we can hardcode things into the autobuilder but we've managed to avoid much of that so far Sep 04 13:10:58 nrossi: it also needs to work with the older release branches Sep 04 13:11:45 nrossi: I also notice that resulttool is lumping sys and usr test results together Sep 04 13:12:12 RP: yes that was something I was going to ask, prefix/suffix them? Sep 04 13:12:35 nrossi: probably suffix then they'll sort next to each other Sep 04 13:13:27 RP: with the selftest args, are you suggesting that selftest itself should handle what is default? Sep 04 13:14:45 nrossi: I don't know what I'm suggesting. I just don't see how this release/qemu-slow tagging is going to get us to where we need to be Sep 04 13:15:32 nrossi: right now the tests would be triggered every build which is too often so we need to fix that. There is then the question of where in a full build they'd hook in Sep 04 13:16:09 RP: ok, but changing the autobuilder templates is undesirable? Sep 04 13:16:56 qschulz: bitbake -e is a life-saver :) Sep 04 13:17:48 nrossi: no, we can change the templates Sep 04 13:20:25 RP: for the sys/usr suffix. Should it suffix both or just usr (aka, gcc-user) Sep 04 13:20:34 qschulz, if you wanna play around, I think its bitbake/lib/bb/checksum.py:get_checksums() that is the interesting one Sep 04 13:21:43 nrossi: just one, -user is probably fine Sep 04 13:22:25 qschulz, see Bitbake commit 66dff37ebcd1dd14ebd6933d727df9cf0a641866 Sep 04 13:26:54 I'm trying to update qt to version 5.12 (currently using 5.11) on my yocto project, but I'm having some trouble trying to find out how to do that. I don't see any tag or branch on the meta-qt5 repo. Anyone here perhaps can give me a hint on that? Sep 04 13:29:51 ricardocrudo: QT_MODULE_BRANCH in qt5-git? Sep 04 13:32:23 to be exact, QT_MODULE_BRANCH_PARAM but it is using QT_MODULE_BRANCH. Sep 04 13:32:52 bad news is that you'll have to have a bbappend for all qt recipes Sep 04 13:33:14 ricardocrudo: also, can't you just take the meta layer at the commit hash where they had qt 5.12? Sep 04 13:37:05 kroon: wow, big pandora box opened now Sep 04 13:38:23 SRC_URI = "file://bar/*.txt" in recipes-foo/foo/foo.bb and then you have recipes-foo/foo/foo/bar/bar.txt recipes-foo/foo/foo/baz/baz.txt Sep 04 13:38:57 in WORKDIR: tmp/work/aarch64-poky-linux/foo/bar AND baz Sep 04 13:39:34 qschulz, well.. not terrible Sep 04 13:39:46 uh? Sep 04 13:39:55 qschulz: some of that code is rather old :/ Sep 04 13:40:32 qschulz, it includes baz.txt in the checksum by mistake. so you get unncesseray rebuilds, right ? Sep 04 13:40:52 qschulz, still rebuilds if bar.txt is modified Sep 04 13:40:56 kroon: nope, not in the list of watched files Sep 04 13:41:34 so 1. it seems there is no support for regexp (at least `*`) in SRC_URI Sep 04 13:41:48 2. why is it fetching baz as well? Sep 04 13:41:58 qschulz, I thought you meant "bar AND baz" meant both was checksummed. So none of them are checksummed ? Sep 04 13:42:05 qschulz: it would be globbing, not regexp Sep 04 13:42:27 kroon: none of them are checksummed but both are fetched (visible in workdir) Sep 04 13:42:55 Thanks (about qt5), that should do it Sep 04 13:43:19 qschulz: do they show up in unpack? file:// urls are odd in that we don't actually need to fetch them Sep 04 13:46:45 * kroon heads home Sep 04 13:54:51 DEBUG: Searching for bar/*.txt in path: /media/qschulz/yocto/oe-dummy-project/recipes-foo/foo/foo/. Sep 04 13:54:54 NOTE: Unpacking /media/qschulz/yocto/oe-dummy-project/recipes-foo/foo/foo/. to /media/qschulz/build/tmp/work/aarch64-poky-linux/foo/1.0-r0/ Sep 04 13:54:59 (from log.do_unpack) Sep 04 13:55:13 RP: ^ Sep 04 13:56:36 Perhaps a simple question, but how can I remove all the old task log files ? Over time they accumulate and take a lot of disk space. Other than completely nuking my build directory tree, what other options are there ? I just don't need multiple (old) copies of every run.do_fetch.xxx, log.do_fetch.xxx, etc. wasting valuable disk space Sep 04 13:56:57 qschulz: right, I just wondered if the unpack sig had the file checksums rather than fetch Sep 04 13:59:29 RP: nothing in sig for do_unpack Sep 04 13:59:38 qschulz: fair enough, just wondered Sep 04 14:00:50 sure np Sep 04 14:05:43 oh no sorry, it's worse /o\ bar and baz dirs are in a bar directory in workdir, it looks like this: WORKDIR/bar/{bar,baz}/{bar.txt,baz.txt} instead of the expected WORKDIR/bar/bar.txt Sep 04 14:10:59 qschulz: I think that is a legacy behaviour :/ Sep 04 14:11:07 qschulz: i.e. things rely on it Sep 04 14:13:46 RP: it does not look like it's from glob though, I quickly tested witha ptyhon3 interpreter and glob was working as expected Sep 04 14:14:55 RP: I'll try to ban the use of regular expression/glob in SRC_URI in our project then Sep 04 14:17:44 RP: going to send out these patches. Just checking again, I can respin them into the series if you would prefer? Sep 04 14:17:58 nrossi: it helps me just to stack them Sep 04 14:18:46 qschulz: It would help a lot to improve bitbake-selftest for this Sep 04 14:19:04 RP: ok, I will just send them as a stand alone series then :) Sep 04 14:19:11 nrossi: thanks Sep 04 14:20:18 RP: oh btw, you should drop the python-pexpect patch when you apply these. As it is no longer required Sep 04 14:20:34 nrossi: right, makes sense, thanks Sep 04 14:21:09 nrossi: realised binutils-cross-testsuite needs the nopackages fix to stop warnings too Sep 04 14:21:37 RP: want me to add that to this series for you? Sep 04 14:22:53 nrossi: its ok thanks, I have it locally Sep 04 14:24:21 RP: np, series sent Sep 04 14:26:02 nrossi: added to -next, thanks! Sep 04 14:29:25 Does anyone have much experience with building recipes that use the Go language ? Sep 04 14:29:52 There doesn't seem to be a whole lot of documentation, just random web pages that I find using Google. Sep 04 14:31:05 ahrg, autobuilder is locked up on some kind of fetch nfs lock :( Sep 04 14:32:42 RP: anyways i managed to produce the "timestamp" issue locally, forcing install task after a clean run produced it, so I will see if i can look into the gcc failures that weren't fixed after your link breaking change. However its past midnight so I am off shortly Sep 04 14:33:06 nrossi: which gcc failures weren't fixed? Sep 04 14:34:47 RP: a number of gcc tests and gcc-runtime tests were failing on this run https://autobuilder.yoctoproject.org/typhoon/#/builders/53/builds/996 which should have your fix Sep 04 14:36:04 nrossi: ah, right. Maybe arch specific headers :/ Sep 04 14:36:18 nrossi: I'd not spotted that yet :/ Sep 04 14:56:59 zeddii, did you see the email about perf on older kernels? Sep 04 15:00:36 zeddii: With stap I got as far as seeing that runtime/linux/access_process_vm.h:#if defined(STAPCONF_LINUX_SCHED_HEADERS) is part of the problem Sep 04 15:00:52 zeddii: it seems STAPCONF_LINUX_SCHED_HEADERS should be defined but isn't. No idea why Sep 04 15:01:10 zeddii: I can change the error if I bypass that Sep 04 15:28:10 RP: Using the BASEHASH in place of the TASKHASH for hash equivalence seems like it would work... except that the BASEHASH doesn't seem to include files (e.g. SRC_URI?) Sep 04 15:33:09 JPEW: right, those are added in later. Not sure that matters though? Sep 04 15:33:44 JPEW: ah, I do see what you mean Sep 04 15:34:25 RP: Unless the actual value of SRC_URI is included in the basehash... in which case any change to it would cover the sources actually changing Sep 04 15:34:36 presumably Sep 04 15:35:01 JPEW: a key use case is change a patch with whitespace/comment, rebuild and get the hash equiv Sep 04 15:35:13 JPEW: if the local file checksums aren't in basehash, that doesn't work Sep 04 15:36:12 RP: Correct. You wouldn't want different source to be detected as equivalent (at least with the default hash detection) Sep 04 15:36:53 JPEW: I don't think detecting different source as equivalent is a problem, that is inevitable Sep 04 15:37:40 Right, as long as the change to source doesn't affect the output? Sep 04 15:37:48 JPEW: exactly Sep 04 15:41:56 JPEW: I think we're going to have to change the code Sep 04 15:42:31 RP: For the basehash or something else? Sep 04 15:43:11 JPEW: Looking at the code, add in a "nodephash" which hashequiv would work off Sep 04 15:43:17 <__angelo> hi sorry if i repeat the help request, but getting crazy. Manual, mailing list etc says that putting DEPENDS += "recipe_2" inside recipe_1 will trig recipe_1 to be processed. It doesn't work for me at all. Sep 04 15:44:07 __angelo: what does bitbake recipe_1 -e show? Sep 04 15:44:21 <__angelo> i am wondering if may be connected my recipe_1 is an initramfs recipe Sep 04 15:44:28 <__angelo> RP, thank, trying Sep 04 15:45:00 __angelo: yes, since images don't have do_configure steps which is where DEPENDS get added Sep 04 15:45:24 __angelo: try do_sometask[depends] += "recipe_2:do_someothertask" Sep 04 15:46:14 <__angelo> bitbake recipe_1 -e shows an infinite python code output Sep 04 15:46:48 __angelo: add | grep DEPENDS.*=, I doubt its infinite Sep 04 15:56:56 <__angelo> ok in bitbake recipe_1 -e i see there is image_2 in the DEPENDS list Sep 04 15:57:16 <__angelo> i try now the do_sometask stuff Sep 04 16:03:46 <__angelo> RP, super thanks. Finally i solved Sep 04 16:04:39 <__angelo> i used do_rootfs[depends] = "my-initramfs-image:do_image" Sep 04 16:05:29 <__angelo> i think i tried before with "my-initramfs-image:do_rootfs" that was not working since probaly initramfs does not call that task Sep 04 16:34:25 zeddii: some progress. When stap is testing its definitions its not finding headers like spaces.h on mips Sep 04 16:34:35 zeddii: It has -I./arch/mips/include/asm/mach-generic missing from its search path Sep 04 16:35:03 zeddii: the rabbit hole beckons... Sep 04 16:35:12 hmm. I was just poking at how it generates defines, etc. but not much progress on it. Sep 04 16:35:26 did we lose our cisco guy that actually knows stap ? :D Sep 04 16:35:38 I could solve the problem with git rm :D Sep 04 16:35:50 zeddii: no, we should ask him Sep 04 16:36:12 zeddii: any luck with kprobes? Sep 04 16:36:22 I've never had that dmesg show up. Sep 04 16:36:39 zeddii: I'm burning a ton of time on this which is going to cost hashequiv :/ Sep 04 16:37:10 buildrun.cxx is kind of nasty in systemtap. we should try and offload that, and I can look at krpobes to try and trigger it here. Sep 04 16:37:26 zeddii: "kind of"? Sep 04 16:37:29 IIRC that's an arch where systemtap runs, and that triggers it. Sep 04 16:37:36 (the kprobes one) Sep 04 16:37:44 zeddii: I ended up on the systemtap channel asking for help Sep 04 16:38:08 ah. Sep 04 16:38:40 zeddii: systemtap on 32bit arm and mips is not tested Sep 04 16:38:51 zeddii: but they welcome patches when we fix it Sep 04 16:39:12 for the 32 bit arches. I was going to suggest we just mark them incompatible. Sep 04 16:39:30 zeddii: don't tempt me Sep 04 16:39:43 when the one 32 bit systemtap user crawls out of a hole, we can tell them the same thing. patches accepted :D Sep 04 17:04:14 Hello guys! I am beginner at yocto project and I would like to know how can I to put my apps and webpage at the linux image? I'm working with legato mangoh. Sep 04 17:12:56 Hello guys! I am beginner at yocto project and I would like to know how can I to put my apps and webpage at the linux image? I'm working with legato mangoh. Sep 04 17:27:40 zeddii: _KBUILD_CFLAGS := $(call flags,KBUILD_CFLAGS) is what the stap Makefile does Sep 04 17:27:55 zeddii: KBUILD_CFLAGS has our flag, _KBUILD_CFLAGS is empty Sep 04 17:28:10 zeddii: is there an easy way to check if the flags filter was removed from the kernel or something? Sep 04 17:31:21 sec. let me check on that. Sep 04 17:31:24 zeddii: https://sourceware.org/git/gitweb.cgi?p=systemtap.git;a=commitdiff;h=e5976ba0af9b828dcc76b3937b5a98fe9c0f6cb8 Sep 04 17:42:48 RP: not currently seeing a difference between 5.0 and 5.2 in that area. looking more. Sep 04 17:43:39 zeddii: I can't even tell what its meant to do. Commenting it out and using raw KBUILD_CFLAGS does seem better Sep 04 17:45:53 I thought it was just me that got lost in that routine. I read all the way to the bottom to see *how* that generated Makefile was used (I assume it is a generated makfile, given the “call” syntax). Sep 04 17:47:52 zeddii: Yes, there is a horrendous make command you end up with. I should write down my "stap debugging tips" and send you a copy Sep 04 17:51:28 I still haven’t found where the ‘call’ make macro is defined. kernel or stap. Sep 04 17:51:44 * zeddii tries a new strategy Sep 04 18:34:22 zeddii: kernel commit cdd750bfb1f76fe9be8cfb53cbe77b2e811081ab Sep 04 18:38:40 zeddii: https://sourceware.org/git/gitweb.cgi?p=systemtap.git;a=commitdiff;h=7cfac6c0f8ba16c9a64c6799dd76ef61ef529f09 fix in systemtap to test Sep 04 18:41:49 well crap. I was limiting my git history foo to a different set of Makefiles. but I wonder why my #@$@#$ git grep "flags =" in the 5.0 kernel missed that. Sep 04 18:42:40 I'm switching to qemuarm64 and the kprobes message then. IIRC it is showing on there. Sep 04 18:42:47 * zeddii finds that email Sep 04 18:43:46 yup. will use arm64 Sep 04 18:52:50 zeddii: My day is feeling like that :/ Sep 04 18:55:03 qschulz: I found my problem, btw. It was because of something I had in local.conf.. DEBUG_PREFIX_MAP="" Sep 04 18:55:36 I added it because setting that to "" avoids adding some compiler flags which gcc7 does not have. Sep 04 20:12:20 zeddii: confirmed that systemtap master fixes qemumips stap Sep 04 20:15:11 nice Sep 04 20:17:39 RP: so nodephash would be an intermediate hash between basehash and taskhash that includes the files? Sep 04 20:18:06 JPEW: yes Sep 04 21:02:11 khem: the update to 9.2 dropped the GLIBC_DYNAMIC_LINKER changes for risc-v again :-) Sep 04 21:02:16 third time now :-) Sep 04 21:02:53 man am i looking forward to being able to move to master once hashequiv is polished off. waiting for a complete rebuild from scratch for no damn reason Sep 04 21:02:56 * kergoth taps foot Sep 04 21:12:09 kergoth: Want to beta test :) Sep 04 21:12:31 kergoth: It's not ready yet though Sep 04 21:12:55 yeah, i know. been poking at it occasionally. even as is it's nice, just not ready to switch production over Sep 04 21:30:58 zeddii: still one qemuarm stap issue left: https://autobuilder.yoctoproject.org/typhoon/#/builders/38/builds/998/steps/8/logs/step1c - much simpler though Sep 04 21:38:55 zeddii: FWIW a straight boot of the qemuarm64 kernel here shows the kprobes message in dmesg Sep 04 21:39:09 zeddii: don't need any of the tests to trigger it Sep 04 21:40:14 zeddii: https://lkml.org/lkml/2019/8/24/107 Sep 04 21:40:20 zeddii: a fix? ^^^ Sep 04 21:44:21 root@qemuarm64:/tmp# dmesg |grep -i kpr Sep 04 21:44:21 [ 2.892795] kprobes: failed to populate blacklist: -22 Sep 04 21:44:21 [ 2.893110] Please take care of using kprobes. Sep 04 21:44:25 yah, was just looking at that. Sep 04 21:45:47 I can easily test that commit to see if it does address it. Sep 04 21:48:32 RP: that fix is queued for 5.2 -stable, which means it is likely the one and I've grabbed it from there. Sep 04 21:48:57 zeddii: cool. It reads like the right thing Sep 04 21:49:11 rebuilding and booting with it applied now. Sep 04 21:49:13 zeddii: so that leaves us stap broken on qemuarm Sep 04 21:49:27 and mips having highmen issues Sep 04 21:49:55 zeddii: 512 pushes mips into highmem range which is likely where the issues come from Sep 04 21:50:18 ah yes. Sep 04 21:50:41 I'll start a 32 bit arm build once I get the kprobes fix built. and then mips on my 2nd builder. Sep 04 21:51:13 zeddii: cool. I think I need to take a break (32 bit arm building as there is a toolchain testsuite issues there too) Sep 04 21:51:39 I'll get myself setup for those last two. Sep 04 22:49:08 RP: kprobes message is gone here. testing systemtap and will send a SRCREV update later. Sep 04 22:49:57 zeddii: sounds good, thanks! Sep 04 23:23:12 best advice on adding something back after it was in _remove? Sep 05 01:02:15 rsalveti:my bad Sep 05 01:02:36 I guess I have to update my patchset **** ENDING LOGGING AT Thu Sep 05 03:00:58 2019