**** BEGIN LOGGING AT Tue Sep 10 02:59:58 2019 Sep 10 11:31:40 Hi folks - is this the correct channel to ask for bitbake recipe help? Sep 10 11:32:34 I'm having problems with a bitbake recipe where dynamically generated content is not being included in my rootfs, but static files in the recipe (that I tell it to) are happily included. Sep 10 12:00:31 Crofton|work: in lapack commit http://git.openembedded.org/meta-openembedded/commit/?id=df411368c0d342ad34094ab8ff022b52caf8fdda you mentioned scipy, is it available somewhere public? I haven't found recipe for it on layer index Sep 10 12:01:47 people have tried Sep 10 12:01:49 damn it Sep 10 12:02:16 I did scipy native with that Sep 10 12:02:40 some todo list I ahve has make a layer to collect the attempts and put lapack there to centralize efforts Sep 10 12:14:22 ok, will create my own then :) Sep 10 12:14:56 you will be a hero if you make scipy work Sep 10 12:37:49 khem: I'm seeing the systemd post-install scriptlets to change /etc/nsswitch.conf fail when building systemd with musl. I'm thinking I must be missing something as that part of the systemd recipe hasn't changed in a long time. Any ideas? Sep 10 12:39:23 heh, suspect I'm a few hours too early for khem Sep 10 12:47:45 Foxdie: you'll need an install rule that handles the dynamically generated content. A relevant example isn't coming to mind but recursive copy/install/tar should work fine. Sep 10 12:48:35 Foxdie: together with proper FILES_$ setting, of course. Sep 10 12:53:06 Thanks for getting back to me chaps Sep 10 12:53:31 I am just testing a build now, I did have this before; Sep 10 12:54:08 FILES_${PN} += "${libdir}/node_modules"FILES_${PN}-dev = "${libdir}/node_modules" Sep 10 12:54:20 It seemed to want a -qa one too Sep 10 12:54:48 (Sorry, that didn't paste very well :) ) Sep 10 13:02:43 Nope, sadly didn't work adding this; FILES_${PN}-qa += "${libdir}/node_modules" Sep 10 13:03:24 I'm at a loss, I keep seeing "DEBUG: Removing manifest" with all the files I've just built in the bitbake logs and no idea how to resolve it. Sep 10 13:03:53 Any advice would be welcomed Sep 10 13:15:10 Foxdie: take a look at the valgrind recipe for the ptest installs? Sep 10 13:39:30 Okay - this one? http://cgit.openembedded.org/openembedded-core/tree/meta/recipes-devtools/valgrind/valgrind_3.15.0.bb Sep 10 13:42:44 Foxdie: yes, it's not installing a list of dynamically created files but the do_install_ptest function does scoop up lots of executables and install them so it might be helpful to you. Sep 10 13:44:01 Foxdie: Actually, I'm sure what you mean by 'dynamically generated content'. Is it similar to the valgrind ptest executables? Sep 10 13:44:27 I should probably better explain what I'm trying to do Sep 10 13:45:24 I'm trying to build nodejs npm packages and install them on my rootfs. No matter which way I try it (using meta-nodejs or manually compiling them), the stuff I built doesn't end up on the rootfs. Sep 10 13:45:54 I decided to try tarballing up the node_modules directory and then extracting it on boot as a hack, but even the tarball I've created doesn't get copied to the rootfs. Sep 10 13:46:28 Any of the static files in that recipe get put on the rootfs fine. It's just ones that are built as part of do_compile() Sep 10 13:47:41 This is the recipe I've written so far; https://pastebin.com/8DnP9MLp Sep 10 13:48:40 (I even tried creating a dummy npm-packages.tar.gz, adding it to SRC_URI with ;unpack=0, updating that with do_compile() and then trying to copy it to the rootfs, it still doesn't work) Sep 10 13:49:06 Admittedly I'm at my wits end, I've been trying to solve this for 3 days now Sep 10 13:49:33 Foxdie: why dont you use 'inherit npm'? Sep 10 13:50:00 Hi rburton, I tried that route, it was met with the same problem, files just wouldn't end up on the rootfs Sep 10 13:50:07 did you verify they're in the packages? Sep 10 13:50:23 bisect the problem: not in rootfs could be "you forgot to install the package" Sep 10 13:50:28 so, are the packages good Sep 10 13:50:31 or is that crazy do_install broken Sep 10 13:50:40 Foxdie: one thing, it's likely not correct to have FILES_${PN} and FILES_${PN}-dev the same Sep 10 13:50:52 Foxdie: it would not surprise me if all your files are in the -dev package Sep 10 13:50:57 oh god well yes that's the problem Sep 10 13:51:04 rburton, how would I verify that? Sep 10 13:51:21 smurray, they were different at one point, the -dev would also include ${includedir} though Sep 10 13:51:24 oe-pkgdata-util list-pkg-files -p [recipename] Sep 10 13:51:47 rburton, to be executed as part of the run or after the build has finished? Sep 10 13:52:07 Foxdie: packaging happens by iterating through PACKAGES and taking files from $D in order. PN-dev is before PN, so PN-dev takes everything in /usr/lib/node_modules Sep 10 13:52:18 Foxdie: its a command you run in the shell after the build Sep 10 13:52:32 by the time PN is packaged, there is nothing left to put in it Sep 10 13:52:38 Ah, it appears to be in -dev Sep 10 13:52:39 -> all your files are in PN-dev Sep 10 13:53:22 now you've solved that, go back to using npm.bbclass. using that class you shouldn't need to set FILES at all. Sep 10 13:53:32 (Not sure how I paste multi-line responses in webchat but I can see it under npm-packages-dev, where npm-packages is my package naturally) Sep 10 13:55:03 Okay I'll have a crack at that again Sep 10 13:57:39 Thanks all, will see how I get on, build shouldn't take long Sep 10 14:19:53 smurray:musl does not implement NSS hence no nsswitch.conf Sep 10 14:20:55 khem: right. However, it used to be installed and the systemd packaging managed to work, e9f751a recently changed base-files to only install it for glibc... Sep 10 14:21:22 khem: so the systemd post-install scriptlet bits should be glibc only, I'd say? Sep 10 14:21:43 heh my last world build is running for 52 hours already and the bitbake log is 502MB already Sep 10 14:21:49 smurray: seems to be the case, submit a patch Sep 10 14:22:12 JaMa: are you building mars world as well Sep 10 14:22:24 khem: okay. I hit it during my testing on upgrading to 243, had me wondering how I'd broken that ;) Sep 10 14:22:43 JaMa: Jeez I thought my 8 hour build was bad :D Sep 10 14:23:10 I think my build swallowed black hole and now it's still trying to digest that Sep 10 14:23:18 smurray: right, sometimes its not your fault :) Sep 10 14:23:42 khem: some fun in 243, someone decided to redo the cpu set stuff in systemd and now use __cpu_mask even though it's supposed to be an internal implementation thing... Sep 10 14:23:59 JaMa: or maybe all big packages are failing and dumping do_compile Sep 10 14:24:27 and it's not just stuck, all 80 cores are busy doing something Sep 10 14:25:10 yes, I'm checking what's wrong, but I'm not trying to load whole 502MB log in jenkins console log :) Sep 10 14:26:28 better to use 256G ram on server Load average: 31.42 26.80 25.07 Sep 10 14:28:18 there were a few failures on different MACHINE, but not so many NOTE: Tasks Summary: Attempted 46845 tasks of which 4846 didn't need to be rerun and 62 failed. Sep 10 14:37:34 is anyone already using some beast with Epyc Rome for builds? I might need a few :) Sep 10 14:39:59 Huzzah! Sep 10 14:40:54 Foxdie: :) Sep 10 14:41:18 3 days of headbanging against the keyboard from one simple fact buried deep in the documentation; "When you add to FILES_PN what you want to happen isn't happening because PN-dev has already taken the files" Sep 10 14:41:46 I simply set this before my main FILES_PN; FILES_${PN}-dev = "" Sep 10 14:42:01 Now I have _all the files_ muhahaha :D Sep 10 14:42:11 Foxdie: was there a warning? rburton: should that result in an error ? Sep 10 14:42:56 There was the usual installed-vs-shipped error but that disappeared when I added this; FILES_${PN} += "${libdir}/node_modules" Sep 10 14:43:16 But I foolishly assumed that it was funneling them to the correct place and not -dev Sep 10 14:43:28 It's always the little things you forget to check, d'oh! Sep 10 14:44:02 Yep, now do you have any expertise in valgrind internals on arm64 ? :) Sep 10 14:44:15 Do have a nice bitbake recipe now that'll simply take `package.json` and do a cross-compiled npm install into the rootfs :) Sep 10 14:44:39 I wish I did, would love to be able to repay the kindness Sep 10 14:52:58 For if it helps people reading the IRC logs in future; https://pastebin.com/Yygc7wfZ (no expiry) Sep 10 14:53:06 (May not be the best way to achieve it, YMMV) Sep 10 14:59:29 what is FILES_${PN}-qa += "${libdir}/node_modules"? Sep 10 14:59:41 also why assume provided, and why are you turning off qa? Sep 10 15:00:51 also why not use npm.bbclass to do the install for you Sep 10 15:01:09 and what is npm install doing if it isn't installing, which it isn't because you're still doing a cp Sep 10 15:01:19 cp preserves ownership, use install Sep 10 15:01:42 (really, just use npm.bbclass) Sep 10 15:33:14 rburton, I'm not sure about the -qa one, I could try without that present, I think it's just habit to include that Sep 10 15:33:32 Foxdie: it doesn't do anything Sep 10 15:33:40 so that's an odd habit Sep 10 15:33:54 I tried using `inherit npm` but it hangs in the do_install() phase, literally doing nothing, no CPU nor MEM being used. Sep 10 15:51:09 you should probably try to figure out why it's doing that rather than reinventing the wheel. use bitbake -e to see exactly what command it's running in do_install Sep 10 15:55:37 Will take a peek tomorrow, end of the day here, have a good evening all Sep 10 18:23:59 Can I change a PACKAGECONFIG in a bbappened? Sep 10 18:26:31 Crofton|work: if you mean PACKAGECONFIG[foo] then you can just define it again Sep 10 18:26:41 if you mean normal PACKAGECONFIG value than of course Sep 10 18:31:00 yeah, why isn't this working! Sep 10 18:32:19 interesting, why is this work ing ..... Sep 10 18:32:48 oh jesus Sep 10 18:32:55 oh shit Sep 10 18:33:05 I am an idiot Sep 10 18:47:53 Crofton|work: and a verbose one! ;-) Sep 10 18:47:58 * vmeson runs!! Sep 10 18:48:22 lolz Sep 10 18:48:27 -DVERBOSE Sep 10 18:48:40 just just jaw droppingly bad mistkae **** ENDING LOGGING AT Wed Sep 11 03:00:21 2019