**** BEGIN LOGGING AT Wed Apr 06 02:59:57 2022 Apr 06 07:47:29 good morning Apr 06 14:44:49 bitbake's dependency tracking seems to be a bit wonky? Apr 06 14:45:35 I have a package A which provides a static library, and I have a package B which uses the static library in A; when I change A's do_compile() to make it build the library with different flags, I expected B to get recompiled, but it doesn't seem to Apr 06 14:53:31 does B depend on A? Apr 06 14:53:40 yes Apr 06 14:53:44 did you bitbake B? Apr 06 14:53:48 yes Apr 06 14:54:10 if you can reproduce that on demand then that's interesting, and see if you can minimise the test case Apr 06 14:54:31 its possible that it can't see the changes you're making to the compile for some reason (some variable usage doesn't get watched automatically) Apr 06 14:55:02 hmm that would be weird though, since it's a change in the commands run in do_compile() for A Apr 06 14:55:13 and A gets recompiled when I bitbake B Apr 06 15:00:00 does bitbake try to be clever and notice that B links against A and thus needs to recompile? If so, maybe bitbake doesn't handle static libraries that well? Apr 06 15:00:21 it would never see that relationship by analyzing any generated elf file after all Apr 06 15:02:32 so it's not an explicit dep? Apr 06 15:03:15 I don't know what you mean Apr 06 15:04:12 A is listed in B's DEPENDS variable if that's what you're asking Apr 06 15:04:58 yeah ... weird then Apr 06 15:07:24 I can reproduce it easily it turns out with dummy recipes Apr 06 15:08:40 https://p.mort.coffee/zWq Apr 06 15:09:04 if I run `bitbake test-b`, then first test-a:do_compile runs for 10 seconds, then test-b:do_compile runs for 10 seconds Apr 06 15:09:23 if I then change test-a to sleep for 5 seconds instead, then run `bitbake test-b`, then test-a:do_compile runs for 5 seconds, then nothing more happens Apr 06 15:09:36 so dependency tracking is definitely broken Apr 06 15:10:05 meaning that it knows it has to rebuild A though Apr 06 15:10:24 yeah, but doesn't know that means it has to rebuild B Apr 06 15:10:37 which is odd Apr 06 15:11:24 my guess is that it analyzes generated elf files to find dependencies and if analyzing B's generated elfs doesn't show that it depends on A, bitbake assumes B doesn't have to be regenerated Apr 06 15:11:29 but that's the wrong model Apr 06 15:24:06 oh, a recent release? Apr 06 15:24:17 its looking at the binary output and seeing they're identical Apr 06 15:24:28 change test-a to write a number to the sysroot instead Apr 06 15:25:01 output hasn't changed -> no need to rebuild stuff Apr 06 15:25:31 really nice when you touch something like glibc but the change doesn't impact the target build, and it doesn't feel the need to rebuild the world Apr 06 15:27:48 mort: bitbake only uses its metadata (i.e., what's in the recipes) to determine what needs to be rebuild. However, if you have the hash equivalence support enabled, it will compare the output of a given task with the output from previous runs of the same task, and if they are identical, then it will consider the tasks equivalent. I.e., if you run task A and it produces some output, and then change task A and rerun it and it still produces the same Apr 06 15:27:48 output, then bitbake will use the sstate for subsequent tasks. Apr 06 15:33:17 ok, test-a now writes 1 to lol in do_compile and installs lol to ${D}/lol in do_install, and has FILES_${PN} = "/lol". I run `bitbake -c cleansstate test-a test-b` to make sure everything's fresh. I run `bitbake test-b`, both test-a and test-b's do_compile runs ofc Apr 06 15:33:37 I change test-a's do_compile to write 2 instead, then `bitbake test-b`, test-a:do_compile and test-a:do_install runs, but test-b:do_compile doesn't run. Apr 06 15:33:54 even though now test-a has changed the content of the file it installs Apr 06 15:35:12 https://p.mort.coffee/sfy Apr 06 15:35:50 changing that number in the `echo 2 > lol` and running `bitbake test-b` causes test-a:do_compile and test-a:do_install to re-run, but doesn't do anything with test-b Apr 06 16:12:31 mort: that file isn't installed to the sysroot, so the other recipe won't be affected by it Apr 06 16:12:41 mort: not everything installed to ${D} is available to other recipes that depend on you Apr 06 16:12:47 only a specific list of paths, or those you add to that list Apr 06 16:12:58 i.e. includedir Apr 06 16:16:26 kergoth: but the problem isn't that stuff isn't available to other recipes Apr 06 16:16:35 B links against the library installed by A, and that works perfectly fine Apr 06 16:16:57 I suppose it affects the test case I made Apr 06 16:17:04 it shouldn't affect the real example though Apr 06 16:17:08 yes, it does. you'r emodifying an unused file, which won't affect checksums :) Apr 06 16:17:13 true Apr 06 16:17:24 i'd suggest a different test case. install it to ${D}${includedir} at least Apr 06 16:17:38 could just treat yocto as buildroot and do a clean rebuild every time I do anything Apr 06 16:17:50 have a 2 hour development cycle Apr 06 16:18:23 there's quite a few of these painful rebuild gotchas in yocto Apr 06 16:18:34 I'd much prefer if it was too eager than the current situation where it's too lax Apr 06 16:19:16 it shouldn't be that hard, if any of recipe R's dependencies have changed, rebuild recipe R Apr 06 16:19:50 anyways, I'll try to make a more realistic small test case tomorrow, my yocto setup is currently busy doing a clean rebuild Apr 06 16:20:14 it is generally too eager, i don't htink i've ever heard anyone report it being too lax on rebuilds Apr 06 16:29:10 By the way, what's FILES_${PN} supposed to be used for, runtime files or both runtime and built-time files? (And which manual is this mentioned in? I didn't find anything in the bitbake user manual) Apr 06 16:34:55 That controls runtime packaging. What files go into the package named ${PN} Apr 06 16:35:14 assuming ${PN} is in the PACKAGES variable, which lists the packages to be created, of course Apr 06 16:35:16 which is the default Apr 06 16:41:27 Right, makes sense Apr 06 16:42:23 The FILES variable is an oe-ism, not bitbake Apr 06 16:42:24 Well, build-time stuff would be part of the -dev package, so I suppose headers and static libs should be included in FILES Apr 06 16:42:37 there are packaging variables which affect bitbake, but only the dependency ones, as those affect what tasks ar erun Apr 06 16:42:41 the rest is oe/yocto, not bitbake Apr 06 16:42:50 they're in FILES_${PN}-dev, yeah Apr 06 16:43:00 aha Apr 06 16:43:24 https://docs.yoctoproject.org/bitbake/bitbake-user-manual/bitbake-user-manual-metadata.html#runtime-dependencies Apr 06 16:43:25 it can be hard to know whether to look to bitbake, openembedded or poky Apr 06 16:44:35 https://docs.yoctoproject.org/overview-manual/concepts.html#package-splitting Apr 06 16:44:35 https://docs.yoctoproject.org/ref-manual/variables.html#term-FILES Apr 06 16:44:50 poky is just a packaging of bitbake and oe-core along with meta-yocto and the poky reference distribution Apr 06 16:44:56 it's still bitbake and oe Apr 06 16:44:58 fwiw, the debian packaging guidelines are well written and basically cover the same packaging basics Apr 06 16:45:03 good point Apr 06 16:45:14 iirc we referenced that when we got started Apr 06 16:45:20 https://docs.yoctoproject.org/ref-manual/classes.html#package-bbclass Apr 06 16:45:23 yeah, there's a lot of debianisms in the guts Apr 06 17:13:11 kergoth: https://p.mort.coffee/neO -- this should be correct, right? 'lol' is installed to /usr/bin, which should be part of list of paths installed to B's sysroot Apr 06 17:13:34 yet it has the same behavior, changing the number in test-a:do_compile causes test-a to be recompiled, but does nothing with test-b Apr 06 17:15:33 well no, /usr/bin wouldn't make sense, /usr/lib would Apr 06 17:16:26 and indeed, when using /usr/lib it does work as expected Apr 06 17:18:12 yeah, usr/bin won't be populated unless it's a native recipe Apr 06 17:18:16 as then we'd be able to run such binaries Apr 06 17:18:20 fgor a target one, wouldnt' make sense Apr 06 17:18:47 yeah Apr 06 17:22:12 hypothesis: when a dependency changes, it runs do_compile but without deleting and re-unpacking, so make gets run but my makefile isn't clever enough to know that the library changes Apr 06 17:37:13 yeah it won't re-unpack if it just needs to recompile Apr 06 17:37:32 this is why out-of-tree builds are Good, so configure can wipe the build tree away Apr 06 17:37:38 if your makefile has a clean target, it will delete things Apr 06 17:37:49 erm, if your makefile has a clean target, it will run it Apr 06 17:38:00 but use a proper build system instead Apr 06 22:47:47 I would love to use a proper build system but all of them suck **** ENDING LOGGING AT Thu Apr 07 02:59:57 2022