**** BEGIN LOGGING AT Mon Aug 12 02:59:58 2019 Aug 12 10:14:53 New news from stackoverflow: Should I specify the OELAYOUT_ABI variable in new yocto distro? Aug 12 10:19:18 Hi o/ Aug 12 10:21:02 I would like to know if I can get information from build log. What task where omitted and taken from sstate-cache or they were just omitted? Aug 12 10:22:26 In my case I have only logs without any debug information and I see that some task are not excuted and the number of following task which are executed increase byt the number of omitted tasks Aug 12 10:23:45 I can't reproduce this behavior even using the same sstate-cache Aug 12 10:33:48 I have a quick question about opkg keyrings. ALL the keys i add to my keyring for use with opkg are marked as unknown. how do i change them to ultimate during the build process? Aug 12 10:37:46 Hello all. I'm having a hell of a run customizing an arm recipe just adding dahdi. It's called mlinux. Anyone familiar? Aug 12 11:13:55 sry for asking a C question (albeit a wicked one): I defined "FUN(TYPE)" as a generic to implement function "fun_##TYPE". gcc now seems to expand FUN(bool) to fun_bool *OR* fun__Bool randomly, which is at least annoying. Can I stop macro expansion in my macro? Aug 12 11:15:01 New news from stackoverflow: Is there a way to install keys into the opkg-keyring during the yocto build process and have them marks as ultimate rather than unknown by gpg Aug 12 11:20:42 stephen: if you ask a little more precisely? Aug 12 11:22:03 LetoThe2nd, certainly. I'm fairly new to yocto, so forgive if I skip important things, and, by all means, please point them out. Aug 12 11:22:41 stephen: start out with what you are trying to do, and where you get stuck :) Aug 12 11:23:10 So, I'm trying to build a cross compile environment to build a specific kernel module. Aug 12 11:24:21 stephen: this sounds like you are trying to manually crosscompile and such. Aug 12 11:24:51 The source for the driver is here: http://amfeltec.com/drivers/USB-FXO/ Aug 12 11:25:34 The device I am trying to compile it for is an ARM device from Multitech Aug 12 11:26:22 Their linux is http://www.multitech.net/developer/software/mlinux/ Aug 12 11:26:30 stephen: your starting point should be https://www.yoctoproject.org/docs/latest/kernel-dev/kernel-dev.html#incorporating-out-of-tree-modules then Aug 12 11:26:38 mlinux is built using yocto. Aug 12 11:27:22 Yeah, the driver requires dahdi, which I realized openembedded supplies a recipe for, and is part of layer-telephony Aug 12 11:27:46 sounds about right. Aug 12 11:29:55 jmiehe: i'd guess this depends on included header before, and/or if its a c or c++ source file Aug 12 11:30:24 jmiehe: trigger word might be stdint.h/stdbool.h, but its pure guessowrk Aug 12 11:30:28 Now I've gone down multiple paths to trying to accomplish this, at it's farthest I was on Ubuntu 16, and have tried using mlinux-5.0.0 and 4.1.9. I installed the mlinux toolchain, added a recipe to extend mlinux-base-image with dahdi-linux. At this point I have issues when I bitbake, as if mlinux is using an old version of python or something? Aug 12 11:31:10 stephen: first and foremore, the releases of all layers in use have to match. Aug 12 11:31:23 At one point I had the linux headers downloaded and I was having to go alter them (or create symlinks) to get the arm includes to be found Aug 12 11:31:51 stephen: i cannot tell on which poky/OE releases this mlinux is based, but common names would be pyro, rocko, sumo, thud, warrior Aug 12 11:32:08 Yeah, adding the layer seemed a bit to heavy, so I ended up trying to just include the recipe Aug 12 11:32:11 stephen: so if you add a layer, make sure its checked out branch matches the base system in use. Aug 12 11:32:21 iy started with a K, one sec Aug 12 11:32:28 stephen: nope, thats outright wrong. layers are meant to be included as-is. Aug 12 11:33:33 So if I want dahdi, I need to include the entire layer that it's included in? Aug 12 11:33:40 yes. Aug 12 11:34:57 Quick aside: I do dual boot, but I have quicker access to WSL Ubuntu 16 right now. Any experience on success factors there? Aug 12 11:35:18 stephen: in a nutshell: don't do it. Aug 12 11:35:24 stephen: WSL is discouraged/unsupported/broken for yocto Aug 12 11:35:32 stephen: as I came to learn, the entire layer should be included and will be parsed, but only your dependencies will actually be built. Aug 12 11:35:57 there are chances that WSL2 might be working some day, but as of today you need a proper linux box. Aug 12 11:38:27 LetoThe2nd: FUN(bool) will expand to FUN(_Bool) -- according to std (c99). Fine for var decls, but broken if I expect consistent function names. Why is bool even a macro and not a typedef?! Well … thinking about using _Bool directly now, even if my function is named fun__Bool then … Aug 12 11:39:07 jmiehe: don't ask me, i was just musing things that came to my mind. Aug 12 11:39:49 … and stdbool is exactly the curse and the blessing here^^ Aug 12 11:39:51 ok, second machine booted and VNC installed Aug 12 11:40:48 jmiehe: just use c++, you get consistent bool types AND even stricter typechecking ;-) Aug 12 11:41:27 stephen: and i just checked, meta-telephony does not seem to be in very good state Aug 12 11:41:34 * jmiehe would much rather directly migrate to Rust Aug 12 11:41:38 stephen: so you are certanly in for A LOT of fun. Aug 12 11:41:45 :-/ Grrr Aug 12 11:42:11 So my main goal here is to build the driver for ARM. I don't need the env afterwards Aug 12 11:42:23 Any thoughts on alternate paths? Aug 12 11:42:28 stephen: now that again is super shortsighted Aug 12 11:42:46 stephen: what happens if they ship driver source with a bugfix next year? Aug 12 11:43:32 LetoThe2nd, publish or perish. I need the driver to upload to hardware we already have working, lacking this one usb device Aug 12 11:43:54 I can fight the driver over the course of that year, and will happily contribute back Aug 12 11:44:02 yes, sure. Aug 12 11:44:34 in that case, try https://www.yoctoproject.org/docs/latest/kernel-dev/kernel-dev.html#building-out-of-tree-modules-on-the-target to just build in target Aug 12 11:44:40 And that's not BS, this is hardware I'm going to need to support for a 5-10 year horizon. Aug 12 11:44:51 yes, sure. Aug 12 11:44:54 lol Aug 12 11:45:14 * stephen sulks and carves in blood "contribute" on his arm Aug 12 11:45:53 Tell me, how did you determine meta-telephony is in such a bad state? Aug 12 11:45:54 don't take it personal, but the combination of "support for 10 years" and "just need it to work once NOW" just does not exist. Aug 12 11:46:12 stephen: i looked at its branch list, and none matches a proper release. Aug 12 11:46:18 I've hacked a MWE for my problem if y'all interested https://repl.it/repls/UnkemptPrestigiousLevel Aug 12 11:46:29 LetoThe2nd, they are supporting an older release in a branch Aug 12 11:46:34 the m-t guys Aug 12 11:47:00 I might look at it if I get bored, would be nice to ahve it work for pi3 etc Aug 12 11:47:05 Actually, I'm going to have to create a build system that will pull this stuff from open source repos as a matter of security review. The build it now driver is for demo purpose Aug 12 11:47:21 Crofton|work: none to be found here: http://git.osmocom.org/meta-telephony/refs/ Aug 12 11:47:32 Crofton|work: so thats what i based my assumption on Aug 12 11:47:52 http://git.osmocom.org/meta-telephony/log/?h=201705 Aug 12 11:47:52 stephen: like i said, follow the in-target building guide Aug 12 11:47:59 I'm not arguing with you Aug 12 11:48:31 stephen: should suffice for "getting it to work once." - and if not, go blame whomever gave that "mlinux" package to you. Aug 12 11:48:34 Now, our final use case doesn't use asterisk, it uses FreeSwitch. As such, should I be heading this up under a different layer, or contributing to meta-telephony? Aug 12 11:48:50 Crofton|work: didn't take it as asguing, just pointing out where the information came from :) Aug 12 11:49:23 Multitech. Part of me feels like they've left it in this state on purpose, but who knows Aug 12 11:49:44 stephen, we always suspect vendors of being evil :) Aug 12 11:49:55 * LetoThe2nd cheers.! Aug 12 11:50:03 * LetoThe2nd works for a HW vendor :) Aug 12 11:51:00 :) Aug 12 11:51:10 some are better than others Aug 12 11:51:42 those are the black sheep Aug 12 11:52:03 stephen, the meta-telephony guys are in #osmocom Aug 12 11:52:55 I really should try merging the branch they use into master and building for a pi3 and see what happens Aug 12 11:54:27 i would love so much to further procrastinate on getting my nodejs builds in shape but i guess i'll finally have to start. meh. Aug 12 11:54:37 lol Aug 12 11:54:39 yeah Aug 12 11:54:52 I need to pack some borads for camp Aug 12 11:54:57 Well now, if you ever need help in the nodejs arena, that's my standard bag Aug 12 11:55:39 stephen: give me a recipe for node8 with intl on armv7 :) Aug 12 11:56:16 (its really more of a build system problem, not exactly a node one) Aug 12 11:56:21 If you're not in a hurry, I'll have it ready before the end of the week Aug 12 11:56:34 you struggling with arm as well? Aug 12 11:56:42 and the leg Aug 12 11:56:45 no more than usual Aug 12 11:56:48 \o Aug 12 11:57:37 this is the arch the toolchain I'm installing claims to be: arm926ejste Aug 12 11:57:53 stephen: armv5, then. Aug 12 11:58:12 Which qemu machine supports that best? Aug 12 11:58:30 pretty much classic qemuarm that is Aug 12 11:58:45 at least until thud or so, we then moved forwards a bit. Aug 12 11:59:42 Krogoth Aug 12 12:00:06 I believe that's the version I was having to anchor on to keep mlinux compatible Aug 12 12:00:25 Might even have to go back to Jethro Aug 12 12:00:36 stephen: not exactly current, but if it works, who cares. Aug 12 12:02:15 For reference, here's my starting point for building the mlinux env: https://www.multitech.net/developer/software/mlinux/mlinux-building-images/building-a-custom-linux-image/ Aug 12 12:03:23 stephen: *sigh* that document mandates quite a bunch of bad practises. Aug 12 12:03:53 Any chance you can point them out? Aug 12 12:04:18 I'll document and write a better one. Aug 12 12:04:39 stephen: sure. 1) do not modify/add anything into a layer provided by anybody else Aug 12 12:04:48 stephen: 2) alsways bundle your stuff in a custom layer Aug 12 12:06:11 3) passing in root password through cli is also quite awkward, but if its their style.... Aug 12 12:07:21 http://git.multitech.net/cgi-bin/cgit.cgi/meta-multitech.git/ Aug 12 12:07:28 heh, thats also a good one: "If a recipe for your desired software is not available, you’ll need to write a custom recipe." Aug 12 12:07:32 looks like they use submodules Aug 12 12:07:37 no link, no explanation on what this means. Aug 12 12:08:20 Crofton|work: *sigh* srsly? http://git.multitech.net/cgi-bin/cgit.cgi/meta-multitech.git/tree/README?id=5.0.0 Aug 12 12:08:38 it probably means that their git clone should include switches to recurse into submodules Aug 12 12:11:18 scrtch that Aug 12 12:12:20 meta-multitech actually looks like a pretty generic bsp layer, little magic. ig uess they rather hide $STUFF in the mlinux download. Aug 12 12:14:19 I suppose I should do some real work Aug 12 12:15:15 New news from stackoverflow: bitbake SRC_URI file:// Aug 12 12:19:45 My problem I think is very simple. I would like to have opkg verify signatures before installing packages from my custom opkg repository. The issue I am having is that the keys I added to the opkg-keyrings yocto recipe are all marked as unknown on the target by gpg. They are all installed though. So when i attempt to install a package form my custo Aug 12 12:19:45 m repository, It fails because there are no trusted keys. I do not believe that this is a yocto bug, but I am running rocko. Aug 12 12:19:46 ficently trusted public keys found. Aug 12 12:21:45 after a little google foo and testing Aug 12 12:22:27 i think i need to stick something like this either in the opkg-keyring recipe or as a start up scripte gpg --list-keys --fingerprint --with-colons |\ Aug 12 12:22:34 Is that a crazy idea ? Aug 12 12:24:07 And see, grr... they supply their sdk, which essentially is a script that sources up a different script, yet the image build script uses oe-*. Along all this I ended up deciding I needed to source oe-blah first, then source their env Aug 12 12:24:15 I have no idea if that is proper Aug 12 12:26:08 stephen: i only had a short peek but their sdk does at least not seem *too* weird. Aug 12 12:26:19 stephen: just documentation is kinda.. meh. Aug 12 12:28:27 right now I've done 2 things: I have a shell with their sdk installed and sourced up Aug 12 12:29:05 and I have a second which it bitbaking mlinux-factory-image from oe-init-build-env Aug 12 12:29:58 if I can get into the image bitbaked, maybe I can opkg in what I need and build the driver? Aug 12 12:30:16 stephen: nope, thats not how it works Aug 12 12:30:28 Yeah, I figured as much Aug 12 12:30:51 but at least I'll know that their instructions at least get me into a virt of their device Aug 12 12:31:19 stephen: look at https://www.yoctoproject.org/docs/latest/kernel-dev/kernel-dev.html#building-out-of-tree-modules-on-the-target Aug 12 12:31:39 As I understand it I need to create another bitbake target which includes the meta-telephony layer extending mlinux-factory-image Aug 12 12:31:55 stephen: that section pretty much exactly outlines what you need to do in order to be able to compile in-target Aug 12 12:32:16 as for doing it properly, you're right. Aug 12 12:32:20 what are *.sdk images? Aug 12 12:32:35 mlinux build as uboot and zImage Aug 12 12:33:02 because the final result is flashed onto a piece of hardware Aug 12 12:33:47 the key about an sdk image is that it brings toolchain, headers, etc. Aug 12 12:33:53 -> in-target compilation Aug 12 12:35:04 you have to decide if you want to do the in-target way or the buildsystem way. mixing probably won't work Aug 12 12:36:09 so if I have mlinux downloaded, how and where is the proper place to download the headers? Aug 12 12:36:36 if you do buildsystem, nowhere. it will take care of it for you. Aug 12 12:36:55 if you do in-target, nowhere. the buildsystem should do it for you and place them in the image Aug 12 12:37:51 buildsystem? Aug 12 12:38:21 if you want bitbake to take care of building stuff Aug 12 12:39:55 So essentially what I'm doing is source-ing into the provided env, then running bitbake. Aside from the layer/recipe edits, am I missing something? Aug 12 12:40:19 no, thats ok. Aug 12 12:40:48 but you have to understand that this is an environment for doing bitbake tasks, and *NOT* for directly crosscompiling. Aug 12 12:40:59 I understand that. Aug 12 12:41:07 The SDK is for cross compiles Aug 12 13:27:10 RP: do you have a quick pointer on how to disable uninative the most elegant way? Aug 12 13:30:00 LetoThe2nd: not sure there is a particularly neat way, wasn't really designed to be disabled easily once enabled Aug 12 13:30:17 LetoThe2nd: mkdir classes; touch uninative.bbclass ? Aug 12 13:30:30 RP: you mean for an existing build? Aug 12 13:30:50 LetoThe2nd: creating an empty class will work on any build Aug 12 13:30:54 Ugh, and here we go Aug 12 13:31:31 try to build dadhi, it thinks h files are missing. Aug 12 13:31:33 RP: let me explain a little. i'm having a very specific kind of fun building node+icu for an older release, and uninative.conf seems to break things because it disables GLIBCXX Aug 12 13:32:13 LetoThe2nd: so alongside your conf directory which has local.conf, create a classes directory with a uninative.bbclass in it Aug 12 13:32:31 (an empty one) Aug 12 13:32:41 stephen: usually this is fixed by adding the provider of the desired headers to the DEPENDS Aug 12 13:32:55 RP: ok, let me give it a spin. Aug 12 13:36:36 LetoThe2nd: you're suggesting bitbake is like a game of chance? Aug 12 13:36:54 RP: how could i dare?!? Aug 12 13:37:01 * LetoThe2nd goes to find beer. Aug 12 13:37:27 LetoThe2nd: :) Aug 12 13:38:24 So, bitbaking mlinux-factory-image worked just fine Aug 12 13:45:58 RP: BTW, specific booze orders for lyon? ;-) Aug 12 13:52:05 RP: did you get quality afk time? Aug 12 13:52:32 kanavin: not at all, spent the weekend staring at runqueue as I was getting seriously worried about the release Aug 12 13:52:42 RP: would it also work if i place the empty class in my own layer? i think it should but i probably need a higher layer priority, right? Aug 12 13:52:57 LetoThe2nd: correct Aug 12 13:53:04 RP: ok, thanks Aug 12 13:54:31 kanavin: its kind of paid off as I was able to figure out the problem and I have some better code in testing locally Aug 12 13:54:47 RP: that's great to hear, but I was a bit concerned about your facebook post Aug 12 13:55:41 kanavin: I was feeling concerned too :( Aug 12 13:56:00 that brought to mind one of my favorite british books Aug 12 13:56:04 This duty done, we refilled our glasses, lit our pipes, and resumed the discussion upon our state of health. What it was that was actually the matter with us, we none of us could be sure of; but the unanimous opinion was that it – whatever it was – had been brought on by overwork. Aug 12 13:56:05 “What we want is rest,” said Harris. Aug 12 13:56:05 “Rest and a complete change,” said George. “The overstrain upon our brains has produced a general depression throughout the system. Change of scene, and absence of the necessity for thought, will restore the mental equilibrium.” Aug 12 13:56:05 George has a cousin, who is usually described in the charge-sheet as a medical student, so that he naturally has a somewhat family-physicianary way of putting things. Aug 12 13:56:34 I followed that advice in July (including the boat bit) with great results :) Aug 12 13:57:24 kanavin: good :) Aug 12 13:58:00 kanavin: this sums up as "get shitfaced when work is done", right? Aug 12 13:58:32 kanavin: the scary thing is I have some insight into those fatigue issues I've been struggling with. Not sure I like knowing what I now know though! Aug 12 13:59:22 RP: I thought you have some kind of poorly-understood chemistry issue in your body? Aug 12 14:00:00 kanavin: yes, I understand it a bit more now :/ Aug 12 14:00:47 LetoThe2nd, I was quoting http://www.authorama.com/three-men-in-a-boat-1.html Aug 12 14:01:08 one of the first complete, original english books I read when I was in school Aug 12 14:01:36 kanavin: never heard of it. :) i was just summing it up into what i thought it shuold read :) Aug 12 14:38:34 RP: ok. result: it works nicely when i comment out uninative in poky.conf. if i place classes/uninative.bbclass in my layer, layer prio 7, it seems to be ignored though Aug 12 14:40:01 RP, I am seeing new build failures on warrior baseline. did you run a warrior build when I was on vacation? Aug 12 14:40:04 LetoThe2nd: what about alongside local.conf? Aug 12 14:40:41 LetoThe2nd: does your layer add to BBPATH? If so append or prepend? Aug 12 14:41:02 armpit: I thought maybe one. Which failures? Aug 12 14:41:24 RP: BBPATH .= ":${LAYERDIR}" Aug 12 14:41:39 LetoThe2nd: try prepending Aug 12 14:41:41 qemuarm64-ptest, oe-selftest fadora Aug 12 14:41:59 armpit: fairly sure the ptest failure happened on the release build? Aug 12 14:42:06 I just found warrior-next on july 29th with failures Aug 12 14:42:13 armpit: there is a small outside chance tgoodwin fixed a bug which was masking config issues too Aug 12 14:42:36 armpit: (in yocto-ab-helper) Aug 12 14:42:50 (or was it yocto-ab-2?) Aug 12 14:42:57 RP: =. breaks parsing because of other dependencies. :( Aug 12 14:44:07 RP, just trying to figure out if I introduced new issue... Aug 12 14:44:10 LetoThe2nd: ok, second layer/sublayer which prepends? Aug 12 14:44:20 RP: and in build/conf next local.conf it also doesn't seem to be picked up. note: we are taking about krogoth :/ Aug 12 14:44:47 LetoThe2nd: you want build/classes/uninative.bbclass Aug 12 14:45:12 LetoThe2nd: BBPATH hasn't changed behaviour in years Aug 12 14:46:20 RP: build/classes picks it up Aug 12 14:50:17 RP: *sigh* layer order as in https://stackoverflow.com/questions/51002891/overwriting-yocto-classes-through-meta-layer Aug 12 14:51:19 LetoThe2nd: layer ordering is a pain :( Aug 12 14:52:28 RP: yeah. reason this time was, for testing i added it through bitbake-layers. yet in CI another script takes care of it, which will place it at the top reproductibly. this is not exactly beautiful, but i think it suffices. Aug 12 15:00:14 * armpit hmm, arm64 is not listed as a tested host in 2.7.1 test results... is confused Aug 12 15:01:47 armpit: I think that failed on the 2.7.1 build Aug 12 15:02:25 odd thing is thud passed the same build Aug 12 15:03:06 in any case I opened bugs for the failures Aug 12 15:05:53 hi Aug 12 15:09:55 does bitbake have a way of returning a list of variables set in a particular bbclass? bb.parse.vars_from_file doesn't seem to do what I need Aug 12 15:12:11 zeddii, thanks for merging the kfrags.. I forgot to include which kver they should have been against. Aug 12 19:00:45 Can anyone recommend an embedded hardware platform with good yocto support/well maintained which supports ethernet, serial, and modem connectivity? Aug 12 19:01:12 Building on Multitech's past mistakes seems like a bad idea Aug 12 19:02:28 osmocom, hmmm Aug 12 19:07:57 compiling an opencv program I get "opencv2/opencv.hpp: No such file or directory" how to fix it in the recipe? I'm cross-compiling for rpi Aug 12 19:11:54 kayterina, have you installed linux-libc-headers? Aug 12 19:12:13 I'm guessing your target is arm or aarch? Aug 12 19:13:09 If so, be aware, source header linkage in that area of the code is a poorly maintained wasteland. In the end you may just have to find the file and correct the link Aug 12 19:34:23 stephen: what file am I looking for to correct the link? Aug 12 19:35:07 Look earlier on that line, or a couple lines before Aug 12 19:35:28 It is compiling ok in the host,do linux-libc-headers need to be in the image too? Aug 12 19:35:41 It will be another h or hpp file which lists opencv.hpp as an include Aug 12 19:36:04 kayterina, No clue, you should research what provides opencv.hpp Aug 12 19:36:41 I run into this a lot when a directory has an asm-common directory but no asm/ directory Aug 12 19:36:57 usually I just link the asm directory to the asm-common dir and try again Aug 12 20:10:45 <|aaron> Hi all. I'm trying to move DL_DIR to an NFS mounted directory. But when I do that, bitbake hangs forever the first time it tries to fetch files. Several 0 byte length .lock files are written to the NFS mount, but the actual files it's trying to fetch never make it there. Aug 12 20:10:58 <|aaron> TMPDIR is still on a regular disk Aug 12 20:23:33 |aaron:which version of release are you on Aug 12 20:23:55 NFS is always finicky Aug 12 20:24:47 <|aaron> BitBake Build Tool Core version 1.40.0 Aug 12 20:25:27 <|aaron> Heh that's an understatement re: NFS lol. I just ran an strace on bitbake and I'm seeing "futex(0x2005600, FUTEX_WAIT_BITSET_PRIVATE|FUTEX_CLOCK_REALTIME, 0, {tv_sec=1565641416, tv_nsec=986858000}, 0xffffffff) = -1 ETIMEDOUT (Connection timed out)" repeated over and over Aug 12 20:54:06 khem: there? may need to be a bit careful with sstate from the hash equiv server :/ Aug 12 20:58:53 <|aaron> Okay I figured it out, although not exactly clear why this was necessary. I ran bitbake with -D and saw that it was hanging at "Testing URL file://...". We had CONNECTIVITY_CHECK_URIS ?= "http://builds.ourdomain.com:88/mirror/sources/" set in our config, I set that var to empty instead and now it works Aug 12 20:59:31 <|aaron> However, now I'm having the same problem when i try to set SSTATE_DIR to the NFS mount. It hangs at "Testing URL file://..." even with that var set to emptuy Aug 12 20:59:44 RP: I wrote a quick fully async hash server you can try if you' Aug 12 21:00:01 *you're still having performance problems on the autobuilder: https://github.com/JPEWdev/aiohashserver Aug 12 21:00:45 That should be able to handle connections until you run out of file descriptors :) . If it's still slow, it's probably sqlite Aug 12 21:05:14 JPEW: perhaps you can help me understand where the bottleneck is. Aug 12 21:05:51 JPEW: I delete the unihash cache file, start a bitbake-hashserv, then run a "bitbake core-image-sato -kn". Aug 12 21:06:16 JPEW: what I then see is 6200 open connection to hashserv for over 30s Aug 12 21:06:50 JPEW: number of connections ~= number of tasks. I have to wonder why. I did already try and context manager around the urlopen() Aug 12 21:08:08 JPEW: definitely having performance problems. Last build was rather a mess :( Aug 12 21:08:18 Is the confusing part that there is are 6200 queries, or that each one is a new connection? Aug 12 21:10:11 JPEW: there are 6200 *connections* open Aug 12 21:10:33 JPEW: so each one is a new connection Aug 12 21:13:24 RP: Are the connections stuck in TIME_WAIT state perhaps? Aug 12 21:13:54 JPEW: yes Aug 12 21:17:49 RP: I would expect to see a lot of connections in the TIME_WAIT state. Each TCP connection from the client is going to enter the TIME_WAIT state after the connection is closed... One solution would be to reuse a persistent connection to the server, but the stock urllib from python doesn't support that AFAIK Aug 12 21:19:29 RP: I don't think that the connections in the TIME_WAIT are the source of the slow down.... they shouldn't really affect anything directly (although, it might strain the systems connection limit). I would be more worried to see a lot of active connections at once Aug 12 21:24:21 JPEW: I need to check what state things are sitting in Aug 12 21:36:59 RP: Really sorry, but I have to head home right now :(. I'll get back on IRC in the morning, or you can e-mail me with what you find. I'll look into trying to write some test programs to try and profile the hash equivalence server; there's probably some ways it can be made faster Aug 12 21:40:27 JPEW: no problem, I will need to sleep shortly too Aug 12 21:40:48 JPEW: I'm a bit fried after the other runqueue stuff tbh Aug 12 21:41:16 JPEW: I'm hoping there is something obvious/easy we're missing and can fix Aug 12 21:45:11 RP: Btw, regarding my mail to the oe-core list regarding the delays with the current bitbake, don't take it as a criticism. I fully understand that correctness is your main concern at the moment. Just take it as a data point from a live environment. Aug 12 21:54:44 Saur: you could see if master-next is any better Aug 12 21:56:49 Saur: clearly there is a problem and its not fit for purpose at the moment :( Aug 12 21:56:50 RP: Now that you mention it, I think the first example in my mail (with the really extreme delays) were with the bitbake changes from master-next cherry-picked. Aug 12 21:57:18 Saur: so you're saying master-next made things worse? :/ Aug 12 21:57:20 However, I do not know if those changes were the cause or it was just a coincidence. Aug 12 21:57:49 That build was weird anyway, so probably best not to give it too much credit. Aug 12 21:59:24 Saur: I now have a dilemma. To try and fix the performance issue with runqueue itself, the one with the hash server, or the correctness issues the autobuilder threw up, or the day to day patch merging Aug 12 22:00:22 Its trying to do everything in parallel that is driving me crazy :/ Aug 12 22:01:13 RP: Wish I could help you. Unfortunately, when it comes to bitbake itself my knowledge is superficial at best. Aug 12 22:01:15 Saur: I think a good benchmark is "bitbake X -n" on a clean build directory fwiw Aug 12 22:01:46 Saur: it might be helpful to figure out where in master we regressed and if that "benchmark" does show it Aug 12 22:46:07 RP: It seems to be commit 1f630fdf in bitbake that introduces the delays. Aug 12 22:54:59 RP: with bitbake master-next update from today, I'm now seeing: /bitbake/lib/bb/data_smart.py:89: ResourceWarning: unclosed not sure if it's related to delays Aug 12 22:55:32 Anyway, I'm off, but maybe that info can give you some idea where to look for the cause of the delays. Aug 12 22:59:02 RP:I was giving a go to hash things, I have disabled it for now Aug 12 22:59:48 RP: building full oe layers with bunch of bsp layers ends up with a world build with 45k tasks Aug 12 22:59:59 so speed is need Aug 12 23:00:02 for now Aug 12 23:01:20 RP: ran -c testimage on rpi3 as well. Runs fairly well except few things, oeqa is very specific to messages coming from tested machines Aug 12 23:33:04 Hello. New to Yocto. I have added some layers to a build that was compiling correctly. And now I am getting libccrypt.a relocation R_x86_64_PC32 against symbol 'ufc_foobar' cannot be sued when making a shared opbject recompile with -fPIC. i added -fPIC to compile and ld flags in the perl-native_5.24.4.bb file (where the problem is reported) to no Aug 12 23:33:04 success. I am now thinking to add that to where libcrypt is defined. But think that I am missing something bigger. Can anyone offer any help? Thank you in advance. Aug 12 23:50:05 you should check the newly added layers for setting global config metadata Aug 12 23:50:11 some layers are notorious Aug 13 00:07:24 khem : thank you for the quick reply. Check for what? thanks. Aug 13 00:13:16 content in conf/layer.conf Aug 13 00:13:38 see if they are changing things are CFLAGS or other compiler global flags Aug 13 02:17:31 New news from stackoverflow: yocto bitbake: how to know a package name from the error message and re-run it **** ENDING LOGGING AT Tue Aug 13 02:59:56 2019