**** BEGIN LOGGING AT Mon Jul 31 03:00:02 2017
Jul 31 09:39:16 <jackmitchell> anyone having issues with bitbake HEAD, running on ArchLinux this morning all I'm getting is
Jul 31 09:39:24 <jackmitchell> NOTE: Starting bitbake server...
Jul 31 09:39:24 <jackmitchell> ERROR: Unable to start bitbake server
Jul 31 09:39:24 <jackmitchell> ERROR: Last 10 lines of server log /mnt/scratch/scratch/manifest-test/abaco-build/build-test/bitbake-cookerdaemon.log:
Jul 31 09:39:41 <jackmitchell> where the cookerdaemon.log is empty
Jul 31 10:05:40 <bluelightning> RP^
Jul 31 10:06:18 <RP> jackmitchell: that is happening repeatedly?
Jul 31 10:06:40 <RP> jackmitchell: is there any bitbake process running?
Jul 31 10:07:04 <RP> jackmitchell: I've seen this in autobuilder logs but not been able to reproduce standalone
Jul 31 10:07:23 <jackmitchell> RP: yes, there is a single bitbake process left running
Jul 31 10:07:31 <jackmitchell> RP: I have to kill it to get the error to repeat
Jul 31 10:08:19 <jackmitchell> RP: otherwise if I try running again bitbake just hands indefinitely
Jul 31 10:08:26 <jackmitchell> s/hands/hangs
Jul 31 10:08:36 <RP> jackmitchell: so you can't get anything to work at all? :/
Jul 31 10:08:53 <jackmitchell> RP: nope, works ok my Ubuntu 14 VM though
Jul 31 10:09:11 <RP> jackmitchell: could you try applying http://git.yoctoproject.org/cgit.cgi/poky/commit/?h=master-next&id=b3bc511a3f5d961807ce6747ff4958e775567397 and see if the daemon log has anything in after that?
Jul 31 10:09:22 <jackmitchell> sure
Jul 31 10:09:32 <RP> jackmitchell: we need to understand what the server is doing and/or where its getting to
Jul 31 10:12:48 <jackmitchell> RP: Starting bitbake server pid 954, then dead
Jul 31 10:13:12 <jackmitchell> RP: still one process left running that I have to kill to re-produce
Jul 31 10:13:51 <RP> jackmitchell: right, I'm not expecting it to change behaviour, just print more logging. Could you add some extra prints into _startServer (see that patch) and see how far its getting?
Jul 31 10:14:08 <RP> jackmitchell: separate question of where the traceback is going remains :/
Jul 31 10:16:55 <jackmitchell> RP: >
Jul 31 10:16:55 <jackmitchell> Starting bitbake server pid 972
Jul 31 10:16:55 <jackmitchell> self.configuration.setServerRegIdleCallback(server.register_idle_function)
Jul 31 10:16:55 <jackmitchell> for param in ('prefile', 'postfile'):
Jul 31 10:16:55 <jackmitchell> self.cooker = bb.cooker.BBCooker(self.configuration, self.featureset, self.readypipein)
Jul 31 10:22:23 <RP> jackmitchell: ok, on the failing line, can you add a try:         except Exception as e:          print(str(e))          raise sequence please?
Jul 31 10:22:48 <RP> jackmitchell: I've added something which makes mine break locally there and I also don't see the traceback. Not sure why
Jul 31 10:22:50 <jackmitchell> RP: i've kept going and I'm currently down in
Jul 31 10:23:09 <jackmitchell> RP:         self.databuilder.parseBaseConfiguration()
Jul 31 10:23:47 <RP> jackmitchell: if you wrap it in that try/except, we hopefully can see the actual error
Jul 31 10:24:04 <jackmitchell> RP: error was blank
Jul 31 10:24:20 <jackmitchell> RP: just printed an empty new line
Jul 31 10:26:11 <RP> jackmitchell: hmm, not good :(
Jul 31 10:26:19 <jackmitchell> RP: eventually down to bb.event.fire(bb.event.ConfigParsed(), self.data)
Jul 31 10:26:29 <jackmitchell> RP: does that head down to ConfigParsed?
Jul 31 10:27:08 <jackmitchell> RP: around line 300, before the while loop
Jul 31 10:27:59 <RP> jackmitchell: that could make sense, sounds like its trying to send data back to the UI/event handlers and perhaps failing somehow
Jul 31 10:28:03 <RP> or hanging there
Jul 31 10:31:39 <RP> jackmitchell: I'd suggest try adding http://git.yoctoproject.org/cgit.cgi/poky-contrib/commit/?h=rpurdie/wip5&id=c8f7d94d2499dadd584a9aefbb3fecc17f6cb443 - locally this started printing sensible tracebacks in my logs
Jul 31 10:32:13 <RP> it also removed the hang on exception issue I was seeing with my local bad line of code I added for testing
Jul 31 10:34:50 <jackmitchell>   File "/mnt/scratch/scratch/manifest-test/jack-build/openembedded/bitbake.git/lib/bb/server/process.py", line 406, in _startServer
Jul 31 10:34:51 <jackmitchell>     self.cooker = bb.cooker.BBCooker(self.configuration, self.featureset, self.readypipein)
Jul 31 10:34:51 <jackmitchell>   File "/mnt/scratch/scratch/manifest-test/jack-build/openembedded/bitbake.git/lib/bb/cooker.py", line 199, in __init__
Jul 31 10:34:51 <jackmitchell>     self.initConfigurationData()
Jul 31 10:34:51 <jackmitchell>   File "/mnt/scratch/scratch/manifest-test/jack-build/openembedded/bitbake.git/lib/bb/cooker.py", line 356, in initConfigurationData
Jul 31 10:34:51 <jackmitchell>     self.databuilder.parseBaseConfiguration()
Jul 31 10:34:51 <jackmitchell>   File "/mnt/scratch/scratch/manifest-test/jack-build/openembedded/bitbake.git/lib/bb/cookerdata.py", line 329, in parseBaseConfiguration
Jul 31 10:34:52 <jackmitchell>     raise bb.BBHandledException
Jul 31 10:34:52 <jackmitchell> bb.BBHandledException
Jul 31 10:35:15 <jackmitchell> pretty much what we'd already printed down to
Jul 31 10:35:50 <RP> jackmitchell: except we now know its BBHandledException
Jul 31 10:36:15 <RP> jackmitchell: that means it printed something into the server's standard logging channels before it died
Jul 31 10:36:29 <jackmitchell> RP: UnboundLocalError: local variable 'e' referenced before assignment
Jul 31 10:36:46 <jackmitchell> cookerdate.py around 443
Jul 31 10:37:04 <jackmitchell>         except (SyntaxError, bb.BBHandledException):
Jul 31 10:37:18 <jackmitchell> it has an except but doesn't log an error
Jul 31 10:37:31 <jackmitchell> ah, wait I see
Jul 31 10:37:44 <jackmitchell> there is no e, so I added the error :D
Jul 31 10:39:39 <jackmitchell> RP: hmm, ok; I've got another traceback here on HOSTTOOLS
Jul 31 10:40:18 <jackmitchell> RP: https://pastebin.com/iDDjKdc4
Jul 31 10:42:28 <RP> jackmitchell: that is helpful, I can try and reproduce with a bb.fatal()
Jul 31 10:44:41 <jackmitchell> RP: yes, ok; so I'm missing some HOSTTOOLS, once they were installed I'm getting a locale issue; I'll fix that and see
Jul 31 10:47:15 <jackmitchell> RP: with the locale issue fixed we're parsing
Jul 31 10:47:41 <jackmitchell> RP: with the locale fix, but with the missing utilities removed again, I get the logging
Jul 31 10:47:50 <jackmitchell> NOTE: Reconnecting to bitbake server...
Jul 31 10:47:50 <jackmitchell> NOTE: Retrying server connection...
Jul 31 10:47:50 <jackmitchell> NOTE: Reconnecting to bitbake server...
Jul 31 10:47:50 <jackmitchell> NOTE: Retrying server connection...
Jul 31 10:47:50 <jackmitchell> NOTE: Reconnecting to bitbake server...
Jul 31 10:47:51 <jackmitchell> NOTE: Retrying server connection...
Jul 31 10:47:51 <jackmitchell> NOTE: Reconnecting to bitbake server...
Jul 31 10:47:52 <jackmitchell> NOTE: Retrying server connection...
Jul 31 10:47:52 <jackmitchell> NOTE: Reconnecting to bitbake server...
Jul 31 10:47:53 <jackmitchell> NOTE: Retrying server connection...
Jul 31 10:47:53 <jackmitchell> NOTE: Reconnecting to bitbake server...
Jul 31 10:47:54 <jackmitchell> NOTE: Retrying server connection...
Jul 31 10:48:23 <jackmitchell> so it could be a mix of the two together
Jul 31 10:48:36 <jackmitchell> but I don't get the HOSTTOOLS message regardless
Jul 31 10:49:31 <RP> jackmitchell: try this http://git.yoctoproject.org/cgit.cgi/poky-contrib/commit/?h=rpurdie/wip5&id=e33cb23aeb69e6164b8cd9cefbd41addd1267d1f
Jul 31 10:49:42 <RP> jackmitchell: this should print the UI messages as well as the traceback
Jul 31 10:50:40 <jackmitchell> RP: yep, that brings the correct message through
Jul 31 10:50:58 <RP> jackmitchell: https://pastebin.com/76j4jJHZ
Jul 31 10:50:59 <jackmitchell> RP: the bitbake server ones were a red herring, I had to remove the .lock and .sock to get bitbake working again at all
Jul 31 10:51:22 <jackmitchell> RP: yep pretty much exactly that
Jul 31 10:51:42 <RP> jackmitchell: I suspect you're in a bit of a mess with the debugging. With the patch and things cleaned up I can get a sensible message from bitbake as in the pastebin
Jul 31 10:52:28 <jackmitchell> RP: ok, sounds good; I can revert the changes and ping me if you want me to give a finished patch a test
Jul 31 10:52:41 <RP> jackmitchell: I think the above is the finished one :)
Jul 31 10:53:30 <RP> really should perhaps suppress the traceback if its a HandledException but its better than where we started with no messages
Jul 31 10:53:47 <RP> jackmitchell: I appreciate the help in figuring this out
Jul 31 10:54:59 <jackmitchell> RP: no problem, traceback and message is visable with everything back to HEAD and only above patch applied, thanks
Jul 31 11:02:41 <RP> jackmitchell: I'd been aware there was something wrong, I'm pleased to have found this!
Jul 31 16:33:33 <ant_work> RP, hi, whuile you're working or rm_work, is there still a reason to unconditionally exclude kernel from it?
Jul 31 16:34:47 <RP> ant_work: yes, the kernel is needed by modules and so on
Jul 31 16:34:54 <ant_work> it is indeed useful if you are fiddling with kernel otherwise it is just a waste of space
Jul 31 16:34:55 <RP> ant_work: no safe way to handle it
Jul 31 16:35:28 <ant_work> you mean external modules?
Jul 31 16:35:45 <RP> ant_work: yes
Jul 31 16:38:12 <ant_work> ok, I suppose these were blocked by the previous (last year?) defaults of rm_work
Jul 31 16:38:53 <ant_work> really, I think the kernel is not yet optimally handled
Jul 31 16:39:12 <RP> ant_work: its not possible to please everyone, that is the fundamental issue
Jul 31 16:39:31 <ant_work> there is a basic problem: you cannot mix/build two versions
Jul 31 16:39:44 <RP> ant_work: today we have one copy of the sources which is used for in and out of tree builds. There is little sstate overhead of it.
Jul 31 16:40:14 <RP> ant_work: we can dial in other compromises - multiple copies of the source, high sstate overhead but cleaner tmpdir
Jul 31 16:40:33 <RP> ant_work: I believe it should be relatively possible to have two versions
Jul 31 16:40:33 <ant_work> I have seen one interesting patchset around, [OE-core] [PATCH v4] kernel: Add support for multiple kernel packages
Jul 31 16:40:54 <RP> ant_work: the pieces of that which did merge have already caused unresolved issues for upgrades :(
Jul 31 16:41:03 <ant_work> hehe
Jul 31 16:41:19 <RP> ant_work: trouble is the original submitter doesn't care, they got their patch in
Jul 31 16:41:40 <kergoth> heh, typical
Jul 31 16:41:59 <ant_work> well, I have been blacklisted because even building linux-yocto and linux-yocto-tiny was not possible
Jul 31 16:42:19 <ant_work> (my recipe included -tiny)
Jul 31 16:42:46 <ant_work> JaMa autobuilder revealed the issue
Jul 31 16:43:57 <ant_work> the conflict is not only about providers
Jul 31 16:46:17 <ant_work> let me solve the gcc7 issue (kernel miscompiled) then I'll retry
Jul 31 16:46:38 <ant_work> I'll let you know, keep on the grat work!
Jul 31 16:47:57 <ant_work> RP, before I forget, some news about do_menuconfig
Jul 31 16:48:17 <ant_work> non yocto kernels fail the task
Jul 31 16:48:30 <ant_work> on Ubuntu 16.04 you need  sudo apt-get install lib32ncurses5-dev
Jul 31 16:49:04 <ant_work> we talked about the peculiarity of ncurses-native not long ago
Jul 31 16:50:50 <ant_work> shouldn't the -dev headers be staged nonetheless?
Jul 31 16:57:06 <RP> ant_work: I don't remember the details offhand, sorry
Jul 31 16:57:13 <RP> ant_work: off to get food right now...
Jul 31 16:57:25 <ant_work> I'll file a bug for Bruce
Jul 31 16:57:30 <ant_work> cheers
**** ENDING LOGGING AT Tue Aug 01 03:00:00 2017