**** BEGIN LOGGING AT Thu Mar 24 02:59:58 2016 Mar 24 05:38:58 morning Mar 24 05:39:06 * DougReeder waves Mar 24 05:39:12 DougReeder: Hi :) Mar 24 05:39:28 We've been fighting with a lot of upgrades lately :P Mar 24 05:39:35 * DougReeder nods Mar 24 05:39:36 Nothing seems to work easily :'( Mar 24 05:44:10 But we'll find a solution Mar 24 05:44:13 :) Mar 24 05:44:20 Just stuff segfaulting is not good :P Mar 24 05:44:39 * DougReeder nods Mar 24 06:33:19 Tofe: Did you see my semi-progress y'day :P Mar 24 06:33:36 I'm trying qemux86 now to see if luna-webappmanager also crashes there Mar 24 06:34:04 But it will be a while since it needs to re-run the whole build :P Mar 24 06:34:37 In the meanwhile I reached out to the systemd patch developer, maybe he has some ideas on how to solve our Mako bootloop Mar 24 08:08:32 morning Mar 24 08:19:56 Morning! Mar 24 08:20:29 Herrie: yes, we'll get it running in the end :) Mar 24 08:21:09 Herrie: do you have an URI to the aforesaid systemd patch? Mar 24 08:24:03 Tofe: Well systemd is in openembedded-core/meta/recipes-core/systemd and we then have our patches in meta-webos-ports/meta-luneos/recipes-core/systemd Mar 24 08:24:15 Fido is on systemd 219, Jethro on 225 Mar 24 08:24:25 ok Mar 24 08:24:29 Patch looks similar, I see no specific issues to be honest Mar 24 08:24:52 Could be something outside the patch, just strange it only affects Mako Mar 24 08:25:50 Doesn't affect maguro? Mar 24 08:26:34 I recall Maguro worked, I can test again tonight Mar 24 08:26:42 Maguro is at home :P Mar 24 08:27:32 I'm trying qemux86 for luna-webappmanager to see if it's maybe something ARM specific Mar 24 08:27:44 But my VirtualBox decided to crash during build :P Mar 24 08:27:51 ok :) Mar 24 08:27:59 So restarted it now, hopefully done tonight when home. Mar 24 08:28:27 Nothing weird in https://github.com/webOS-ports/meta-smartphone/tree/webOS-ports/master/meta-lg/recipes-core/systemd ? Mar 24 08:29:17 Anyway the with the !bound_ check on l29 has been changed in newer Chromium, so it might disappear by itself once we go to 5.6 (or not), but worth the try. Mar 24 08:31:01 Tofe: The diff for the patch between Fido and Jethro doesn't show anything suspicious to me. Mar 24 08:31:16 mmh Mar 24 08:31:38 can we start systemd in some sort of debug mode? Mar 24 08:32:26 or, maybe, create by hand the log storage directory, so that it becomes persistent across boots Mar 24 08:33:02 I could guide you through doing that, if you're willing to give it a try Mar 24 08:33:19 Tofe: Sure :) Let's do that tonight. Mar 24 08:34:09 ok! Mar 24 08:40:19 For PulseAudio 6 reverting to the Fido revision seems to do the trick as well. There were quite some changes in the recipes between Fido and Jethro some of which seem to be related to systemd as well. Mar 24 08:40:35 The PulseAudio recipes go over my head anyway :P Mar 24 08:41:04 And the log doesn't give me any clues it seems as to why it has memory violations. Mar 24 11:45:11 JaMa: Some slight progress. Reverting PA 6 to Fido version makes audio work again. Still have luna-webappmgr crashing hard though. Mar 24 11:45:33 So for now downgrading systemd & pa6 to Fido from Jethro solves 2/3 blockers Mar 24 11:47:24 Will do some testing tonight with Tofe to see if we can log why systemd bootloops Mar 24 11:53:43 Herrie|Veer: ok, can you move these downgrades to meta-luneos layer, so that we can switch to Jethro and then remove these backports when the issue is resolved? Mar 24 11:54:10 Herrie|Veer: I don't want to use oe-core fork just because of 2 downgraded recipes Mar 24 11:56:44 JaMa: That's my idea but we need to fix the luna-webappmgr hard crash first before we can switch Mar 24 13:38:09 JaMa: The luna-webappmgr hard crash might be solved with Qt 5.6 upgrade since underlying Chromium code was changed quite a bit, however we have other issues with qtwayland, libhybris on our Android targets. Tofe is looking into that. Mar 24 13:45:39 Herrie|Veer: do you want to upgrade to Qt 5.6 at the same time as Yocto or after? Mar 24 13:47:01 JaMa: They can be separate. We'd like to do both, really depends on how quickly we can sort issues. Mar 24 13:47:45 Herrie|Veer: my Qt 5.6 builds relies on a couple of Jethro commits, so it would be better to wait a bit Mar 24 13:48:07 I'm just debugging it now because that will have to be done anyway Mar 24 13:48:45 I might check Jethro with QT 5.6 on qemu after I've checked luna-webappmgr with qemu with 5.5.1 to see if it's ARM specific or not. Mar 24 13:49:49 It could be that we'll still have the issue with luna-webappmgr with 5.6 too and need to address it anyway. In that case might be able to do fix that, upgrade to Jethro and then to 5.6 Mar 24 13:53:58 Tofe: You don't have any principal rejection for Jethro right if we don't have regressions? Mar 24 13:55:05 s/rejection/objections Mar 24 13:55:44 JaMa: Jethro would have priority, mainly so you have less to maintain :) Mar 24 13:56:10 5.6 is a nice to have we'd like for the project because we'd expect it to solve some long outstanding issues. Mar 24 13:59:51 what surprises me the most that luna-webappmgr crash is caused by Jethro upgrade which didn't change the Qt or chromium AFAIK Mar 24 14:00:13 so even if the upgarde to Qt 5.6 might fix it, it would still be interesting to find the root cause of the issue Mar 24 14:00:59 JaMa: It points to a very specific line in Chromium code Mar 24 14:01:06 I'm not sure what it is... Mar 24 14:01:14 the chromium in qtwebengine build, right? Mar 24 14:01:25 It's there with my Fido PulseAudio6 as well Mar 24 14:01:25 so it should be the same in Fido and Jethro based build Mar 24 14:01:58 We first thought it might be related to PulseAudio, but when I downgraded it audio started to work, but webengine still hard crashing Mar 24 14:02:22 OK Mar 24 14:02:38 thanks for doing all this work! Mar 24 14:02:46 It should be the same.... Mar 24 14:02:50 Let me double check Mar 24 14:02:58 With QT stuff I mean Mar 24 14:03:23 it's using the same meta-qt5 revision Mar 24 14:03:32 I've just compared webos-ports-setup branches Mar 24 14:04:11 and meta-webos-ports git diff fido jethro also doesn't show any changes in chromium/webengine Mar 24 14:07:56 JaMa: It can be some other component that's updated in Jethro... Just the clues are minimal Mar 24 14:08:04 Need to look into gdb I guess Mar 24 14:08:11 But that's another can of worms :P Mar 24 14:08:16 yes, it could be even toolchain :) Mar 24 14:08:31 does it happen with qemux86 as well? Mar 24 14:08:50 That's what I need to check Mar 24 14:08:59 Put a build this morning Mar 24 14:09:04 Hopefully done when home Mar 24 14:09:39 But my VirtualBox already locked up once, so not 100% sure it will. I guess I could just take latest unstable from 2+ weeks ago to test? Mar 24 14:21:36 JaMa: Unstable is Jethro or master? Luna-webappmgr is OK there Mar 24 14:23:14 unstable is currently (and usually) == jethro Mar 24 14:23:26 it was only temporarily switched to master when testing the build on jenkins Mar 24 14:26:37 Ok let me just test an older unstable for verification Mar 24 14:34:49 That also seems to work Mar 24 14:35:11 Might still be pulseaudio somehow? Not sure if we use that in qemux86? Mar 24 14:48:37 Herrie|Veer: PS is installed and built in qemux86 builds as well, so I think it's used the same as on other devices Mar 24 14:48:41 PS Mar 24 14:48:43 PA Mar 24 14:49:19 JaMa: Well if qemu is fine for luna-webappmgr then it might be the cross compilation indeed? Mar 24 14:49:31 Which might also explain the pa6 issues? Mar 24 14:50:09 We have no audio working in virtualbox for already a while I guess. But that's another issue altogether I guess. Mar 24 14:53:23 Let me check qemu journalctl for pulseaudio mentions Mar 24 15:01:43 JaMa: PulseAudio doesn't crash there either Mar 24 15:01:55 So I guess you might be right with the cross compiler Mar 24 15:02:22 In yesterday's log I posted the exact codeline in chromium that caused the hard crash in luna-webappmgr Mar 24 16:32:57 JaMa: Any specifics I can test for you on the mako where we have the PA 6/8 and luna-appmgr issues? Mar 24 16:33:10 I.e. any specific gdb I can run or something? Mar 24 16:33:34 testing on the mako? like, the unreleased phone? Mar 24 16:33:48 :) Mar 24 16:33:57 mako = nexus 4 for us :P Mar 24 16:34:01 ah Mar 24 16:34:08 Anything legacy has too little ram for us :P Mar 24 16:34:22 TP with 1GB is really the minimum to be fair Mar 24 16:34:51 because of all the extra waste in QT5 or the new webkit? Mar 24 16:34:56 (waste of RAM) Mar 24 16:35:01 Yeah as well Mar 24 16:35:15 512MB on the Pre3/Veer never was a lot Mar 24 16:35:26 "Out-of-cards" :P Mar 24 16:35:44 WebEngine eats a fair bit too but nothing too drastic Mar 24 16:35:47 lol, you're talking to someone who owns both a Pandora and a GP2X. You can do a surprising amount with little ram Mar 24 16:35:56 as long as you are smart about it Mar 24 16:36:30 I will agree that having 2-4 GB of RAM would be ideal, tho' Mar 24 16:36:54 GodGinrai: Yeah if you write assembly :P Mar 24 16:37:02 You can do with 64MB :P Mar 24 16:37:20 Hell even Bill Gates said 640KB is enough :P Mar 24 16:38:28 Not just assembly. Back on the GP32, Squidge managed to get Beats of Rage, which had a minimum RAM requirement of 256 MB to run using the GP32's 8MB of RAM Mar 24 16:39:12 Now, that can't be done to just about everything, but there *is* a lot of unoptimized code within desktop source Mar 24 16:40:37 Oh we have too I'm sure, just with current prices of RAM I don't think we'll be having issues with 2GB+ devices :D Mar 24 16:41:53 oh, I understand that Mar 24 16:41:56 as I said Mar 24 16:42:00 2-4 GB is ideal Mar 24 16:43:11 I'm curious to see how LuneOS would run on a 2016 flagship :D Mar 24 16:44:28 As much as I would like it, I still feel that none of the current phone hardware would be as enjoyable as my Pre 3 Mar 24 16:44:40 The design was just so good Mar 24 16:44:48 even if the insides are outdated Mar 24 16:45:04 Well was holding a S7 and S7 Edge today during lunch, nice phones, feel great Mar 24 16:45:19 I'd miss the hw kb though. I hate vkb... Mar 24 16:45:25 I'm not a fan of the whole flat phone thing Mar 24 16:45:30 If we'd only had a rooted BB Priv :P Mar 24 16:45:49 the only reason I owned a Note was because it was the best option at the time since no good phones had a physical keyboard Mar 24 16:46:02 at least with the note, I could handwrite my text Mar 24 16:46:52 LOL Mar 24 16:47:10 We added word prediction recently in LuneOS :) Mar 24 16:47:20 Long wanted feature in webOS :P Mar 24 16:50:05 word prediction? Mar 24 16:51:05 Yeah the stuff that when you enter a few letters it will suggest words :p Mar 24 16:51:19 ah Mar 24 16:51:44 I was never bothered by that, since I had a physical keyboard Mar 24 16:51:53 I could see people wanting it with a software keyboard, tho' Mar 24 16:52:39 Herrie: for me with russian keyboard it predicts only words that i wrote before Mar 24 16:53:07 nizovn: probably because no russian words are in the prediction dictionary Mar 24 16:54:51 yes, i thought it will be solved along with locales(when choosing right dictionary) Mar 24 16:57:59 nizovn: Hmmz it should work... Mar 24 16:58:05 We do provide a dictionary Mar 24 16:58:15 It worked for English and French Mar 24 16:58:28 Could be some codepage issue somehow Mar 24 16:58:43 Dictionary files should be utf8 though Mar 24 16:59:09 Not sure about maliit, webos-keyboard and qt5 but quite sure they also are :s Mar 24 17:00:13 ok, i will try latest emulator image Mar 24 17:01:16 If it doesn't work we'd need to debug a bit :) Mar 24 17:16:39 Herrie|Veer: it doesn't work, and word is covered by black rectangle until i start a new word Mar 24 17:18:43 Yeah black rectangle I noticed too. Will do some digging on that. It's probably something in webos-keyboard's WordRibbon.qml or like that. Mar 24 17:19:00 I'll take a screenshot, pick color in image editor and search for it. Mar 24 17:20:02 Word Ribbon anyway needs some tweaking. Not a big fan of it :P Mar 24 17:21:17 for some reason word prediction is not available in browser's addressBar Mar 24 17:45:55 nizovn: Could be by design, didn't test it yet Mar 24 17:48:02 Herrie: ok, just tell me when you'll want to debug the N4 Mar 24 17:51:32 Tofe: You can start Mar 24 17:51:39 It might be cross compilation Mar 24 17:52:51 Well, on my side, I don't have a N4 image with jethro Mar 24 17:53:06 I'm fully busy with Qt 5.6 :p Mar 24 17:58:04 Tofe: GDB for LunaWebAppManager gives me: Program received signal SIGILL, Illegal instruction. 0x4408e4c8 in ?? () from /lib/libcrypto.so.1.0.0 Mar 24 17:59:01 And the bt: https://bpaste.net/show/95b2b81ce7ab Mar 24 18:00:23 Herrie: just execute "c" Mar 24 18:01:58 but i guess it's better to attach to webappmanager, not to run it from gdb Mar 24 18:02:03 nizovn: https://bpaste.net/show/1da654913c80 Mar 24 18:06:41 gdb attach `pidof LunaWebAppManager` Mar 24 18:07:29 nizovn: Now reflashing image where PulseAudio is also crashing so can do for both LunaWebAppMgr & PulseAudio :) Mar 24 18:07:41 ok Mar 24 18:11:27 nizovn: SInce it's not running how I get pid? Mar 24 18:11:35 It continuously crashes ;) Mar 24 18:13:16 /usr/sbin/LunaWebAppManager && gdb attach `pidof LunaWebAppManager` Mar 24 18:15:12 nizovn: https://bpaste.net/show/1dac47b38faa Mar 24 18:16:11 Which refers to https://github.com/webOS-ports/qtwebengine-chromium/blob/webOS-ports/master/chromium/content/common/mojo/service_registry_impl.cc#l29 Mar 24 18:16:18 sorry Mar 24 18:16:26 /usr/sbin/LunaWebAppManager & gdb attach `pidof LunaWebAppManager` Mar 24 18:16:47 Which has been changed quite a bit in more recent Chromium Mar 24 18:17:38 OK: https://bpaste.net/show/62d252d87422 Mar 24 18:17:47 there is strange commit https://github.com/scheib/chromium/commit/59a3a63b021eddf1b64e23aec7c3f07913e322f3 Mar 24 18:18:31 Herrie: now you can type c Mar 24 18:19:47 https://bpaste.net/show/7d7fe3eb2f58 Mar 24 18:21:13 Herrie: another c :) Mar 24 18:22:07 OK get a few times the same, then exits: https://bpaste.net/show/9c308ed4db52 Mar 24 18:24:00 as that check was added to find bug, i think we need to update chromium Mar 24 18:24:33 (where it's already solved) Mar 24 18:25:26 Just change it back to DCHECK? Mar 24 18:27:13 if it's possible, we should update chromium, else we will have unknown bug Mar 24 18:29:39 nizovn: Strange thing is that it works OK in current builds, but when we move from Fido to Jethro it causes issues Mar 24 18:29:49 Could be gcc update somehow broke it Mar 24 18:30:05 Because we use exactly the same revision of qtwebengine and qtwebengine-chromium Mar 24 18:36:30 maybe it's due to that sigpipe Mar 24 18:41:55 Herrie: for the latest "c", you should have made "bt" instead, when it stopped at "raise" Mar 24 18:42:18 that way we'd have had the call stack Mar 24 19:10:26 Tofe: Like this? https://bpaste.net/show/ea9a3fd3b4f1 Mar 24 19:10:43 Just had dinner ;) Mar 24 19:20:18 Herrie: yes, like that; unfortunately, when doing gdb directly on the device, you don't have the qtwebengine debug symbols Mar 24 19:20:51 There are two ways to get the debug symbols, if you are interested Mar 24 19:22:10 Herrie: here are my notes about how to do it https://bpaste.net/show/b16349253f3d Mar 24 19:49:33 Tofe: Can try Mar 24 19:49:45 But might be easier to debug PulseAudio? Might have same cause Mar 24 19:50:21 the bootloop you mean? Mar 24 19:50:38 Tofe: No PulseAudio in Jethro has similar error as lunawebappmgr Mar 24 19:50:48 JaMa said it might be toolchain related Mar 24 19:50:56 ok Mar 24 19:50:58 Which could also be the cause of systemd eventually Mar 24 19:51:06 Or not of course Mar 24 19:53:04 Tofe: you can also unpack all -dbg packages on your build machine and then debug interactively with remote debugger Mar 24 19:53:19 ah you do in 2nd part :) Mar 24 19:53:40 with newer yocto even the sysroot files are stripped Mar 24 19:53:52 so unpacking -dbg packages is way to go for debug symbols Mar 24 19:54:13 ok, good to know Mar 24 19:58:03 JaMa: New unstable build, any use to test? Mar 24 20:03:14 Herrie: no big changes, I've just updated oe-core a bit and rebased jethro branches to have latest changes you did in fido Mar 24 20:14:06 JaMa: OK I'll test anyway **** ENDING LOGGING AT Fri Mar 25 02:59:59 2016