**** BEGIN LOGGING AT Fri Apr 29 02:59:58 2016 Apr 29 14:23:39 Good morning everyone! Apr 29 16:43:35 I am -><- this close to screaming at the UEFI code Apr 29 16:43:49 I hate doing first-article bring-up of new chips sometimes Apr 29 16:44:01 Booting can be a real pain in the ass Apr 29 16:46:45 Anyone here have extensive experience with UEFI and arm64? Apr 29 16:50:44 Martyn: I have some Apr 29 16:50:53 HEY! You're online! Apr 29 16:51:35 Martyn: what trouble are you having? Apr 29 16:52:08 okay, so I am trying to bring up on a Juno board Apr 29 16:52:29 Those have upstream support in EDK2, so what exactly are you trying to do? Apr 29 16:53:36 I'm trying to boot over PCIe Apr 29 16:53:50 so I need to initialize the PCIe bus early, in UEFI Apr 29 16:54:17 ( this is on the A57 ) Apr 29 16:55:14 However, once I do that initialization and copy/unpack the kernel into a bootable memory address -- The kernel fails to boot as -it- tries to re-initialize it Apr 29 16:56:44 Martyn: which Juno revision? Apr 29 16:57:20 r2 Apr 29 16:57:31 r2p0 Apr 29 16:58:25 oh wait -- that's wrong Apr 29 16:58:35 I have Cortex A57 - r1p1 Apr 29 16:58:45 and the a53 is r0p3 Apr 29 16:58:55 so it's juno r1 Apr 29 16:59:43 Hmm, that _should_ work. Which FW, which kernel, and are you using ACPI or DT? Apr 29 17:00:36 I'm using devicetree, kernel 4.6-rc5 ( current mainline ), and unsure which firmware Apr 29 17:01:32 Also, I've tested booting from SATA just to make sure that I'm not having some weird XAUI issue, and I can boot from SATA just fine Apr 29 17:01:39 also I can boot from 10GigE ( also XAUI ) Apr 29 17:02:19 Martyn: So which PCIe device(s) do you have plugged in in the failing case? Apr 29 17:02:37 but when I try to cluster boot over PCIe .. somehow initializing the PCIe bus early results in a deadlock condition once the mainline kernel starts to boot. I don't have a 1wire debug available, or KEIL debug over JTAG, so I'm kind of stuck Apr 29 17:03:10 Martyn: I was under the impression that the SATA was over PCIe, so what do you mean by "boot over PCIe"? Which device? Apr 29 17:03:19 just a PCIe debug board I built. I'm clocking the data in, and all the data is well ordered Apr 29 17:04:28 I'm trying to do something San Mehat did back at VA Linux Systems back in the early 2000's ... I am putting together a TCP/IP stack over PCIe, so that I can use one a64 card to boot another directly. Apr 29 17:05:05 Martyn: given that SATA works (over PCIe), the issue sounds specific to your debug board; I'm not sdure I can be of much help. :/ Apr 29 17:06:03 so the board that's plugged into the PCIe bus is fairly simple -- a SPEAr 1340 w/ single channel PCIe 2.0 RC/EP port Apr 29 17:06:34 and all it does it clock in the data -- the kernel does load correctly into memory, and I have uncompressed it into RAM and can even start booting it Apr 29 17:07:06 the -problem- is that the entire PCIe bus locks up right afterwards. I don't think the PCIe bus likes being re-initialized by when the kernel reboots Apr 29 17:07:16 reboots/boots Apr 29 17:07:47 Is there a way to prevent the kernel from attempting to re-initialize the PCIe bus, given that I've already done it during UEFI? Apr 29 17:12:25 oh .. interesting. I just tried to checkout a fresh copy of EDK2 -- issues with git Apr 29 17:12:27 warning: remote HEAD refers to nonexistent ref, unable to checkout. Apr 29 17:12:39 Martyn: I don't follow why that would only be a problem when your board is plugged in? Surely it re-initialises it anyway? Apr 29 17:13:16 That's what I was thinking... PCIe has to be initialized to get SATA working, so I figured this would be safe as houses Apr 29 17:13:41 however, I'm not doing anything significantly more complex than what happens during SATA boot Apr 29 17:14:21 I am copying data from the card I built, the data copies in, and as soon as the kernel does a PCIe probe -- BANG Apr 29 17:14:27 deadlock Apr 29 17:15:58 Bah .. I need to get JTAG on this Apr 29 17:19:05 It may be worth asking on the linux-arm-kernel mailing list; this sounds like a PCIe issue, and people there are more familiar with the ARM/ARM64 details for that than I am Apr 29 17:19:55 Yeah. With the A57/v8 chips starting to get a lot more popular out there ... I figured that y'all might be doing some unusual cluster booting techniques like I am here Apr 29 17:19:57 Will do Apr 29 17:20:02 LKML can be ... contentious :) Apr 29 17:20:17 LAKML is also .. contentious :) Apr 29 17:22:07 I boot my Juno with GRUB and TFTP, which works well enough for me Apr 29 17:23:23 There are some nice people on LAKML (and LKML), if you can ignore the noise Apr 29 17:25:27 Heh .. this work doesn't end with the Juno, of course. just like with Smooth-Stone/Calxeda -- early steps Apr 29 17:25:42 Well, for now I'll stick to booting on 10gigE Apr 29 17:25:51 But I need to crack this PCIe issue Apr 29 19:03:31 mrutland : GAH! solved it. Apr 29 19:04:35 mrutland : It was an out-of-order transaction happening on the PCIe bus... for some weird reason, the PCIe IP on the Juno isn't allowing it. Apr 29 19:05:14 IS there a limitation on the a57's architecture that breaks when there are out-of-order transactions on PCIe? ( like there was on the A9's? ) Apr 29 19:06:05 Is relaxed ordering allowed? Apr 29 19:06:46 ( If I turn off the relaxed ordering bit in the TLP, things work ) Apr 29 19:07:04 that would be bad news for things like 8 and 16 lane graphics cards... **** ENDING LOGGING AT Sat Apr 30 02:59:58 2016