**** BEGIN LOGGING AT Tue Mar 30 02:59:56 2021 Mar 30 05:00:10 added a project idea for beaglewire on the project ideas page Mar 30 05:00:58 under a new and most awesome "FPGA based projects" section Mar 30 05:02:25 if there is a brave soul I would even consider creating new hardware like we did for the beaglewire project Mar 30 05:03:44 I have been toying with the idea of a ECP5 based FPGA cape for a while Mar 30 05:22:02 I’m in for ECP5+Beagle, whether within or out of GSoC Mar 30 05:23:07 Abhishek[m]_, I could also revive the Akita project Mar 30 05:23:13 Ah yes Mar 30 05:23:20 I remember that one Mar 30 05:23:25 which was ECP5+Octavo Mar 30 05:23:44 Let’s start with cape first Mar 30 05:23:57 We could design and get HW ready Mar 30 05:24:25 okay, lets do this Mar 30 05:24:36 And focus on SW gateware support on for GSoC Mar 30 05:24:52 indeed Mar 30 05:25:05 that is what we did for beaglewire Mar 30 05:25:13 it worked quite well Mar 30 05:25:29 I have a HaD badge or we could potentially get the student a BBAI + ECP5 board like the Radiona one Mar 30 05:27:38 yeah for newer boards we will have to ditch the GPMC interface Mar 30 05:28:02 since it is not broken out Mar 30 05:28:17 I’ve always toyed with the idea of PRU+ECP5 or iCE40 as a potential BeagleLogic extension Mar 30 05:28:32 Never implemented or built anything though Mar 30 05:32:42 I find the PRU just gets in the way of an FPGA Mar 30 05:33:09 but many a buffered bus like interface could work Mar 30 05:44:41 PRU for the data path, maybe control path can be separate Mar 30 05:45:28 Something like a register based SPI soft core on the FPGA side Mar 30 05:45:37 PRU for fast data path Mar 30 05:48:48 and JTAG/SPI for bitstream load from ARM core Mar 30 05:50:11 will think about it a bit Mar 30 05:52:39 Yep, let’s chat in the evening today my timr Mar 30 05:52:49 *time Mar 30 05:53:41 For the students - if any of the terminology above in the conversation is new, please feel free to look up the terms Mar 30 05:57:38 :) Mar 30 07:19:28 Count me in also for cape design :) Mar 30 07:21:13 m_w: create a channel on your discord for this ? Mar 30 08:14:03 I understand that ECP5 is open source, as opposed to Xillinx/Intel and that is why it is a tempting addition? Mar 30 08:14:53 Weird, still cannot see messages I sent from native client... Mar 30 08:15:52 Not the FPGA, the toolchain Mar 30 08:16:09 And what are Zephyr related projects btw? Mar 30 08:16:25 Think those are for BeagleV? Mar 30 08:16:37 Or the RISC-V core? Mar 30 08:16:49 Right, recently I read that Xillinx was opening up parts of its toolchain, HLS + LLVM backend Mar 30 08:16:56 oh, okay Mar 30 08:16:59 Yeah, I’ve too Mar 30 08:17:21 Zephyr is an open source RTOS backed by the Linux Foundation Mar 30 08:18:04 Never tried it myself yet Mar 30 08:20:11 It’s interesting, but for an OSS flow you’d also need the FPGA bitstream format, block mapping, timing and stuff which is proprietary at the moment Mar 30 08:20:29 There are reverse engineering efforts underway for Artix-7 Mar 30 08:20:53 Project X-Ray under SymbiFlow if anyone’s interested Mar 30 08:20:57 Yeah of course, was often bottlenecked by the `free` versions of the software family, especially Intel Quartus Mar 30 08:21:35 I saw symbiflow's GSoC tasks but they were very toolchain specific and I got scared :D Mar 30 08:22:07 There’s no WASM support for ARMv7 otherwise you could theoretically compile bitstreams for ECP5 on the Beagle itself Mar 30 08:22:59 Or cross/compile the OSS toolchain using arm gcc - not sure how portable the Yosys/Nextpnr stack is Mar 30 08:24:12 Don’t be! You’d get to learn if you have the interest, and you’ll have mentors Mar 30 08:26:19 It’s a huge upramp, yes. Mar 30 08:28:31 In any case it is good to be having open source effort with FPGAs since they are great for learning hardware programming and quick reconfigurable acceleration Mar 30 08:28:53 Maybe offtoping slightly though Mar 30 08:28:55 Also, if you feel scared or intimidated, highly likely other students feel so as well Mar 30 08:29:06 It’s a niche skill Mar 30 08:29:29 Like when I applied for Beagle in 2014 Mar 30 08:29:55 Not many applicants that year and sincr Mar 30 08:30:33 I barely knew what PRUs and kernel modules do, tried some PRU assembly and filed my proposal for BeagleLogic Mar 30 08:31:10 In Month 2 of GSoC I built kernel for the first time and wrote my first module Mar 30 08:31:51 It’s pretty much a prerequisite now for almost every GSoC project because the selection bar goes higher every year Mar 30 08:33:14 We expect your proposals to exhibit a high degree of pre work and engagement on IRC Mar 30 08:34:23 I know nothing about compilers too, when I applied last year too 😂😂 Mar 30 08:34:27 If you get stuck, you should be able to figure your way out and get unstuck yourself (worst case). Of course we’ll be here to give pointers Mar 30 08:35:30 Oh, that is filling me with more optimism :) I am still kind of in a quandary with my proposal though. I would truly like to add something for _future_ but the removal of support for TIDL is kind of a discouragement, and as ds2 said TFLite is a pain to use Mar 30 08:35:50 But you learnt it really quickly and did a good job so :) Mar 30 08:36:06 Hence I am not sure whether I should be continuing digging around YOLO deployment or focus on something else Mar 30 08:36:39 It’s unfortunate but TFLite is more or less universal - you can run it on MCUs and DSPs and everything Mar 30 08:36:47 TIDL is proprietary Mar 30 08:37:10 Ah and also I saw some proposals from the previous year, they weren't top quality but they weren't bad as well, are there any specific criteria making the proposal outstanding? Mar 30 08:37:26 ds2: mentioned you cannot run TFLite on GPU? Mar 30 08:37:32 something of that sort Mar 30 08:37:32 yes and there's 2 months still to start the project, so don't worry jduchniewicz Mar 30 08:37:32 You’re asking proprietary information here :) Mar 30 08:37:57 haha true, so best I do my own research on the topic Mar 30 08:38:31 It is just that some were quite solid and still got rejected Mar 30 08:38:43 jduchniewicz: Link to example? Mar 30 08:38:51 I think you should have good discussion and communication with mentors, that is what matters the most. Mar 30 08:39:56 jduchniewicz: Actually TensorFlow is a full framework Mar 30 08:40:37 I answered my previous question by skimming the proposal after yesterday's discussion :) https://elinux.org/BeagleBoard/GSoC/2020Proposal/PrashantDandriyal Mar 30 08:40:48 It looks all right but is `hollow` Mar 30 08:41:03 nothing tangible included Mar 30 08:41:27 As in (I have limited working TF/ML knowledge) - but TF is a way of expressing compute operations on Tensors Mar 30 08:42:00 It doesn’t dictate “on what” you perform your actual computation Mar 30 08:42:30 It can be your CPU, or GPU or a custom DSP, or even an FPGA accelerator you design yourself Mar 30 08:43:13 As long you supply the backend implementation on some chip, a TF Model can run anywhere Mar 30 08:43:19 hmm ds2 said that darknet (YOLO backend library) is not really a good companion for TFLite Mar 30 08:43:55 I need to dig around more TFLite then and see how difficult would it be Mar 30 08:44:08 Yeah, go ahead and dig :) Mar 30 08:44:18 as the main goal of my project would be acceleration by whatever means possible Mar 30 08:44:32 I would also suggest trying to study the architecture of the TI DSPs Mar 30 08:44:38 Figure out its strengths Mar 30 08:44:51 The main point is using EVEs Mar 30 08:45:02 and DSPs as well Mar 30 08:45:11 If not YOLO, what would shine on these DSPs/EVEs? Mar 30 08:45:27 that is also a good question, thanks Mar 30 08:46:03 AFAIK YOLO is a goto ANN architecture for CV and deployment on constrained devices Mar 30 08:46:27 thank you for ideas and some room for thought! Mar 30 08:47:07 No worries! Mar 30 08:47:51 I’m looking at the EVE architecture too and have some observations, let me know when you want to compare notes or share observations Mar 30 08:48:23 Sure, do you have a link to it? Seems like there is scarcity of knowledge on this topic Mar 30 08:48:34 documentation* Mar 30 08:48:42 Start here Mar 30 08:48:58 https://www.ti.com/lit/wp/spry251/spry251.pdf?ts=1617090324980&ref_url=https%253A%252F%252Fwww.google.co.in%252F Mar 30 08:49:36 I did a search for “ti eve” and this PDF was the first result I got Mar 30 08:50:40 https://www.ti.com/lit/ug/tidueb6/tidueb6.pdf?ts=1617090616414&ref_url=https%253A%252F%252Fduckduckgo.com%252F Mar 30 08:50:52 also there is a reference design if anyone is interested Mar 30 08:53:14 This document shows step-by-step process but doesn’t talk about how the inference actually happens Mar 30 08:54:40 It just says something like “here’s our exe, train your model, we’ll convert it to something that can execute on the EVE using magic” Mar 30 08:56:01 yup, will dig into the other pdf then Mar 30 08:56:10 Happy digging :) Mar 30 09:22:52 Hmm TFlite on GPU? context? Mar 30 09:26:34 it seems that it is truly not working (at least with NVIDIA GPUs) https://stackoverflow.com/questions/57170737/cannot-run-tflite-model-on-gpu-jetson-nano-using-python Mar 30 09:39:14 One thing striking me is that TFLite effectively uses automatic partitioning and allocation of resources using TIDL under the hood, thus it seems like if TI is to ditch TIDL, they will need to provide a different backend for TFLite Mar 30 09:39:25 ds2: this might be valuable Mar 30 12:46:09 ds2: I am wondering which framework we would be likely using for the initial YOLO implemention? There are some implementations in the most popular frameworks and these are then easily translatable to TIDL Mar 30 12:46:57 ds2: As for the darknet impletentation we would need to do the translation anyway Mar 30 13:12:17 ds2: did some more research, but still some topics are unclear to me. First of all it seems like YOLO is fully deployable in the TIDL library, the only question is the upsampling layers (they are not listed in http://downloads.ti.com/processor-sdk-linux/esd/docs/latest/linux/Foundational_Components/Machine_Learning/tidl.html#neural-network-layers-supported-by-tidl) Mar 30 13:15:09 ds2: Seems like taking a model from TF and translating it to TIDL is the best idea (as we will benefit the most from various optimizations this library offers). Mar 30 13:16:50 ds2: The AM57X devices can address 4GiB of physical SDRAM space (according to https://www.ti.com/lit/ug/tidueb6/tidueb6.pdf -> the end of chapter 1.1.1) Mar 30 13:19:31 ds2: not sure though how much memory would be required during inference (would need a hand in estimating this) Mar 30 13:20:11 ds2: I have a reference implementation in TF https://itnext.io/implementing-yolo-v3-in-tensorflow-tf-slim-c3c55ff59dbe Mar 30 13:21:33 ds2: Lastly, it seems like EVEs favour sparse models so it could be retrained in order to be sparse. Also some optimizations are mentioned along the lines, including grouped convolutions and depth-separable convolutions Mar 30 13:23:12 ds2: So I have much more information but still not enough to create a complete proposal, with planning etc. It looks like using the EVEs in conjunction with DSPs may be much more complex than with EVEs or DSPs only, but that is probably the fun part! Mar 30 15:23:29 Good to see progress, jduchniewicz Mar 30 15:32:05 Abhishek regarding beagle config, I was thinking maybe from hosts side we can use the hotplug events to load a kernel module that can enable an additional interface to share internet. Mar 30 15:32:05 Like after plugging in pocketbeagle, on usb host's side a beagle-config host event will trigger to ask user to enable forwarding packets Mar 30 15:32:52 I’d like the cross platform experience to be as seamless as possible Mar 30 15:33:12 (Also, I couldn’t take a look at your doc, will get back later) Mar 30 15:33:30 I have some Qt experience Mar 30 15:33:36 There should ideally be no driver install required Mar 30 15:33:44 Or even any software Mar 30 15:34:52 This is easy on Linux, but tricky on Windows Mar 30 15:35:10 That's true🤔 Mar 30 15:35:43 A helper utility on Windows side could be useful Mar 30 15:35:59 For example, be able to enable ICS automatically Mar 30 15:36:31 On Linux, execute iptables and enable IPv4 forwarding Mar 30 15:36:52 ICS - internet connection sharing Mar 30 15:37:38 Learn about how the current USB Composite descriptors are Mar 30 15:37:49 Yes. That's what i had proposed but i didn't mention anything on the hosts side in the proposal as i am not sure of the timeline that'll take longer maybe Mar 30 15:38:24 Give your best estimate Mar 30 15:38:33 Yes, I'll do that for windows especially Mar 30 15:39:05 Let me dig more about windows Mar 30 15:39:35 Okay Mar 30 15:45:29 It's possible through cmd definitely Mar 30 15:46:23 I believe it will become a complete 1 click experience Mar 30 15:47:56 I will devise a new timeline Mar 30 15:48:07 In the docs too Mar 30 15:48:10 Okay Mar 30 16:11:37 I wouldn't mind using electron js if Qt could cause license issues Mar 30 16:28:16 use ImGUI, or wxwidgets Mar 30 16:28:41 using electron is adding a lot of dependency, also the size is pretty bjg Mar 30 16:30:29 Qt is LGPL afair, should not cause license issues, since this would be an open source project anyway Mar 30 16:34:39 Those are fair statements. But what's faster in development ? Mar 30 16:35:17 Qt docs are nice but mehh Mar 30 16:41:40 I would not use ImGui for anything else than debug widgets when we dont care about performance Mar 30 16:42:01 from my experience it is quite costly Mar 30 17:23:33 oh, I found it easy to use though Mar 30 17:27:43 yeah it is very simple :) Mar 30 17:30:09 Hello students! Hello mentors! Mar 30 17:40:16 Hi cwicks Mar 30 17:44:18 Hi Mar 30 17:55:23 jduchniewicz1: framework yes. but which ones are accelerated? the ARM core does not cut it Mar 30 17:56:25 jduchniewicz1: no retraining, please. Mar 30 18:00:58 Hello students. Do you have any questions about filling out the template? Mar 30 18:04:09 jduchniewicz1: I think a requirement is - take the config and the weights that work in darknet and have a process to run them on the AI or x15 Mar 30 18:25:47 Not right now. Mar 30 18:34:03 is the mentor Steve Arnold in this chat ? Mar 30 18:35:07 Do we have any project groups for Privacy and Security on the beagle bone right now ? Mar 30 19:59:49 @ds2 which ones? what do you mean by that? Mar 30 20:00:09 ds2: no retraining then :) that is just what TI recommends Mar 30 20:01:44 ds2: I think it should be checked whether translation from darknet to TF to TIDL would be better than taking an implementation in TF and feeding TIDL with it Mar 30 20:01:56 anyone having trouble with the new bbb.io/gsocml ? Mar 30 20:04:05 ds2: the last issue is as I wrote above, not knowing if the model will fit in the RAM during inference Mar 30 20:04:35 ds2: and if these two issues are addressed I can proceed with the proposal Mar 30 20:11:27 VaishnavMA[m]: any chance you'd be a mentor this year? I hope to attract someone to help with BeagleConnect (Zephyr, wpanusb, greybus, mikrobus), though there is little out there to attract them. :-/ Mar 30 20:12:47 looks like https://elinux.org/Category:GSoCProposal2021 currently has 3.5 proposals, kinda. Mar 30 20:13:24 er, 2.5. Just realized one of them was the Template. Mar 30 20:13:46 * jkridner really wants to see people to use that template before submitting to Google. :-( Mar 30 20:51:26 JAYANTBENJAMIN[m: i'm here Mar 30 20:54:18 JAYANTBENJAMIN[m: feel free to add some details to your forum post Mar 30 20:54:32 or just ask away in here... Mar 30 21:06:51 test guest login on chat.beagleboard.org Mar 30 21:07:06 odd there is no history. Mar 30 21:08:34 ok, link at bbb.io/gsocchat changed to chat.beagleboard.org Mar 30 21:08:45 still using matrix.org bridge. :-( Mar 30 21:11:56 * jkridner redirected bbb.io/gsoc to the forum page. Mar 30 21:24:07 https://elinux.org/BeagleBoard/GSoC/2021_Proposal/beagle_config Mar 30 22:06:57 > <@satacker:matrix.org> https://elinux.org/BeagleBoard/GSoC/2021_Proposal/beagle_config Mar 30 22:06:57 > Do I need to move it on another page ? Mar 30 22:06:57 I don’t know if you need to move that one, but add the category tag. Mar 30 22:07:29 I search using https://elinux.org/Category:GSoCProposal2021 Mar 30 22:59:16 ds2: while researching regarding the project i came across hardware based volume rendering , can we include it in any way in the project OpenGLES acceleration for DL.It has massive scope in the medical domain. Mar 30 22:59:20 * ds2: while researching regarding the project i came across hardware based volume rendering , can we include it in any way in the project OpenGLES acceleration for DL. It has massive scope in the medical domain. Mar 30 23:32:33 possibly a better answer for the web interface: https://webchat.freenode.net/ Mar 30 23:32:56 they even prettied it up recently Mar 30 23:34:40 ds2: I think compute shaders must be used to perform the computation required.Also can you please explain how can the Darknet CNN Framework be possibly used for this? Since you have stated it in the project description. Mar 30 23:43:10 https://web.engr.oregonstate.edu/~mjb/cs519/Handouts/compute.shader.1pp.pdf Mar 30 23:43:54 This describes more or less all the aspects of the compute shader Mar 31 00:01:25 ds2: With respect to performing convolutions i came across the EXT_convolution—The Convolution Extension that allows filtering of images by convolving the pixel values in a one or two dimensional image with a convolution kernel. Mar 31 00:04:28 ds2: It is given here - http://www-f9.ijs.si/~matevz/docs/007-2392-003/sgi_html/ch10.html along with various other extensions and functions that can be used for various applications. Mar 31 00:07:57 ds2: Also regarding the timing and efficiency,I am searching a lot but there is nothing with respect to the Dual-core PowerVR SGX544 3D GPU on the BB AI,or any similar gpu of that range.Most of the stuff I am getting is with respect to android devices. Mar 31 00:38:32 * ds2: With respect to performing convolutions i came across the EXT_convolution—The Convolution Extension allows filtering of images by convolving the pixel values in a one or two dimensional image with a convolution kernel. Mar 31 00:52:17 jduchniewicz1: I don't have enough visiblity to say if it will fit. The ARM only version does run on the Pi and Jetson though **** ENDING LOGGING AT Wed Mar 31 02:59:56 2021