**** BEGIN LOGGING AT Wed Mar 18 02:59:58 2020 Mar 18 11:42:51 I have chosen the vendor specific solution (what ds2 didn't prefer) ...by opting to use TIDL instead of leveraging an acceleration library like NNPack. I felt that choosing such a method would mean that the acclerators (on-board) wouldn't necessarily be used. This is beacause the short duration would not allow me to modify both : the NNPack for BB + use accelerators with or without TIDL Lib .. Mar 18 11:43:43 Can anyone suggest some compromise ? Mar 18 11:59:17 * pratimugale[m] sent a long message: < https://matrix.org/_matrix/media/r0/download/matrix.org/gBFWwcKMpBvhmnJCCoQknouJ > Mar 18 12:47:52 * pradan[m] sent a long message: < https://matrix.org/_matrix/media/r0/download/matrix.org/oWVosBcqwlrMHcZXrGISWQTX > Mar 18 16:30:19 who all is here? Mar 18 16:30:28 our weekly meeting is starting. Mar 18 16:30:36 ping cwicks Mar 18 16:30:43 ping pdp7 Mar 18 16:31:05 howdy lorforlinux ! Hope my reply was useful. Mar 18 16:31:19 * hendersa waves. Mar 18 16:32:19 final proposals are due by the 31st. Mar 18 16:32:43 a little less than 2 weeks. Mar 18 16:33:28 Hi Mar 18 16:34:25 one proposal has been submitted to the Google system and 4 are up on elinux: https://elinux.org/Category:GSoCProposal2020 Mar 18 16:36:50 me0w Mar 18 16:37:33 hi ds2 Mar 18 16:38:11 lorforlinux as a student applying for the first time this year, why do you think so few students are engaging with us this year? Mar 18 16:38:41 also, please avoid private/direct messages with me. they get me in a bad mood. Mar 18 16:41:13 buildroot is awesome.... great cross-building platform that is easy to understand. Mar 18 16:41:20 Hi everyone Mar 18 16:41:22 not as flexible as Yocto Project, but way easier to build. Mar 18 16:41:30 howdy Abhishek_ ... Mar 18 16:43:47 * pratimugale[m] sent a long message: < https://matrix.org/_matrix/media/r0/download/matrix.org/oipPebARjFePVMvbltElzuJe > Mar 18 16:44:49 pratimugale You might avoid the long quotes that prevent non-Riot users from seeing the text. Mar 18 16:44:56 I think TIDL is meant to be used. Mar 18 16:45:28 lorforlinux: :-D Mar 18 16:45:44 jkridner: Ok Mar 18 16:46:52 so, attendees include hendersa ds2 pratimugale Abhishek_ and lorforlinux Mar 18 16:47:18 and me, so 5 mentors and 1 student. nice ratio! :-D Mar 18 16:47:53 pradan: are you still around for this meeting slot? Mar 18 16:48:05 Hello everyone! Mar 18 16:49:03 howdy ymdatta ! Mar 18 16:49:12 better late than never. Mar 18 16:49:59 lorforlinux: was my question about why you don't think more students are here seen? anyway I should rephrase it? Mar 18 16:50:21 My college was in lockdown and i have been travelling, so I wasn't able to ask things in past days. Mar 18 16:50:58 lorforlinux: do you have a Linux desktop computer? You should be able to install dtc and the Linaro toolchain for cross-compilation. Your pull-request task should show you know how to cross-compile. Mar 18 16:51:13 * jkridner[m] checks the pull-requests. Mar 18 16:52:02 I have submitted my PR Mar 18 16:52:47 * jkridner[m] checks out https://github.com/jadonk/gsoc-application/pull/130 Mar 18 16:53:32 ugh.... more examples of matrix/freenode bridge breakage. Mar 18 16:54:02 lorforlinux[m] doesn't even show up here. Mar 18 16:54:11 for students that have talked to here before - be sure to upload a final app by the deadline Mar 18 16:54:34 jkridner (@freenode_jkridner:matrix.org): any suggestions on breaking up the mikroBus driver project into small tasks? Mar 18 16:54:46 ds2: any idea why lorforlinux[m] doesn't show up? Mar 18 16:55:49 ymdatta: some ideas, but I know I need to document a lot of what needs to be done for that task. we've started bi-weekly calls with Mikroelektronika to educate them a bit on what needs to be done for this bus driver. Mar 18 16:56:03 gateway may be slow... last week it took almost 10 minutes Mar 18 16:56:37 some of what needs to be broken down is validation of sufficient platform data to be able to instantiate more device drivers over different bus drivers. Mar 18 16:56:57 lorforlinux: yes, but your messages aren't showing up on freenode for some reason. Mar 18 16:57:24 I try to say logged into both so I can see what is going on. Mar 18 16:57:45 pradan: Does the first proposal (featherCNN one) involve the use of onboard accelerators? There is a weekly meeting on Wednesdays 10pm IST Mar 18 16:57:45 ymdatta: starting with iio drivers over i2c/spi would be one of the early phases. Mar 18 16:58:08 I’d much rather prefer IRCCloud if using IRC. Not very happy with Riot/Matrix but I don’t want to be skipping messages when they (IRCCloud) log me off Mar 18 16:58:10 here I am.. Mar 18 16:59:14 ymdatta: https://github.com/vaishnav98/gbsim/wiki left some clues for a starting point. Mar 18 16:59:32 * jkridner[m] wishes vaishnav98_ was around this year. Mar 18 16:59:52 jkridner (@jkridner:matrix.org): I went through vaishnavs project Mar 18 17:00:57 10min matrix delay seems about right, just saw pradan[m]'s message Mar 18 17:01:00 https://github.com/vaishnav98/mikrobus_device/blob/master/mikrobus_i2c_device.c was rather a hack. Mar 18 17:01:16 good for him at the time as this wasn't the problem he was originally asked to solve. Mar 18 17:01:36 but one that we only understood as he got into the execution of his project. Mar 18 17:01:53 jkridner (@jkridner:matrix.org): from what we discussed, data present in the manifest files would be enough Mar 18 17:02:02 all makes a lot of sense in retrospect, though I still need to better document what needs to be done. Mar 18 17:02:10 we don't want to ultimately put it in the manifest files..... Mar 18 17:02:27 But the problem is how do we parse the data once we get it thorough eeprom Mar 18 17:02:35 we want to tackle it a bit differently, a bit more like cape bus, but not needing any device tree overlays. Mar 18 17:03:48 Thank you for letting me know.. I am still going through the initial messages. Feather CNN is in itself, a big task, to port for the BB AI. So, I don't think using accelerators would be possible in the timeframe Mar 18 17:04:22 this table (https://github.com/vaishnav98/mikrobus_device/blob/master/mikrobus_i2c_device.c#L57) shouldn't necessarily need the IRQ number since that should be defined by the mikroBus instance. Mar 18 17:04:24 jkridner: yes, since in that case we only need to read an identifer from EEPROM and we would get the overlay from firmware, here we are planning to read entire platform data from EEPROM, so we need to parse it (so we need a protocol). Mar 18 17:04:45 * ymdatta i actually asked this question before, didn't get any replies Mar 18 17:06:01 by that i mean cape bus Mar 18 17:07:08 I am using Riot and everything works fine till now Mar 18 17:07:20 ymdatta: vaishnav98_ made these module parameters. I think that's fine, but for mainline, they'd likely need to be defined through DT... but that would be a DT definition for the mikroBus slot, not for the Click boards connected to it. Mar 18 17:07:43 pradan: you are showing up. for some reason, lorforlinux isn't showing up on freenode. Mar 18 17:08:17 wonder if freenode is throttling connections from matrix Mar 18 17:09:09 pradan[m]: do you have drafts of the apps ready? Mar 18 17:09:17 ds2: maybe. your last message is also slow to show up on matrix. Mar 18 17:09:57 jkridner: right. We need something for the clickboard's data. Once we read from EEPROM, we need to parse it to extract platform data? Or is my line of though wrong? and is there any other way? Mar 18 17:10:00 freenode and matrix just continue to make my life harder.... but I can't quite stomach going closed source with Slack or Discord. Mar 18 17:10:38 xmpp and find a gateway? :D Mar 18 17:11:20 ymdatta: the EEPROM might have nothing more than a board name/revision, though we could try to ask for more. any additional data required for the platform data structure should be implicit where at all possible. Mar 18 17:11:21 I don't know why that happens. Me and lorforlinux tried to replicate my setup on Riot, but to no success Mar 18 17:11:21 ds2: Yes, I fininshed with the second app this morning and added the link here.... https://elinux.org/BeagleBoard/GSoC/2020Proposal/PrashantDandriyal_2 Mar 18 17:11:47 ymdatta: potentially, I could see incoding it as a manifest binary. Mar 18 17:12:03 pradan[m]: I don't see anything on the official gsoc site yet Mar 18 17:12:36 ymdatta: I just want to avoid putting in parameters like the GPIO to use for interrupts or reset when it is directly defined in the mikroBus spec. Mar 18 17:13:06 ...and our goal is to get as many working Clicks as possible with as little coding effort or fragility as possible. Mar 18 17:13:31 ds2: I haven't submitted to gsoc till now. I haven't got any feedbacks yet Mar 18 17:13:37 jkridner: yeah, i understand what you mean. You don't want them as module parameters but something which comes from EEPROM Mar 18 17:14:28 ymdatta: anyway, after iio (I2C/SPI), we'd want to expand to tinydrm and other devices that could be implemented across mikroBus. There is now a serial bus driver, which we should use as well. I wonder what that will do to how GNSS is typically consumed, if anything at all. Mar 18 17:14:34 ds2: did you see this ? Mar 18 17:14:58 pradan[m]: no Mar 18 17:16:00 > <@pradan:matrix.org> I have chosen the vendor specific solution (what ds2 didn't prefer) ...by opting to use TIDL instead of leveraging an acceleration library like NNPack. I felt that choosing such a method would mean that the acclerators (on-board) wouldn't necessarily be used. This is beacause the short duration would not allow me to modify both : the NNPack for BB + use accelerators with or without TIDL Lib .. Mar 18 17:16:00 * pratimugale: ds2: did you see this ? Mar 18 17:16:08 pradan[m]: mobilenet is another model... so Collect mobilenet and Caffe versions of YOLO v2 models (may stick with just one if it works fine... does not make sense Mar 18 17:17:05 pradan[m]: nope...just saw it now... the only concern I have for that is a support plan if you get stuck.... TIDL is well.. very TI specific and.... Mar 18 17:18:06 I fear that almost half of my messages never reach up to my mentor... Mar 18 17:18:21 Oh sorry, I meant to refer MobileNet-YOLO Caffe as mentioned here: https://github.com/eric612/MobileNet-YOLO Mar 18 17:18:58 pradan[m]: probally true... I don't see much from you in my logs Mar 18 17:19:25 lorforlinux pradan ymdatta as prospective students, can you provide me with some feedback why you think other students might not be showing up? Were you almost turned-away somehow by how we are positioned? Mar 18 17:20:06 pradan[m]: i think that is yolo trained with the mobilenet data? Mar 18 17:24:36 jkridner: In the starting days, many students came (my friend was also interested), but since there was no activity..many left in searching for other orgs Mar 18 17:25:17 jkridner: hello, I had a conflict earlier Mar 18 17:25:17 From what i remember till the first week of this month, there was no discussion about projects on the channel. I tried posting few times, but there was no reply. Mar 18 17:26:34 hi pdp7 ! Mar 18 17:26:35 jkridner: those were some of the reasons i found why many people weren't discussing Mar 18 17:26:54 * jkridner[m] just had a cgroups fail and needed to switch machines. :angry: Mar 18 17:27:42 ymdatta: k, so responding quickly to the firsts posts might encourage others to post? Mar 18 17:29:42 maybe closed school == homeless, no internet access? Mar 18 17:30:51 ds2: I experienced another glictch in riot... caught up now Mar 18 17:32:08 Yeah, Caffe perhaps. just noticed now Mar 18 17:33:08 pradan[m]: how are you planning for support w/TIDL? Mar 18 17:34:32 The part of TIDL that worries me is the retraining YOLO model using the Caffe-Jacinto framework Mar 18 17:35:15 Using the accelerators has good docs and e2e is quite responsive too ds2 Mar 18 17:35:51 That's the concern I share too. TIDL is TI specific but guarantees that I can leverage the accelerators.... but the NNPack+accelerator approach demands work in both the NNPack porting as well as writing OpenCL backend. I sent those messages to seek help on comparing both the approaches. But still doubtful if the messages even got sent .... ds2 pratimugale Mar 18 17:36:09 pradan ds2 Can we use something like darkflow to convert the YOLO model to tensorflow? Mar 18 17:36:26 or similar conversion of the model to caffe? Mar 18 17:36:28 pradan[m]: there are conversion tools Mar 18 17:36:59 pratimugale[m]: I think so...there are quite a few tools... I got things converted to caffe before but never tried it as I got stuck dealing with the TIDL config (txt) files Mar 18 17:36:59 Whatever library we use (if we do), it seems very bad to not use the 4 accelerators Mar 18 17:37:33 TIDL is just one way to use the DSP Mar 18 17:37:43 you can also use OpenCL (if you can figure it out) on the DSP Mar 18 17:37:48 EVE is trickier Mar 18 17:38:26 NNpack is mostly just ARM NEON plus a change in strategy - instead of just bruteforce convolution, transform to another space and multiple Mar 18 17:38:28 ds2: Ohh Mar 18 17:39:03 I am ready to learn OpenCL if mentors agree with the NNPack(or any good lib) + accelerator (using OpenCL and not TIDL) Mar 18 17:39:44 ds2: You mean to implement the math behinf NNPack from scratch ? Mar 18 17:40:51 pradan[m]: no...just pointing it what it NNpack boils down to... if you can run all the layers on EVE/DSP, then NNpack doesn't matter...but for anything on the ARM (or if DSP/EVE gets overwelmed)... Mar 18 17:41:04 Also, the If I recall, NEON is provided with the BB black and not BB AI Mar 18 17:41:10 TIDL is very attactive Mar 18 17:41:40 no, NEON is on the AI... it is the ARM version of MMX/SSE Mar 18 17:41:54 vector math extensions Mar 18 17:42:04 ds2: attactive? Mar 18 17:42:08 attractive? Mar 18 17:42:34 or however it is spelled... just wondering if you are saying it looks like the way to go or not. Mar 18 17:44:50 attractive Mar 18 17:44:55 bad typo Mar 18 17:45:34 putting it another way... EVE/DSP accel on TIDL seems very enticing but I am worried about support when we (and we will) find issues Mar 18 17:45:50 E2E can be fast but I have seen it take a long time for hard issues Mar 18 17:46:08 To mention a use case of overload distribution Mar 18 17:46:08 > All layers except the detection layer were placed in EVEs. Floating-point operations are required for the ArgMax layer which is faster in Mar 18 17:46:08 DSP compared to EVE. Mar 18 17:46:19 E2E seems to be a polished "business" response vs engineers tossing out ideas Mar 18 17:47:59 yes, I feel the same. Mar 18 17:50:27 pradan[m]: I Haven't updated my tree yet (I have mods) - has the TIDL stuff exposed/explains/or demonstrated how to run just 1 or a few layers on EVE/DSP? Mar 18 17:50:36 lorforlinux pradan thanks for having the patience to wait for folks to engage. I think #beagle is much more active, but very much less focused on people new to beagle and gsoc. Mar 18 17:51:02 ...and ymdatta Mar 18 17:52:07 jkridner (@jkridner:matrix.org): Not a problem, since we are talking now Mar 18 17:52:58 jkridner (@jkridner:matrix.org): so regarding project, we can divide it into two parts Mar 18 17:53:40 1. A parser to parse data received Mar 18 17:53:40 2. Filling the parsed data into platform specific data structures. Mar 18 17:58:25 ds2: Mar 18 17:58:25 >TIDL API supports running as many different network models concurrently as there are TIDL accelerators Mar 18 17:58:25 available on the AM57x device. The example can be seen here: https://www.ti.com/lit/ug/tidueb6/tidueb6.pdf Mar 18 18:00:42 On distributing layers, they suggest last 2-3 layers to be executed on DSP, especially if they involve FP32 calculations (like SoftMax). Mar 18 18:01:11 The above link was to a guide co-authored by manisha.agrawal Mar 18 18:02:12 pradan[m]: I know what they say... but what is it in reality Mar 18 18:03:57 All the resources that I will be working with (NNPack, TIDL, etc) are new for me. So, all I have is the docs and demos (sometimes). Mar 18 18:05:22 pradan: Regarding milestone #6, aren't there 4 EVEs on the AI? So ds2 Can we use all 6(4EVEs and 2 DSPs), the TIDL website has demonstrated an example using 2 DSP and 2EVE on AM5749 Mar 18 18:06:31 If the new things make me and the mentors doubtful, I would rather go with something not new for the mentors Mar 18 18:07:34 Yes pratimugale the milestone got messed up as I mentioned about the AM5749 mistakenly Mar 18 18:08:39 Ok Mar 18 18:08:47 np Mar 18 18:09:02 ds2: using OpenCL for BB AI would again be vendor specific... please correct me if wrong Mar 18 18:10:34 pradan[m]: sort of.... OpenCL is a standard. Problem is OpenCL con most other hardware is just for the GPU Mar 18 18:11:16 it would be so much easier if @#$#@$!@$!@#@!$!@%$@!# the SGX folks would just provide their OpenCL libs instead of forcing everyone to dance around Mar 18 18:11:42 or if TI used a different GPU.... but things are written in stone so that's all moot Mar 18 18:11:58 well... maybe sand or glass Mar 18 18:12:24 (side note, this part makes the IMX look good :D) Mar 18 18:13:26 I hope you did notice that I also planned to re-train the YOLO model using the framework by TI... I know it may ruin your mood Mar 18 18:13:37 why ? Mar 18 18:14:09 and do you have equipment to complete training within the GSoC timeframe? Mar 18 18:14:13 "why" on runining the mood or on re-training the model? Mar 18 18:14:22 why do you need/want to retrain? Mar 18 18:14:56 that all to compress the model by 80 % sparse-ification Mar 18 18:15:22 So, a faster inference, with some drop in accuracy Mar 18 18:15:37 and do you have equipment to do it? Mar 18 18:16:19 * pradan[m] uploaded an image: sitara-processor-comparison-table.png (27KB) < https://matrix.org/_matrix/media/r0/download/matrix.org/wpNwEQSINmulBwfeyFXhHvAH > Mar 18 18:16:32 I hope this explains the need/benefit Mar 18 18:16:52 I plan to use cloud tools Mar 18 18:17:11 pradan[m]: do you have money to pay for the cloud tools? GPU hours are not cheap Mar 18 18:18:11 pradan[m]: put it another way... do you have an estimate of how long training will take? Mar 18 18:18:30 Nope. But the free resources can be fine if we drop the number of classes from 1000 to <100.... I know I sound stupid Mar 18 18:19:24 i would strongly advise against retraining Mar 18 18:20:06 yes it can be done but if you are changing the number of classes, etc; then all of a sudden you are not quite doing an apples to apples comparism Mar 18 18:20:27 it maybe an apples to pears comparism so not as bad as apples to oranges Mar 18 18:20:29 * apart from the subject of conversation Mar 18 18:20:29 I am doing what is **difficult** yet possible... Mar 18 18:20:29 * Now returning to the topic Mar 18 18:24:42 that's my comments on training... I don't see any convincing argument for doing it and doing it in the timeframe Mar 18 18:26:45 This was difficult to engulf but i feel you meant that there will be a rise in generalization of detected objects and distinct objects may not be detected as distinct Mar 18 18:27:44 pradan[m]: no... unless you are planning to also retrain in darknet, I don't think this is a fair comparism Mar 18 18:28:17 Ok, that's the end of all my thoughts (for good). What do you suggest w.r.t to the timeframe.... ? Mar 18 18:30:34 Or should I add more ideas and get them benchmarked by the mentors ? Mar 18 18:35:31 training can take weeks Mar 18 18:35:56 it is more or less a brute force approach and each training may result in something slightly different Mar 18 18:36:32 feel free to toss out ideas... everything seems somewhat plausible except for the training Mar 18 18:37:02 there are 3 evals... it would be hard to do a eval for me if you are just stuck doing training which is really anncillary to this project Mar 18 18:38:46 The training was introduced to sparsi-fy the model... we also have some pre-trained sparse models by Intel (YOLO)... I will look into it. Mar 18 18:40:34 I would prefer it to be working on the stock trained data Mar 18 18:41:05 otherwise, you are really making this useful for a specific case vs getting an optimized framework for the YOLO model working Mar 18 18:41:16 it makes it less useful overall Mar 18 18:42:47 ok. I will keep that in mind. But the primary goal of utilising the accelerators stays unclear Mar 18 18:43:54 other people have custom trained models that works on other hardware (x86, CUDA, etc) Mar 18 18:44:06 it would be nice if they didn't have to retrain and test it again Mar 18 18:45:39 The YOLO v2-tiny is port-able AFAIK till now... Mar 18 18:45:40 I need to make sure the ported model can reap all the benefits as the Caffe-Jacinto one trained from scratch Mar 18 18:49:07 Do you suggest me to go for the acceleration of 4 EVEs + 2DSPs using TIDL by trusting TI ? OpenCL will neglect the EVEs if I manage to port it to use the DSPs in the timeframe Mar 18 18:52:10 There have been some references to the accelerator usage Mar 18 18:52:10 http://openaccess.thecvf.com/content_ICCVW_2019/papers/LPCV/Jose_Real-Time_Object_Detection_On_Low_Power_Embedded_Platforms_ICCVW_2019_paper.pdf and Mar 18 18:52:10 http://openaccess.thecvf.com/content_cvpr_2017_workshops/w4/papers/Mathew_Sparse_Quantized_Full_CVPR_2017_paper.pdf Mar 18 19:12:13 maybe this might be helpful - Mar 18 19:12:38 first vocabulary - there are frameworks...this is like TIDL, Darknet, tensorflow, caffe, etc Mar 18 19:13:04 there are models, this is YOLO-N-tiny, MobileNet, etc... it defines layers and what happens Mar 18 19:13:25 there are the trained models - weights that determine how the network is connected Mar 18 19:14:13 next - usage - since training doesn't work well on embedded devices, it would be good to be able to train on a big machine/GPU...take the binaries with some conversion, and put it on the embedded device to do inference Mar 18 19:15:04 with that in mind - the flow i was thining of is - at the end of the project, we have a way (wiki page, script) that will take the trainned model and let you run it at full speed on the BBAI/x15 HW Mar 18 19:16:09 each time you train, you introduce variation...training as I said is a brute force w/a random seed...how (if) a particular training session converges can change and this convergence can mean better confidence in one thing over another Mar 18 19:16:53 in addition, training can run for days/weeks for a large dataset...having a user retrain just to run it on another device is not helpful...esp if they have to rent GPU time **** ENDING LOGGING AT Thu Mar 19 03:00:02 2020