**** BEGIN LOGGING AT Tue Nov 22 02:59:57 2005 Nov 22 11:02:27 [g2]: i sure have hell of a time getting with beewoolie Nov 22 11:02:37 [g2]: seems like we are off by 12 hours Nov 22 11:08:40 hehehe Nov 22 11:09:56 vmaster: trying to get some patches added for apex Nov 22 11:10:24 vmaster: beewoolie is driving me nuts with short responses via email Nov 22 11:11:45 HA Nov 22 11:11:55 speak of the devils minions Nov 22 11:12:00 beewoolie-afk: ho ho ho Nov 22 11:12:09 prpplague: Hey man Nov 22 11:12:20 beewoolie-afk: whats the latest? Nov 22 11:12:33 I've got a sick slug. It's more green than usual. Nov 22 11:12:44 hehe Nov 22 11:12:50 Do you have a copy of the latest openslug firmware binary you can share? Nov 22 11:12:56 <[g2]> prpplague beewoolie-afk is right here :) Nov 22 11:13:03 beewoolie-afk: you get my last email with the question about the sdram boot stuff? Nov 22 11:13:06 The link online was broken last night. Nov 22 11:13:12 Let me check.... Nov 22 11:14:19 prpplague: I don't see it. I'm checking the spam folder. Nov 22 11:14:43 Not there either. Nov 22 11:14:46 odd Nov 22 11:14:49 * prpplague resends Nov 22 11:16:02 beewoolie-afk: otw Nov 22 11:16:11 <[g2]> beewoolie-afk is there a kenel debug flag to be verbose about what's happening at calling init ? Nov 22 11:16:19 <[g2]> right after mounting the rootfs Nov 22 11:16:29 I've been clearing out space so that I can run the monotone VCS and build openslug. Nov 22 11:16:51 [g2]: only by enabling kernel debugging and adding a printascii to the printk function Nov 22 11:16:57 [g2]: I don't know what you mean. Calling init is just a matter ofstarting a program. Nov 22 11:17:11 [g2]: What are you trying to do? Nov 22 11:17:14 <[g2]> beewoolie-afk I know but it's failing on my uClibc rootfs Nov 22 11:17:19 <[g2]> init=/bin/sh Nov 22 11:17:25 [g2]: or atleast thats the only way i know of Nov 22 11:17:26 <[g2]> doesn't put me in the shell Nov 22 11:17:34 Usually, it's a library problem. Did you check ldd? Nov 22 11:18:03 If it doesn't work, sometimes I insert a printk (no DEBUG_LL needed) to tell me what programs it is starting. Nov 22 11:18:21 Generally, it's just that the executable cannot be started due to a missing link library. Nov 22 11:18:32 <[g2]> no I didn't check ldd but I think it's a library problem too Nov 22 11:18:42 [g2]: BTW, what happenedto the openslug binary download? Nov 22 11:18:52 <[g2]> dunno. Nov 22 11:19:08 <[g2]> I've kinda been out of the openslug loop Nov 22 11:19:14 Oh. It came back. Nov 22 11:19:45 [g2]: How are you bootstrapping? Nov 22 11:20:22 <[g2]> bootstrapping the Loft ? Nov 22 11:20:54 <[g2]> I'm in Nirvana with the Loft Nov 22 11:20:56 Well, we have been using a flash filesystem as a recovery tool and bootstrap into a full install. loft or slug. Nov 22 11:21:32 <[g2]> with the Loft I stick a CF in and go load -r -v -b 0x1600000 -m disk hda1:/boot/zImage Nov 22 11:21:53 <[g2]> and that loads the kernel from the ext2/3 CF Nov 22 11:21:59 Oh. You've got an extra input method. Still, don't you put something in flash for recovery? Nov 22 11:22:25 <[g2]> well besides 16MB of strataflash, I've got JTAG Nov 22 11:22:35 <[g2]> and serial Nov 22 11:22:40 True. Do you use flash? Nov 22 11:22:44 <[g2]> 2 serial ports actually Nov 22 11:22:50 <[g2]> sometimes Nov 22 11:22:54 <[g2]> fis load zimage Nov 22 11:22:56 Ok OK. So you're phat. Nov 22 11:23:00 <[g2]> fis load zimage26 Nov 22 11:23:15 <[g2]> It's almost like GRUB Nov 22 11:23:31 <[g2]> I've got fconfig too for scripting Nov 22 11:24:01 In other words, you don't use openslug at all anymore?? Nov 22 11:24:20 <[g2]> not for a couple months Nov 22 11:24:31 <[g2]> I've got BE sarge running Nov 22 11:24:37 Hmm. The binary I downloaded appears to be broken. Nov 22 11:24:51 right. But I still need something to get sarge booted. Nov 22 11:25:12 <[g2]> just a kernel for me Nov 22 11:25:27 <[g2]> when running sarge it's often Nov 22 11:25:32 <[g2]> fis load kernel Nov 22 11:25:34 Yeah. It's easier and safer without the slug's limits. Nov 22 11:25:53 <[g2]> exec -c ".... root=/dev/sda1 rootdelay=12 .... Nov 22 11:26:15 I noticed that someonee took my work on the intel access library and has produced a kernel patch for LE support. Nov 22 11:26:18 <[g2]> My web site is running and old Debian Slug Nov 22 11:26:32 <[g2]> yeah guys are running LE Nov 22 11:26:36 That's what I'm trying to do, but the slug has gone south. Nov 22 11:26:53 I think that the OS is corrupted. Nov 22 11:27:10 <[g2]> do you have serial ? Nov 22 11:27:22 Yeah. Nov 22 11:27:25 I'm not lost. Nov 22 11:27:33 <[g2]> then just reload a kernel Nov 22 11:27:37 I just want to do another setup to see if a different slug has the same problems. Nov 22 11:27:46 That would also load a new kernel. Nov 22 11:28:00 <[g2]> reflash is your friend Nov 22 11:28:06 Oh, I know. Nov 22 11:28:17 The thing is, it's a little tough to get the pieces to flash. Nov 22 11:28:30 Besides, I doubt that the kernel in flash is wrong. Nov 22 11:28:39 I suspect that something on the HD changed. Nov 22 11:28:48 <[g2]> then just unplug Nov 22 11:29:10 <[g2]> don't mind me I'm just typing too fast :) Nov 22 11:29:31 <[g2]> it does fall back to the jffs2 partition though Nov 22 11:29:42 unplug? Oh. That works fine. The troble is that I don't know why the debian system crashes. Nov 22 11:30:10 <[g2]> set -x in it linuxrc Nov 22 11:30:19 <[g2]> s/it// Nov 22 11:30:33 Let me be moreclear. The system boots and I can get a login. Nov 22 11:30:54 I can even mount it via NFS. But when I start using it, the system crashes. Nov 22 11:31:10 <[g2]> Ok so NFS has issues Nov 22 11:31:21 <[g2]> if you don't mount NFS then it's stable ? Nov 22 11:31:53 Don't know. Nov 22 11:32:08 I've been using NFS for more than a month without incident. Nov 22 11:33:12 <[g2]> does debian have an install log ? Nov 22 11:33:24 <[g2]> like gentoo's emerge.log Nov 22 11:33:46 I think there is a way to see what's been installed. I've never looked. What are you thinking? Nov 22 11:34:49 <[g2]> I'm wondering if you can just see what you installed in the last 24/48 hours Nov 22 11:35:17 <[g2]> I've been playing around with SD Nov 22 11:35:22 Hmm. Nov 22 11:35:45 <[g2]> there's a physical Lock mechanism on most Nov 22 11:35:58 <[g2]> a hw RO system Nov 22 11:37:09 <[g2]> it would be nice to play with unionfs Nov 22 11:37:20 <[g2]> then allow changes to be saved each day Nov 22 11:37:42 Sounds reasonable. Nov 22 11:38:07 <[g2]> then you could "rollback" Nov 22 11:38:27 <[g2]> that'd be a good name for the disto Nov 22 11:38:33 <[g2]> Rollback linux Nov 22 11:38:37 Well, I shutdown all of the new services. Nov 22 11:39:12 <[g2]> right but obviously something else is effected Nov 22 11:39:28 But this is a kernel panic. Nov 22 11:39:41 That isn't a user-space kind of problem. Nov 22 11:40:16 <[g2]> so it's a panic not a silent crash ? Nov 22 11:42:58 beewoolie-afk: basically i was just curious about whether you though i really needed to init the rs-232 interface even during a sdram boot Nov 22 11:43:01 I see a kernel panic message. Nov 22 11:43:57 where is it panicking? Nov 22 11:46:03 lennert: it ishard to say. I'm still getting the spurrious irq messages. Nov 22 11:46:10 It happens right after one. Nov 22 11:46:28 spurious irqs on ixp4xx is hard if not impossible to solve Nov 22 11:48:12 I thought that the solution was to ignore them. Nov 22 11:48:38 well, yeah, the warning needs to be silenced because there is no good solution Nov 22 11:49:16 Right. I just mean that I'm running a kernel that doesn't suppress them. Nov 22 11:50:03 I could get an ooops report and see where there crash is happening. Nov 22 11:50:22 Yeah. My 'spurious irqs on ixp4xx is hard if not impossible to solve' was a random thought in context. Nov 22 11:50:30 that would help Nov 22 11:50:42 Let me boot it up. Nov 22 11:58:01 Hmm. Not crashing now. I think that part of the problem was that I had some hanging NFS sessions. Nov 22 11:58:33 okay Nov 22 11:58:38 if you see it again, do let me know Nov 22 11:59:28 k. Nov 22 12:02:14 lennert: I was running a wiki on this little box, so I'll try it again. Nov 22 12:02:23 beewoolie-afk: do you want me to resubmit the patch with the rs-232 init code prior to the sdramboot section? Nov 22 12:04:04 prpplague: yeah. The idea is that APEX should work completely if it is run from SDRAM. Init of the uart and other devices should be correct. You may not know the state of the system. Nov 22 12:04:16 prpplague: Moreover, the uart init ought to be in the driver and not in the init- code. Nov 22 12:04:35 beewoolie-afk: hmm Nov 22 12:05:14 beewoolie-afk: just out of curiosity how would you be running apex directly from sdram without already having the rs-232 init'd ? Nov 22 12:05:18 prpplague: the uart init in the startup routine is there for debug only. Nov 22 12:05:26 JTAG. Nov 22 12:06:00 You also don't know what performed the init. It may not be compliant with what the system needs. Nov 22 12:06:06 beewoolie-afk: ahh, thats something i don't normally do, i always get flash working first Nov 22 12:06:26 The idea is that we want to guarantee, to the best of our ability, that the system is in the state that we need. Nov 22 12:06:49 prpplague: That's you, but there is no guarantee that others will do the same. Nov 22 12:07:25 beewoolie-afk: i meant only that i didn't think of it due to the fact that we don't dev in that way, not that its not a feasible item Nov 22 12:08:01 The contract is this: after APEX starts, the system is in a known state for the OS. Nov 22 12:08:26 Anything that we don't do could mean that the system won't run. Nov 22 12:09:25 UART init belongs in the driver. Nov 22 12:11:49 beewoolie-afk: ok Nov 22 12:17:26 beewoolie-afk: btw, another "feature" i'm working on is adding a micro load option, which builds just enough of apex to init an interface like the sd/mmc and copy the bootloader, kernel, and rootfs to flash Nov 22 12:17:51 beewoolie-afk: is that something you want submitted? or should i not bother? Nov 22 12:22:40 beewoolie-afk: updated patch otw Nov 22 12:31:39 beewoolie-afk: still there d00d? Nov 22 12:38:00 yeah. Nov 22 12:38:26 How small is micro? Nov 22 12:38:31 1-2k Nov 22 12:38:40 3max Nov 22 12:38:45 I'm not sure how that's apex at all. Nov 22 12:39:05 beewoolie-afk: its not really, just helps to have it all in one place Nov 22 12:39:12 Is this for keeping a NAND flash image small? Nov 22 12:39:28 beewoolie-afk: keeping the initial load via jtag small Nov 22 12:39:46 beewoolie-afk: when you are building 5000 units, jtagin takes time Nov 22 12:39:49 For faster manufacturing? Nov 22 12:39:55 yea Nov 22 12:40:06 How long does 16K take to download? Nov 22 12:40:21 3 minutes Nov 22 12:40:31 Ouch. That's slow jtag. Nov 22 12:40:40 Why so slow? Nov 22 12:40:59 several reasons Nov 22 12:41:20 parallel port, 16 bit flash Nov 22 12:41:57 If you are manufacturing, don't you think it makes sense to use a faster JTAG interface? Nov 22 12:42:10 This isn't to say that it is a bad idea to write a boot-strap. Nov 22 12:42:23 I just wonder how it makes sense to put this in apex. Nov 22 12:42:32 It sounds like a targeted application. Nov 22 12:42:44 one sec phone Nov 22 12:42:48 k. Nov 22 12:43:47 beewoolie-afk: yea we have discussed designing a faster jtag, but we have lots of legacy devices and we already have the jtag dongles built Nov 22 12:43:58 right. Nov 22 12:44:10 Classic legacy problem. Nov 22 12:44:23 beewoolie-afk: my main thought in doing it with apex is that i will have about 5-6 different processors that will eventually be supported in apex Nov 22 12:44:36 beewoolie-afk: i would like to use the same microloader design for all of them Nov 22 12:44:44 How is the user going to select the right startup? Nov 22 12:44:53 beewoolie-afk: hence trying to standardize the process so i do have to keep writing it from scratch Nov 22 12:45:14 Ok. what in APEX is valuable for this project? Nov 22 12:45:42 The biggest components are the FS drivers and the mechanisms used to load services and drivers. Nov 22 12:45:49 beewoolie-afk: severalt things, first, the support for the different file systems, second, the support for the automated copies Nov 22 12:45:52 printf is really big. Nov 22 12:46:04 You aren't going to get an FS driver in 3K. Nov 22 12:46:21 hmm Nov 22 12:46:29 Look at the object files. Nov 22 12:46:38 one sec conference call just started Nov 22 12:46:41 The only thing I've been trying to do is get thumb instructions. Nov 22 12:46:43 k. Nov 22 12:48:25 beewoolie-afk: ok done, sorry, short call Nov 22 12:48:59 beewoolie-afk: well i could get by with something small_er like 8-10k Nov 22 12:49:17 prpplague: what are those 5-6 different processors? variants of arm? or different architectures? Nov 22 12:49:27 all arm Nov 22 12:49:54 and currently you flash them using jtag-tools? Nov 22 12:49:56 however there was talk of doing a mips board Nov 22 12:50:17 vmaster: no i usually write a small app to do the job Nov 22 12:50:35 vmaster: although i usually get jtag-tools working with the boards Nov 22 12:50:50 ok, but using the boundary-scan approach? 3 minutes for 16k sounds like it Nov 22 12:50:56 vmaster: i usually optomized the flash routines Nov 22 12:51:12 prpplague: This is with single-byte programing, too. Right? Nov 22 12:51:36 basically just bitbanging the flash interface Nov 22 12:51:52 beewoolie-afk: it's not so much single-byte vs. block, but about using boundary-scan vs. processor logic Nov 22 12:52:03 vmaster: yea no processor logic Nov 22 12:52:45 vmaster: I think they also cannot write blocks Nov 22 12:53:15 i suspose if i make an extra small version of apex that would probably be enough Nov 22 12:53:37 prpplague: IMHO, this deserves more attention. Nov 22 12:53:51 prpplague: Perhaps you'd be better off writing something that could activate a serial transfer. Nov 22 12:54:00 That could be really small. Nov 22 12:54:17 even my "naive" flashing achieves 500-1000 bytes / sec using a wiggler Nov 22 12:54:28 beewoolie-afk: yea, i already have an app that is just 2k that inits the sd/mmc interface and copies the other items from there, but no fs support Nov 22 12:55:04 i was considering adding some very small fat16 code that i have Nov 22 12:55:17 You could init SDRAM and the serial interface, add xmodem and be able to download the loader over the wire. Nov 22 12:56:05 beewoolie-afk: yea Nov 22 12:56:12 that would be only 3K. Nov 22 12:56:18 xmodem is kinda big, too. Nov 22 12:56:29 how much is the whole download? Nov 22 12:56:46 As I wrote, I'd really like to get thumb working to reduce the code size. Nov 22 12:56:51 beewoolie-afk: roughly 1.5mb with bootloader, kernel, and rootfs Nov 22 12:56:56 beewoolie-afk: yea Nov 22 12:56:57 vmaster: the whole loader could be 30K. Nov 22 12:57:23 beewoolie-afk: i was refereing to what prpplague has to download (1.5mb) Nov 22 12:57:29 ah, Nov 22 12:58:05 vmaster: the initial load of the device is a diagnostic kernel and ramdisk that we will use to do a burning Nov 22 12:58:08 burnin Nov 22 12:58:41 vmaster: prior to shipment a util in the ramdisk will fetch a new kernel and ramdisk via a wireless interface Nov 22 12:58:45 prpplague: I'd be inclined to do a setup that gives you network access so that the whole process could be automated. Nov 22 12:59:15 hmm, ok - i wanted to see if doing the whole download through jtag would save you a lot, but with >1mb you'd need more than parport, no matter if you use processor logic and ram buffers or not Nov 22 12:59:21 beewoolie-afk: yea we are considering that as well Nov 22 12:59:32 vmaster: yea Nov 22 13:00:16 beewoolie-afk: problem we have is not all of our devices have the same net interface and some don't even have net Nov 22 13:00:24 beewoolie-afk: but they all have sd/mmc Nov 22 13:01:23 prpplague: From my perspective, the big win is in automation. You could reduce some of the steps, but the more you can do in batch and without intervention, the faster you can do a large batch. Nov 22 13:01:29 Henry Ford did it first. Nov 22 13:04:56 prpplague: are they all arm7/arm9? or xscale, too? Nov 22 13:05:27 beewoolie-afk: yea the main boards are plugged into a test bed that already has the sd card in them Nov 22 13:05:53 vmaster: arm7 and arm9 right now, with arm11 in the future Nov 22 13:06:27 vmaster: no xscale Nov 22 13:06:50 vmaster: intel is a beoych to work with for companies our size Nov 22 13:07:29 mhh, with a ft2232c i can write to target ram at 25kb/s - with the overhead for transfering the flash code and the actual flash write, you could probably flash at >20kb/s Nov 22 13:08:13 < 90s until all of the 1.5mb are inside your flash, with no human interaction Nov 22 13:08:17 vmaster: yea i actually have a ez-usb based device that i get great performance with, but we have a legacy issue Nov 22 13:08:28 vmaster: management doesn't want to change the jtag interface Nov 22 13:08:33 hehehe, ok Nov 22 13:08:54 didn't get what "legacy" meant 30min ago Nov 22 13:08:58 ahh Nov 22 13:09:26 vmaster: we already have all the parallel port dongles and test beds made Nov 22 13:09:29 prpplague: Do you have to use the legacy device for the test-bed? Nov 22 13:09:35 Oh. Nov 22 13:09:55 ok, gotta go - cya Nov 22 13:10:01 beewoolie-afk: they represent a pretty big investment Nov 22 13:11:18 beewoolie-afk: i'll probably present 3 options to they, custom micro loader, micro version of apex, and just an extra small version of apex Nov 22 13:11:40 beewoolie-afk: after discussing with you guys i suspect that just the extra small apex will be acceptable Nov 22 13:12:46 beewoolie-afk: btw, does the ";" function work now? Nov 22 13:13:21 prpplague: For your constraints, I'd recommend a second build target. Nov 22 13:13:32 You can use the code in apex to link in the drivers you need. Nov 22 13:13:57 Really, there is no need to link in all of the UI. Nov 22 13:14:04 beewoolie-afk: agreed Nov 22 13:14:29 It's an interesting problem. Nov 22 13:14:41 What might work is a script that performs the loads you need. Nov 22 13:15:05 Might be straightforward. Nov 22 13:16:02 yea Nov 22 13:16:07 <[g2]> lennert you saw the post from the Atheros guy on l-a-k about the "nobody cared" problem right ? Nov 22 13:16:41 beewoolie-afk: a stripped down build of apex with just the options i need is 16k Nov 22 13:18:20 prpplague: I've spent a lot of time getting it small. Nov 22 13:18:30 Part of the problem is that there are limits on what I can remove. Nov 22 13:18:32 beewoolie-afk: yea i know Nov 22 13:18:43 One of the big pieces is the printf/sprintf code. Nov 22 13:18:50 <[g2]> bbl Nov 22 13:19:00 If we remove the UI, that stuff can go. Nov 22 13:19:01 [g2]: later Nov 22 13:19:13 [g2]: later Nov 22 13:19:32 with thumb, it drops about 2K right now, but it doesn't run anymore. Nov 22 13:19:47 And that is with only a small portion of the code in thumb. Nov 22 13:20:37 There are a couple of things to chase on the thumb path: the call-by-r3 library function is missing; the code doesn't even run now. Nov 22 13:21:24 beewoolie-afk: i plan on tinkering this weekend, so i'll let you know Nov 22 13:21:36 beewoolie-afk: you got the latest s3c2410 patch correct? Nov 22 13:24:34 I don't think so. Let me check the receipt log. Nov 22 13:25:08 It hasn't even been received. Nov 22 13:25:10 hehe you already have my address set to send to the trash bin Nov 22 13:25:30 Tee hee. No. I have a log that show everything received. Nov 22 13:25:40 Even before the auto-trashing procmail gets it. Nov 22 13:28:47 beewoolie-afk: the latest one has the serial changes you suggested and doesn't include the meminfo removed Nov 22 13:32:56 OK. Still waiting. Nov 22 13:34:08 hmm Nov 22 13:34:11 * prpplague resends Nov 22 13:34:42 beewoolie-afk: otw Nov 22 13:34:52 yahoo delivery schedule sucks Nov 22 13:35:31 beewoolie-afk: i've been wondering why it takes so long for you to respond, hehe, must be crappy yahoo mail Nov 22 13:35:58 I'd be lying if that was the only reason. Sometimes, I get the mail and it takes a couple of days before I can review it. Nov 22 13:36:17 hehe Nov 22 13:37:02 beewoolie-afk: my boss is considering no longer submitting patches, since we've had horrible responses in the last 2 months Nov 22 13:38:51 Then your boss is naive. Nov 22 13:39:11 Open source doesn't run on corporate schedules, but it does deliver results. Nov 22 13:39:12 beewoolie-afk: why is that? Nov 22 13:39:27 Open source is about relationships. Nov 22 13:39:35 Relationships take time. Nov 22 13:39:58 prpplague: There may be something wrong with my email. i'm not getting anything. Nov 22 13:40:22 beewoolie-afk: yea well, they have never submitted patches before untill i started there Nov 22 13:41:10 Look at reiser. that guy is *nuts* about getting his code in the kernel, but his behaviour has shown that he's untrustworthy Nov 22 13:41:46 The kernel folks are cautious, with good reason. Nov 22 13:45:48 beewoolie-afk: well, most of the time adding our stuff to the main trees Nov 22 13:48:11 I'm still not getting email, so I suspect something has gone awry. Nov 22 13:49:24 bummer, Nov 22 13:49:34 beewoolie-afk: well, most of the time adding our stuff to the main trees isn't very beneficial to us Nov 22 13:49:50 It's been running for, gosh, years without trouble. Nov 22 13:49:51 beewoolie-afk: i'm mainly doing it to add back to community Nov 22 13:50:02 It can be tricky. Nov 22 13:50:39 If it's specific to your hardware, and you don't care about updating your patches as the kernel progresses forward, then there is no reason to worry. Nov 22 13:51:01 However, you get somebenefit from putting your code into the tree as code improvements are, kinda, free. Nov 22 13:57:43 beewoolie-afk: yea normally our builds are very static Nov 22 14:07:51 prpplague: bbs, I'm going to reset this machine. Nov 22 14:08:06 beewoolie-afk: ok Nov 22 14:14:18 prpplague: Still no mail from you. I'm getting other email. Nov 22 14:15:07 resending Nov 22 14:31:28 prpplague: I found the problem... Nov 22 14:49:23 beewoolie-afk: hehe and? Nov 22 14:53:14 beewoolie-afk: gotta run, let me know about the patch Nov 22 21:42:01 [g2]: yes, i saw that. same problem, ixp4xx irq madness. **** ENDING LOGGING AT Wed Nov 23 02:59:56 2005