**** BEGIN LOGGING AT Sat Jan 02 02:59:57 2010
Jan 02 09:18:42 <lool> cwillu_at_work: It's in lucid now
Jan 02 09:18:49 <lool> cwillu_at_work: I updated the bug, thanks
Jan 02 09:19:18 <cwillu_at_work> lool, I noticed a bunch of neon stuff in pixman git head , mid/late december
Jan 02 09:19:21 <cwillu_at_work> would those be included?
Jan 02 09:24:50 <lool> cwillu_at_work: This bug only mentions earlier NEON bits though
Jan 02 09:25:35 <lool> cwillu_at_work: We currently have upstream release 0.16.2
Jan 02 09:26:00 <cwillu_at_work> which might have the neon stuff simply disabled, if I read the mailing lists correctly
Jan 02 09:26:15 <lool> Apparently the latest release 0.16.4 is very similar to .2 and was out mid december
Jan 02 09:26:29 <lool> cwillu_at_work: Sorry could you be more specific?
Jan 02 09:26:36 <lool> Which NEON bits and which mailing-list?
Jan 02 09:27:10 <cwillu_at_work> lool, a moment, just checking a head build
Jan 02 09:27:18 <lool> Our pixman is from Sep 2009 so it definitely doesn't have anything from December commits
Jan 02 09:32:15 <cwillu_at_work> head still shows some screen corruption of the same sort I saw on 0.15 (and don't on 0.16.2)
Jan 02 09:32:36 <cwillu_at_work> doing a quick reboot to check a different display depth
Jan 02 09:34:08 <cwillu_at_work> lool, sanity check, is it sufficient to upgrade libpixman/replace /usr/lib/pixman-1-0.so, or does cairo or X or anything need to match it?
Jan 02 09:35:20 <lool> It should be sufficient to replace it, but it's not a good idea to replace the /usr/lib lib
Jan 02 09:35:32 <lool> Install to /usr/local or create an updated pixman .deb
Jan 02 09:35:41 <cwillu_at_work> yep
Jan 02 09:35:42 <lool> Otherwise your changes will be lost wiht next upgrade
Jan 02 09:35:56 <cwillu_at_work> not my first time around the block ;)
Jan 02 09:37:49 <cwillu_at_work> well, first time around the block with pixman, but you know what I mean ;p
Jan 02 09:40:13 <cwillu_at_work> giving lucid's a shot
Jan 02 09:41:30 <lool> It's probably enough to update just libpixman-1-0 from a karmic install
Jan 02 09:41:37 <cwillu_at_work> yep
Jan 02 09:42:56 <cwillu_at_work> restarting x
Jan 02 09:44:59 <cwillu_at_work> okay, looks like the relevant patches are in lucid's version
Jan 02 09:45:33 <cwillu_at_work> or at least, I get the same display corruption under lucids as I did under head and on 0.15
Jan 02 09:47:27 <lool> So you have a regression with NEON enabled pixmans?  Not an issue in 0.14.x?
Jan 02 09:47:31 <lool> What's your hardware?
Jan 02 09:47:47 <cwillu_at_work> lool, for reference, libpixman-1-0_0.16.2-1em1_armel.deb from emdebian.org doesn't have the display cheese
Jan 02 09:47:56 <cwillu_at_work> while lucid's and head's does
Jan 02 09:48:03 <cwillu_at_work> beagleboard
Jan 02 09:48:19 <cwillu_at_work> nope, 0.14.x is fine
Jan 02 09:49:20 <cwillu_at_work> running firefox, scrolling a relatively simple page.  I get squares of varying patterns obscuring the page as I scroll, different for each redraw
Jan 02 09:49:53 <cwillu_at_work> the obscured sections appear to line up with the existing page items, and don't always obscure the entire item
Jan 02 09:50:15 <cwillu_at_work> I can post a photo if that's useful
Jan 02 09:52:18 <lool> cwillu_at_work: Looks like a toolchain issue
Jan 02 09:52:29 <lool> Is libpixman-1-0_0.16.2-1em1_armel.deb patched in any way?
Jan 02 09:54:28 <cwillu_at_work> no idea, it's just a random version I found on the web
Jan 02 09:55:09 <cwillu_at_work> the head test was compiled on the board itself, not sure if that's relevant to toolchains
Jan 02 10:01:16 <cwillu_at_work> (uploading images)
Jan 02 10:02:06 <lool> cwillu_at_work: If you grabbed libpixman from http://emdebian.org/grip/pool/main/p/pixman/ then its exactly the same source as in Debian/Ubuntu and only toolchain differ
Jan 02 10:02:19 <cwillu_at_work> okay, yes, that's what I grabbed
Jan 02 10:02:25 <lool> cwillu_at_work: Could you try rebuilding with gcc 4.3?
Jan 02 10:02:34 <lool> (Was the default in Debian until recently)
Jan 02 10:02:46 <lool> Either head or lucid's package
Jan 02 10:02:59 <cwillu_at_work> k, that'll take a couple minutes
Jan 02 10:04:04 <cwillu_at_work> photos at http://cwillu.com/files/arm/
Jan 02 10:05:07 <cwillu_at_work> forgive the filesize, although the pattern is visible in the corruption at that resolution
Jan 02 10:06:48 <lool> cwillu_at_work: So the red stripes are corruption?
Jan 02 10:06:55 <cwillu_at_work> yes
Jan 02 10:07:08 <lool> Oh and number 4 is pretty clear too
Jan 02 10:09:54 <cwillu_at_work> the corruption changes from frame to frame, and not every frame is corrupted (in fact, I'd say only about a third of the frames are corrupted, although they can be bunched up
Jan 02 10:11:14 <cwillu_at_work> the colour, region, and direction of the pattern change (i.e., angled left vs right)
Jan 02 10:11:40 <cwillu_at_work> and it's not clear in the photos, but the corruption begins and ends partway through a given horizontal line
Jan 02 10:12:29 <cwillu_at_work> (build is running)
Jan 02 10:12:49 <cwillu_at_work> hmm
Jan 02 10:12:58 <cwillu_at_work> ./autogen.sh dies with "Illegal instruction"
Jan 02 10:13:09 <cwillu_at_work> should I run gcc and company from karmic instead of lucid?
Jan 02 10:13:31 <lool> Either should work
Jan 02 10:13:33 <cwillu_at_work> (original head build was done with gcc from karmic
Jan 02 10:14:02 <lool> But I'm interested in gcc-4.3's build result in any case
Jan 02 10:14:12 <cwillu_at_work> well, autogen.sh doesn't even finish
Jan 02 10:14:16 <lool> What's your kernel?
Jan 02 10:14:34 <cwillu_at_work> one of rcn's
Jan 02 10:14:41 <lool> Which upstream version?
Jan 02 10:14:41 <cwillu_at_work> Linux overo 2.6.32.1-x1.0 #1 PREEMPT Thu Dec 17 02:23:37 UTC 2009 armv7l GNU/Linux
Jan 02 10:14:51 <lool> That's quite good hmm
Jan 02 10:15:11 <lool> cwillu_at_work: I'm a bit scared that you get an illegal instruction with lucid + custom 2.6.32
Jan 02 10:15:23 <lool> cwillu_at_work: It's definitely another bug, but I'd like to take a look at that one too
Jan 02 10:15:35 <lool> cwillu_at_work: Could you set -x autogen and see which command triggers that?
Jan 02 10:16:33 <cwillu_at_work> yep, one sec
Jan 02 10:18:24 <cwillu_at_work> autoreconf: configure.ac: tracing
Jan 02 10:18:24 <cwillu_at_work> Illegal instruction
Jan 02 10:18:42 <lool> cwillu_at_work: autoreconf -v should tell you what it's running exactly
Jan 02 10:19:19 <cwillu_at_work> + autoreconf -v --install
Jan 02 10:19:19 <cwillu_at_work> autoreconf: Entering directory `.'
Jan 02 10:19:19 <cwillu_at_work> autoreconf: configure.ac: not using Gettext
Jan 02 10:19:19 <cwillu_at_work> autoreconf: running: aclocal
Jan 02 10:19:19 <cwillu_at_work> autoreconf: configure.ac: tracing
Jan 02 10:19:19 <cwillu_at_work> Illegal instruction
Jan 02 10:21:17 <cwillu_at_work> hmm, that might have been a redherring
Jan 02 10:21:24 <lool> ?
Jan 02 10:21:32 <cwillu_at_work> what's the right way to use 4.3 instead of 4.4?
Jan 02 10:21:41 <lool> CC=gcc-4.3 should work with most sources
Jan 02 10:22:10 <lool> If you're building by hand; if you're building packages, you might have to -eCC or set it in rules instead
Jan 02 10:22:16 <cwillu_at_work> one moment
Jan 02 10:25:46 <cwillu_at_work> okay, things are proceeding now
Jan 02 10:26:30 <cwillu_at_work> (I mean, why use an environment variable when I could just install gcc-4.3 and remove 4.4? :p)\
Jan 02 10:29:26 <cwillu_at_work> make[3]: Entering directory `/home/cwillu/work/pixman/pixman'
Jan 02 10:29:26 <cwillu_at_work>   CC     pixman-access.lo
Jan 02 10:29:26 <cwillu_at_work>   CC     pixman-access-accessors.lo
Jan 02 10:29:26 <cwillu_at_work>   CC     pixman-cpu.lo
Jan 02 10:29:26 <cwillu_at_work>   CC     pixman-gradient-walker.lo
Jan 02 10:29:27 <cwillu_at_work> ../libtool: line 970:  6075 Segmentation fault      gcc-4.3 -DHAVE_CONFIG_H -I. -I.. -g -O2 -Wall -fno-strict-aliasing -fvisibility=hidden -MT pixman-gradient-walker.lo -MD -MP -MF .deps/pixman-gradient-walker.Tpo -c pixman-gradient-walker.c -o pixman-gradient-walker.o > /dev/null 2>&1
Jan 02 10:29:31 <cwillu_at_work> make[3]: *** [pixman-gradient-walker.lo] Error 1
Jan 02 10:29:33 <cwillu_at_work> make[3]: Leaving directory `/home/cwillu/work/pixman/pixman'
Jan 02 10:29:35 <cwillu_at_work> make[2]: *** [all] Error 2
Jan 02 10:29:37 <cwillu_at_work> lool,
Jan 02 10:30:15 * lool whistles
Jan 02 10:30:35 <lool> cwillu_at_work: This is kind of unfortunate
Jan 02 10:30:59 <lool> cwillu_at_work: You could do the reverse, build under sid + gcc 4.3 and then rebuild against gcc 4.4
Jan 02 10:31:36 <lool> cwillu_at_work: Basically, it's always the same source, so it's the build env which differs; I really suspect gcc here, and it just switched in Debian, but pixman did build with 4.3 in Debian
Jan 02 10:37:00 <cwillu_at_work> sorry, I'm confounding things
Jan 02 10:37:20 <cwillu_at_work> ran make clean (which I could have sworn I ran before), and now the build proceedeth
Jan 02 10:40:16 <cwillu_at_work> build finished, installing
Jan 02 10:41:58 <cwillu_at_work> and restarting x...
Jan 02 10:43:59 <cwillu_at_work> lool, rebuild under 4.3 still has corruption
Jan 02 10:44:09 <cwillu_at_work> git head
Jan 02 10:45:09 <lool> Hmm
Jan 02 10:45:12 <cwillu_at_work> checkout 0.16.2 and rebuild?
Jan 02 10:45:37 <lool> Yeah, it's probably safer to always use the same source
Jan 02 10:48:45 * cwillu_at_work rebuilds
Jan 02 10:49:47 <cwillu_at_work> geez
Jan 02 10:49:58 <cwillu_at_work> oh, no, good
Jan 02 10:50:09 <cwillu_at_work> thought I had rebooted and did that last test against 4.4
Jan 02 10:52:15 <lool> So lucid + gcc-4.3 pixman 0.16.2 works?
Jan 02 10:52:26 <cwillu_at_work> hasn't finished building yet
Jan 02 10:52:42 <cwillu_at_work> I wasn't sure if I had run head against gcc-4.3, but I had
Jan 02 11:01:26 <cwillu_at_work> build finished
Jan 02 11:05:07 <cwillu_at_work> lool, gcc-4.3 pixman 0.16.2 still has corruption
Jan 02 11:05:22 <lool> Hmm
Jan 02 11:05:39 <cwillu_at_work> this is on an otherwise largely stock karmic
Jan 02 11:05:55 <lool> cwillu_at_work: Well you could try the opposite approach of starting from squeeze which has 4.3, and then rebuilding with 4.4
Jan 02 11:06:08 <cwillu_at_work> lool, not sure I follow
Jan 02 11:06:12 <lool> I don't quite know what else than gcc could cause a different in the runtime lib causing corruption
Jan 02 11:07:20 <lool> cwillu_at_work: All libs you used were built from the same source, but some have display corruption and some have not; I proposed changing the corrupted build env into a working one by using gcc 4.3, but that didn't work, we could try changing a working build env (squeeze) into a broken one by upgrading gcc to 4.4
Jan 02 11:07:25 <lool> cwillu_at_work: Does that make sense?
Jan 02 11:07:58 <cwillu_at_work> okay, so I need a squeeze rootfs?
Jan 02 11:08:12 <lool> yeah; just debootstrap it from beagleboard
Jan 02 11:08:29 <lool> I'd guess you'll need around 400MB counting build-deps
Jan 02 11:08:49 <lool> debootstrap squeeze /path/to/new-chroot
Jan 02 11:08:55 <lool> (as root obviously)
Jan 02 11:09:12 * cwillu_at_work would have used fakeroot :p
Jan 02 11:09:25 <lool> Then chroot into it and apt-get source pixman + apt-get build-dep pixman + apt-get install build-essential gcc-4.3
Jan 02 11:09:32 <lool> s/4.3/4.4
Jan 02 11:09:48 <lool> You mean fakechroot
Jan 02 11:10:10 <cwillu_at_work> make any sense to use a git checkout of pixman instead of the apt-get source pixman?
Jan 02 11:10:19 <lool> That's possible; it's a debootstrap variant; I don't usually bother fighting this additional layer  ;-)
Jan 02 11:10:22 <cwillu_at_work> alternatively, should I redo my current work based on an apt-get source checkout?
Jan 02 11:10:31 <lool> No, I recommend you stick to a single source, the Debian/Ubuntu source package
Jan 02 11:10:34 <cwillu_at_work> (and what's the deb-src line for ubuntu ports anyway?)
Jan 02 11:10:52 <lool> If you change the source tree, you should redo all your tests to see which works and which don't (i.e. don't change multiple things at once)
Jan 02 11:11:03 <lool> cwillu_at_work: deb-src for ports is just like deb
Jan 02 11:11:11 * cwillu_at_work points out that he's been using a git checkout
Jan 02 11:11:20 <lool> i.e. http://ports.ubuntu.com/ubuntu-ports karmic main restricted universe multiverse
Jan 02 11:11:30 <cwillu_at_work> hmm, I tried that, didn't work
Jan 02 11:11:37 <lool> cwillu_at_work: Well you also said you have been using binaries from emdebian and that *these* work
Jan 02 11:11:46 <cwillu_at_work> yep
Jan 02 11:11:54 <lool> The deb line certainly works for me; did you apt-get update?
Jan 02 11:11:56 <cwillu_at_work> just making sure you're not under the impression that I did something that I didn't do
Jan 02 11:12:02 <cwillu_at_work> oh, probably not
Jan 02 11:12:06 <cwillu_at_work> although I thought I did
Jan 02 11:12:15 <cwillu_at_work> (if memory serves, it was the update that didn't like it)
Jan 02 11:12:49 <cwillu_at_work> okay, so I'm going to debootstrap squeeze, and then build pixman from apt-get source
Jan 02 11:13:00 <lool> Yup
Jan 02 11:13:15 <cwillu_at_work> and in the mean time find out if my btrfs root is really as good at load-levelling as I hope it is :p
Jan 02 11:13:24 <lool> You might also want to test squeeze's libpixman to be on the safe side
Jan 02 11:13:50 <lool> load-levelling?  does that relate to wear-levelling?
Jan 02 11:13:54 <cwillu_at_work> I'll snag it out of the debootstrap after it finishes
Jan 02 11:14:02 <cwillu_at_work> sorry, wear-levelling is what I meant to say
Jan 02 11:14:31 <cwillu_at_work> I always start by thinking "load-balancing, no that's not right, load-levelling, that's right"
Jan 02 11:14:36 <lool> Not sure btrfs is what you want for wear levelling
Jan 02 11:14:53 <lool> It usually causes more writes than other filesystems
Jan 02 11:15:03 <cwillu_at_work> not a whole lot of good options for an sd card;  mounted with ssd_spread it seems to do a decent job
Jan 02 11:15:22 <lool> Oh right, I remember there's an SSD mode in btrfs now
Jan 02 11:15:30 <cwillu_at_work> has been for a while now actually
Jan 02 11:15:44 <lool> Too bad we don't have the underlying flash device easily accessible   :-/
Jan 02 11:16:24 <cwillu_at_work> btrfs generally provides some nice-to-haves
Jan 02 11:16:39 <cwillu_at_work> compression is useful, although at some point I want to get it using lzo instead of gzip
Jan 02 11:16:56 <cwillu_at_work> and the checksumming provides a nice way of telling when my sd cards are starting to go bad
Jan 02 11:17:04 <lool> Doesn't it make your IO slower on beagleboard during builds when you're CPU bound?
Jan 02 11:17:28 * lool didn't dare the switch to btrfs just yet, but has been tempted
Jan 02 11:17:37 <cwillu_at_work> it would, although you can always remount without compress;  existing data stays compressed until you write to it
Jan 02 11:17:51 <lool> Nice
Jan 02 11:17:52 <cwillu_at_work> I've been using it on a couple machines with good backups, haven't run into anything scary yet
Jan 02 11:18:06 <lool> What would be cool is per file or directory attributes to achieve compression or not
Jan 02 11:18:06 <cwillu_at_work> not sure you can do that without a reboot currently though
Jan 02 11:18:10 <lool> Like chattr's
Jan 02 11:18:10 <cwillu_at_work> yep
Jan 02 11:18:31 <cwillu_at_work> sub-volumes would allow it pretty easily
Jan 02 11:18:53 <cwillu_at_work> (shares the storage pool, but mounted separately with whatever args)
Jan 02 11:19:03 <cwillu_at_work> haven't played with that end of things yet though
Jan 02 11:19:22 <cwillu_at_work> I've got a rootfs working, call me easily satisfied :)
Jan 02 11:20:49 <cwillu_at_work> they do some braindead thing with df though:  df against the mountpoint of a btrfs raid reports the total capacity of the underlying storage devices, not the available space in the raid
Jan 02 11:22:01 <cwillu_at_work> I'm also using a 1gb swapfile on that btrfs partition
Jan 02 11:22:24 <cwillu_at_work> which I find kinda hilarious :)
Jan 02 11:23:10 <cwillu_at_work> 2.0gb usage (including the 1gb swap file) ends up using 663mb on the card
Jan 02 11:23:41 <lool> Oh nice
Jan 02 11:23:51 <lool> So you get compressed swap for free
Jan 02 11:24:00 <cwillu_at_work> that's the hilarious part :)
Jan 02 11:24:20 <cwillu_at_work> _wear_levelled_ compressed swap for free :)
Jan 02 11:26:09 <cwillu_at_work> the #beagleboard folks never seem impressed by that for some reason :p
Jan 02 11:26:17 <cwillu_at_work> #beagle rather, but ya
Jan 02 11:28:44 <kblin> nice
Jan 02 11:28:57 <kblin> but what do you do that uses swap? :)
Jan 02 11:29:27 <cwillu_at_work> I'm lazy about removing things from my image that I don't need
Jan 02 11:29:33 <lool> 512 MB isn't that much for large builds such as oo.o or xulrunner and then at runtime GNOME + firefox is pretty big too
Jan 02 11:29:47 <lool> At least IME
Jan 02 11:30:39 <cwillu_at_work> kblin, I usually only have 10 megs or so in swap, although I have to figure that on a beagleboard gaining 10mb ram is a win
Jan 02 11:30:41 <kblin> oh, right, compiling would do it
Jan 02 11:30:57 <cwillu_at_work> well, I meant in normal usage
Jan 02 11:31:01 <cwillu_at_work> browser session really
Jan 02 11:31:19 <kblin> ah, my beagles run as headless servers :)
Jan 02 11:32:14 <cwillu_at_work> I end up with 10 megs or so of used swap;  I haven't done much in the way of rigorous benchmarking
Jan 02 11:32:15 <kblin> but at least on the revB, ram is a bit on the short side if I start up samba4 and bind
Jan 02 11:32:37 <cwillu_at_work> the compression + ssd_spread is a win just for the storage, as long as the interface itself remains usable, which it does
Jan 02 11:33:16 <cwillu_at_work> I basically see swap as the means by which I can extend my working set size
Jan 02 11:33:32 <cwillu_at_work> firefox always ends up with a couple megs paged out with no visible delay in the ui
Jan 02 11:33:37 <cwillu_at_work> so right there it's a win
Jan 02 11:33:43 * kblin nods
Jan 02 11:34:04 <cwillu_at_work> re: compression of the filesystem, there's definitely a cost, although my particular use isn't heavy on the reads or writes
Jan 02 11:34:29 <cwillu_at_work> I'm interested in an lzo compression to replace gzip though for that reason
Jan 02 11:34:42 <kblin> I'm running a fileserver :)
Jan 02 11:34:51 <cwillu_at_work> lzo being an order of magnitude faster than gzip on the compression side, and I think 50% quicker or so on decompression
Jan 02 11:36:05 <cwillu_at_work> not even sure what a good way of benchmarking throughput would be in this case
Jan 02 11:36:23 <cwillu_at_work> hard to get away from the data dependence
Jan 02 11:43:53 <cwillu_at_work> I: Unpacking the base system...
Jan 02 11:55:12 * cwillu_at_work plans to head home after this finishes, and as such starts his car
Jan 02 11:58:19 <cwillu_at_work> I: Base system installed successfully.
Jan 02 12:00:02 * cwillu_at_work grabs squeezes pixman
Jan 02 12:04:00 <cwillu_at_work> lool, sorry, you wanted me using gcc-4.4 under the squeeze chroot?  or should I start with the default, confirm that it works, and then do gcc-4.4?
Jan 02 12:05:54 <cwillu_at_work> lool, squeeze's binary package works without corruption
Jan 02 12:07:15 * cwillu_at_work waits for apt to finis
Jan 02 12:17:52 <lool> cwillu_at_work: You might want to do a test rebuild before trying with 4.4 + squeeze
Jan 02 12:34:03 * cwillu_at_work waits for debian/rules to do its thing
Jan 02 12:53:22 <cwillu_at_work> installing squeeze-chroot-built 4.3
Jan 02 12:53:58 <cwillu_at_work> lool, ^
Jan 02 12:55:58 <cwillu_at_work> lool, no corruption on 4.3, rebuilding with 4.4
Jan 02 13:07:56 <cwillu_at_work> lool, incidently, it looks like a simple CC=gcc-4.4 works with debian/rules
Jan 02 13:11:33 * cwillu_at_work waits for 4.4 to build
Jan 02 13:22:53 <cwillu_at_work> built, installing
Jan 02 13:24:12 <cwillu_at_work> restarting x
Jan 02 13:27:36 <cwillu_at_work> lool, still around?
Jan 02 13:28:08 <cwillu_at_work> squeeze's 0.16.2-1 works fine (no corruption) against gcc 4.4 and 4.3
Jan 02 13:29:01 <lool> Gah
Jan 02 13:29:21 <lool> cwillu_at_work: Well I have no idea what part of the toolchain / build env breaks it then   :-(
Jan 02 13:29:26 <cwillu_at_work> heh
Jan 02 13:29:47 <cwillu_at_work> I'm going to do a build against the git tree, although I'm going to go to bed first
Jan 02 13:29:51 <lool> The builddeps are pretty short
Jan 02 13:29:51 <lool> Build-Depends: debhelper (>= 5), automake, autoconf, libtool, pkg-config, quilt
Jan 02 13:30:07 <lool> It certainly is not one of these
Jan 02 13:30:16 <lool> It could be binutils, but it would be kind of weird
Jan 02 13:30:19 <cwillu_at_work> is there anything relevant to assembling the neon code?
Jan 02 13:30:44 <cwillu_at_work> (the blit optimizations are handcoded iirc)
Jan 02 13:30:53 <lool> yeah, the assembler is in binutils
Jan 02 13:31:02 <lool> Oh right, it's manual assembly
Jan 02 13:31:17 <lool> Well you could try installing Ubuntu's binutils under squeeze and see if that regresses the build
Jan 02 13:31:46 <lool> But these quWe have 2.20-4ubuntu4 and squeeze/sid have 2.20-4, but we have some custom patches
Jan 02 13:32:12 <cwillu_at_work> yep
Jan 02 13:32:31 <cwillu_at_work> okay, I'll try that when I get back
Jan 02 13:33:36 <cwillu_at_work> I have to play hockey in 8 hours, so it's time for bed :p
Jan 02 13:33:40 <cwillu_at_work> thanks for the help, I'll poke you again
Jan 03 02:03:41 <gregoiregentil> Ping ogra?
**** ENDING LOGGING AT Sun Jan 03 02:59:58 2010