**** BEGIN LOGGING AT Sat Nov 12 02:59:58 2005
Nov 12 08:19:25 <key2> anyone knows how If there is a way to get 128GB of RAM for a NFS on a computer ? is there any appliance doing that ?
Nov 12 08:27:19 <[g2]> key2 I think there are some big servers that support lots of memory
Nov 12 08:28:02 <[g2]> there are the 36-bit extensions
Nov 12 08:28:15 <lennert> intel x86 is 64G max
Nov 12 08:28:24 <[g2]> which only 16*4 or 64G
Nov 12 08:28:27 <vmaster> opteron server could possibly use that much
Nov 12 08:28:50 <lennert> you'll need 64 DIMM slots
Nov 12 08:29:02 <lennert> you'll probably get 7 other CPU sockets with it
Nov 12 08:29:23 <lennert> 'for free'
Nov 12 08:29:54 <[g2]> right, I don't understand why you'd put that on 1 box unless it has 10G
Nov 12 08:30:20 <[g2]> 1G top out under 128MB/sec
Nov 12 08:31:06 <[g2]> and my nfore4 Ultra would probably serve that up with a good PCI-E card
Nov 12 08:31:21 <[g2]> it does 600Mbs+ with a $19 card
Nov 12 08:31:35 <lennert> with 10G it'll still take 100 seconds at 10G line rate to fill your DRAM
Nov 12 08:31:52 <[g2]> nod
Nov 12 08:32:30 <[g2]> so unless it's one big DB file that all transations are using, it could probably be split across boxes
Nov 12 08:32:56 <[g2]> but I think there are some Opteron boxes out there
Nov 12 08:33:22 <lennert> 128G is useful to keep your entire database/fileset in ram, for low latency access
Nov 12 08:33:23 <[g2]> or lennert can whip you up an FPGA that has 4 PPCs on it and supports 128G memory :)
Nov 12 08:33:51 <lennert> [g2]: the problem here is pin count :)
Nov 12 08:34:13 <lennert> and drive strength
Nov 12 08:34:18 <[g2]> they make some pretty big FPGA
Nov 12 08:34:30 <lennert> yup
Nov 12 08:34:32 <[g2]> I'm sure you'd be cascading some of the memory
Nov 12 08:34:43 <lennert> nallatech's boards are only $30k :)
Nov 12 08:35:29 <[g2]> well what's a 1G of 400Mhz DDR go for these days ? One would probably want ECC
Nov 12 08:35:51 <[g2]> like for nallatech ?
Nov 12 08:36:55 <[g2]> I'm at nallatech.com
Nov 12 08:37:27 <lennert> i think the BenBLUE-V4 was $30k or so
Nov 12 08:37:42 <lennert> i have their price list somewhere.. nothing is under $10k, pretty much
Nov 12 08:38:19 <[g2]> I'm surprised they don't have PCI-E
Nov 12 08:38:31 <key2> vmaster: and with a FPGA, it wouldnt be possible to make a board with 128 RAM slot and make an RAMDISK NFS of 128GB ?
Nov 12 08:38:56 <lennert> [g2]: 1gb pc3200 ddr400 400mhz - $61
Nov 12 08:39:29 <key2> so it's fine
Nov 12 08:39:32 <lennert> [g2]: hmm, i thought they had pci-e..
Nov 12 08:41:17 <[g2]> ok so $61*128= 8K
Nov 12 08:42:25 <vmaster> key2: why would you want to spend that much money just to have a nfs share on a ramdisk?
Nov 12 08:42:42 <[g2]> lennert I think you'd have an extra 14 cores :)
Nov 12 08:43:06 <[g2]> http://extended64.com/Article30.x64
Nov 12 08:43:50 <[g2]> http://www.theinquirer.net/?article=23518
Nov 12 08:43:56 <[g2]> that's the one I remember
Nov 12 08:44:26 <lennert> [g2]: looks like a microsoft astroturf site
Nov 12 08:44:36 <lennert> [g2]: "see we have a community too"
Nov 12 08:44:44 <vmaster> http://h18004.www1.hp.com/products/servers/proliantdl585/
Nov 12 08:45:01 <[g2]> lennert I just meant the iwill board
Nov 12 08:45:13 <lennert> oh
Nov 12 08:46:17 <lennert> 128GB at 266mhz, or 48GB at 333mhz, or 32GB at 400mhz
Nov 12 08:48:12 <[g2]> well it's not like the 266mhz would matter on NFS unless you've got 10G pipes
Nov 12 08:48:25 <[g2]> in which case you've got other issues
Nov 12 08:48:38 <lennert> [g2]: key2 didn't say yet what he wants to do with it :)
Nov 12 10:49:50 <key2> sorry
Nov 12 10:49:51 <key2> back
Nov 12 10:51:16 <key2> It was basically for a Video On Demand service, a hard disk would be too low for sending 300 streams to 300 different people
Nov 12 10:52:08 <lennert> how about 16 machines with 8G each?
Nov 12 10:52:56 <key2> do you know how much that costs ?
Nov 12 10:53:13 <vmaster> i'd bet it's a lot less than one machine with 128G
Nov 12 10:53:34 <vmaster> how much bandwidth does one stream take?
Nov 12 10:53:36 <key2> vmaster: I need 200 of those machine
Nov 12 10:53:42 <key2> 3Mb
Nov 12 10:53:53 <vmaster> as in megabyte? or millionbit?
Nov 12 10:54:02 <ka6sox> 3Megabits/sec
Nov 12 10:54:03 <key2> bit
Nov 12 10:57:30 <vmaster> shouldn't a disk array be fast enough to handle this? i mean, even the most expensive disks are going to be a lot less than the memory you're thinking of
Nov 12 10:59:56 <key2> like how many disks
Nov 12 11:00:25 <ka6sox> vmaster, not really
Nov 12 11:00:25 <ka6sox> if everyone was watching the SAME content yes....
Nov 12 11:00:33 <ka6sox> and if they started at the *same* time.
Nov 12 11:00:46 <key2> no 
Nov 12 11:00:51 <key2> they don't start at the same time
Nov 12 11:00:59 <ka6sox> having to have the physical heads move all over the disk is the problem.
Nov 12 11:01:04 <vmaster> hmm, yeah
Nov 12 11:01:08 <key2> basically
Nov 12 11:01:55 <key2> I wanna have at least 400 movies 
Nov 12 11:02:20 <key2> and all should be watched at the same time
Nov 12 11:02:23 <key2> how can I do
Nov 12 11:04:20 <ka6sox> get a Z900 mainframe and go from there.
Nov 12 11:04:34 <ka6sox> they are optimized for bandwidth.
Nov 12 11:05:40 <key2> Z900
Nov 12 11:05:40 <key2> ?
Nov 12 11:06:00 <ka6sox> ibm System 390 Z mainframe computer
Nov 12 11:07:10 <key2> it would cost more than a single board I would made with 128Gb of ram
Nov 12 11:07:16 <key2> and few fpga
Nov 12 11:07:19 <key2> wouldn't it ?
Nov 12 11:07:35 <key2> -made + make
Nov 12 11:07:58 <ka6sox> one BIG ram disk eh?
Nov 12 11:08:12 <key2> well today 1Gb of ram doesn't cost shit
Nov 12 11:08:43 <key2> and since I need a lot of those device (+200) and need them to be quiet small (2U)
Nov 12 11:08:51 <key2> i would think about makin my own
Nov 12 11:09:35 <key2> unless there is a better solution cheaper
Nov 12 11:10:06 <ka6sox> you are streaming to settop boxes?
Nov 12 11:10:24 <key2> yeah
Nov 12 11:10:31 <key2> and I can't put them all in a datacenter
Nov 12 11:10:36 <key2> it would make too much of trafic load
Nov 12 11:10:43 <key2> i better put one or two on each DSLAM
Nov 12 11:11:50 <ka6sox> yep
Nov 12 11:12:01 <ka6sox> at 3Mb/sec you would have to.
Nov 12 11:12:23 <key2> I mean our backbone is made with Gb optic fiber
Nov 12 11:12:52 <key2> but sending all those video from one point through the whole network is stupid
Nov 12 11:12:55 <key2> it would slow it down
Nov 12 11:13:21 <key2> it's better to have those very big in each telco center and send once for good the movie when it's needed
Nov 12 11:15:31 <lennert> key2: what company you working for?
Nov 12 11:15:43 <key2> i'm in france
Nov 12 11:15:48 <key2> so probably don't know it :)
Nov 12 11:15:51 <key2> it's called free
Nov 12 11:16:01 <key2> we do 24Mb dsl for 29.9Eur/month
Nov 12 11:16:35 <lennert> ah, i think i've heard of them
Nov 12 11:20:48 <key2> now I have to figure out how much a virtex would cost :)
Nov 12 11:21:15 <key2> and If I have to put them all on the same bus or make different buses to speed up the thing
Nov 12 11:23:10 <lennert> a virtex is useless for you
Nov 12 11:23:25 <lennert> fpgas won't help a bit for what you're trying to do
Nov 12 11:25:20 <lennert> you need i/o bandwidth and lots of spindles
Nov 12 11:25:41 <ka6sox> or lots of RAM disks to eliminate latency
Nov 12 11:26:15 <lennert> lots of RAM is impractical if you ever want to reboot
Nov 12 11:26:29 <lennert> if you have 128G of movies in-memory, and you want to reboot.. it'll take you an hour to get everything back in RAM
Nov 12 11:26:46 <ka6sox> thats why you only have 2 movies in any one box.
Nov 12 11:26:48 <lennert> (assuming you can read them back in at 35 meg/second)
Nov 12 11:26:52 <ka6sox> 10GB of ram/box
Nov 12 11:26:56 <lennert> yeah
Nov 12 11:27:08 <lennert> either that or a few big boxes with lots of disk
Nov 12 11:27:33 <ka6sox> ram disks will be more expensive but faster and very low latency
Nov 12 11:27:42 <lennert> if you're seek-bound (and you will be, for a certain number of clients), it's better to have 1000 9G disks than 30 300G disks
Nov 12 11:32:02 <key2> lennert
Nov 12 11:32:37 <key2> let say I have only 8 movies of 800MB each
Nov 12 11:32:48 <key2> and 5000 people are watching each of them
Nov 12 11:32:52 <key2> how do I have to deal
Nov 12 11:33:02 <key2> do you think the Head of the hard disk can go that fast ?
Nov 12 11:33:34 <key2> and on only 2U
Nov 12 11:33:42 <key2> you won't be able to fins cheaper
Nov 12 11:33:53 <key2> it doesn't matter if it takes 1hour to load it back
Nov 12 11:34:01 <key2> it's not supposed to reboot every hour
Nov 12 11:34:18 <ka6sox-away> sure..just use WinCE
Nov 12 11:34:31 <key2> wince ?
Nov 12 11:34:40 <lennert> 128G of ram is $8000 while 128G disk is less than $60
Nov 12 11:35:14 <key2> lennert: but you can't have 100 people watching the same thing on the $60 hard drive
Nov 12 11:36:41 <key2> so you can't deal with hard drive unless you have 1000 of them
Nov 12 11:36:48 <key2> which takes lot of room
Nov 12 11:37:09 <key2> you can't go in each DSLAM center and put one full cabinet of hard drive
Nov 12 11:37:34 <key2> plus the power
Nov 12 11:37:43 <key2> plus the temperature
Nov 12 11:37:43 <key2> ...
Nov 12 11:37:47 <key2> it's a pain in the ass
Nov 12 11:37:50 <key2> relaly
Nov 12 11:38:28 <key2> why do you think most companies that do Pay Per Vew do multicast streamin the same movie every 10min
Nov 12 11:38:37 <vmaster> wouldn't you have to used buffered ram?
Nov 12 11:38:50 <key2> what u mean
Nov 12 11:38:58 <vmaster> if you put 128G in one machine
Nov 12 11:39:05 <key2> yeah
Nov 12 11:39:08 <key2> what would be the pb ?
Nov 12 11:39:20 <vmaster> buffered ram is even more expensive
Nov 12 11:39:21 <key2> it would be a big FPGA dealin with htose
Nov 12 11:39:47 <key2> buffered ?
Nov 12 11:39:50 <key2> for what reason
Nov 12 11:39:50 <key2> ?
Nov 12 11:40:21 <key2> a DDR should make it
Nov 12 11:41:37 <vmaster> as far as i know there's a problem with line drive strength when having too many ram modules on a single bus
Nov 12 11:41:46 <vmaster> that's why you use buffered memory
Nov 12 11:41:51 <vmaster> with a buffer for every line
Nov 12 11:42:14 <vmaster> reducing the capacity that needs to be driven
Nov 12 11:43:18 <key2> what do you call line drive strenght
Nov 12 11:43:52 <key2> that the power is not enough ?
Nov 12 11:44:15 <vmaster> mhh, not power, but current
Nov 12 11:44:35 <vmaster> [16:34] < lennert> [g2]: the problem here is pin count :)
Nov 12 11:44:35 <vmaster> [16:34] < lennert> and drive strength
Nov 12 11:44:41 <vmaster> lennert mentioned that already
Nov 12 11:44:57 <key2> well pincount shouldnt be a probleme
Nov 12 11:45:09 <key2> if they are all on the same bus and multiplexed
Nov 12 11:45:28 <key2> but drive strenght
Nov 12 11:45:33 <key2> dunno what would be the probleme
Nov 12 11:45:57 <bullet> key2: check out servers. they use registered (buffered) dimms.
Nov 12 11:46:51 <key2> bullet: but why wouldnt it work
Nov 12 11:47:00 <key2> if you have a chip, you can chose it with the CE
Nov 12 11:47:02 <vmaster> if you use buffered ram it could work
Nov 12 11:47:17 <key2> and what's the difference
Nov 12 11:47:32 <key2> if you put 4 of them or 100 of them
Nov 12 11:47:45 <vmaster> the buffer - the memory controller doesn't have to drive as much current as it would have to
Nov 12 11:47:51 <vmaster> without the buffers
Nov 12 11:48:13 <bullet> http://en.wikipedia.org/wiki/Registered_memory
Nov 12 11:49:27 <key2> ok
Nov 12 11:50:09 <key2> so it means that when you have that amount of ram on the same time, all of them would use a certain amount of the current even if the CE is not enabled ?
Nov 12 11:51:17 <key2> so the controller would have to deliver more current to "talk" to all of them
Nov 12 11:51:18 <key2> ?
Nov 12 11:51:38 <vmaster> i think so
Nov 12 11:52:08 <key2> well it doesn't cost that much to put a little buffer in front of them
Nov 12 11:52:14 <key2> just to bring the current hight
Nov 12 11:52:16 <key2> higher
Nov 12 11:53:07 <lennert> the spartan3 can drive like 24 mA per pin before it goes pop
Nov 12 11:53:27 <key2> lennert: ok but let's take it like that
Nov 12 11:53:47 <key2> if you put a little buffer in front of the line for every 4 of them
Nov 12 11:53:50 <key2> then it's fine
Nov 12 11:56:44 <key2> but then I might get some timing issue
Nov 12 12:01:47 <key2> INPUT LEAKAGE CURRENT
Nov 12 12:01:47 <key2> Any input 0V = VIN = VDD, VREF pin 0V = VIN = 1.35V
Nov 12 12:01:48 <key2> (All other pins not under test = 0V)
Nov 12 12:01:48 <key2> Command/
Nov 12 12:01:48 <key2> Address, RAS#,
Nov 12 12:01:48 <key2> CAS#, WE#
Nov 12 12:01:50 <key2> II
Nov 12 12:01:52 <key2> -36 36
Nov 12 12:01:56 <key2> so 36uA
Nov 12 12:02:27 <key2> 24mA is a lot compared to the 36uA each input line uses
**** ENDING LOGGING AT Sun Nov 13 02:59:57 2005