Giga what?

I’ve been testing my network speed again, trying to figure out why I’m not really seeing gigabit speeds.

I’ve eliminated the switches from being the problem as using a bit of crossover between two PC’s made no difference at all.

I think its a either a limitation of the RTL8169 cards or the PCI bus, as I tried from the Pentium4’s built-in Intel Pro 1000CT and speeds from iperf went up from about 67MB/s to about 82MB/s – although that was still to another RTL8169 PCI card on the other end. I tried a combination of “pv < /dev/zero | nc -u 5000" and "nc -u -l 5000 > /dev/null” to create a client/server pair with no filesystem access, and was getting similar results to iperf.

I was considering making the Pentium4 into the fileserver, although it seems like extreme overkill to have a 3GHz/3Gb machine serving files to XBoxes etc for 15-20MB/s difference, plus its a bit of a noisey box in comparison to the Athlon 2.4GHz/1.5Gb.

I guess 67-82MB/s is not bad without jumbo frames enabled, as the maximum would be about 119MB/s with a 9k MTU – which wouldn’t work with the 100Mbit routers, printserver and XBoxes.

I’ve also turned off hard disk spindown (hopefully it has worked) on the fileserver, as that causes quite a nasty hang of Nautilus whilst the NFS server is figuring out why it can’t read from the disk: “hdparm -S0 /dev/sda”, plus it’s probably killing the disks spinning up/down all the time.

Rotten Sunday

I’ve got a cold today, so taking it easy. I’ve watched some films as I don’t feel like doing much, but they’ve all been a bit naff.

Lakeview Terrace, about an arsehole cop who makes the lives of those who live around him a nightmare.

Transporter 3. Pretty good, but I really think its time they broke the mould, as the Transporter films seem to have a recipe:

  • Someone’s daughter gets kidnapped and Frank has to transport her;
  • She’s a pain in the arse, but still falls for Frank;
  • Frank’s client tries to kill him;
  • Frank’s house gets destroyed;
  • Frank has a fight scene or two where he kicks the arses of a dozen or so guys;
  • Lots of fast driving scenes – although this time it was an Audi, not a Mercedes.

At least it was better than Transporter 2 I guess, back to the Transporter 1 roots.

Finally I watched Ghost Town, which could be described as a comedy version of Ghost or Sixth Sense – some guy can see ghosts and has to help them, whilst falling in love. The problem was it wasn’t very funny, and the romance bit featured more than the ghosts bit.

I’ve also been playing with my LAN bandwidth again today. I’ve been testing three Linux boxes with gigabit cards and can get up to about 90MBytes/s, or around 700Mbits/sec using iperf, which is not bad, although I never see that kind of speed when transferring files over rsync, NFS or sftp. I also had another go at enabling jumbo frames with an MTU of 7168, but it still doesn’t work. I guess disk is the bottleneck as only the 1Tb drive in my Core2Quad can manage 120Mbps reads, the 250/500Gb drives are maxed out at about 70Mbps, and probably less for write operations. Maybe I’ll try RAID striping a couple of the 250Gb’s in my Pentium4 sometime.

Update: just found a really useful Firefox plugin called Firebug, it lets you inspect the HTML/CSS/Javascript of any element on a web page – and even change them. Very handy for when you’re trying to modify a bit of a WordPress stylesheet but can’t figure out if the element is an h2 or a sidebar.caption

NFS performance

I noticed when transferring files from the new Core2Quad to the fileserver that there seems to be a big burst of file transfer (like 500Mb in a couple of seconds) and then the remaining 200Mb or whatever seems to take forever – so much so that the windows grey out and Nautilus becomes unresponsive. I’ve got a feeling that this is a Compiz or Fedora9 issue, as it doesn’t present itself with Fedora7 – well some unresponsiveness maybe, but that’s probably due to the not-that-great drives or SATA controller in the fileserver. Hell it could be that the Quad is so much faster than the fileserver that it is waiting for it to catchup!

Anyway I’ve been playing around with NFS settings and have set the async bit in /etc/exports, which seems to give a few percent performance increase (copying a 700Mb file takes 50-ish seconds instead of ~1m5s with sync set). None of the /etc/fstab rsize or wsize settings made any difference. I guess that’s still not bad as it works out at 112Mbps – which includes the write time for a 700Mb file, not just the network speed.

I also tried enabling jumbo frames again on my ethernet cards – well that didn’t work, so I tried a piece of crossover cable in case it was my switches or routing setup, but that made no difference. I even tried using the Intel e1000 card with 9k frames instead of the Realtek r8169 cards with 7k frames – still no go. So I’ve come to the conclusion that MTU>1500 just doesn’t work on Linux 2.6 kernels!

Rsync oddities

I’m trying to sync my “useful software” directory on my desktop with my fileserver, and as there’s so much on there, doing it via NFS and drag’n’drop is a bit hit and miss, so I thought I’d try rsync’ing it.

When I tried using SSH as the transport I was getting strange “Corrupted MAC on input” errors, which nobody seems to know the cause of, its hinted at iffy hardware such as consumer-grade switches. Its nothing to do with MAC addresses, but actually SSH’s checksumming system. I thought it may be due to the VirtualBox bridging setup, so disabled br0 and went with plain old eth0, but that didn’t fix it.

I also tried enabling jumbo frames by issuing “ifconfig eth0 mtu 7200” to enable the RTL8169 chipset’s maximum frame size of 7K, which seems to work fine for NFS, but causes rsync to just sit there doing nothing at the “Receiving file list….” bit.

So I setup a basic rsync server on my desktop machine by creating an /etc/rsyncd.conf file and running rsync –daemon; this stops rsync using a remote shell (such as SSH/RSH) and forces it to use the rsync protocol instead.

Now everything seems to work fine – no SSH MAC issues, and its running up to twice as fast as SSH at about 40MB/s, jumbo frames still don’t work though and I still get a “failed verification” every now and then, although I guess that’s normal enough as I used to find that a lot when mirroring the Fedora Updates repository over the internet – at least it retries rather than giving up as SSH does. I also seem to get a few “permission denied” messages when it tries to transfer PDF files, which seems very odd.

Update: VirtualBox 2.0.2 is out, with many bugfixes, although to me it seems less stable than 2.0.0 as my Vista VM got hosed when I installed teh gest additions, although to be fair I did try to install ServicePack1 at the same time.

VMWare Server 1.0.7 is out too – just some security fixes though.

I watched Finding Amanda, which was a totally awful movie about a gambler who tries to get his hooker niece into rehab, although its him that needs the rehab.