 | | | NFS Help! Terrible performance with sync, fast performance with async | NFS Help! Terrible performance with sync, fast performance with async 2006-11-21 - By Stephen John Smoogen
Back
On 11/20/06, Christopher McCrory <chrismcc@(protected)> wrote:
> Hello...
>
Sorry for misparsing which Chris(topher) I was talking to.
> On Mon, 2006-11-20 at 15:06 -0700, Stephen John Smoogen wrote:
> > On 11/20/06, Christopher McCrory <chrismcc@(protected)> wrote:
> > > Hello...
> > >
> > > On Sun, November 19, 2006 1:58 am, Chris Wornell wrote:
> > > > I've got a problem that I've spent quite a bit of time on, though I'm
> > > > not an expert at NFS. In summary, operations that require meta-data
> > > > changes (such as file/directory creations/deletions), perform extremely
> > > > slow over sync, but over 10x faster using async.
> > > >
> > >
> > > The same thing happens with RHEL4; software raid and lvm/lvm2 make it
worse.
> > > I opened a support ticket with RH, but it didn't really go anywhere.
> > > I'm considering switching my NFS server to FreeBSD 6.2
> >
> > I am going to assume that you did this a couple of days ago.. as
> > opening a ticket on a Sunday nigth and expecting a resolution by
> > Monday morning is a bit tight.
>
> Actually I opened a ticket last June. It went something like:
> Me: examples ( like below)
> RH: We replicated in lab, please do these other tests
> Me: ( busy ) figured if they can see it in their lab, why do they need
> me? There must be lots of other people seeing the same thing. I
> probably should have followed up.
>
yeah. The reason for usually asking for more tests is trying to figure
out what might be causing it if there are multiple ways it could be
triggered.
The problems look to be in the kernel hardware interaction. THe fact
that your system gets slower on software-raid(SR), LVM, SR+LVM, etc
shows that the NFS server is having a hard time navigating the
interactions.
To be honest, this may be a place where Linux is just going to suck.
The NFS server stuff has a lot of issues to deal with.. and reading
http://nfsworld.blogspot.com/ (which I am guessing is from Sun ) there
are a lot of places where it hasnt been optimized.
> ( Again, I was not the OP but,) here are my numbers:
> ( done on live systems with people using them so MMV )
> nfs mount opts: wsize=8192,rsize=8192,hard,tcp
> GigE connects
> NFS export is LVM2 over software raid
> lots of 15k U320 scsi drives
> with RHEL3 client and RHEL4 server
> roughly the same with RHEL5
>
> summary: untar file with overwrite takes same amount of time as copying
> a single file 10x as large. File creation seems to be the bottleneck,
> not data transfer.
>
> Not shown, but non-raid, non-lvm filesystems are worse than single
> drives, but single drives are still painful over NFS. e.g. "cvs co" or
> "svn co"
>
>
> # copy from NFS server to local disk: 2.1G iso
> [chrismcc@(protected) iso]$ time cp
> RHEL5-Server-20060830 (See http://ver-20060830.ora-code.com).1-SRPMS-DVD.iso /tmp/
>
> real 1m17.591s
> user 0m0.340s
> sys 0m19.680s
>
> # copy from local disk to NFS server 2.1G iso
> [chrismcc@(protected) iso]$ time
> cp /tmp/RHEL5-Server-20060830 (See http://ver-20060830.ora-code.com).1-SRPMS-DVD.iso
> RHEL5-Server-20060830 (See http://ver-20060830.ora-code.com).1-SRPMS-DVD.iso.copy
>
> real 3m25.563s
> user 0m0.140s
> sys 0m16.000s
>
> # untar from local disk to NFS server 214M uncompressed tarball
> [chrismcc@(protected) iso]$ time tar
> -xf /tmp/mysql-debug-5 (See http://bug-5.ora-code.com).0.24a-linux-i686.tar
>
> real 2m43.387s
> user 0m0.040s
> sys 0m1.890s
>
> # untar from local disk to NFS server 214M uncompressed tarball with
> overwrite
> [chrismcc@(protected) iso]$ time tar
> -xf /tmp/mysql-debug-5 (See http://bug-5.ora-code.com).0.24a-linux-i686.tar
>
> real 3m26.998s
> user 0m0.050s
> sys 0m2.300s
>
>
> ## just for comparison, async exports:
> # async iso test:
> chrismcc@(protected) async]$ time
> cp /tmp/RHEL5-Server-20060830 (See http://ver-20060830.ora-code.com).1-SRPMS-DVD.iso
> RHEL5-Server-20060830 (See http://ver-20060830.ora-code.com).1-SRPMS-DVD.iso.copy
>
> real 0m41.678s
> user 0m0.270s
> sys 0m13.380s
>
> # async tar test
> [chrismcc@(protected) async]$ time tar
> -xf /tmp/mysql-debug-5 (See http://bug-5.ora-code.com).0.24a-linux-i686.tar
>
> real 0m8.259s
> user 0m0.050s
> sys 0m1.540s
>
--
Stephen J Smoogen. -- CSIRT/Linux System Administrator
How far that little candle throws his beams! So shines a good deed
in a naughty world. = Shakespeare. "The Merchant of Venice"
--
Taroon-list mailing list
Taroon-list@(protected)
https://www.redhat.com/mailman/listinfo/taroon-list
|
|
|