 | | | RHEL + SAN feedback [long] | RHEL + SAN feedback [long] 2005-12-02 - By ShinYuh Yong
Back
Hi guys,
I am also implementing something similar as mentioned below and being a
Linux nut, I am very loss about the whole thing. Can someone please help to
advise me?
The project: To have 5 linux machines (RHEL4 Update 2) connected to EVA8000.
Oracle 10g RAC to be setup in EVA8000. Intended FS to be used is OCFS2 on
affected LUN in EVA8k
Problem: Oracle Cluster File System 2 does NOT support normal file, other
than oracle data files, oracle archival log, etc. We have requirement for
oracle cronjobs to be run and output to the EVA8k. Since OCFS 2 does not
support normal files, then I am STICK!!!! These output files must be
available in the EVA8k for access from any of the 5 linux Proliant Servers
(running AMD 64bit Linux OS)
Can anyone share how, what is the right way to go about doing this? I have
being tasked to do this just today and to give solution in 3 days time.
Things about using what LVM, any other FS that is cluster aware??
Please, really appreicate any advise/ help given.
Thank you.
Best Rgds
YONG
####################################################
On Mon, Jun 27, 2005 at 06:40:07PM -0400, Al Tobey wrote:
>On Mon, 2005-06-27 at 12:38 -0700, Tobias Speckbacher wrote:
>I haven't seen Dag Wieers or Ed Wilts pipe in on this thread yet, but
>I know they both have experience with these arrays and HBAs, so you
>might find some wisdom from them (highly recommended!) if you search
>the archives - try "qlogic" and "HP EVA".
Sheesh, can't a guy take a vacation? :-). I just came back from a
lengthy well-deserved holiday and am just starting to catch up in the
middle of a thunderstorm (one last week felled a tree that I had to cut
up this morning).
> > We are currently evaluating the implementation of a SAN to work in
> > conjunction with an Oracle 10gR2 RAC. The host OS will be RHES 3 or 4.
> > The hba's of choice are going to be QLogic cards.
> > > The SAN products we have narrowed our choice down to are the HP
>EVA-6000
> > and the EMC Cx500.
> > > This will be our first SAN, thus I do not have any working experience
> > with either system.
> > > If you have worked with either of these (EVA-6000 is pretty new,
> > EVA-3000 or 5000 exp should do) I would appreciate your feedback.
I've got 2 RHEL3 systems running off multiple EVA 5000 controllers. We
don't have any EVA 6000s yet (an 8000 is likely later this year). My
RHEL4 environment is limited to non-SAN systems so far.
>From what I've read, RHEL4 is not quite SAN-ready yet, at least as far
as the EVA5000 goes. The 5000 is quite a different beast from the 6000
- the 3/5 controllers are active/passive whereas the 4/6/8 are
active/active - a HUGE difference when considering failover scenarios.
> > My main criteria are in order of importance:
> > > 1. Management
>
>I have a 9i RAC cluster on RHEL 3 on EVA 5000 storage. We use Qlogic
>(HP re-branded FC2312's - two GBICs on one PCI card) with HP/Brocade
>switches. I typically run HP's driver, which is just a release from
>Qlogic that they've blessed. Redhat typically picks those drivers up
>fairly soon after HP and I've also had no trouble with RedHat's
>shipped driver. Maintaining a driver binary outside Redhat's is a
>bit tedious and not explicitely supported by them, so I recommend
>using the on included with your distro - it'll work fine.
My first system was set up using HP's provided Qlogic driver - it's
downloadable from their web site. It worked fine for me. My second
system used out-of-the-box Red Hat drivers and they're flawless. I
removed the HP version of the drivers from my first system and both are
now running the stock kernels.
(so much for just the thunderstorm - the tornado sirens just went off).
OK, after huddling in the basement for a while, I'm back...
I'm using stock Qlogic HBAs (I can't remember the model number off the
top of my head and I'm not at work right now).
I did some testing of the failover capabilities by toggling the SAN
switch ports while hammering the file system with bonnie tests. Bonnie
was happy but naturally whenever the switch ports were toggled I saw
messages into the system logs.
>I was recently trained on the EVA's management interface. It took
>about 5 minutes, most of which was explaining how we laid out the disk
>rather than how to use the tool. Allocating disk takes less than 5
>minutes per/lun if you take your time. Setting up multipathing on
>the Qlogic card is very easy using HP's fibreutils package (search for
>FCA2214 on HP's driver page).
Agreed. The package has an easy tool to rescan the SCSI bus if you add
drives later and don't want to reboot. I tested this too since I had to
add a volume to my system while it was already live.
>Dag Wieers has a tool that sets this
>stuff up, too: http://dag.wieers.com/home-made/qla-autoconf/.
I haven't tested this. If memory serves my right, Dag has IBM SAN gear.
> > 2. Scalability
>
>Currently, the EVA 3000 and 5000 do not support active-active at the
>array level, although I believe the EVA 6000 does. Failover support
>for the Qlogic+EVA has worked flawlessly without much effort to set it
>up. So, if you're looking to do more than 2GB/s to your servers,
^^ 2Gb/sec (bits, not bytes)
>you'll want to find out more about active-active support on the EVA
>6000.
Although my SAN-connected RHEL3 servers are mainly FTP servers, bonnie
did perform very well on the EVA volumes. I can't remember the numbers,
but I think we were sustaining 80 MB/sec, certainly much more than my
servers would ever see in production.
>I would not recommend Veritas on Red Hat as you'll be better
>off in the long run using Qlogic with failover and waiting for RH to
>support active-active with device-mapper+CLVM. Then again, I have no
>experience with Veritas on Linux, so take that with a grain of salt.
I don't have Veritas DMP experience on Linux either but the Solaris
admins were quite happy when we were still an EMC shop (before my Linux
systems were SAN connected). DMP supports only active/active paths so
it's not an option for the EVA 3000/5000 controllers but I would expect
it to work well on the 4/6/8000 line.
>You should contact Redhat or Oracle (or both!) to determine if you can
>set up something on RHEL4 that is supported. RHEL4 has a lot of
>features that make management much easier (like on-line resizeable
>filesystems). The main hold-back is that OCFS is not supported on
>the 2.6 kernel, OCFS2 is not released yet, and AFAIK GFS is not
>certified for RAC yet. So, from a support point of view, you'll
>probably want to go RHEL3+OCFS, if only to make sure you have a
>certified clustering file system. I know there are other CFS's out
>there, but have no experience with them and I'm intentionally ignoring
>raw disk.
RHEL3 is certainly proven today. RHEL4 is a little more leading-edge,
but if you can get certified pieces, you should be in great shape. You
didn't say when you needed it working, but I know that there is active
development to get this all working quickly for RHEL4. You should log a
formal call with Red Hat to get the latest status.
> > 3. Support
>
>My personal opinion is that the HP array is the way to go in the long
>run because you can easily stay away from binary kernel modules. Both Red
>Hat and Oracle have a much better chance of supporting you if
>they don't have to wonder about what binary modules are doing to your
>kernel. AFAIK, the Qlogic drivers are fully supported by both
>Redhat _and_ HP. Once CLVM is available, you'll have a
>fully-supported cluster LVM, too.
Agreed. For Red Hat support, though, you'll need at least a standard
level of support - the basic contract doesn't cover the SAN components.
> > 4. Performance
>
>This is largely dependent on how you configure your array, but I have
>had no complaints about performance on our Linux/EVA disk. Then
>again, our RAC instance on Linux is low-volume. We do have a number
>of dev/test HP-UX/Oracle instances and our Exchange server on the same
>array and they also are rarely the subject of performance complaints.
The EVAs are very fast and we haven't been able to saturate them and
we're trying by hammering them with systems running Windows, Linux,
Solaris, and HPUX with both normal file systems and Oracle databases.
>In closing:
>
>If I sound like a guised HP sales person, I'm not. I just work at a
>shop that has done well by having an all-HP environment, from a
>support perspective. Over all, I've been very happy with the EVA5000
>+ Qlogic + HP combination.
Ditto (and I don't work for HP either).
>The EMC array is probably comparable on almost all points except for
>the driver support.
EMC wants you to run Powerpath, a closed-source driver. If you see the
archives from the redhat lists, there have been a few issues reported
and Red Hat's comments have typically been "they have our source - we
don't have theirs. Contact EMC".
>If you need a full 4GB/s load balanced pipe to the array, you'll
>probably get there faster with EMC PowerPath. I believe this will be
>available at some point for the EVA 6000 if it is not already.
>Failover support on EVA+Qlogic is free and quite reliable.
EMC PowerPath won't support the EVA6000. HP's version, SecurePath, has
effectively been discontinued - HP's approach today is to use native
drivers whenever possible (Veritas DMP on Solaris, MPIO on Windows,
Qlogic failover on Linux, etc., native on OpenVMS and Tru64).
>Lastly, dual pathing is probably the single most important "best
>practice" when implementing a SAN. In my experience, most of the
>problems people have with their SAN are due to having some single
>point of failure in the path between the host and disk, be it a
>switch, an HBA, or an array controller. If you go through and count
>all the pieces of your SAN and come up with an odd number, something
>isn't quite right.
>2 HBAs/server, 2 switches, 2 array controllers
I manage the SAN in our shop too (100TB on 4 EVA controllers and growing
at a rate of a new EVA every quarter or so). I dual-path whenever I
can.
>Many shops (like mine) even go with 2 arrays (one production, one D.R.).
I've got a few applications that do a software mirror between arrays for
protection against array failures (yup, they do happen - I've seen them
on both EMC Symmetrix and HP EVA).
We've been an EMC shop and an HP shop. Both generally work and you'll
find cases where people have had issues with one or the other (or both).
Both companies are reputable and stand behind their arrays. I
personally think that HP is more dedicated to Linux and the open source
community than EMC is. EMC supports Linux but doesn't seem to give back
to the community like HP does (that's my perception which doesn't always
match reality).
.../Ed
--
Taroon-list mailing list
Taroon-list@(protected)
https://www.redhat.com/mailman/listinfo/taroon-list
Earn $52 per hosting referral at Lunarpages.
|
|
|