Subject: tuning Solaris 2

From: [email-masked] (Charles Hedrick)
Date: 9 Oct 93 22:44:26 GMT

We've recently been through a crisis that points up the importance of
tuning with Solaris 2.  I'm sharing our experience in hopes that
others will be moved to do something similar.  I've also requested our
salesman to see if he can get Sun to issue a tuning document giving
some of their experience.

We have a Sparccenter 2000, which is intended for general student
timesharing.  The original goal was to run 2000 simultaenous users.  I
personally have some qualms about that.  It's hard to see how we'd go
about managing a system where "who" goes on for 100 screens, but
that's the goal.  At the moment I don't think we have even 100 users
on at once, but it's still a fairly large-scale system.  Anyway,
this system has turning up a number of interesting scaling problem.

We brought it to its knees last week after increasing the number of
pty's.  First, we found that with more than about 440 pty's, it won't
come up.  OK, so we left it at 440 pty's.  It came up, but performance
was abysmal.  It turns out to be due to ttyname.  A lot of our
interactive software uses ttyname.  Since it's never been thought of
as a an expensive call, some software calls it several times during
startup.  Well, on our system ttyname was taking 10 sec.  of real
time.  Simple interactive programs would take 30 sec. to start.  The
reason is sort of obvious: ttyname works by looking through all of
/dev for a terminal that matches the one it was passed.  In the old
days with 32 pty's that wasn't a problem.  But on a system with 440
pty's, linear search is not a reasonable approach.  All those stats
were bringing the system to its knees.

Anyway, to make a long story short, we've managed to work around the
problem by using the parameters someone recently posted.  They came
indirectly from Sun, as part of an NFS benchmark.  They increase
various caches.  Presumably the effect is to keep that search of pty's
from having to go to disk for every one.  I include the parameters we
used, in case you missed the earlier posting.  As we've got plenty of
memory on the machine, I haven't tried tweaking the parameters
individually to see which ones have the most effect.  These parameters
took ttyname from 10 sec back down to its usual fraction of a second.

Here's the previous posting I mentioned.  Note that I'm currently
using a reduced version of these parameters on my Classic, in hopes of
fixing a serious performance problem on it.  (I can't use these
parameters as is, because they're clearly intended for a machine with
more memory.)  Over the course of a few days, the system slows down,
until starting emacs take 20 seconds.  I'm about to throw the thing
out the window and get a PC.  (My 486 box at home running Linux is
wonderful.)  No, I can't move to 4.1.3.  I have to know if Solaris 2
can be made to work, since otherwise we've got to keep our faculty
from buying any Sun hardware that might depend upon Solaris 2.

----

The *documented* list of /etc/system parameters is in the manual
section "Administering SunOS 5.2 Security Performance and Accounting -
Appendix A - Tuning Kernel Parameters".

Some benchmarks require that tuning parameters are disclosed, so the
SPEC LADDIS report "documents" some tweaks for Solaris 2.3alpha that
helps NFS performance on an SS1000 using FDDI.

set maxusers=32 ( some of the dependent parameters are set explicitly below)
set ufs_ninode=32768
set ncsize=32768
set nsegkp=1024
set nhbuf=256
set p_nbuf=2048
set bufhwm=20480
set bf:bf_nbufs=2048
set bf:bf_nrmds1=256

---

From: [email-masked] (David Robinson)
Date: 10 Oct 1993 23:24:21 GMT

>helps NFS performance on an SS1000 using FDDI.

Beware of using LADDIS tunings.  It is a highly specilaized benchmark
that only mildly resembles a real world NFS server environment.  Using
it's tunings for an interactive timeshare machine may not produce
the desired results.  For example, for an entire LADDIS run not
a single system call is made!  As you can see below almost all of these
tunings are to maximize disk caches.

>set maxusers=32 ( some of the dependent parameters are set explicitly below)
Normally maxusers is tuned to the amount of memory in the system.  Most
of the tunings are based off of this number.  Think of it as a global
knob.

>set ufs_ninode=32768
This is the size of the in core inode cache.  The larger it is the more
memory is used and the less frequently you will have to go to disk
to get a file's attributes.  Tune this to reflect the number of files
that are commonly in use.

>set ncsize=32768
This is the directory name cache size.  It caches file names in diretories
to save disk I/O.  Use vmstat -s, if the hit rate is below 90% you may want to
tune it up.

>set p_nbuf=2048
No longer does anything in Solaris 2.3.  Does almost nothing in Solaris 2.2.

>set bufhwm=20480
The maximum number of buffers for buffered disk I/O.

>set bf:bf_nrmds1=256
These are tunings for the SUN Sbus FDDI card, if you don't have it it
is not useful.