BrandonHutchinsonNotes

Miscellaneous Solaris notes from http://www.brandonhutchinson.com/Miscellaneous_Solaris_notes.html

netstat -k

Determine total amount of bytes (not just packets) sent and received over a network interface: netstat -k | less /hme0 (or your network interface)

hme0: ipackets 1450233797 ierrors 5183 opackets 1312917784 oerrors 0 collisions 122831 24 defer 58 framing 0 crc 0 sqe 0 code_violations 0 len_errors 0 ifspeed 100 buff 5063 oflo 0 uflo 0 missed 107 tx_late_collisions 0 retry_error 0 first_collisions 55296 nocarrier 0 inits 246 nocanput 917395 allocbfail 0 runt 13 jabber 0 babble 0 tmd_error 0 tx_late_error 0 rx_late_error 0 slv_parity_error 0 tx_parity_error 0 rx_parity_error 0 slv_error_ack 0 tx_error_ack 0 rx_error_ack 0 tx_tag_error 0 rx_tag_error 0 eop_error 0 no_tmds 0 no_tbufs 0 no_rbufs 0 rx_late_collisions 0 rbytes 1773527647 obytes 3313629004 multircv 321597 multixm t 0 brdcstrcv 328712706 brdcstxmt 162426 norcvbuf 726653 noxmtbuf 0

netstat -k provides a method to display kernel statistics recorded by the kstat facility.

prtconf usage Display device aliases (along with other information) without being at the Open Boot Prompt ("ok" prompt):

prtconf -pv

Determine current boot-device (i.e. it is possible that the first entry in the EEPROM boot-device order is not necessarily the current boot-device): prtconf -pv | grep bootpath

Here is a script I wrote to convert the "bootpath" into cxtxdx (logical) format. This procedure should work with Solaris 2.6 and later. If it does not work on your system, please send me the prtconf output above along with the full device path to your boot disk.

#!/bin/sh

  1. Determine the Solaris boot disk in cxtxdx format.

BOOTPATH=`prtconf -pv | grep bootpath | tr -d "'" | awk '{print $2}'`

if [ -n "`echo $BOOTPATH | grep "/disk"`" ] ; then # The bootpath contains "disk," but the /devices block device contains # either "sd" or "ssd" BOOTPATH=`echo $BOOTPATH | sed 's/disk@//'` BOOT_DISK=`ls -l /dev/dsk | sed -e 's/ssd@//' -e 's/sd@//' \ | grep "$BOOTPATH" 2>/dev/null | awk '{print $9}' | sed 's/s[0-7]//'` else BOOT_DISK=`ls -l /dev/dsk | grep "$BOOTPATH" 2>/dev/null | \ awk '{print $9}' | sed 's/s[0-7]//'` fi

if [ -n "$BOOT_DISK" ] ; then echo "Your boot disk is ${BOOT_DISK}." else echo "Unable to determine logical boot disk." fi

More information at sun-managers mailing list.

Determine minfree

minfree is the percentage of a file system reserved for the superuser or privileged processes. The value is 10% by default on UFS file systems. /usr/sbin/fstyp -v raw_device | grep minfree

ex.

  1. /usr/sbin/fstyp -v /dev/rdsk/c0t0d0s0 | grep minfree minfree 10% maxbpg 2048 optim time

"Blackhole"/null routing a destination IP address: route add destination_IP 127.0.0.1 -blackhole

Note that this seems to work without the -blackhole flag, and can be used on most Unices.

All network traffic destined to destination_IP will not leave the network interface, as it is given a "next hop" of 127.0.0.1 (localhost). Note that destination_IP can still reach the host, although response traffic will not reach destination_IP, making it appear that communication is broken.

Multiple devalias entries with eeprom ex.

  1. eeprom "nvramrc=devalias c0t0d0 /sbus@2,0/SUNW,socal@d,10000/sf@0,0/ssd@w220000203714f2ee,0:a devalias c3t6d0 /sbus@6,0/SUNW,socal@d,10000/sf@1,0/ssd@w2200002037093df0,0:a"

Make sure to also set use-nvramrc? to true. These device aliases will be stored after a system reset.

  1. eeprom "use-nvramrc?=true"

SENA disk serial numbers Script to display disk serial numbers for fibre channel devices and devices in a Sun Enterprise Network Array (SENA):

#!/bin/sh

for disk in `format < /dev/null | egrep "[0-9]+\." | awk '{print $2}'` do serial=`luxadm display /dev/rdsk/${disk}s2 2>/dev/null | grep Serial | awk '{print $3}'` echo "$disk\t$serial" done

Example output: c0t0d0 LAV27428 c0t2d0 LAY18677 c1t0d0 LAV00875 c1t1d0 LAY19210 c1t2d0 LAY18927 c1t3d0 LAY17116 c1t4d0 LAY19231 c1t6d0 LAY18239 c1t16d0 LAY19212 c1t17d0 LAY18981 c1t18d0 LAY19339 c1t19d0 LAY16957 c1t20d0 LAY17615 c1t22d0 LAY15851

"configuring IPv4 interfaces:ifconfig: setifflags: SIOCSLIFFLAGS: interface: Cannot assign requested address interface"

An /etc/hostname.interface file exists, but the value in /etc/hostname.interface file is not in /etc/hosts. Add the IP address and hostname listed in /etc/hostname.interface to /etc/hosts.

Determining RPM of a Sun disk Run mkfs -m /dev/rdsk/cxtxdxs2 and multiply the rps= value (revolutions per second) by 60 to determine the RPM.

ex. # mkfs -m /dev/rdsk/c0t0d0s2 mkfs -F ufs -o nsect=133,ntrack=27,bsize=8192,fragsize=1024,cgsize=16,free=10,rps=90,nbpi=2049,opt=t,apc=0,gap=0,nrpos=8,maxcontig=16 /dev/rdsk/c0t0d0s2 412964

90 (rps) * 60 (seconds) = 5400 RPM.

"UX: groupadd: ERROR: group is already in use. Choose another." If you attempt to add a group with a name longer than 8 characters, you may see this error message with Solaris even if group is not in /etc/group or the NIS group map. You may manually add groups longer than 8 characters to /etc/group to overcome this limitation, although grpck will complain about these entries.

Solaris routing with multiple interfaces In this example, we have an hme2 network interface with an IP address of 65.201.213.117, a network address of 65.201.213.112, and a subnet mask of 255.255.255.240. The interface was initially enabled using the default subnet mask and broadcast address.

ifconfig hme2 hme2: flags=1000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4> mtu 1500 index 3 inet 65.201.213.117 netmask ff000000 broadcast 65.255.255.255

Routing Table: IPv4 Destination Gateway Flags Ref Use Interface


-------------------- ----- ----- ------ --------- 65.0.0.0 65.201.213.117 U 1 8 hme2

To correct the routing table:

1. Specify the correct subnet mask and broadcast address with ifconfig. ifconfig 65.201.213.117 netmask 255.255.255.240 broadcast 65.201.213.127

2. Add an /etc/netmasks entry for the correct netmask to be used after system reboot. 65.201.213.112 255.255.255.240

Corrected routing table:

Routing Table: IPv4 Destination Gateway Flags Ref Use Interface


-------------------- ----- ----- ------ --------- 65.201.213.112 65.201.213.117 U 1 1 hme2

"/usr/share/man/windex: No such file or directory" Run catman -w as root to rebuild the "windex" database that is used by the whatis and man -f and -k options.

List installed packages by size The following script will list all installed Solaris packages with an estimate of their size using the ouput of pkginfo. A better implementation would be a Perl script that would look at the /var/sadm/install/contents file.

#!/bin/sh

printf "%-10s %-10s %-s\n" "Name" "Size (k)" "Description" printf "%-10s %-10s %-s\n" "----" "--------" "-----------"

pkginfo | while read line do set $line category=$1 name=$2 description=`pkginfo -l $name | grep DESC: | awk -F: '{print $2}'` size=`pkginfo -l $name | grep "blocks used" | awk '{print $1}'` [ -n "$size" ] && size_in_kbytes=`echo "$size * 512 / 1024" | bc` printf "%-10s %-8d %-s\n" "$name" "${size_in_kbytes}" "$description" 2>/dev/null done

A "block" (disk block) referred to in pkginfo -l output or df output is 512 bytes. By default, Solaris UFS file systems are created using 8K blocks (filesystem blocks).

"Clock board TOD does not match TOD on any IO board."

Run the command copy-clock-tod-to-io-boards from the Open Boot Prompt ("ok" prompt).

patchadd return codes The "Return codes" listed when installing the Sun Recommended Patch Clusters are listed toward the top of the patchadd script. Here are some common return codes:

  1. 2 Attempt to apply a patch that's already been applied
  2. 8 Attempting to patch a package that is not installed
  3. 25 A required patch is not applied

Mounting an ISO image The following instructions are for Solaris 8 and later. You must be root to mount the ISO image.

lofiadm -a absolute_path_to_ISO mount -F hsfs -r /dev/lofi/1 mount_point

ex. mounting /home/hutch/cdrom.iso to /tmp/mnt lofiadm -a /home/hutch/cdrom.iso /dev/lofi/1

mount -F hsfs -r /dev/lofi/1 /tmp/mnt

"WARNING: Timed out waiting for NIS to come up" If a Solaris system is configured as a NIS client and there is a problem reaching a NIS server, the system will hang indefinitely when booting. To fix,

1. Perform a send brk to get to the Open Boot Prompt ("ok" prompt). 2. Boot to single-user mode with boot -s 3. Temporarily disable RPC services or remove /etc/defaultdomain to prevent NIS from starting at system boot. mv /etc/rc2.d/S71rpc /etc/rc2.d/NOS71rpc -- or -- rm /etc/defaultdomain 4. Type exit to continue booting your system to your default runlevel.

The system will now boot. Begin troubleshooting problems reaching the NIS server with /etc/init.d/rpc start (if you disabled RPC) or domainname your_NIS_domainname;/usr/lib/netsvc/yp/ypstart (if you removed /etc/defaultdomain).

Once you determine the problem, re-enable RPC services with mv /etc/rc2.d/NOS71rpc /etc/rc2.d/S71rpc or re-create /etc/defaultdomain, depending on which method of disabling NIS you chose.

"Blackhole"/null routing a destination IP address route add destination_IP 127.0.0.1 -blackhole

Note that this seems to work without the -blackhole flag, and can be used on most Unices.

All network traffic destined to destination_IP will not leave the network interface, as it is given a "next hop" of 127.0.0.1 (localhost). Note that destination_IP can still reach the host performing the null routing, although response traffic will not reach destination_IP, making it appear that communication is broken.

"umount: I/O error" umount: cannot unmount /mount_point I received this error on a Solaris 8 system when the disk containing a Veritas volume failed. After killing all the processed identified with fuser -c /mount_point and confirming no other processes were in use in the file system with lsof +D /mount_point, I had to issue the umount -f /mount_point command to forcibly unmount the file system:

  1. umount /file_system umount: I/O error umount: cannot unmount /file_system
  2. fuser -c /file_system /file_system:
  3. lsof +D /file_system
  4. umount -f /file_system
  5. mount | grep /file_system #

For Solaris users, only Solaris 8 and later provide the umount -f ("force") flag. Other Unices may have the "force" flag for umount.

Volume Management (vold) problems Starting vold via /etc/init.d/volmgt start or /usr/sbin/vold will occasionally fail.

  1. /usr/sbin/vold Mon Dec 22 14:49:07 2003 fatal: mounting of "/vol" failed
  2. cd /vol NFS server for volume management (/vol) not responding still trying NFS getattr failed for server for volume management (/vol): error 23 (RPC: Unitdata error) /vol: bad directory

To fix the problem, unmount the /vol directory (you may have to use the -f "force" flag in umount for Solaris 8 and later) and restart volume management.

"No directory! Logging in with home=/" This error message may occur on Solaris systems when the automount daemon (automountd) is already running when NIS is started (/usr/lib/netsvc/yp/ypstart). To fix this, restart the automount daemon via /etc/init.d/autofs stop;/etc/init.d/autofs start and re-login.

Sun Explorer The Sun Explorer Data Collector is a tool that gathers extensive system information primarly for the use of Sun engineers. The latest version of the tool may be downloaded at http://sunsolve.sun.com/diag/explorer/SUNWexplo.tar.Z

Problems starting vold If the volume management service will not start via /etc/init.d/volmgt start or /usr/sbin/vold, and the following errors are listed in /var/adm/messages and /var/adm/vold.log, you may need to unmount /vol. Run fuser -c /vol to see which process ID(s) are using /vol. Kill these process(es), and restart volume management.

/var/adm/messages errors: /usr/sbin/vold[9808]: svc_tli_create: could not bind to requested address: No Error /usr/sbin/vold[9808]: svc_tli_create: Cannot create server handle

/var/adm/vold.log errors: fatal: svc_tli_create: Cannot create server handle

Booting off an external CD-ROM To boot off an external CD-ROM, determine the device path with probe-scsi-all from the Open Boot PROM (OBP or "ok" prompt). Note: you may want to temporarily perform a setenv auto-boot? false and reset-all if OBP warns you that executing probe-scsi-all could hang your system.

Relevant probe-scsi-all output:

/sbus@b,0/SUNW,fas@3,8800000 Target 3 Unit 0 Removable Read Only device PLEXTOR CD-R PX-R820T 1.0811/21/00 16:00

In this example, the SCSI target ID ("Target") is 3, and the physical path to the external CD-ROM is /sbus@b,0/SUNW,fas@3,8800000. To boot a Solaris Operating Environment CD using this external CD-ROM, you would issue the command:

{e} ok boot /sbus@b,0/SUNW,fas@3,8800000/sd@3,0:f

In the command above, sd@3,0 refers to the SCSI target (3) and LUN/Unit (0).

Note: when connecting an external CD-ROM device, make sure that the CD-ROM drive and system are both powered off. Connect the CD-ROM, and power on the CD-ROM and then the system. Failure to follow this procedure may result in the following errors:

{e} ok boot /sbus@b,0/SUNW,fas@3,8800000/sd@3,0:f Boot device: /sbus@b,0/SUNW,fas@3,8800000/sd@3,0:f File and args: Bad magic number in disk label Can't open disk label package

Can't open boot device

Repairing a hard disk with bad sectors The format command may be used to repair a hard disk that is experiencing read/write errors.

Example error:

WARNING: /sbus@3,0/SUNW,fas@3,8800000/sd@0,0 (sd120): Error for Command: read Error Level: Retryable Requested Block: 577712 Error Block: 577796 Vendor: SEAGATE Serial Number: 9737K75183 Sense Key: Media Error ASC: 0x16 (data sync mark error), ASCQ: 0x0, FRU: 0xd2

While the format command allows you to repair an individual block, it is probably best to run analyze and perform at least a read test, as there may be more than one bad block on the disk.

The analyze command will automatically repair errors it finds when scanning the hard disk for bad blocks.

analyze> read Ready to analyze (won't harm SunOS). This takes a long time, but is interruptable with CTRL-C. Continue? y

        pass 0WARNING: /sbus@3,0/SUNW,fas@3,8800000/sd@0,0 (sd120): Error for Command: read                    Error Level: Retryable Requested Block: 577800 Error Block: 577800 Vendor: SEAGATE                            Serial Number: 9737K75183 Sense Key: Media Error ASC: 0x16 (data sync mark error), ASCQ: 0x0, FRU: 0xd2 Medium error during read: block 577800 (0x8d108) (267/8/0) ASC: 0x16 ASCQ: 0x0 Repairing hard error on 577800 (267/8/0)...ok.

Back to brandonhutchinson.com. Last modified: 08/24/2004