Apache 2 Brainbench Certification

This is my newest cert.

SSH Public Key Authentication

SSH Public Key Authentication

Putty and Pageant
see also http://docs.google.com/Doc?id=dfxjbxcc_12hc8k38

-----------------------------------------------------------------------------------
On the SSH Server
Verify that the following settings are defined in /etc/ssh/sshd_config

#Protocol 2,1
Protocol 2
#AddressFamily any
ListenAddress 192.168.0.10 # substitute correct IP here

HostKey /etc/ssh/id_dsa # Defines your private key name

# Logging
# obsoletes QuietMode and FascistLogging
#SyslogFacility AUTH
SyslogFacility AUTHPRIV
#LogLevel INFO

# Authentication:

LoginGraceTime 2m
#PermitRootLogin yes
PermitRootLogin no
#StrictModes yes
MaxAuthTries 3

RSAAuthentication no
PubkeyAuthentication yes
AuthorizedKeysFile .ssh/authorized_keys

MaxStartups 5

# no default banner path
Banner /etc/ssh/banner

# override default of no subsystems
Subsystem sftp /usr/libexec/openssh/sftp-server
--------------------------------------------------------------------------------

Verify that there are no settings in /etc/ssh/ssh_config that prevent PubkeyAuthentication:

i.e. you should not see
PubkeyAuthentication no in the file anywhere

--------------------------------------------------------------------------------

On Windows - Use puttygen to generate a SSH-2 DSA key and save the public and private keys.

YOU must then copy the public key directly from the puttygen key window and paste it directly into file called ~/.ssh/authorized_keys on the remote linux box (There can be no extra white space or newline characters in the file!)

chown user.user authorized_keys
chmod 755 .ssh
chmod 644 authorized_keys

On Unix

$ ssh-keygen -t dsa -b 1024 -f /home/bb/.ssh/id_dsa

### note that you can change the properties ############

# IdentityFile ~/.ssh/identity
# IdentityFile ~/.ssh/id_rsa
# IdentityFile ~/.ssh/id_dsa So you could add something like ~/.ssh/servername_dsa
#### in /etc/ssh/ssh_config ########################
Generating public/private rsa key pair.
Enter passphrase (empty for no passphrase): leave blank
Enter same passphrase again: leave blank
Your identification has been saved in /home/localuser/cron/id_dsa.
Your public key has been saved in /home/localuser/cron/id_dsa.pub.
The key fingerprint is:
2e:28:d9:ec:85:21:e7:ff:73:df:2e:07:78:f0:d0:a0 localuser@localhost

Now we have a key in the two files mentioned above. Make sure that no other unauthorized user can read the private key file (the one without the '.pub' extension). Chmod 600 on the id_dsa file.

This key will not work until we put the public portion (id_dsa.pub) into the 'authorized_keys' file on remotehost, specifically the one for remoteuser:

/home/remoteuser/.ssh/authorized_keys

chmod 644 on authorized_keys

Use scp to copy the file to the remotehost:

$ scp /home/localuser/cron/id_dsa.pub remoteuser@remotehost:/home/remoteuser/

SSH to the remotehost:

$ ssh remoteuser@remotehost

Make sure the necessary directory and files exist to authorize connections with this key:

$ if [ ! -d .ssh ]; then mkdir .ssh ; chmod 755 .ssh ; fi
$ mv localhost-rsync-key.pub .ssh/
$ cd .ssh/
$ if [ ! -f authorized_keys ]; then touch authorized_keys ; chmod 644 authorized_keys ; fi
$ cat localhost-rsync-key.pub >> authorized_keys

DenyHosts

DenyHosts Install DenyHosts from Sourceforge rpm download or tar.gz i.e.: rpm -Uvh Deny*.rpm copy /usr/share/denyhosts/denyhosts.cfg-dist to /usr/share/denyhosts/denyhosts.cfg Edit the options in denyhosts.cfg i.e. admin email etc.... Add denyhosts to init.d for automatic startup and shutdown: copy daemon-control-dist to /etc/init.d/name-of-your-choice chkconfig --levels 345 name-of-your-choice on To start the daemon: service daemon-control-dist start # to start the daemon which will monitor, notify, and update hosts.deny file

RSYNC Setup and RSYNC with SSH Setup

RSYNC no SSH /etc/rsyncd.conf looks like the following: use chroot = yes hosts allow = 10.0.0.1 [mysql] path = /apps/data/mysql uid = mysql gid = mysql read only = false comment = MySQL backup (make sure 'hosts allow' is set to the secondary network interface of the opposite system). If you invoke the following command on the primary system,

/usr/bin/rsync -auv --delete /apps/data/mysql/ 10.0.0.2::mysql/

you should see a copy of all the mysql files on the secondary in /apps/data/mysql. If not, then your rsync installation isn't correct yet. RSYNC WITH SSH requirements:

rsync
openssh
cron (or vixie-cron)

Make sure that a remoteuser has read permissions to a /remote/dir/ on a remotehost, and that a local user has write permissions to /local/dir/ on localhost. Also, 'rsync' and 'ssh' should be in the local user's path (use "which ssh" and "which rsync"), 'rsync' should also be in remoteuser's path, and 'sshd' should be running on the remotehost. Test rsync with ssh with: $ rsync -avz -e ssh remoteuser@remotehost:/remote/dir /local/dir/ Generate a private/public pair of keys to allow a 'ssh' connection without asking for a password. This may sound insecure, and it is, but it is better than storing a user password (or key password) as clear text in the script. Furthermore, I can put some limitations on what connections made with this key may do. Anyway, I generate the key I will use on localhost as localuser with: $ ssh-keygen -t dsa -b 1024 -f /home/localuser/cron/id_dsa Generating public/private rsa key pair. Enter passphrase (empty for no passphrase): leave blank Enter same passphrase again: leave blank Your identification has been saved in /home/localuser/cron/id_dsa. Your public key has been saved in /home/localuser/cron/id_dsa.pub. The key fingerprint is: 2e:28:d9:ec:85:21:e7:ff:73:df:2e:07:78:f0:d0:a0 localuser@localhost Now we have a key in the two files mentioned above. Make sure that no other unauthorized user can read the private key file (the one without the '.pub' extension). Chmod 600 on the localhost-rsync-key file. This key will not work until we put the public portion (id_dsa.pub) into the 'authorized_keys' file on remotehost, specifically the one for remoteuser: /home/remoteuser/.ssh/authorized_keys

chmod 755 on authorized_keys Use scp to copy the file to the remotehost: $ scp /home/localuser/cron/id_dsa.pub remoteuser@remotehost:/home/remoteuser/ SSH to the remotehost: $ ssh remoteuser@remotehost Make sure the necessary directory and files exist to authorize connections with this key: $ if [ ! -d .ssh ]; then mkdir .ssh ; chmod 755 .ssh ; fi $ mv localhost-rsync-key.pub .ssh/ $ cd .ssh/ $ if [ ! -f authorized_keys ]; then touch authorized_keys ; chmod 600 authorized_keys ; fi $ cat localhost-rsync-key.pub >> authorized_keys Now the key can be used to make connections to localhost, but these connections can be from anywhere (that the ssh daemon on remotehost allows connections from) and they can do anything (that remoteuser can do). To provide better security edit the 'authorized_keys' file (with vi) and modify the line with 'localhost-rsync-key.pub' information on it. Just add a few things in front of what is already there, changing the line from this: ssh-dss AAAAB3NzaC1kc3MAAAEBAKYJenaYvMG3nHwWxKwlWLjHb77CT2hXwmC8Ap............................ to this: from="10.1.1.1",command="/home/remoteuser/cron/validate-rsync" ssh-dss AAAAB.............................. where "10.1.1.1" is the IP address of localhost, and "/home/remoteuser/cron/validate-rsync" is a script similar to: #!/bin/sh case "$SSH_ORIGINAL_COMMAND" in *&*) echo "Rejected" ;; *(*) echo "Rejected" ;; *{*) echo "Rejected" ;; *;*) echo "Rejected" ;; *<*) echo "Rejected" ;; *`*) echo "Rejected" ;; rsync --server*) $SSH_ORIGINAL_COMMAND ;; *) echo "Rejected" ;; esac Make certain that the 'validate-rsync' script is executable by remoteuser on remotehost and test it. Now that the key is in place and configured, test it out before putting it in a cron job. Exit from the ssh session to remotehost and try: $ rsync -avz -e "ssh -i /home/localuser/cron/localhost-rsync-key" remoteuser@remotehost:/remote/dir /local/dir/ Finally, create a cron script like" #!/bin/sh RSYNC=/usr/bin/rsync SSH=/usr/bin/ssh KEY=/home/localuser/cron/localhost-rsync-key RUSER=remoteuser RHOST=remotehost RPATH=/remote/dir LPATH=/local/dir/ $RSYNC -az -e "$SSH -i $KEY" $RUSER@$RHOST:$RPATH $LPATH Then save the script as something like "rsync-remotehost-bak" Use 'crontab -e' to insert a line for this new cron job that with run at 2AM every day: 0 2 * * * /home/localuser/cron/rsync-remotehost-bak

IPTABLES SETTINGS to Allow Tivoli

# Tivoli Ports to accept from any
iptables -A INPUT -p tcp -m tcp –sport 9494:9495 -j ACCEPT
iptables -A INPUT -p tcp -m tcp –dport 9494:9495 -j ACCEPT

# Tivoli Ports limited by TIVOLI chain
iptables -A INPUT -p tcp -m tcp –sport 9494:9495 -j TIVOLI
iptables -A INPUT -p tcp -m tcp –dport 9494:9495 -j TIVOLI

iptables -A TIVOLI -s src-tiv-IP -j ACCEPT
iptables -A TIVOLI -d dst-tiv-IP -j ACCEPT
iptables -A TIVOLI -j REJECT -- reject-with icmp-port-unreachable

Netbackup Tips

Stopping Netbackup

/etc/init.d/netbackup stop --> graceful shutdown
/usr/openv/netbackup/bin/bpps -a --> check for any remaining processes
/usr/openv/netbackup/bin/goodies/bp.kill_all ---> kills all remaining netbackup processes, not necessarily graceful
/usr/openv/netbackup/bin/bpps -a --> check for any remaining processes
kill -9 <pid> for any remaining. NOTE: unkillable processes may require a reboot

Starting Netbackup

/etc/init.d/netbackup start --> after bp.kill_all, to restart

Common Tasks

Starting the Administration GUI

java from the windows client
java from the server - /usr/openv/java/jnbSA &

Checking Backup Status

Activity Monitor or
/usr/openv/netbackup/bin/admincmd/bpdbjobs -report

Cleaning a tape manually

Identify the drive name to be cleaned

tpclean -L

Manually clean the drive:

tpclean -C <drive name>

Determining what tapes were used for a backup

Backup and Restore --> Find the file system --> Preview Media Button

Find the correct backup images

bpimagelist -U -client <CLIENT> -d <STARTDATE> -e <ENDDATE>

Find the media used for those images

bpimagelist -U -client <CLIENT> -d <STARTDATE> -e <ENDDATE> -media

Listing the files in a backup

bpflist --help --> undocumented netbackup command to list files from a binary .f file

Inventory the Robot

Inventory Robot --> /opt/openv/volmgr/bin/vmcheckxxx -rt tld -rn <robot_number> -list
Inventory Robot and Update Configuration --> /opt/openv/volmgr/bin/vmupdate -rt tld -rn <robot_number> -list

Listing Properties of the Volume Pools

vmpool -listall

Scratch Tapes

Count scratch tapes: /usr/openv/volmgr/bin/vmquery -pn Scratch | grep -c "robot slot"
Moving tapes to the scratch pool

If Needed - Expire the tape

bpexpdate -ev <TAPE ID> -d 0 -force -host <Media Manager>

Move the tape

vmchange -p 2 -m <TAPE ID>

Checking Drive Usage

/usr/openv/volmgr/bin/vmoprcmd -d

Taking a drive down or up

/usr/openv/volmgr/vmoprcmd -down <drive index>
/usr/openv/volmgr/vmoprcmd -up <drive index>

Performing a Restore

From the GUI

user backup & restore --> configuration --> client
user backup & restore --> configuration --> client to restore
directory to search
directory depth
date range
file --> browse backups for restore

Adding New Tapes to the Library

Using the GUI

Media Management --> Actions --> New --> Single Volume . . -->
Media Type (ie DLT)
Robot Type (ie TLD)
Media ID (from Inventory)
Slot Number (from Inventory)
Robot Number (ie 0)
Volume Group
Volume Pool (ie Scratch)

Using the CLI

vmadd -m <media id> -mt <tape type> -verbose -rt tld -b <barcode> -rn <robot number> -rc1 <slot> -p <pool number> -mm <max mounts>

vmpool -listall --> lists all pools, both name and number

For example: vmadd -m Q100L1 -mt hcart -verbose -rt tld -b ECQ100L1 -rn 0 -rc1 8 -p 2 -mm 0

Re-using Tapes from other systems or older Netbackups

Expire the media

bpexpdate -ev MEDIA_ID -d 0 -force -host HOST

Deassign the media

vmquery -deassignbyid MEDIA_ID 4 0

Move to the scratch pool

vmchange -m MEDIA_ID -p POOL#

Relabel the media

bplabel -ev EVA019 -d 8mm -p Scratch

Changing the attributes of media

Changing the barcode

vmchange -barcode CYM100D -m CYM100

Changing the Volume Pool

vmchange -m MEDIA_ID -p POOL#

To expire media

bpexpdate -ev <medai id> -d 0 -force -host <media server>

To unfreeze media

List the frozen media

/usr/openv/netbackup/bin/goodies/available_media | grep -i FROZEN

Unfreeze the media

bpmedia -unfreeze -ev <media id> -h <media server>

To relabel a tape

bplabel -ev <media id> -d <tape density> -p <pool name>
bplabel -ev 000687 -d hcart -p TriVrgt_OFFSITE

To remove media from the Netbackup database

Verify that there are no images on the tape

bpimmedia -mediaid 000687 -L

Expire the tape

bpexpdate -ev 000687 -d 0 -host scorpius -force

Get the status and pool number of the tape

vmquery -m 000687

Deassign the tape

vmquery -deassignbyid <media id> <pool number> <status code from vmquery -m>
vmquery -deassignbyid 000687 4 0x0

Delete the tape

vmdelete -m 000687

Installing the Netbackup Client

/update_clients -ForceInstall -ClientList /tmp/clients.lst

requires that TMPDIR and TEMPDIR be set correctly

Excludng files from backup on a client

Create /usr/openv/netbackup/exclude_list
Put the file specifications of the files/directories to be excluded

/mnt/directory/*

Displaying Information about a Tape

vmquery -m <media id> --> Displays attributes about a particular tape
bpmedialist -U -mcontents -ev 000687 --> Displays media contents
bpmedialist -U -mlist --> List of all media
bpmedialist -U -mlist -ev CYM966 --> Listing of a particular media id
bpimmedia -mediaid 000687 -L --> Listing of images on a tape

Robtest Commands

Starting robtest

robtest
1 --> to select TLD 0

Getting help

Looking at contents of the tape drives

Looking at the contents of the library

Moving a tape from a drive to a library slot

s d --> to identify drive number that has tape (Contains Cartridge = yes, Barcode=XXXXXX)
s s --> to identify an empty slot in the tape library (Netbackup will need to be re-inventoried)
m d# s# --> from from drive # to slot #
s d --> verify the tape drive is empty
s s --> verify the library slot has the tape

Configuration Files

/usr/openv/netbackup/bp.conf

configuration file, sets backup server and backup clients
force statement must be correct
client to browse from
client to restore to

/usr/openv/volmgr/vmconf

Logfiles

To utilize logfiles, create the corresponding directory in /usr/openv/netbackup/logs

Server Logfile directories:

admin - adminstrative commands
bpbrm - backup and restore manager
bpcd - client daemon
bpdbjobs - database manager program process
bpdm - disk manager process
bpjava-msvc - Java application server authentication service
bpjava-usvc - process that services Java requests
bprd - request daemon process
bpsched - scheduler process that runs on master servers
bptm - tape/optical media management process
user-ops - required directory for use by Java programs
xbpadm - X based administration utility
xbpmon - X based job monitor process

Client Logfile directories:

bp - client user interface process
bparchive - archive program
bpbackup - backup program
bpbkar - program that generates golden images
bpcd - client daemon
bpjava-msvc - Java application server authentication service
bpjava-usvc - process that services Java requests
bplist - program that lists backed up and archived files
bpmount - program that determines local mountpoints and wildcard expansion for multiple streams
bphdb - Oracle database backup program start process
db_log - database specific extension log
tar - tar process log during restores
user_ops

Media Manager logging automatically goes to the system log using syslogd logging facility

.Logging will only occur if these directories are created. These directories will generate a lot of data and should be deleted when no longer necessary.

To increase the amount of logging information set VERBOSE=2 in /usr/open/netbackup/bp.conf (default is VERBOSE=1)

Processes

ltid
acsd
vmd

Useful Commands

bpcllist - list classes
bpclinfo <class> -L --> displays info about a class
vmpool - volume pools
vmpool -listall
vmpool -listscratch
bplabel -ev <media id> -d hcart
bpbackup db --> backs up the catalog
bpclclients <policy> --> lists the clients for a particular policy (class)

Troubleshooting

bperror -statuscode <-- displays information about the netbackup error.

No Backups are running:

Check system log file for error messages
Stop and restart all the netbackup processes
Look for a downed drive

/usr/openv/volmgr/bin/vmoprcmd -d
/usr/openv/volmgr/bin/vmoprcmd -up 0 --> this will bring up drive 0 if it's control shows as down

Look for pending requests

/usr/openv/volmgr/bin/vmoprcmd –d or gui --> device management
If there is a pending request either re-assign it to a drive, or deny the request

Downed drive does not come back up or does not stay up

Check for a hardware problem by looking for messages on the tape library
Make sure there is not a tape stuck in the drive

Use robtest (described above) to look at the drives

If there is a tape stuck in the drive, try to remove it using robtest
If robtest fails, then you must manually remove it.

Verify the Client is communicating properly:

bpclncmd -ip <ip address> --> from both client and server
bpclntcmd -hn <hostname> --> from both client and server
bpclntcmd -pn --> from client only

Device Actions

Device Management --> info about tape drives

8mm
hcart (LTO)

Netbackup Client

To check things out do this:

It could be a couple things. Mostly DNS, bp.conf, or something stupid. On
the client run this command

/usr/openv/netbackup/bin/bpclntcmd -pn

/usr/openv/netbackup/bin/bpclntcmd -server "server name"

/usr/openv/netbackup/bin/bpclntcmd ip "ip_address"

One of these usually fails and your able to fix it right off

1074 ./bpclntcmd -hn corpbu1
1075 ./bpclntcmd -ip 10.194.1.129
1076 ping 10.194.1.129
1077 ./bpclntcmd -hn corpldv1
1078 ./bpclntcmd -hn corpbu1.corporate.vox.net
1079 ping corpldv1
1080 ./bpclntcmd -ip 10.194.1.120

Must be able to resolve correctly from the master server and the client or it will not work!!!

Netapp Putty and RemoteShare

pavsan01 Filer view pavsan01> vol options evs08_vmp3 nosnap=on, nosnapdir=off, minra=off, no_atime_update=on, nvfail=off, ignore_inconsistent=off, snapmirrored=off, create_ucode=on, convert_ucode=on, maxdirsize=167690, schedsnapname=ordinal, fs_size_fixed=off, guarantee=volume, svo_enable=off, svo_checksum=off, svo_allow_rman=off, svo_reject_errors=off, no_i2p=off, fractional_reserve=100, extent=off, try_first=volume_grow pavsan01> vol options evs_cluster2_vol1 no_atime_update on

remote share

C:Documents and Settingsmsxgdt>rmtshare \pavsan01

Share name Resource Remark

----------------------------------------------------------------------

--------- IPC$ Remote IPC ETC$ C:etc Remote Administration HOME C:volvol0home Default Share C$ C: Remote Administration vol0$ C:volvol0 Remote Administration vol0bu$ C:volvol0 Backup Administrative

Share evs01_log_vol1$ C:volevs01_log_vol1 MS Exchange Logs

evs02_log_vol1$ C:volevs02_log_vol1 MS Exchange Logs

evs01_log_qtree1$ C:volevs01_log_vol1evs01_... MS Exchange Logs

paveasp1_ea_vol1$ C:volpaveasp1_ea_vol1 MS EMail Archive

paveasp1_ea_qtree1$ C:volpaveasp1_ea_vol1pave... MS EMail Archive

pavmvs01$ C:volpavmvs01 pavMVS01 Test LUN pavmvs01_disk$ C:volpavmvs01pavmvs01_disk pavMVS01 Test LUN

pavmvs01_qtree$ C:volpavmvs01pavmvs01_qtree pavMVS01 Test LUN

evs03_log_vol1$ C:volevs03_log_vol1 MS Exchange Logs

evs02_log_qtree1$ C:volevs02_log_vol1evs02_... MS Exchange Logs

evs03_log_qtree1$ C:volevs03_log_vol1evs03_... MS Exchange Logs

evs01_db_vol1$ C:volevs01_db_vol1 MS Exchange Storage

Group 1

evs01_db_qtree1$ C:volevs01_db_vol1evs01_d... MS Exchange Storage

Group 1

evs03_db_vol1$ C:volevs03_db_vol1 MS Exchange Storage

Group 3

evs03_db_qtree1$ C:volevs03_db_vol1evs03_d... MS Exchange Storage

Group 3

evs06_vmp$ C:volevs06_vmp3 Exchange EVS06

Transaction Log ... evs08_vmp$ C:volevs08_vmp3 Exchange EVS08

Transaction Log ... evs07_vmp$ C:volevs07_vmp3 Exchange EVS07

Transaction Log ...