1. Configure operating System
Make sure the OS is configured properly and all necessary OS patches are installed.
Perform Windows Update for Critical updates only and (deselected IE7 2/15/07)
Boot.ini add /3GB /userva=3030 to end of line
In Windows Explorer, navigate to the system partition. This is the partition that holds hardware specific Windows files such as Boot.ini and NtlDR.
If you cannot see the Boot.ini file, it could be because the folder options are set to hide protected operating system files. If this is the case, in the Explorer window, click Tools, Folder Options, and then click View. Clear the Hide protected operating system files (Recommended) check box. When prompted, click Yes.
Right click the Boot.ini file and select properties and deselect Read Only checkbox.
After the Boot.ini file is visible in Explorer, right-click the file, click Open With, and then click Notepad to open the file.
Add the /3GB and /userva=3030 switches to the end of the line starting with "multi" under the [Operating Systems] section.
Save the changes and close Notepad.
Change boot.ini timeout values for system startup: set 1 node to 10seconds and all other nodes to 90 seconds.
At the command prompt, type: bootcfg /timeout# Where # specifies the time in seconds after which default operating system will be loaded.
If disks are not already partitioned as below go to Computer Management > Disk Management > and create partitions
C: 12GB - Under Properties verify it is named Boot
D: 40.32GB - Under Properties verify it is named Exchanged
E: 16384MB or 16G – Under Properties verify that it is named Pagefile, Trend, Temp
Adjust Pagefile by right clicking My Computer – selecting properties > Advanced > Performance > Settings button – go to Advanced Tab > Virtual Memory > Change button and set C: 512 min 512 max Pagefile on E: 4096 min 4096 max for 4GB
Application Event log increased to 40MB or 40960KB by going to Computer Management > Event Viewer > right-click Application > select Properties and update Maximum Log Size value.
Set Security and System Event Log to Maximum Log Size of 16384KB
Install Windows 2003 sp1 Resource Kit and Support Tools
2. Install SnapDrive 4.1
Install Emulex storport driver 1.20a3 if required
First check Start > Programs > Emulex – if it does not exist then install driver below
Run distribution file at netappstorportminiportkit_1.-20a3-1g.exe 2007.1.23.1139 or newer then Extract to c:program filesemulex.
Run AutoPilot Installer at conclusion of distribution file extraction
Select correct version number as the driver to upgrade to
Click Next and installation begins
When complete, go to Device Manager – SCSI and RAID Controllers and check if HBA is now at the correct version
Reboot if prompted
Install new FCP HAK 3.0
Double-click ntap_windows_fcp_3.0_setup_x86.exe file
Click Next and accept all defaults
Install directory should be c:program filesnetappwak
Select Complete Installation method
Select Install to begin installation
Install Snapdrive 4.1
Double-click snapdrive4.1.exe file
Select all default options
Snap Drive
SDR-MPIO
VYZMIQINJVBTEE
SDR-WIN LUN
JMOFBEWOJFHEYA
SME2K3
EQZKPWA
User Account
msxpv
If SnapDrive must be uninstalled for any reason ALWAYS use the snapdrive4.1exe to do so and select the option to remove MPIO drivers when prompted.
If you cannot see the initiators when trying to create a LUN do the following:
Stop the SnapDrive service from services.msc (DO NOT USE COMPUTER MANAGEMENTSERVICES)
Remove or rename the old HBAAPI.dll (version 2.2.1.0) in the SnapDrive directory (by default, c:Program FilesNetAppSnapDrive), then Start the SnapDrive service
Install Snapdrive 4.2.1
Install Data ONTAP DSM 3.0 for Windows
Double-click ntap_win_mpio_3.0_setup_i386.exe
Select Single Instance Image Mode (cfmode=single_image)
Snap Drive
SDR-MPIO
DZFUPANSZYHXCF
Double-click snapdrive4.2.1 exe
Select all default options
Snap Drive
SDR-WIN
VKCOHUMRBFAAAA
SME2K3
HTFNBGOGJNDCMA
User Account
msxpv
If you cannot see the initiators when trying to create a LUN do the following:
Stop the SnapDrive service from services.msc (DO NOT USE COMPUTER MANAGEMENTSERVICES)
Remove or rename the old HBAAPI.dll (version 2.2.1.0) in the SnapDrive directory (by default, c:Program FilesNetAppSnapDrive), then Start the SnapDrive service
3. Create the Cluster
See section 5 for LUN and drive letter details (PUT IN SCREEN SHOTS)
Create the evs_cluster_vol1/evs_cluster_qtree1/evs_cluster_quorum.lun first on node 1 (pvmsgpp06) according to table below
evs_cluster_vol1/evs_cluster_qtree1/evs_quorum.lun
500
If you cannot see the initiators when trying to create a LUN see end of section 2 above.
Create the \pvsanpp02evs_cluster_qtree1evs_ quorum.lun LUN for the Quorum on node 1
DO NOT PERFORM THESE STEPS ON SUBSEQUENT NODES
Setup Cluster Services
Verify that NICs have been assigned appropriate IP’s on all Nodes
NIC1 for pvmsgpp02 = 132.189.91.102 pvmsgpp03 = .103 pvmsgpp04= .104
GW= 32.189.91.254, SM= 255.255.255.0, DNS=132.189.91.10 and 132.189.8.28
ClassC
Heartbeat Nic SM=255.255.255.192 GW=
pvMSGPP02 = 10.0.36.2
pvMSGPP03 = 10.0.36.3
pvMSGPP04 = 10.0.36.4
Cluster IP 132.189.91.121 SM=255.255.255.0
MSDTC 132.189.91.122 SM=255.255.255.0
EVS02 132.189.91.123
EVS03 132.189.91.124
Rename the network connections Production Network Port (1) and Cluster Heartbeat Port (1)
Right click the cluster Heartbeat Port connection properties, select TCP/IP, click properties, click Advanced, click DNS tab.
Remove the checkbox “Register this connection’s address in DNS”
Click WINS tab and select “Disable NETBIOS over TCP/IP”
Open Network Connections, on the Advanced menu, click Advanced settings.
On the “Adapter and Bindings” move the production network port to the top.
On the “Provider” tab move the “Microsoft Windows Network” to the top.
Run cluadmin on first node evs01
Select File > Create new cluster (FIRST NODE ONLY)
Enter the IP address for the cluster
Make sure Q: is the quorum
If there are any issues with Cluster you can remove and cleanup by removing nodes then running c:cluster node pvmsgpp02 /forcecleanup and starting over by running Create New Cluster option
Verify that console:i:1 is added to first line of RDP connection shortcut via Wordpad (required to see QUORUM on nodes 2 and 3) or use mstsc /console
Connect Quorum Drive on subsequent nodes pvMSGPP03 and 04 in Cluster
In SnapDrive right-click Disks and select connect
Enter full path to Quorum LUN \pvsanpp02evs_cluster_vol1$evs_cluster_qtree1evs_quorum.lun
Add subsequent nodes to cluster – in cluadmin go to File > New > Node
Click add and then select Advanced and chose minimal configuration and continue selecting next through all warnings and finish
Finish the wizard.
After node is added do a move group command and verify that Q: is available from Windows Explorer
Move Group back to the passive node
Perform steps k – r on the remaining nodes
Open the cluster administrator and right click on the cluster name (pvMSGC1) properties, click Network Priority tab and move the cluster heartbeat port up. Click properties of the cluster heartbeat port and select “Internal cluster communications only”
Create the MSDTC
Create MSDTC Drive on pvMSGPP04 in Cluster
In SnapDrive right-click Disks and select create
Enter full path to MSDTC LUN \pvsanpp02evs_cluster_vol1$evs_cluster_qtree1evs_MSDTC.lun
Click Next, select Shared (Microsoft Cluster Services Only)
Select each server and add the initiators and click Next
Create a new group for MSDTC and click Next
Click Finish.
Open cluster Administrator and go to MSDTC group.
Add IP Address, Network and DTC resources. Click add new resource.
Right click and bring MSTDC resource online
Right click and bring the resource online
Right click and bring the resource online.
Remove “Affect Group” from the DTC advanced properties.
4. Create the Volume Mount Point
Create the Volume Mount Point Root LUNS
First create VMP root LUNs according to table below.
In SnapDrive right-click on disks and select Create Disk
Enter Virtual disk path. \pvsanpp01evs_cluster_vmp_roots$evs_cluster_vmp_roots_qtree1
Enter a lun name. evs01_root_f.lun
Select “No under snapshot”
Size 32MB
Select Shared
Create a Cluster Services Group, VMP-Roots
Finish the wizard
Repeat steps a thru I for creating remaining root luns based upon pv_exchange_NetApp_config.xls spreadsheet
Create the database LUNs
In SnapDrive right-click on disks and select Create Disk
Enter Virtual disk path.
\pvsanpp01evs01_vmp1$evs01_sg1_db1_qtree1
Enter a lun name. evs01_sg1_db1.lun
Select “Yes under snapshot”
Size 65700 MB
Select Shared
Select Use Volume Mount Point and enter a name i.e. F:evs01_sg1_db1_vmp as in this example
Finish the wizard
Repeat steps a thru h for creating the other database LUNs, change the highlighted values. See the table below for the details about the virtual disk path and LUN names.
Database Virtual Disk Path | Database LUN | Mount Points-65700MB |
evs01_sg1_db1.lun | F:evs01_sg1_db1_vmp | |
evs01_sg1_db2.lun | F:evs01_sg1_db2_vmp | |
evs01_sg1_db3.lun | F:evs01_sg1_db3_vmp | |
evs01_sg1_db4.lun | F:evs01_sg1_db4_vmp | |
evs01_sg2_db1.lun | F:evs01_sg2_db1_vmp | |
evs01_sg2_db2.lun | F:evs01_sg2_db2_vmp | |
evs01_sg2_db3.lun | F:evs01_sg2_db3_vmp | |
evs01_sg2_db4.lun | F:evs01_sg2_db4_vmp | |
evs01_sg3_db1.lun | F:evs01_sg3_db1_vmp | |
evs01_sg3_db2.lun | F:evs01_sg3_db2_vmp | |
evs01_sg3_db3.lun | F:evs01_sg3_db3_vmp | |
evs01_sg3_db4.lun | F:evs01_sg3_db4_vmp | |
evs01_sg4_db1.lun | F:evs01_sg4_db1_vmp | |
evs01_sg4_db2.lun | F:evs01_sg4_db2_vmp | |
evs01_sg4_db3.lun | F:evs01_sg4_db3_vmp | |
evs01_sg4_db4.lun | F:evs01_sg4_db4_vmp | |
|
|
|
evs02_sg1_db1.lun | G:evs02_sg1_db1_vmp | |
evs02_sg1_db2.lun | G:evs02_sg1_db2_vmp | |
evs02_sg1_db3.lun | G:evs02_sg1_db3_vmp | |
evs02_sg1_db4.lun | G:evs02_sg1_db4_vmp | |
evs02_sg2_db1.lun | G:evs02_sg2_db1_vmp | |
evs02_sg2_db2.lun | G:evs02_sg2_db2_vmp | |
evs02_sg2_db3.lun | G:evs02_sg2_db3_vmp | |
evs02_sg2_db4.lun | G:evs02_sg2_db4_vmp | |
evs02_sg3_db1.lun | G:evs02_sg3_db1_vmp | |
evs02_sg3_db2.lun | G:evs02_sg3_db2_vmp | |
evs02_sg3_db3.lun | G:evs02_sg3_db3_vmp | |
evs02_sg3_db4.lun | G:evs02_sg3_db4_vmp | |
evs02_sg4_db1.lun | G:evs02_sg4_db1_vmp | |
evs02_sg4_db2.lun | G:evs02_sg4_db2_vmp | |
evs02_sg4_db3.lun | G:evs02_sg4_db3_vmp | |
evs02_sg4_db4.lun | G:evs02_sg4_db4_vmp | |
|
|
|
Create the Transaction Log LUNs
In SnapDrive right-click on disks and select Create Disk
Enter Virtual disk path.
\pvsanpp01evs01_vmp3$evs01_sg1_tl_qtree1
Enter a lun name. evs01_sg1_tl.lun
Select “Yes under snapshot”
Size 23760 MB
Select Shared
Select Use Volume Mount Point and enter a name i.e. T:evs01_sg1_tl_vmp as in this example
Finish the wizard
Repeat steps a thru I for creating the other database LUNs, change the highlighted values. See the table below for the details about the virtual disk path and LUN names.
tl Virtual Disk Path | tl LUN | Mount Points 23760MB |
evs01_sg1_tl.lun | F:evs01_sg1_tl_vmp | |
evs01_sg2_tl.lun | F:evs01_sg2_tl_vmp | |
evs01_sg3_tl.lun | F:evs01_sg3_tl_vmp | |
evs01_sg4_tl.lun | F:evs01_sg4_tl_vmp | |
|
|
|
evs02_sg1_tl.lun | G:evs02_sg1_tl_vmp | |
evs02_sg2_tl.lun | G:evs02_sg2_tl_vmp | |
evs02_sg3_tl.lun | G:evs02_sg3_tl_vmp | |
evs02_sg4_tl.lun | G:evs02_sg4_tl_vmp | |
|
|
|
Create the SMTP LUNs
In SnapDrive right-click on disks and select Create Disk
Enter Virtual disk path.
\pvsanpp01evs01_vmp3$evs01_smtp_qtree1
Enter a lun name. evs01_smtp.lun
Select “No under snapshot”
Size 4096 MB
Select Shared
Select Use Volume Mount Point and enter a name i.e. S:evs01_smtp_vmp as in this example
Finish the wizard
Repeat steps a thru I for creating the other database LUNs, change the highlighted values. See the table below for the details about the virtual disk path and LUN names.
SMTP Virtual Paths | SMTP LUNs | Mount Points 4GB |
evs01_smtp.lun | F:evs01_smtp_vmp | |
evs02_smtp.lun | G:evs02_smtp_vmp |
Maintenance LUNs on SAN01 aggregate 1
evs_cluster_mnt_vol2/evs_cluster_mnt_qtree1/evs_cluster_mnt1.lun
400000
5. Install Exchange Server 2003 on Each Node
Installing and Enabling Required Windows Services
Exchange Server 2003 Setup requires that the following components and services be installed
and enabled on the server:
.NET Framework
ASP.NET - verify that this is set to Auto Start
Internet Information Services (IIS)
World Wide Web Publishing Service
Simple Mail Transfer Protocol (SMTP) service
Network News Transfer Protocol (NNTP) service
Install Exchange Server 2003 on all cluster nodes
Make sure that the Cluster service is running on each node.
Install and enable the required Windows services.
Install Microsoft Distributed Transaction Coordinator (MSDTC).
Run Exchange Server 2003 Enterprise Setup.
Ignore error stating there is a compatibility issue with this version of windows.
Create D:Exchsrvr on all nodes
Change Path of installation to D:Exchsrvr and chose typical install on all nodes.
Install SP2 on all nodes
Run Windows Update on all nodes
Post-deployment steps – Tuning Parameters
Set the system pages to 0 (DWORD)
HKEY_LOCAL_MACHINESystemCurrentControlSetControlSession ManagerMemory ManagementSystemPages = 0
Set HeapDeCommit Threshold to 262144 (decimal) (DWORD)
HKLMSystemCurrentControlSetControlSession Manager HeapDeCommitFreeBlockThreshold = value of 262144 (decimal)
Set Guid-Replid Caching to 0 (DWORD)
HKEY_LOCAL_MACHINESYSTEMCurrentControlSetServicesMSExchangeIS"server name""each private database"
Set the MinUserDC set to number GCs – the PDC emulator = 6 (in the current environment). (DWORD)(Decimal)
HKLMSystemCurrentControlSetServicesMSExchangeDSAccessProfilesDefault
Set the msExchESEParamLogBuffers
Go to ADSIEDIT.msc and set the value to 9000
Start ADSI Edit.
Double-click the Configuration container, expand CN=Services, expand CN=Microsoft Exchange, and then expand CN=<ExchangeOrganizationName>.
Expand CN=Administrative Groups, expand CN=<AdministrativeGroupName>, and then expand CN=Servers.
Expand CN=<ServerName>, expand CN=InformationStore, right-click CN=<StorageGroupName>, and then click Properties.
In the Attribute Editor, click the msExchESEParamLogBuffers attribute, and then click Edit.
In the Edit Attribute box, set the value to 512 for Exchange 2000 Server or 9000 (for example, clear the value) for Exchange Server 2003.
Click Apply, and then click OK.
Close ADSI Edit and then restart the Microsoft Exchange Information Store service for the change to take effect.
2) PagedPoolSize = DWORD value of FFFFFFFF
hklm-system-currentcontrolset-control-session manager-memory management
3) DynamicMemory = Add DWORD value of 1
hklm-system-currentcontrolset-control-session manager-memory management
4) PoolUsageMaximum = Add DWORD value of 40
hklm-system-currentcontrolset-control-session manager-memory management
5) MsExchAgingKeepTime = value of 345600 ???? (May be do not need this) (default is 40 days)
set at each DB in ADSIEdit
6) msExchMaxCachedViews = value of 20 (default is 11)
set at each DB in ADSIEdit
http://www.microsoft.com/technet/prodtechnol/exchange/Analyzer/2c732be4-eb00-4cde-8c90-f5f1427575eb.mspx?mfr=true
7) SkipWildcardSearchInGC = value of 1
HKLMSoftwareTrendMicroScanMail for ExchangeCurrentVersion
8) QuerySender = value of 0
HKLMSoftwareTrendMicroScanMail for ExchangeRealTimeScan
9) ScanningThreads = should be twice the number of processors plus one
HKLMSystemCurrentControlSetServicesMSExchangeISVirusScan
10) MaxMessageCount = value of 500
HKLMSoftwareTrendMicroScanMail for ExchangeCurrentVersion
11) MaxScanningThreadCount = value of 25
HKLMSoftwareTrendMicroScanMail for ExchangeRealTimeScanScanOption
12) ScanningThreadPoolCount = value of 25
HKLMSoftwareTrendMicroScanMail for ExchangeRealTimeScanScanOption
13) TempFileThreshld = value of 102400 (or max message size)
HKLMSoftwareTrendMicroScanMail for ExchangeRealTimeScanScanOption
6. Creating the Exchange Virtual Servers
Create the IP Address and Network Name resources for all Virtual Servers EVS01 and 02
Create System Attendant Resource
Make sure you select the LUN where you want to the SMTP/MTA to reside. Exchange will create a mailbox store and public store in the same LUN. Move the mailbox store and delete the public folder store.
If an error occurs then move the computer account from the computers OU to Saint PaulServers OU in AD. Make sure the computers are in the correct OU.
Remove the “affect the group” setting on all resources except network name, IP address, Information Store and System Attendant resources on each virtual server.
Change the Threshold to 1 and the Period to 120 on the advanced tab for Information Store and System Attendant resources.
Change the value of Threshold to 1 on Exchange system attendant and information store on all virtual servers.
Disable EventLogReplication on each virtual server
Cluster.exe /prop EnableEventLogReplication = 0
Add the MsgHandleThreshold registry key on all the nodes.
http://technet.microsoft.com/en-us/library/aa998105.aspx
Disable the MTA service on all nodes and set the following registry keys to disable MTA events for each database
HKEY_LOCAL_MACHINESYSTEMCurrentControlSetServicesMSExchangeISServerNameGateway In Threads – (DWORD) (0)
HKEY_LOCAL_MACHINESYSTEMCurrentControlSetServicesMSExchangeISServerNameGateway Out Threads – (DWORD) (0)
Securing Clusters
http://technet.microsoft.com/en-us/library/bb123629.aspx
http://technet.microsoft.com/en-us/library/bb124140.aspx
7. Create Storage groups and databases.
Each Exchange virtual server has 4 storage groups and 4 databases. The first two databases are journaled in each storage group. None of the servers will contain any public folder stores. Delete any default public folder stores using the following steps.
Start ADSI Edit.
Double-click the Configuration container, expand CN=Services, expand CN=Microsoft Exchange, and then expand CN=<ExchangeOrganizationName>.
Expand CN=Administrative Groups, expand CN=<AdministrativeGroupName>, expand CN=Servers, expand CN=<ServerName>, expand CN = InformationStore, and then expand CN=<Storage Group Name>
Right-click CN=<Public Folder Store Name> and then click Delete.
Double-click the Configuration container, expand CN=Services, expand CN=Microsoft Exchange, and then expand CN=<ExchangeOrganizationName>.
Expand CN=Administrative Groups, expand CN=<AdministrativeGroupName>, expand CN= Folder Hierarchies
Right-click CN=<Public Folders> and then click Properties.
In the Attribute Editor, click the msExchOwningPFTreeBL attribute, and then click Edit.
In the Edit Attribute box, make sure the public folder store is not listed
Click Apply, and then click OK.
Close ADSI Edit and then restart the Microsoft Exchange Information Store service for the change to take effect.
Install and Configure Snap Manager for Exchange 3.2
Permissions for Blackberry Enterprise Server
Setting the required permissions in Exchange for the BES Service accounts on the new mail servers which will be hosting BlackBerry-enabled accounts.
These permissions must be applied at the Storage Group or Server Level!
The required permissions are:
Administer Information Store
View Information Store
Receive As
Send As (Probably no longer a requirement due to how permissions are evaluated, but this would keep us consistent with other servers).
I would recommend adding all BES Service accounts for now, then removing the old management accounts at a later date, once all migrations have been completed.
4 comments:
Paul, nice notes! What SAN are you connecting to NetApp or IBM or ?
This configuration was setup using a NetApp SAN
Paul, do you have any links for NetApp community for posting questions, etc? I need to get help with adding a second Fiber switch and the cfmode recommendations..
Thanks!
I store my Exchange data on a flash card, but was the day when I lost my data on the flash and PC. I was disappointed but quickly observed the solution on the Inet. It maybe will help in such situation - edb repair.
Post a Comment