Archive for November, 2012

Configuring NTP for VMWare vSphere Environment


NTP is a critical component in every network environment, especially when it comes to Virtualization. Because the entire Virtual Machines are relaying on the host to get the time. When the Domain Controller itself running as a Virtual Machine, cares must be taken of time synchronization between hosts, Virtualized Domain Controllers and other Virtual Machines.

In my environment, since the server vLAN has got access to the Public Internet, this wasn’t a big issue to get the synchronization done properly to the Domain Controllers.

Luckily, I have got two Domain Controllers one running as a Physical Server (PDC) and the second running as a Virtual Machine (ADC). In the initial stage, all the FSMO roles was configured on the Virtual Domain Controller, so the Virtual DC it’s holding the PDC emulator which is responsible for the domain hierarchy which means it’s responsible of the time service as well.

To get this done without the hassle of Time Protocols errors, below is the entire configuration.

  1. Backed up the Virtual DC with Veeam Backup & Replication”.
  2. Transferred the FSMO Roles from the Virtual DC to the Physical DC
  3. Configured the Physical DC to be responsible for the time service as per the MS article http://support.microsoft.com/kb/816042
  4. Configured the Virtual DC to obtain time from Physical DC.
  5. Now, since the majority of our servers are virtual running on VMWare vSphere the ESX configured to obtain the time from the same Physical DC
  6. In vSphere Client, configured the ESX with the Physical Domain Controller as NTP Server.
  7. Ran the ntpdate -q 192.168.10.20 in ESX Server against the Physical Domain Controller
  8. Restart the ESX NTP Service ” service ntpd restart”
  9. Then watch “ntpq -p” to watch time drift in ESX Console.
  10. Domain Members Virtual Machines has been configured not to obtain the time from ESX host, but it’s configured on other VMs that are not member of the Domain.
  11. The time is probably synchronized from the Internet to our Virtual and Physical Servers.

, , , , , , ,

Leave a comment

Steps to Upgrade Veeam Backup & Replication v6.1.x to v6.5.0.109


Hello,

I don’t want to go through a step by step procedure to upgrade from your current version of Veeam Backup & Replication to the latest version, but I would like to go through the procedure that I have followed in my environment and hope you will be benefit.

1. Stop Veeam Services.
2. Backup SQL Database
3. Upgrade SQL Express 2005 to SQL Express 2008 R2 by initiating this in the CMD SQLEXPR_x64_ENU.exe /action=upgrade
4. Once it’s done, start veeam Services.
5. Upgrade Enterprise Manager first if you are using it.
6. Upgrade Veeam Backup & Replication.
7. Launch Veeam Console and you will be prompt it with agent upgrade to upgrade the Repository Servers and Proxy Servers.
8. Just make sure everything is back to normal, jobs, logs, configuration run a job test to make sure everything is fine.

, , , ,

Leave a comment

Cannot add Windows PowerShell snap-in VeeamPSSnapin


You may encounter an error Cannot add Windows PowerShell snap-in VeeamPSSnapin when you run PowerShell scripts on Veeam Backup & Replication to start Backup Jobs after an upgrade from Veeam  Backup v6.x to v6.5

Symptoms:

  • PowerShell Scripts scheduled via Task Scheduler doesn’t start automatically.
  • If you run the script manually via Veeam PowerShell snap-in VeeamPSSnapin, you will receive an error. Cannot add Windows PowerShell snap-in VeeamPSSnapin. But the job will start.

Solution:

Upgrade to PowerShell 3.0 by installing Windows Management Framework 3.0 (http://www.microsoft.com/en-us/download … x?id=34595) which supports .Net 4.0 natively.

, , , , , , , ,

Leave a comment

Backup job failed. Cannot create a shadow copy of the volumes containing Exchange writer’s data.


Been facing the Exchange VSS Writer issue  when I backup my Exchange Server 2007 with Veeam Backup & Replication for quite sometime, and the only way to clear this out is by rebooting the Exchange Server or sometime it will pass through from the second or third job retry.

Last Friday 16th Nov, 2012 my Veeam Backup jobs suppose to run as Full Active Backup as per my configuration, Backup went fine for all the Backup Jobs, except the Backup Job which contains the Exchange Mailbox Server, it gives the usual error;

11/16/2012 8:43:20 PM :: Unable to release guest. Error: Unfreeze error: [Backup job failed.
Cannot create a shadow copy of the volumes containing writer's data.
A VSS critical writer has failed. Writer name: [Microsoft Exchange Writer]. Class ID: [{76fe1ac4-15f7-4bcd-987e-8e1acb462fb7}]. Instance ID: [{8ea7190d-337c-448f-b264-3401303b586b}]. Writer's state: [VSS_WS_FAILED_AT_FREEZE]. Error code: [0x800423f2].]

I have reboot it, and retry the job but no joy it didn’t help. I rebooted second and third time, but no joy The error is persistent.

I searched and searched for a solution but the usual result showed up, either reboot the server to clear the VSS writer Timed Out or restart the Microsoft Information Store which will clear the VSS Timed Out. I have done, both but without any luck.

I fed up from troubleshooting during my holiday and I left home for fishing. While I’m at the sea, my mind triggered out that why don’t you Exclude the C:\Drive of the Exchange VM and select only those drives which contains the Exchange Database?

Hummm, it seems it’s a brilliant idea. As soon as I reached home, I immideltly, logged in remotely again and I Excluded the C:\Drive VMDK SCSI (0:0) from the job and selected only those which contains the Exchange Database, “the disks are vRDM”;


SCSI (0:1)
SCSI (0:2)
SCSI (0:3)
SCSI (0:4)
SCSI (0:5)
SCSI (0:6)
SCSI (0:8)
SCSI (0:9)
SCSI (0:10)
SCSI (0:11)
SCSI (0:12)
SCSI (0:13)
SCSI (0:14)
SCSI (0:15)
SCSI (1:0)
SCSI (1:1)
SCSI (1:2)
SCSI (1:3)
SCSI (1:4)
SCSI (1:5)
SCSI (1:6)
SCSI (1:8)

Imagine, the Job passed the Snapshot Process so fast and I’m surprised that the job is started reading :) and the Exchange Database has been put in Backup Mode and the backup Speed is a bit faster 1.6 TB finished in 4 Hours :)

Wondering:
Why Excluding the SCSI (0:0) which is the C:\SystemDrive of the Exchange 2007 Virtual Machine and Including only the vRDM SCSI Drives the process will pass successfully without VSS Error or Timed Out?

What is the restoration impact of Backing up only the those drives where the Exchange Database resides without the the C:\Drive SCSI (0:0).

Will it be possible configuring another job against the Exchange VM to backup only the C:\Drive SCSI (0:0) without the rest of other drives? And when I want to restore it, I have to restore the Job which contains the C:\Drives and followed by the other Job which backed up the Database Drives?

Update:
I can confirm that, I have Selected All Disks to process under the Disk Exclusions and the job Failed. But I have amended the Selection and I included SCSI (0:0) which contains the Virtual Machine System Drive and the backup successful.
I think the reason why it didn’t work when the All Disks radio button is selected but it does when the SCSI Disks are selected including the SCSI (0:0) which includes the System Drive.

The Virtual Machine is limited to 60 Virtual SCSI Controllers/Targets and when Veeam Backup is initiating job to process the Backup, it creates a Loop within the SCSI Controllers selected in the VM. If any drives / SCSI Controller is presented, it will be added for the Backup. While Exchange is a bit sensitive to wait for the entire process of Veeam Backup Job to go from SCSI (0:0) till SCSI (3:15) which is the last SCSI Controller, it failed because VSS Snapshot it didn’t pass within the time frame that specified by Veeam.
But by selecting the correct SCSI Controller/Disks which are presented within the Virtual Machine, I think Veeam Intelligent enough to pickup only those SCSI Disks which are presented/Selected.

, , , , , , , , , , , ,

1 Comment

How to Power-Off / Shutdown Virtual Machine on VMWare ESXi5


How to Power-Off / Shutdown Virtual Machine on VMWare ESXi5

 

Symptoms

You are experiencing these issues:

  • You cannot power off an ESXi hosted virtual machine.
  • A virtual machine is not responsive and cannot be stopped or killed.
  • You cannot power off an ESXi because it’s hanged with lots of SCSI Storage I/Os

You can start by obtaining the <VMID> and you can run the power action against the VMID generated by the below command;

vim-cmd vmsvc/getallvms – list all VMs registered on the host

vim-cmd vmsvc/power.getstate <vmid> – show power state of a VM
vim-cmd vmsvc/power.shutdown  <vmid>  – shutdown a VM (shutdown guest)
vim-cmd vmsvc/power.reset  <vmid>  – reset a VM
vim-cmd vmsvc/power.off <vmid>  – power off a VM
vim-cmd vmsvc/power.on <vmid> – power on a VM
vim-cmd vmsvc/power.reboot <vmid> – reboot a VM
vim-cmd vmsvc/get.summary  <vmid>  – get summary information for a VM
vim-cmd vmsvc/unregister  <vmid>  – unregister a VM from a host

Example;

# vim-cmd vmsvc/getallvms

Vmid   Name             File                  Guest OS       Version   Annotation

2      VM01   [vESXi5-01] VM01/VM01.vmx   winLonghornGuest   vmx-08

~ # vim-cmd vmsvc/power.getstat 2

Retrieved runtime info

Powered on

~ # vim-cmd vmsvc/power.shutdown 2

(vim.fault.ToolsUnavailable) {

dynamicType = <unset>,

faultCause = (vmodl.MethodFault) null,

msg = “Cannot complete operation because VMware Tools is not running in this virtual machine.”,}

In the above example, it gives me an error that cannot Shutdown the VM as the VMWare Tools is not running.

But by running the command to forcibly power off the VM with Power.Off command, it’s immediately powered off the named VM ID. (2)

~ # vim-cmd vmsvc/power.off 2

Powering off VM:

~ # vim-cmd vmsvc/power.getstat 2

Retrieved runtime info

Powered off

~ #

Commands available under vmsvc/:

~ # vim-cmd vmsvc/
Commands available under vmsvc/:
acquiremksticket                 get.spaceNeededForConsolidation
acquireticket                    get.summary
connect                          get.tasklist
convert.toTemplate               getallvms
convert.toVm                     gethostconstraints
createdummyvm                    login
destroy                          logout
device.connection                message
device.connusbdev                power.getstate
device.disconnusbdev             power.hibernate
device.diskadd                   power.off
device.diskaddexisting           power.on
device.diskremove                power.reboot
device.getdevices                power.reset
device.toolsSyncSet              power.shutdown
device.vmiadd                    power.suspend
device.vmiremove                 power.suspendResume
devices.createnic                queryftcompat
get.capability                   reload
get.config                       setscreenres
get.config.cpuidmask             snapshot.create
get.configoption                 snapshot.dumpoption
get.datastores                   snapshot.get
get.disabledmethods              snapshot.remove
get.environment                  snapshot.removeall
get.filelayout                   snapshot.revert
get.filelayoutex                 snapshot.setoption
get.guest                        tools.cancelinstall
get.guestheartbeatStatus         tools.install
get.managedentitystatus          tools.upgrade
get.networks                     unregister
get.runtime                      upgrade
get.snapshotinfo
~ #

, , , , , , , , , ,

Leave a comment

View Mailboxes in Exchange Server 2007 who Exceed Quota Limit


Mailbox quotas help control the size of mailboxes, For example, would you like to know which mailboxes have reached their prohibit send receive quota, Or view the mailbox size and quota status of a specific mailbox?

You can do all that with the Get-Mailbox and Get-MailboxStatistics Exchange PowerShell cmdlets.
Get-Mailbox -ResultSize Unlimited | Get-MailboxStatistics | where {$_.StorageLimitStatus -notlike "BelowLimit*"} | select DisplayName,LegacyDN,DatabaseName,StorageLimitStatus|Export-Csv c:\ExceedQuota.csv

The above script will loop through each Mailboxes and will get each Mailbox Statistics where their StorageLimitStatus not BelowLimit.

The Get-MailboxStatistics has a property where you can fetch each a property with Where statement and braces bracket {} to get the StorageLimitStatus Property.

The result of the above script is the below table;

DisplayName LegacyDN DatabaseName StorageLimitStatus
Full Display Name /O=Organization/OU=FIRST ADMINISTRATIVE GROUP/CN=RECIPIENTS/CN=Alias Students-DB10 ProhibitSend

Manual work has to be done to remove the LegacyDN and keep only the Alias CTRL+H find what and Replace All is a good to delete the entire LegacyDN name and keep only the Alias.

You can use another script to move the mailbox from one Database which has got a quota limited quota to a higher Database which has higher quota.

In order to move the Mailbox to higher quota Database, you need to know the Database Storage Limit Size. You can achieve this by running the Get-MailboxDatabase cmdlet in Exchange PowerShell and pipe it with the property of ProhibitSendReceiveQuota
 Get-MailboxDatabase | select Name, StorageGroup, ProhibitSendReceiveQuta | Export-CSV C:\DBQuota.csv

Name StorageGroup ProhibitSendReceiveQuota
Staff-DB1-H MAILSERVER\Staff-SG 2355MB
Staff-DB2-N MAILSERVER\Staff-SG 1111520KB
Staff-DB3-L MAILSERVER\Staff-SG 515520KB
Students-DB1 MAILSERVER\Students-SG 1000MB
Students-DB2 MAILSERVER\Students-SG 100MB
Students-DB3 MAILSERVER\Students-SG 200MB
Students-DB4 MAILSERVER\Students-SG 311999KB
Students-DB5 MAILSERVER\Students-SG 1300000KB
Students-DB6 MAILSERVER\Students-SG2 400MB
Students-DB7 MAILSERVER\Students-SG2 522000KB
Students-DB8 MAILSERVER\Students-SG2 800MB
Students-DB9 MAILSERVER\Students-SG2 919200KB
Students-DB10 MAILSERVER\Students-SG2 1024160KB
Staff-DB4-O MAILSERVER\Staff-SG 5000MB
Students-DB11 MAILSERVER\Students-SG3 200MB
Students-DB12 MAILSERVER\Students-SG3 200MB
Students-DB13 MAILSERVER\Students-SG3 200MB
Students-DB14 MAILSERVER\Students-SG3 200MB

Now it’s time to move the Mailboxes from their current Database where has got limited quota to the higher Databases which has higher quota.

In order to move the require users from one Database to another, you need to put their Alias/EmailID into a text file to be called by another script and start moving the mailbox to the target Database.

You can filter the first CSV export by Database Column to get all the users in the limited quota database, copy only the  Alias/EmailID to a text file and use the below script to move them to the target Database;


#############################
# Move-Multiple-Mailbox.ps1 #
# Created By Assaf Miron #
# Date : 06/11/07 #
#############################

Param(
[String]$TargetDB,
[String]$FileToOpen,
[Boolean]$ValidateOnly=$False
)

function IsHelpRequest
{
param($argument)
return ($argument -eq "-?" -or $argument -eq "-help");
}

# Function that displays the help related to this script following
# the same format provided by get-help or -?
Function Usage
{
@"

NAME:
`tMove-Multiple-Mailbox.ps1 [-FileToOpen][-TargetDB]

SYNOPSIS:
`tMoves a list of mailboxes listed in an Excel File to
`ta specific Database.

SYNTAX:
`tMove-Multiple-Mailbox.ps1
`t`t[ -FileToOpen ]
`t`t[ -TargetDB ]
`t`t[ -ValidateOnly [True/False] ]

PARAMETERS:
`t-FileToOpen (required)
`t`tAn Text File wich Contains Principle Names of Users to Move
`t-TagretDB (required)
`t`tTarget Databse Name
`t-ValidateOnly (optional)
`t`tDoesn't move the mailboxes but checks if they can move to the target DB

`t-------------------------- EXAMPLE 1 --------------------------

C:\PS> .\Move-Multiple-Mailbox.ps1 -FileToOpen "C:\UsersToMove.txt" -TargetDB "Students-DB6"

`t-------------------------- EXAMPLE 2 --------------------------

C:\PS> .\Move-Multiple-Mailbox.ps1 -FileToOpen "C:\UsersToMove.txt" -TargetDB "Students-DB6" -ValidateOnly $true

REMARKS:
`tThe Move Mailbox accours with the Max threads Parameter
`tit will not move mailboxes that have errors
`tor mailboxes that need to preserve thier size.

"@
}

####################################################################################################
# Script starts here
####################################################################################################

# Check for Usage Statement Request
$args | foreach { if (IsHelpRequest $_) { Usage; exit; } }

$arrMailboxes = @()

# Validate the parameters
If ($TargetDB -eq "") { Usage; exit; }
Else {
If ($FileToOpen -eq "") { Usage; exit; }
Else {
write-host $FileToOpen

$arrMailboxes = Get-Content $FileToOpen

Write-Host "About to move"$arrMailboxes.Count"Mailboxes to"$TargetDB
$Ans = Read-Host "is This information Correct ? `n[Yes,No]`nDefault(Yes)"
If ($Ans.Contains("n")) { exit; }
If ($ValidateOnly -eq $True)
{
$arrMailboxes | Move-Mailbox -TargetDatabase $TargetDB -MaxThreads 16 -ValidateOnly
}
Else
{
$arrMailboxes | Move-Mailbox -TargetDatabase $TargetDB -MaxThreads 16
}
}
}

, , , , , , , ,

Leave a comment