CommVault 8 and Exchange 2010

One of my customers upgraded to Exchange 2010 this year, but their managed backup team could not upgrade their backup solution from CommVault 8 to CommVault 9. CommVault 8 does not support backups on passive DAG copies, but they tested it and got it working… sort of.  The first big issue they had was that CommVault would not truncate their logs after a “successful” backup. I guess I should mention that the only reason they believe the backups are successful is because the CommVault consoled reported it “successful”. They do experience multiple backup job failures for unknown reasons, which require them to kick off a new backup job. The failed backup creates an interesting side effect which we’ll discuss, but first let’s discuss the first issue.

ISSUE #1: Failure to Truncate Logs: With Exchange 2010 in a Database Availability Group, Microsoft will support circular logging if you have more than 3 copies of your databases in the DAG. There are some other caveats to this strategy, which are not in scope of this article. They enabled circular logging on their databases and now their database log drives are not at risk of filling up due to a failed backup, or 10.

ISSUE #2: When the CommVault backup jobs run, they create a shadow VSS copy of the data they are attempting to back up. The location of the VSS copies can be seen by looking at the Shadow VSS tab of the hard-drive properties. When a CommVault job fails it does not clean up the VSS copy that it was creating. This presented an interesting issue. The hard-drive where the customer’s databases were located appeared to be almost full, but if you looked at the Exchange Database and Logs it was only a third the size of the drive. If we granted ourselves permission to the System Volume Information (hidden) folder, then you could see a number of files with a strange name. Based on the time stamp the files definitely appeared to be failed backups. The VSSadmin function described in this link it would fail and report that the snapshots needed to be deleted through the backup tool.

This wasn’t an option because we didn’t have access to the CommVault console, since it was managed by another team. We could also not grant ourselves rights to the files and simply delete them.

We ended up changing the shadow storage size (link). This cleaned out the storage immediately. To prevent issues in the future we set it back to unlimited.vssadmin resize shadowstorage /for=<ForVolumeSpec> /maxsize=<maxsize>

This solved our issue temporarily. The end goal is to upgrade to a backup solution that supports backing up passive copies of Exchange 2010.