vApp not powering on after failover test

I had a very frustrating issue and I wanted to post the problem and solution here in hopes of 1) possibly helping someone out if they run into this and 2) providing exposure to it in hopes that Zerto’s engineering/QA teams will test this and possibly work with VMware if needed

We had an issue where a client was performing test failovers into our vCD-backed Zerto DRaaS and the vApp wouldn’t power on.

After looking at the vcloud-container-info.log (grepping case insenstive for ‘error’) I found these:

ERROR | storage-fabric-activity-pool-976 | SpbmCallable | Storage profile is not associated with VM disks.

Looking around there I found moref ID pointing to the new VM so I new it was relevant. I checked the Storage Policy at the VPG and the Org vDC resource, both were the same, both were picked up by vCD from vcenter. I even refreshed the storage policies in vCD. I checked Cloud Settings in ZVM and all datastore were marked as available for everything. But why is this failing to power on at test failover when all the settings were valid?! Note: I could manually power on the VM in vcenter or the vApp in vCD.

I was in the middle of a storage migration and I had my suspicion this had something to do with it. Our existing datastore cluster is VMFS5 and we need VMFS6 to prepare for vsphere 7. So I turned up a new datastore cluster with fresh VMFS6 datastores, tagged everything properly and made sure they were checked in Cloud Settings.

I triple checked this was right; it was. Finally I was able to finish the svmotion of all VRAs off the old DS cluster to the new and removed the tags on the old datastores. This made them be removed from Cloud Settings.

So, now only the new datastore cluster was available in Cloud Settings. Were the only ones tagged. Fail over tests worked perfectly; the vApps powered back on.

When working with support it was indeed showing the hook for zerto calling on vCD to power up the vApp and it throwing an exception. This suggests it’s an issue vCD side.

However, moving VRAs entirely onto a separate storage pool and not split across both and leaving only one pool available in Cloud Settings suggests it may be a Zerto issue somehow (possibly in calling the hook somehow).

All I can say is, if you’re in the middle of storage migration, and I assume many of you service providers are or will be soon as it is a requirement for vsphere 7.0 (VMFS 6 is, unless your storage pools aren’t that old) and you’re getting failover powerup failures, this post should help.

I encourage Zerto lab this out and test it to see if they can reproduce and go down that rabbit hole to either fix, or reach out to VMware official channels to fix on their end. It’s bug somewhere.