After about 2 weeks of frustration and testing I finally figured out why 2 of my VMs were not being backed up and hopefully this helps someone else out.
We run our Veeam server as a VM and use the Virtual Appliance mode to backup all of our VMs. At first, all I did was add my vCenter server into the list of servers. I set up a few jobs and I would have 2 VMs continuously fail, the vCenter and vCenter Database VMs. To alleviate this issue, these 2 VMs cannot be backed up through vCenter, but instead you have to add the ESX(i) server to the list of servers and then add the VM to the backup job through the particular host.
Backups then ran without a problem for about 2 weeks. Then one day I patched the ESX servers in our environment which made me migrate our VMs all around. I have my vCenter and vCenter Database VMs set to partial DRS because it's best practice to always know where these VMs are located in case of a failure so i moved these VMs back to their original host. After that day, backups stopped working for these 2 VMs. Every single VM continued to run fine, but these 2 VMs would fail with the error message:
5 of 6 files processed
Total VM size: 80.00 GB
Processed size: 12.84 KB
Processing rate: 0 KB/s
Backup mode: HOTADD with changed block tracking
Start time: 3/30/2010 1:41:28 PM
End time: 3/30/2010 1:43:48 PM
Duration: 0:02:20
Backing up file "[FAS2020a_Node1_NFS_A] XXXvCenter/XXXvCenter-flat.vmdk"
BackupDisk failed
Client error: Failed to open VDDK disk [[FAS2020a_Node1_NFS_A] XXXvCenter/XXXvCenter.vmdk] ( is read-only mode - [true] )
Logon attempt with parameters [VC/ESX: [XXXESX01];Port: 902;Login: [root];VMX Spec: [moref=96];Snapshot mor: [96-snapshot-85];Transports: [hotadd]] failed because of the following errors:
Server error: End of file
The Veeam log shows as if it is a login error. After removing the individual ESX(i) hosts and re-adding them, the problem still persisted. As a last moment of testing, I tried to see if the VM location on the host was the problem, sure enough it was. I vMotioned my Veeam VM to the same host as the vCenter and vCenter Database VMs and backups ran without any errors.
To Make sure there were no problems in the foreseeable future, I set DRS for the Veeam VM to be partially automated and set a DRS rule to always keep these 3 VMs together.
Conclusion: If you are backing up a VM using Virtual Appliance Mode through a particular ESX(i) host and not vCenter, be sure that the VM you want to backup and the Veeam VM are living on the same host.