About a week ago another person in our IT department installed KB3211320 (Update for Windows Server 2016 for x64-based Systems) on our DPM servers.
Since this was a Saturday i did not notice until Monday that our backups had failed on both DPM servers. Giving different errors on 2016 and 2012 R2 servers. And i was unaware that windows update was run on the backup server. As the instruction is that i am to be let known of that.
The error on 2016 was
On 2012 R2 the error was
So i thought ok hmm strange. I tried to reboot DPM server, and wanted to reboot hosts. But as i was trying to live migrate VM’s off one host it failed. Could not do it. I shutdown one vm i could during day time and tried to offline migrate it. It moved, but VM was gone from Hyper-V manager. Now this was strange. Would not come back up again. So i had to wait until after production hours. I shut down all vm’s and rebooted 1 HyperV host after the other. Then the VM i tried to offline migrate would pop up again in Hyper-V manager. Some strange lock on the config i think. But not sure.
Now i tried doing backup again and it failed quite fast, and i could not live migrate vm’s again. Now i had contacted Microsoft support at this time. I spent the next 5 days trying to sort this out. I noticed on Thursday that a patch was installed the same day the backups started failing. So i decided to uninstall the patch and reboot 1 DPM server. I had 1 HyperV host that i could live migrate to so i tried to backup the vm’s from that host. That worked.
Today a week later i had the time to uninstall the patch on the 2nd dpm server and reboot all our HyperV hosts 5 clusters in total. And now backup was once again working.
The issue has been escalated to the DPM and HyperV team at Microsoft and they are looking into this. But somehow something with KB3211320 for Windows Server 2016 is breaking DPM 2016. Can’t say if this relate to DPM2012 R2 installed on 2016 tho.
So make sure this patch is not installed on your DPM servers as of now.
To remove the KB follow this guide.
https://windowsexploredforu.blogspot.com/2017/01/cant-uninstall-update-kb3211320-solved.html
