Exchange upgrades and forgotten servers

Hello Everyone! 

Got a guest post from a colleague of mine today. Erick Purkins is a Microsoft consultant out of the Houston Texas area and he did a write up of a recent issue he saw. Enjoy.

--------------------------------------------------------------------------------------

I just wanted to share an experience and issue with everyone this morning.

I am currently working with a customer to upgrade their Exchange 2010 infrastructure to Exchange 2016. During our discussions, we talked about correct service pack levels and OS’s required, etc. One thing I didn’t think to talk about was “FAILED EXCHANGE SERVERS”. Just curious if anyone brings this up?

This is important because during the installation of their first 2016 server I received a rather odd error.

It was the “Update-RmsSharedIdentity -ServerName $RoIeNetBIOSName was run: "Microsoft.ExchangeData.DataVaIidationException: Database is mandatory on UserMaiIbox.” That led me to the issue.

Apparently at one time or another they had an Exchange server go belly up. Instead of fixing the issue they turned it off and forgot about it, eventually having someone go in and remove the server through ADSI.

Now normally this wouldn’t have been much cause for alarm, but after reviewing the error message and a little google-fu I realized they had no arbitration mailboxes and this was what the error referred to.

• So How did I fix this issue?

First, I reviewed how to recreate Arbitration mailboxes. Something I have done before but not in a while.

https://social.technet.microsoft.com/wiki/contents/articles/5317.recreate-and-enable-missing-arbitration-user-accounts-and-mailboxes-in-exchange-server-2010.aspx

• OK seems easy, right? Wrong.

Since I had previously run the Exchange 2016 setup it had ran /prepareAD and updated the schema. So I could not run the Exchange 2010 SP3 with /PrepareAD to recreate the mailboxes.

• Where to next? I guess I’ll have to use the Exchange 2016 Media.

I hope you’ll never have to do this but with Setup.exe there is a /mode switch which you can use to remove a failed Exchange install. This is the only way, you cannot remove the install through add/remove programs. The command looks like this “Setup.exe /mode:uninstall /iacceptexchangeserverlicenseterms”  

Learn more at: https://technet.microsoft.com/library(EXCHG.150)/ms.exch.setupreadiness.InstallWatermark.aspx 

or use setup.exe /?

After successfully removing the Exchange installation I removed the AD objects associated with Arbitration mailboxes and re-run Setup.exe /prepareAD. All the correct mailboxes were recreated in the Default Users container as they are supposed to.

• Now it’s time to enable those mailboxes…

After recreating the mailboxes with the Exchange 2016 Media I followed what I would normally do and re-enable them through the shell. Okay new error, WTH? You mean to tell me I can only do this through the Exchange 2016 Shell, but I haven’t gotten a server even installed yet. Now we have a Chicken or the egg situation.

• Do we try and install Exchange 2016 again? You Feeling Lucky?

That was the only thing I could think of to do and the internet was no help with that question. So I ran setup again and prayed to the Microsoft gods while crossing everything and holding every lucky charm I could find.

Must have been the lucky rabbits foot, because this time around we were successful at installing Exchange 2016.

After rebooting the server all arbitration mailboxes appear on the new Exchange server.  

• Moral of the story

Talk to your customers about failed or improperly decommissioned servers. Double check your arbitration mailboxes prior to any upgrade. It may just save you a few hours of Google-fu. Also, always keep a lucky rabbits foot close at hand.