I am so honored to be voted vExpert again this year!  Congratulations to all of the Class of 2014 vExpert’s!
The list can be found here- http://blogs.vmware.com/vmtn/2014/04/vexpert-2014-announcement.html
The neatest vExpert announcement I have seen is from Tintri,
http://www.tintri.com/congratulations-2014-vexperts-you-are-rock-stars
The marquee with all the vExperts is very cool!
My thanks to the vExpert program team for all their work!

With the release of vSphere 5.5, the vCenter Server Appliance now supports 100 hosts and 3000 virtual machines.
If you are looking to reduce the number of Windows servers used for your management, the vCenter appliance is a great option.
And if you have stateless hosts, or are looking to use the new VSAN, you can use Auto Deploy to boot your hosts, thus freeing up your local disk drives.  I have done both Auto Deploy and boot from SAN, and feel Auto Deploy is much easier to use, and it does not use up your SAN storage.
To use Auto Deploy you need the vCenter server, an Auto Deploy server, a tftp server, PowerCLI, and a DHCP server with reservations for your hosts.
There is a Auto Deploy Service integrated in the vCenter Appliance, shown below, but you need to start the service.

The vCenter Appliance also has a tftp service embedded in it, however it is not started by default, or when you start the Auto Deploy service.
If you SSH to your appliance and check the "atftp" service it shows as unused.

So you will need to start the atftp service, then update chkconfig to make sure it starts after reboots. I use chkconfig --level 2345 atftp on
Once the atftp service is running, you can use the vCenter appliance as your Auto Deploy server and tftp server!

In the next post, I will cover creating the image profiles and DHCP reservations for deploying your servers!

The 2013 vExperts were announced in May and I am honored to be one of them!
The complete list can be found here
One perk to being selected as a vExpert for 2013 is Trains Signal has offered a free 1 year subscription to all of their on line training videos!  You other vExperts can find the info here .  I would recommend the Train Signal video training to everyone working on a new or updated certification, whether for VMware, Cisco Microsoft or many others.
I want to thank all my co-workers  who nominated me and told me to apply!  Hope to be on the vExpert list in the coming years!!

Here is an interesting issue I ran into recently-
-Alarms showing loss of path redundancy to storage
-Several hosts disconnect from a cluster
-Cannot access the host via vSphere client or SSH
-One or more datastores shows dead and cannot be accessed
-CPU on several hosts is at or near 100 percent
I saw these issues just after several hosts reported the loss of redundant path to storage alarms.
The storage is managed by a separate team, so I had them check the fabric and storage presented to the cluster, they didn’t see any issues except an alarm around the same time as the first loss of redundant path alarms… So what is the next step, try a rescan of the storage, well I did that and the rescan ran for several minutes and timed out, then that host disconnected from the cluster!  Going back to the storage team I had them check the one of the LUN ID of the datastore that showed dead, they said it showed on line and didn’t see a problem.  Finally they removed and re-presented the LUN to the clusters hosts.
I tried another rescan and again it took forever and failed.  So the next step, reboot a hosts?  I had one that only had one VM on it, rebooted it and the previous dead datastore was back.  A few minutes later the hosts that were previously disconnected from the cluster reconnected and appeared fine..??  
I remember back on a 4.0 environment when someone powered off an iSCSI array the hosts disconnected from the cluster, so I assumed that having the storage pulled out from under the hosts is still an issue in vSphere 5.0.
After doing some research and opening a case with VMware, this still can be an issue.
The link below is to a KB that explains a Permanent Device Loss and  All Paths Down error.  One note on the KB is-
“As the ESXi host is not able to determine if the device loss is permanent (PDL) or transient (APD), it indefinitely retries SCSI I/O, including:

• Userworld I/O (hostd management agent)
• Virtual machine guest I/O”

That explains why the hosts disconnected and why the CPU on some showed 100 percent. The hostd process just peaks trying to retry I/O, that slows the management agents so you can’t connect directly, and of course running a rescan of the storage just compounds the problem.
Click here for a link to the KB article.
The KB also notes that the only way to recover is to resolve the storage access issue and reboot the hosts.  Nice…
It turns out there are some settings that can be added to alleviate this issue from happening in 5.1 and in 5.0
Update 2. 
For more details see Cormac Hogans great info on the storage features in 5.1 starting here-
(Hope he doesn't mind me sharing this link)
Another KB states that if Storage I/0 Control is enabled, a host cannot remount the datastore.
In my case SIOC was enabled on all of the datastores.
The KB details steps to stop the SIOC service on a host to allow the removal of the datastore.
Access this KB here-
In my case I think rebooting the hosts was the only option to clear the I/0 to the lost datastore.  Of course what caused the issue on the storage side is still a mystery.
I have since added the settings to each of the hosts and to the cluster, if there is another issue like this one I am hoping it makes a difference.
If you have experienced this or a similar issue please share your experiences.....