Vol. 27, #8 - April 29, 2013 - Issue #927
Troubleshooting Hyper-V Storage
- Editor's Corner
- Troubleshooting Hyper-V Storage
- Tip of the Week
- Recommended for Learning
- Quote of the Week
- Admin Toolbox
- Admin Tools We Think You Shouldn't Be Without
- Events Calendar
- Webcast Calendar
- Ignoring Users Hasn't Worked: Tactics for managing application settings
- Register for Webcasts
- Tech Briefing
- Hyper-V storage-related articles
- VIDEO: Microsoft Certified Career Day 2013: Windows Server 2012 Product Session
- Blade Server Networking Options
- DVS Enterprise XenDesktop Reference Architecture
- Installing an Exchange 2013 Test Environment on Windows Azure
- Master System Center 2012 SP1 Configuration Manager with this shortlist of FREE Learning Resource
- Virtualization in Windows 8 - What is it Good For? (Part 2)
- Migrate Hyper-V Cluster to Windows Server 2012 Hyper-V via Cluster Migration Wizard
- Cannot remove Hyper-V Host from SCVMM
- Files Vs. Objects: the difference is in the value
- Introduction to Identity Management and Forefront Identity Manager 2010 R2 SP1 (Part 1)
- Windows Server News
- Is cloud-based storage the right option for you?
- Taking stock of zero-client hardware options for VDI
- Improving business continuity via a virtual disaster recovery strategy
- VMware Fusion vs. Workstation: Choosing an end-user virtualization tool
- WServerNews FAVE Links
- This Week's Links We Like. Fun Stuff.
- WServerNews - Product of the Week
- Free Hyper-V Backup Download for WServerNews subscribers
- SAVE THIS NEWSLETTERso you can refer back to it later for helpful tips, tools and resources!
- FORWARD THIS NEWSLETTER to a colleague who you think might find it useful!
- SEND YOUR FEEDBACK to [email protected] if you have any comments or suggestions!
This week's newsletter is all about troubleshooting Hyper-V storage issues and includes an excerpt from my soon-to-be-released ebook Optimizing and Troubleshooting Hyper-V Storage from Microsoft Press. This ebook leverages the experience of Premier Field Engineers, Support Escalation Engineers and other experts at Microsoft to demonstrate how to troubleshooting various storage-related issues in Hyper-V environments and optimize specific storage scenarios. Click here to pre-order this new title from O'Reilly:
There's also a companion ebook coming out soon called Optimizing and Troubleshooting Hyper-V Networking which you can pre-order as well by clicking here:
Oops, I almost forgot this week's comic! Let's see what xkcd has in store for us today:
Troubleshooting Hyper-V Storage
The Windows event logs are one of the first places you should look for clues when your systems or applications are not behaving in the way you expect them to behave. In the old days of Windows Server 2003 there were only a few event logs you had to check, with the System, Application, and Security logs being the main ones. Then beginning with Windows Server 2008 the number of different event logs and types of logs jumped into the hundreds, and although you can filter and search these logs using Event Viewer and Windows PowerShell, many long-time server admins still often feel overwhelmed by the sheer amount of information in Windows Server logs and wonder how to discover and identify what might be relevant for the particular scenario they are trying to optimize or troubleshoot.
To help get you oriented, Thomas Roettinger, a Program Manager with the Partner and Customer Ecosystem Team at Microsoft, explains some basics and also provides two examples below.
Hyper-V storage event logs
Windows Server provides several different event log categories where you can look for Hyper-V related issues. You can open the event log viewer either through the modern UI by using the key combination Windows logo key+X, or by launching the MMC snap-in directly by typing eventvwr.msc.
If you expand Application and Services Logs, Microsoft, and Windows, you will find specific application and services logs for Windows components. For example, you will see the Hyper-V logs if the Hyper-V Role is installed.
The storage-related logs are called:
- Hyper-V-SynthFC (Virtual FC Adapter)
- Hyper-V-SynthStor (SCSI Controller)
Figure 1: A screenshot of the Event Viewer console that shows the different event logs when you expand Applications And Services Logs, then Microsoft, and then Windows in the console tree.
When you start a virtual machine, a virtual machine worker process (VMWP.exe) is launched for each virtual machine. The Hyper-V-Worker log provides all necessary information related to start, stop action, and the run time of a virtual machine. The worker process connects to the virtual machine management services (VMMS). In the example below you will notice that you can look at several Hyper-V logs for troubleshooting and that sometimes it is required to find the root cause of an outage.
Example: Missing virtual hard disk
Patricia the administrator received an error message when she tried, to start one of her virtual machines, but she did not pay attention to it:
Figure 2: A screenshot showing a dialog box with the message "MissingDisk failed to start."
To investigate the problem, Patricia opens Event Viewer and checks the related Hyper-V event logs. She looks at the Hyper-V-VMWP logs first because the issue happened when she tried to power on the virtual machine. She finds the following event entry:
Event ID 32902
"Missing Disk" Synthetic SCSI Controller: Failed to Power on with Error 'The system cannot find the file specified.'
Patricia next looks at the Hyper-V-SynthStor logs to see if she can find more information. She finds the following:
Event ID 12240
'Missing Disk': Attachment" could not be found due to error 'The system cannot find the file specified.'
Patricia now needs to identify which file is missing, so she checks the Hyper-V-VMMS logs and finds the following entry:
Event ID 32902
The absolute path 'd:\missingdisk.vhdx' is valid for the "Hard Disk Image pool, but references a file that does not exist.
Patricia knows that a virtual hard disk file is missing on drive D. She uses Windows Explorer to find the file. She then discovers that drive D is missing entirely:
Figure 3: A screenshot of Windows Explorer with the Computer node selected. Only one hard drive (drive C) is visible.
This error could happen if you lose storage connectivity, no matter how it is presented to your host. There are techniques to provide multiple paths to your storage when using SMB, FC, or ISCSI using MPIO or SMB multichannel to provide availability.
In this section you have seen the available Hyper-V event logs for storage. But you also read about their relationship to other important Hyper-V logs such as the Virtual Machine Worker Process and the Virtual Machine Management Service.
Example: Unsupported Fibre Channel adapter
Patricia needs to configure a virtual machine with a virtual Fibre Channel adapter. The Hyper-V host has a Fibre Channel adapter installed, so Patricia opens Virtual SAN Manager to create a new Fibre Channel SAN. She notices that she is not able to select the physical HBA and that the status is "The device or driver does not support virtual Fibre Channel."
Figure 4: A screenshot showing the Virtual SAN Manager properties page with the message "The device or driver does not support virtual Fibre Channel" displayed.
She ignores the message and continues to add a virtual HBA to the virtual machine. When she tries to start the virtual machine, she receives the following error message:
Figure 5: A screenshot showing a dialog box with the message "vHBA failed to start."
Patricia looks at the Hyper-V-SynthFC event log and discovers the following entry:
Event ID: 32161
'vHBA': Operation for virtual port ……failed with an error: No physical port available to satisfy the request
Patricia opens Windows PowerShell to validate the driver. She does a WMI query to see if the driver provides support for Hyper-V. She runs the following command:
gwmi –Namespace "root\virtualization\v2" –Class msvm_externalfcport |select *hyper*
The resulting output from this command looks like this:
While this output indicates that the driver does not support Hyper-V, Patricia knows that the physical adapter is capable of NPIV and should support Hyper-V. This disparity can happen because some older driver versions do not populate the necessary information correctly.
After Patricia downloads and installs the latest driver from the HBA vendor, everything works as expected.
About Thomas Roettinger
Thomas Roettinger is a Program Manager at Microsoft. Thomas is part of the Partner and Customer Ecosystem Team and works with technologies like Hyper-V and System Center Virtual Machine Manager. His team runs the Windows Server TAP Program and collects very early technology best practices. Before he joined the Product Group he was the EMEA Virtualization Lead in Microsoft Premier Field Engineering. During this time he was responsible for various services including the Hyper-V Risk Assessment Program and the Implementing Hyper-V Workshop. He has rich experience in cloud implementations across various business segments such as Hosters and Enterprises. Thomas maintains a personal blog at:
and also contributes to his team blog at:
Pre-order these ebooks today!
These two ebooks by Mitch Tulloch and the Windows Server Team can be pre-ordered from Amazon:
- Optimizing and Troubleshooting Hyper-V Storage
- Optimizing and Troubleshooting Hyper-V Networking
Send us feedback
Got comments or questions concerning anything in this issue? Let us know at [email protected]
Tip of the Week
Here's another tip from my book Training Guide: Installing and Configuring Windows Server 2012 from Microsoft Press that shows how to display a list of all installed roles and features on Windows Server 2012:
To display a list of all installed roles and features on the server, pipe the output of the preceding command into the Where-Object cmdlet and use Where-Object to filter out everything except roles and features whose InstallState property is equal to Installed:
Get-WindowsFeature -ComputerName SEA-SRV-1 | Where-Object InstallState -eq Installed
You can narrow your results even further by using the -name parameter of the Get-WindowsFeature cmdlet to select only roles and features that begin with “Print” like this:
Get-WindowsFeature -Name Print* -ComputerName SEA-SRV-1 | Where-Object InstallState -eq Installed
Contact me at [email protected] if you have a tip you'd like to share with our readers.
Recommended for Learning
Here are two titles that crossed our desk this week:
A Practical Guide to Advanced Networking, Third Edition (Pearson IT Certification)
A pretty cool book that uses a hands-on approach to teach you about routers and routing, network security, IPv6, and how the Internet works. Includes chapters on configuring Juniper routers and Linux networking, using WireShark to analyze network traffic, and lots more.
Programming Windows, Sixth Edition (Microsoft Press)
This classic title has been expanded to teach you how to use your existing C# programming skills with XAML to create apps for the new Windows 8 platform.
Quote of the Week
"It always seems impossible until it's done." --Nelson Mandela
Note to subscribers: If for some reason you don’t receive your weekly issue of this newsletter, please notify us at [email protected] and we’ll try to troubleshoot things from our end.
Admin Tools We Think You Shouldn't Be Without
Running Hyper-V? Download Altaro Hyper-V Backup – Free for 2 VMs for WServerNews subscribers, forever. No catch. No sales pitch.
Discover. Profile. Scan. – It's that easy. Download StealthSEEK for free and start discovering sensitive data in minutes – download now:
Free download! SolarWinds free Exchange Monitor continuously monitors Microsoft Exchange to deliver real-time insight into Exchange services, mail queue sizes, and host server health.
Email Archiving made easy – Exclaimer Mail Archiver provides you with all the benefits of email archiving in a package that's simple to install, easy to maintain and low cost to own.
Build and deploy policies to help identify, classify and protect critical data on your file servers:
- Microsoft TechEd North America on June 3-6, 2013 in New Orleans, USA
- Microsoft Build on June 26-28, 2013 in San Francisco, USA
- Microsoft Worldwide Partner Conference on July 7-11, 2013 in Houston, USA
- Microsoft TechEd Europe on June 25-28, 2013 in Madrid, Spain
Add your event
PLANNING A CONFERENCE OR OTHER EVENT you'd like to tell our 100,000 subscribers about? Contact [email protected]
Ignoring Users Hasn't Worked: Tactics for managing applications settings
You've got users on desktops, laptops, and now VDI sessions. But your management of their experience is still scattershot. You think they're secure and well managed only to learn they're getting get prompted for annoying "application upgrade pop-ups" and different people on the same team are out of IT compliance and insecure.
Meanwhile, while all this is going on, you're crossing your fingers, gambling that your users won't introduce a security breach.
Join desktop management expert and Microsoft Group Policy MVP, Jeremy Moskowitz, for some pro tips on how to manage and secure your desktop and applications. This webinar takes place multiple times a week, please click the link below to see which day / time is best for you! (Registration is on the top right of the page)
In this webinar, Jeremy will show you:
- How to deliver, enforce and remediate important settings in popular applications such as Firefox, Java, Adobe Flash and dozens more
- How to turn off harmful and annoying update behavior so users are no longer prompted for updates
- How to lock down your desktops, applications and Windows operating systems
The vast majority of administrators say they're "too busy" to take the time to get things done right. If you're ready to get this problem handled, here's where to sign up and try something new.
Register for Webcasts
Add your Webcast
PLANNING A WEBCAST you'd like to tell our 100,000 subscribers about? Contact [email protected]
We'll start with some links to resources about storage in virtualized environments built using Hyper-V.
Hyper-V storage-related articles
- Optimizing Windows Server 2012 storage management via PowerShell for both performance and resiliency (Building Clouds)
- Updated Links on Windows Server 2012 File Server and SMB 3.0 (Jose Barreto's Blog)
- Configuring Storage Fabric in Virtual Machine Manager - Build Your Private Cloud (IT Pros ROCK! at Microsoft)
- Iron Networks shows a complete private cloud pod at MMS 2013 with Windows Server 2012 (Storage Spaces, SMB3, Hyper-V) and System Center 2012 SP1 (Jose Barreto's Blog)
- SMB 3.0 and ODX both appear in EMC VNX mainstream release (Hyper-V.nu)
- Tip-of-the-Day: USB on Hyper-V Revisited with Windows-To-Go USB Drives (IT Pros ROCK! at Microsoft)
- MMS 2013 Demo: Hyper-V over SMB at high throughput with SMB Direct and SMB Multichannel (Jose Barreto's Blog)
Now on to some other stuff...
VIDEO: Microsoft Certified Career Day 2013: Windows Server 2012 Product Session (YouTube)
Senior Technical Evangelists Rick Claus and Joey Snow discuss how to get the training needed to transform IT operations and deliver a whole new level of business value with Windows Server 2012.
Blade Server Networking Options (Kevin Houston / IT Pro)
In this blog Kevin gives an overview about current network technologies that can be used in a blade or modular environment. The overview includes all major vendors like Dell, HP, IBM and Cisco.
DVS Enterprise XenDesktop Reference Architecture (Dell TechCenter)
Florian Klaffenbach has found some guides for DVS Enterprise XenDesktop Reference Architecture based on Dell gear that provide a good overview for people who want to build up their own environment.
- DVS Enterprise XenDesktop Reference Architecture
- DVS Enterprise XenDesktop Reference Architecture - Rack Shared Storage
- DVS Enterprise XenDesktop Reference Architecture - Rack Local Storage
Installing an Exchange 2013 Test Environment on Windows Azure (MSExchange.org)
Rui Silva shows how to install and configure an Exchange Server 2013 test environment on Windows Azure Virtual Machines.
Master System Center 2012 SP1 Configuration Manager with this shortlist of FREE Learning Resource (Keith Mayer / Microsoft Evangelist/IT Pro)
On his official Microsoft Blog page, Keith collected a lot of free resources to prepare for the MCSE: Private Cloud exam.
Virtualization in Windows 8 - What is it Good For? (Part 2) (WindowsNetworking.com)
Brien M. Posey concludes the series on running Hyper-V on the desktop by examining the deployment process and discussing some of the many benefits to using Hyper-V.
Migrate Hyper-V Cluster to Windows Server 2012 Hyper-V via Cluster Migration Wizard (Thomas Maurer / MVP/IT Pro)
Thomas wrote a short guide how to easy migrate your Windows Server 2008 R2 Hyper-V Cluster to a Windows Server 2012 Hyper-V Cluster without any downtime.
Cannot remove Hyper-V Host from SCVMM (Thomas Maurer MVP/IT Pro)
With this blogpost Thomas gives a solution for an everyday VMM problem. What can I do when I'm not be able to remove a VM Host via GUI.
Files Vs. Objects: the difference is in the value (Enrico Signoretti / IT Pro)
In his blog, Enrico gives an overview about file and object based storage and when and how to use it, how to access the data etc.
Introduction to Identity Management and Forefront Identity Manager 2010 R2 SP1 (Part 1) (WindowsNetworking.com)
Scott D. Lowe provides you with a foundation on which to understand the importance of identity management and begins to introduce you to FIM 2010 R2 SP1.
Thanks to Florian Klaffenbach for providing some of the items in this section. Be sure to check out Flo's Datacenter Report:
Is cloud-based storage the right option for you?
While cloud-based storage is said to improve the flexibility and efficiency of data backups, it may not be right for every organization. Inside this tip, explore the various benefits – and drawbacks – associated with this approach to determine whether or not the pros outweigh the cons for your IT environment.
Taking stock of zero-client hardware options for VDI
Many organizations are considering hosting their virtual desktops with zero clients as an alternative to traditional thin clients. Access this resource to learn about the advantages these that new appliances can deliver and discover detailed insights on a variety of popular zero client hardware offerings.
Improving business continuity via a virtual disaster recovery strategy
Having an optimal disaster recovery strategy in place is essential for every organization, and leveraging virtualization can ease many of the top pain points that IT pros face. Inside this guide, explore the top disaster recovery and business continuity benefits a virtual disaster recovery strategy can provide.
VMware Fusion vs. Workstation: Choosing an end-user virtualization tool
If you’re evaluating VMware Fusion and VMware Workstation, there are many key similarities and differences between the two that you should keep in mind. Find out the benefits and drawbacks associated with each offering so you can determine which is right for your organization.
This Week's Links We Like. Tips, Hints And Fun Stuff
GOT FUN VIDEOS or other fun links to suggest you'd like to recommend? Email us at [email protected]
Two dogs play "Dueling Banjos," the classic bluegrass song. They are obviously very talented musicians.
Hyperlapse - a technique combining time-lapse and sweeping camera movements - created by using Google Street View photos:
Creative Home Engineering can build Hollywood-style secret passages for your home or office:
Although they are graduates of the renowned Vienna College of Music, "Mnozil Brass" show that playing music is not just serious business.
WServerNews - Editors
Mitch Tulloch is Senior Editor of WServerNews and is a widely recognized expert on Windows administration, deployment and virtualization. Mitch was lead author of the bestselling Windows 7 Resource Kit from Microsoft Press and has published hundreds of articles for IT pros. Mitch is also a seven-time recipient of Microsoft's Most Valuable Professional (MVP) award for his outstanding contributions in support of the global IT pro community. Mitch owns and runs an information technology content development business based in Winnipeg, Canada. For more information see www.mtit.com
Ingrid Tullochis Associate Editor of WServerNews and was co-author of the Microsoft Encyclopedia of Networking from Microsoft Press. Ingrid is also Head of Research for our content development business and has co-developed university-level courses in Information Security Management for a Masters of Business Administration program.