Vol. 28, #8 - May 6, 2013 - Issue #928
- Editor's Corner
- RAID Controllers
- Tip of the Week
- Recommended for Learning
- Quote of the Week
- Admin Toolbox
- Admin Tools We Think You Shouldn't Be Without
- Events Calendar
- Webcast Calendar
- Ignoring Users Hasn't Worked: Tactics for managing application settings
- Register for Webcasts
- Tech Briefing
- Data Tiering Strategies
- Choosing Hyper-V Storage: Physical Disks
- Optimizing Performance on Mellanox QDR and Intel Westmere Platforms
- Design Considerations For Converged Networking On A Budget With Switch Independent Teaming In Windows Server 2012 Hyper-V
- SMB 3.0 Multichannel Auto Configuration in Action with RDMA / SMB Direct
- Can't add a Hyper-V host to cluster using VMM 2012 SP1
- Server Hardware Explained
- Top 5 Tips for Windows 8 Networking
- Cannot remove Hyper-V Host from SCVMM
- Group Policy Changes: Windows Server 2012, Windows 8 and Window RT
- Three free Hyper-V tools
- Windows Server News
- Why are people so hung up on Apache Hadoop?
- VDI comparison: Citrix XenDesktop vs. VMware View
- Hypervisors offer built-in disaster recovery management features
- Stripping a Microsoft Office password from a document with DocRecrypt
- WServerNews FAVE Links
- This Week's Links We Like. Fun Stuff.
- WServerNews - Product of the Week
- Beat server downtime with anytime server backup.
- SAVE THIS NEWSLETTERso you can refer back to it later for helpful tips, tools and resources!
- FORWARD THIS NEWSLETTER to a colleague who you think might find it useful!
- SEND YOUR FEEDBACK to [email protected] if you have any comments or suggestions!
This week's newsletter is all about different types of RAID controllers and how to choose the right form of RAID for your Windows servers with a guest editorial by Mike Pepe who works as a Service Engineer for the Bing Information Platform at Microsoft.
RAID is all about redundancy. How do I know? Because it starts with an R of course. An engineer should always think in terms of redundancy, and I think Dilbert probably said it best:
And now on to our guest editorial by Mike Pepe...
Choosing the right way to protect your data can be a daunting task. Many system administrators may simply opt to add more drives to a system and implement mirroring and consider the task done. However there are many options available to you and understanding the implications of one data protection scheme over another will help you make the best choice.
The basic forms of RAID
RAID (Redundant Array of Independent Disks) has been around for almost as long as there have been hard drives. The most commonly encountered RAID types are defined by their number; 0, 1, 5 and (most recently) 6. Let's quickly review these RAID levels and what they really mean:
- RAID Level 0 (that's a zero) is sometimes called "non-RAID" or simply "striping". In this scheme, data is read and written across some number (n) disks simultaneously. This improves read and write performance up to n times that of a single drive, and also gives you n times the capacity of a single drive. The big downside to RAID 0 is that the failure rate of such an array increases n times that over a single drive. When a drive does fail, the array will become unavailable and the data irrevocably lost. In some circumstances, however, when raw performance or capacity are the only concern: RAID 0 may be a good choice.
- RAID Level 1 is also often called "mirroring". A volume using RAID-1 will contain (at least) two drives, and the data will be read and written to all drives simultaneously and in the same order, often at the sector level. Read performance may be improved up to n times, however, as there are now multiple copies of the data to potentially read from. Write performance may suffer, as it may take n times as long to commit a write to all the copies when compared to a single standalone drive. Capacity remains fixed to the size of one drive, no matter how many copies you make.
- RAID Level 5 may also be called "parity" by some folks. A RAID-5 array must consist of at least three drives. In this type of array, data is written to n-1 drives, and a "parity" unit is calculated and written to the remaining drive. The drive holding the parity chunk per stripe is rotated through the physical drives, distributing it evenly across the drives in the array. The primary advantage of using a RAID-5 array is that the failure of any single drive does not produce any data loss; the missing data can be reconstructed from the parity and the array can continue to operate, albeit with some degradation in performance. Capacity of the array is n-1 drives, and read performance can be improved by up to n-1 times that of a single drive. Write speeds can suffer in RAID-5 arrays since there must be a parity computation before the stripe can be committed to disk.
- RAID Level 6 improves upon the ideas of RAID-5 by providing another, different parity calculation and distributing it across the available drives. You need at least 4 drives for RAID-6, and the capacity of the array will be n-2. The chief advantage is that a RAID-6 array can sustain two disk failures without loss of data, again at the penalty of having to computationally reconstruct the missing data. Writes similarly suffer as with RAID-5 due to the need to compute two different parity chunks for every write to the array.
RAID-10 and compound RAID levels
RAID Level 10 is a compound RAID level. More precisely, it's RAID 1+0 (or sometimes, RAID 0+1) – and combines both striping and mirroring. A RAID-10 must consist of at least 4 drives: (two mirrors of two two-drive stripes is the minimum) but can consist of any number of drives, which we'll call stripe width (n) multiplied by the number of mirrored copies (m).
RAID-10 arrays combine excellent performance characteristics as well as good data integrity. There are potentially a great number of drives to pull data from, meaning there is a theoretical read performance of n * m times over that of a single drive. Their biggest downfalls are in capacity: which is only that of n drives, cost: since you must purchase n times m drives, and in write performance depending on how well you data stripes across the drives, which is something we will explain later.
Other "compound" RAID levels are possible, for instance, striping across multiple RAID-5 arrays (RAID 50) or mirroring two RAID-5's (RAID 51) although not every controller supports these more complex scenarios. These compound RAIDs are not officially defined and therefore may not be portable across systems or controllers.
Different types of RAID controllers
Now that we've reviewed the different ways which we can use multiple disk drives for varying degrees of performance, capacity and reliability you may have already decided on what the best scheme is for your application: but RAID type is only one part of the equation. How you control these disks is also important. We can bundle RAID controllers into three distinct categories.
Hardware RAID controllers offer the best performance since they are, in effect, self-contained computers dedicated to running RAID arrays. The controller manages all the aspects of the RAID, and the host system is free to do other tasks while the RAID controller manages everything behind the scenes. Hardware RAID controllers often have their own cache to improve performance, and often have an option for a battery back-up to prevent data loss if the contents of a write cache were not written to disk. All this power has a price, however in this case literally. The best high-end RAID controllers can be very expensive. There are other potential pitfalls as well, which we will discuss a little later.
Software-based RAID uses your host operating system to virtualize your storage into RAID (or RAID-like) groups. For instance, creating a mirror (RAID-1) of your boot disk in Windows Disk Administrator is a simple example of a software RAID. On the other end of the complexity spectrum, Windows Server 2012 introduces a storage management system called Storage Spaces. Using Spaces, you can make a pool out of your storage and apply different protection schemes to your data on a folder-by-folder basis rather than at the partition or disk level. Software RAID has the advantage of being the least expensive option in most cases since the functionality is part of the operating system and requires no additional hardware, or the addition of relatively low-cost host bus adapters to connect disks to your system if you need more ports. Software RAID also has the potential to be the most flexible. For instance, it is possible in Windows to create a RAID-1 mirror using half the capacity of two disks, and then create a RAID-0 volume out of the remaining storage. You'd then have a volume for data that needs protection and one for data that's not critical: all on the same two disks. The main disadvantage of a software RAID setup is that your operating system must manage it, therefore performance may suffer as your CPU time is used for disk I/O rather than for your application. We'll also examine the real-world implications of this later.
Somewhere in the middle are "hybrid" RAID controllers. These sort of controllers are marginally more expensive (or in some cases, the same price as) non-RAID host bus adapters. They generally have firmware that host CPU actually runs to provide the RAID controller functionality, and OS drivers that do the same. In that sense, they are not much different than a software RAID. However, these devices may have some form of caching or dedicated hardware to help speed up operation of a RAID array: for instance, a hardware parity calculator for RAID-5 and 6 arrays. So these devices sit somewhere in between the functionality provided by software and hardware and therefore the pros and cons of both may apply.
Choosing the right RAID controller
So which one is best? Most people would assume that a high-end hardware RAID controller is obviously the best choice, but that's not always the case. At the entry-level server spectrum, a high-end RAID controller can be more costly than an entire server! Some of them have their own out of band network configuration and can be rather complex devices for the non-techie to get working. Interchangeability also is a potential issue for the hardware and hybrid RAID controllers: if your controller fails, you'd likely need at least something from the same product family with similar firmware installed to insure you can read/recover the disks. Good luck to the system administrator who has to try and track down a specific version of a RAID controller that hasn't been made in half a decade!
Contrast this to a software RAID where there's a very good chance that any machine running the same operating system can have transplanted disks from a failed server back up and running very quickly. Recoverability in the event of a crisis may be better here, unless you keep a spare RAID controller card handy. The software/hybrid solutions do utilize your system's resources to a much greater degree than the hardware solutions, but except in the most demanding and critical systems the few percentage points of processor utilization is hardly likely to be noticed.
Price versus performance is a second key decision point, but let's talk more about recoverability. We touched on this earlier with a key advantage of a software-based RAID: the RAID volume should be readable in any machine running the same operating system, whereas with a hardware RAID controller there's a good chance that your RAID volume would not be readable with another brand or type of controller. However there is one exception to this; a RAID-1 that consumes an entire disk; often these volumes are simple block-by-block copies of what would normally be written to a single disk. In many cases it is indeed possible to take one of the copies of a failed mirror volume and put it into any random machine and read it.
Recovery time and reliability
Recovery time and reliability are another point of consideration. As of today, a 4TB drive is the largest available capacity. Average transfer rate on a drive of this size is somewhere around 180 megabytes per second, which means it would take, on average, over seven and a half hours to completely fill this drive up. (In the real world, the time to rebuild an active RAID-5 using drives of this size would be two or more times that!)
Why is this important? Let's consider a RAID-5 built with five 4TB drives. One drive fails and is replaced, and the rebuild process begins. Since hard drives are electro-mechanical devices, there is an engineered in error rate. In this case our drives have a 1 in 10E14 chance of an uncorrectable bit error during any read. In order to reconstruct the RAID-5, we must read a total of 16TB of data, which is 1.28x10E14 bits! There's a very real chance that during the rebuild, we'll encounter an uncorrectable error on one of the remaining drives: if the controller deems that drive bad, we'll have a RAID-5 array with two dead drives and the entire array will fail, and our data disappears.
RAID-6 will help here, since it will continue operating even if two drives fail. However given the high likelihood of an error, even RAID-6 starts to look less and less attractive.
The value of triple redundancy
Given that there is a statistical chance of catastrophic failure of a parity-based RAID group, you should always remember a few things; first and foremost: RAID is not a replacement for a sound backup (and recovery) strategy. Make sure you have backups in place, and test them periodically to make sure that they are recoverable. Secondly, consider triple-redundant options using RAID-10 striping and mirroring.
It's probably safe to say that many people have encountered random silent corruption in their daily lives. It's that picture that won't display anymore, or the video that's broken at some point in playback. Sure these things can happen with single drives and single copies of data, but they do appear even when disk mirroring is in place. Why would that be? Consider the following scenario: a server running a RAID-1 array with two drives crashes or loses power. A random spurious write corrupts a random sector on the hard drive. When the machine comes back up, the controller detects a dirty shutdown and re-mirrors the drive, and encounters a data difference. Which block is the correct one? It's entirely possible the RAID controller doesn't know, and there's potentially a 50% chance that it'll guess wrong, permanently corrupting the file.
What if there were three copies instead of two? Well, in that case, the RAID can take a vote; if two of the blocks agree, it's probably the "right" data. Add a checksumming filesystem, such as Windows Server 2012's ReFS and Storage Spaces on top of that with triple mirroring, and the chances of silent corruption in your data drop dramatically.
Stripe size and RAID performance
Also consider performance of your stripes. RAID types that stripe data across disks have what is known as a "stripe size". A common stripe size is 64k, meaning that data is written to each drive in 64k chunks. As an example, a 4 drive RAID-5 would then commit data to disks in chunks of 256k (4 drives, 64k each). There is nothing wrong with this, as long as your files are generally larger than 256k. If they are not, updating the smaller files within this stripe will require a read of all 256k, a modification to the data, recalculation of parity, and then a 256k write back to all the drives! If you have a lot of very small files, the performance penalty to write or modify them can be enormous.
A few guidelines concerning RAID
Armed with these basic guidelines, the data protection scheme you choose is a balance between needed capacity, performance, and the ever-present constraints of budget. However here are some guidelines based on some real world experience:
- Mirrors of single, whole drives is simplest. The ability to take one drive out of an array and read it elsewhere can be a real timesaver over restoring from backup.
- If your application demands utmost performance, consider investing in a hardware-based RAID controller and RAID-10. Otherwise, a hybrid or software-based RAID-10 solution may be sufficient.
- If capacity needs are high and performance and budget requirements are low, a parity-based solution may be a good fit. Consider using RAID-6, particularly if the array will have large numbers of high capacity drives.
- If data integrity is the utmost importance, consider a three-copy mirror and ReFS using Storage Spaces. Background data scrubbing and majority-vote-wins concepts will significantly reduce the chance of spurious data corruption.
- And most importantly: Make sure you have a good backup strategy, and you know you can restore!
About Mike Pepe
Mike Pepe joined Microsoft in 2006 after working in the IT field for ten years providing clustering, backup, and storage solutions for the telecommunications industry. He is currently a Service Engineer working on datacenter-scale automation and service design for Bing.is a Service Engineer for the Bing Information Platform at Microsoft.
Send us feedback
Got any comments or stories concerning RAID solutions and controllers? Let us know at [email protected]
Tip of the Week
PowerTip: Use PowerShell to see remaining battery run time
This week Microsoft Scripting Guy Ed Wilson shows how you can use Windows PowerShell to see the remaining battery runtime on your laptop.
Question: How can you use Windows PowerShell to determine how much time you have left on your battery on your laptop?
Answer: Use the Win32_Battery WMI class and query the EstimatedRunTime property. This technique appears here (NOTE: if your laptop is plugged in while you run the query, the results are inaccurate).
Ed Wilson is the bestselling author of eight books about Windows Scripting, including Windows PowerShell 3.0 Step by Step, and Windows PowerShell 3.0 First Steps. He writes a daily blog about Windows PowerShell called Hey, Scripting Guy! that is hosted on the Microsoft TechNet Script Center; for more PowerTips Check out the Hey, Scripting Guy! blog.
Contact me at [email protected] if you have a tip you'd like to share with our readers.
Recommended for Learning
Here's an announcement from the Microsoft Virtual Academy:
Windows Azure for IT Professionals Jump Start May 16
Join a new expert-led Jump Start from Microsoft Virtual Academy (MVA) that will help you understand how to use Windows Azure Infrastructure Services such as Virtual Machines and Virtual Networks to migrate, extend, run, manage and monitor common workloads in the cloud.
Quote of the Week
"When you have no rules, the mind starts to slack." --Jason Statham from an article in Men's Fitness Magazine
Until next week,
Note to subscribers: If for some reason you don’t receive your weekly issue of this newsletter, please notify us at [email protected] and we’ll try to troubleshoot things from our end.
Admin Tools We Think You Shouldn't Be Without
Free Tool: Idera Server Backup Free – fast, disk-based continuous data protection for Windows and Linux servers – backs up and restore files in seconds
Running Hyper-V? Download Altaro Hyper-V Backup – Free for 2 VMs for WServerNews subscribers, forever. No catch. No sales pitch.
Discover. Profile. Scan. – It’s that easy. Download StealthSEEK for free and start discovering sensitive data in minutes – download now:
Manage email on-premise or in the cloud. A hybrid archive solution is both scalable and compliant. Take advantage of cost-efficient cloud storage whilst maintaining compliancy.
This free tool shows you the health of your hard disks:
- Microsoft TechEd North America on June 3-6, 2013 in New Orleans, USA
- Microsoft Build on June 26-28, 2013 in San Francisco, USA
- Microsoft Worldwide Partner Conference on July 7-11, 2013 in Houston, USA
- Microsoft TechEd Europe on June 25-28, 2013 in Madrid, Spain
Add your event
PLANNING A CONFERENCE OR OTHER EVENT you'd like to tell our 100,000 subscribers about? Contact [email protected]
Ignoring Users Hasn't Worked: Tactics for managing applications settings
You've got users on desktops, laptops, and now VDI sessions. But your management of their experience is still scattershot. You think they're secure and well managed only to learn they're getting get prompted for annoying "application upgrade pop-ups" and different people on the same team are out of IT compliance and insecure.
Meanwhile, while all this is going on, you're crossing your fingers, gambling that your users won't introduce a security breach.
Join desktop management expert and Microsoft Group Policy MVP, Jeremy Moskowitz, for some pro tips on how to manage and secure your desktop and applications. This webinar takes place multiple times a week, please click the link below to see which day / time is best for you! (Registration is on the top right of the page)
In this webinar, Jeremy will show you:
- How to deliver, enforce and remediate important settings in popular applications such as Firefox, Java, Adobe Flash and dozens more
- How to turn off harmful and annoying update behavior so users are no longer prompted for updates
- How to lock down your desktops, applications and Windows operating systems
The vast majority of administrators say they're "too busy" to take the time to get things done right. If you're ready to get this problem handled, here's where to sign up and try something new.
Register for Webcasts
Add your Webcast
PLANNING A WEBCAST you'd like to tell our 100,000 subscribers about? Contact [email protected]
Data Tiering Strategies (WindowsNetworking.com)
Mitch Tulloch describes some strategies for implementing a data tiering solution for enterprise storage.
Choosing Hyper-V Storage: Physical Disks (Aidan Finn, MVP/IT Pro)
In this blog post Aidan compares the different disk types for Microsoft Hyper-V Virtual Machine Storage. He also summaries the advantages and disadvantages.
Optimizing Performance on Mellanox QDR and Intel Westmere Platforms (University of Cambridge)
This guide provides best practice studies, how to optimize an Infiniband environment based on experiences of the University of Cambridge.
Design Considerations For Converged Networking On A Budget With Switch Independent Teaming In Windows Server 2012 Hyper-V (Didier van Hoye, MVP/IT Pro)
In this guide Didier give an overview how you can design a converged network solution with a low budget.
SMB 3.0 Multichannel Auto Configuration in Action with RDMA / SMB Direct (Didier van Hoye, MVP/IT Pro)
Didier gives with this blog a good intro and best practice on RDMA and SMB Direct.
Can't add a Hyper-V host to cluster using VMM 2012 SP1 (Kristian Nese MVP/IT Pro)
In this post Kristian shows a solution when you cannot add a Hyper-V host to your VMM 2012 SP1 cluster.
Server Hardware Explained (WindowsNetworking.com)
A series of articles by Brien Posey explaining the basics of server hardware.
Top 5 Tips for Windows 8 Networking (WindowsNetworking.com)
David Davis puts together his 5 top tips for Windows 8 networking.
Group Policy Changes: Windows Server 2012, Windows 8 and Window RT (WindowsNetworking.com)
Deb Shinder examines the changes in Group Policy and how you can best use them to your benefit in keeping your Windows network as secure as possible.
Three free Hyper-V tools (VirtualizationAdmin.com)
Scott D. Lowe describes three free tools that you can use with the newest version of Hyper-V.
Thanks to Florian Klaffenbach for providing some of the items in this section. Be sure to check out Flo's Datacenter Report:
Why are people so hung up on Apache Hadoop?
When it comes to the cloud, Apache Hadoop is one buzz word on many IT pros’ minds; however, most fail to really understand what this cloud model is and the top use cases where it can be effectively applied. Uncover the facts about Apache Hadoop and find out what benefits – if any – it can offer your business.
VDI comparison: Citrix XenDesktop vs. VMware View
While there are many virtual desktop infrastructure (VDI) offerings to choose from, for many IT admins, it comes down to two options: Citrix XenDesktop and VMware View. Inside this tip, explore a detailed comparison of these two VDI products and find out which one comes out on top.
Hypervisors offer built-in disaster recovery management features
Ensuring business continuity is a top priority – and pain point – for most IT pros, and as a result, many are turning to virtualization to simplify this pressing task. Access this exclusive guide to learn how you can leverage the built-in options offered in hypervisors to ease disaster recovery management.
Stripping a Microsoft Office password from a document with DocRecrypt
If an employee forgets their Microsoft Office password or leaves the company, admins often turn to expensive password-cracking products to break into the doc – but that’s all about to change thanks to Microsoft’s new free tool, DocRecrypt. Find out more about this advanced platform inside.
This Week's Links We Like. Tips, Hints And Fun Stuff
GOT FUN VIDEOS or other fun links to suggest you'd like to recommend? Email us at [email protected]
A model rocket crashes at sea over Peconic Bay, Long Island, USA (with mission control chatter).
A modern, simple and efficient way to sweep a chimney, as practiced in Zarinsk, Russia.
After their viral success of 'Roller Skating Babies' in 2009, Evian is back with a new version titled 'Baby and Me.'
Diver and dolphin work together to get a fishing hook and line off the bottlenose dolphin's fin.
Juggling comedian Michael Davis performs at the historic Ford's Theater for the former President and Mrs. Ronald Reagan:
WServerNews - Editors
Mitch Tulloch is Senior Editor of WServerNews and is a widely recognized expert on Windows administration, deployment and virtualization. Mitch was lead author of the bestselling Windows 7 Resource Kit from Microsoft Press and has published hundreds of articles for IT pros. Mitch is also a seven-time recipient of Microsoft's Most Valuable Professional (MVP) award for his outstanding contributions in support of the global IT pro community. Mitch owns and runs an information technology content development business based in Winnipeg, Canada. For more information see www.mtit.com
Ingrid Tullochis Associate Editor of WServerNews and was co-author of the Microsoft Encyclopedia of Networking from Microsoft Press. Ingrid is also Head of Research for our content development business and has co-developed university-level courses in Information Security Management for a Masters of Business Administration program.