Disk Virtualization
Disk drives have been virtualized for years. Disk firmware transforms the cylinders, heads and sectors (CHS) address into consecutively numbered logical blocks, known as LBA (Logical Block Addressing). File systems can use disks and determine disk size simply by determining the number of logical blocks.

LUNs are a further abstraction (or virtualization) of disk drives connected on a SCSI bus (parallel or serial, which can be Fibre Channel or Serial Attached SCSI). A  LUN commonly refers to the defined logical storage (using LBA), but technically is an identification or address for storage. A LUN may refer to a single disk, a subset of a single disk, or an array of disks.

Tape Virtualization
Using disk space to virtualize tape has become very popular lately. With Disk-to-Disk-to-Tape or Backup–to–Disk, the whole backup job goes to disk that looks like tape to the backup software and OS drivers. This is called a Virtual Tape Library. When the time to go to tape comes, if it does, the data can stream onto tape at a constant high rate, avoiding that nasty “shoeshining” effect you get when congested backup networks keep data from streaming through your tape drive at the required speeds.

File System Virtualization
File systems: do you really want to manually create a map of inodes or File Allocation Tables to tell your applications where to write data? Of course not, even if you could! File systems virtualize the underlying storage to allow you to save your data without worrying about the underlying complexity. Networked file systems take it a step further: use of remote file systems such CIFS (Windows shares) and NFS allow you to hide data location, a concept also known as location transparency.

Disk Array Virtualization
We all know about hard drives. We first started combining them in “dumb” enclosures called JBOD, or Just a Bunch Of Disks. We then left it up to Logical Volume Managers (LVMs), software running on the host, to manage this storage.
Then came RAID.

RAID (redundant array of independent disks) is a way of storing the same data in different places (thus, redundantly) on multiple hard disks. This improves performance and also increases fault tolerance. A RAID appears to the operating system to be a single logical hard disk. RAID can be software–based (LVM) or managed by hardware controllers. Again, virtualization is a means to simplify management and add new capabilities to the storage system.
The table below defines the most common types of RAID.

One more type is RAID–6: this RAID level is similar to RAID–5 but includes a second parity scheme that is distributed across different drives and thus offers extremely high fault– and drive–failure tolerance.

Block Virtualization
The newest technology in virtualization is centered on block level disk services, and takes virtualization a step beyond what we discussed earlier. Block virtualization aggregates several physical disks to present a single logical disk. RAID is an example of this. However, the implications lead much further, as do modern applications of this.

The goal of block level virtualization is to control physical storage devices and combine them to provide logical storage devices to meet the needs of storage consumers without all the unnecessary (and confusing) low–level detail. Block level virtualization creates virtual storage from physical devices, which are as large, fast, and available.
Capacity, performance, and reliability. All you really need. Block level virtualization is the path to this.

• Add capacity in the background by extending or concatenating LUNs in the background, without applications noticing.
• Enhance performance by striping the file system across more disks (spindles, to you DBAs) to improve I/O without applications noticing.
• Increase the availability of the storage by clustering, RAID, mirroring, and replication. 
• Now, do all of that across multiple arrays and over distance—without the end user or application noticing. That’s virtualizing storage!

Thin Provisioning
This is one of the best examples of block level virtualization. Thin provisioning is an advanced storage virtualization capability that helps prevent low storage utilization by preventing over–allocation of capacity. Physical storage is allocated on demand from a shared pool, but only when needed. For example, a DBA demands a 200 GB LUN, but only really needs 10 GB to start with (you know those DBAs, always greedy for storage). With thin provisioning, you trick the database server (and thus the DBA!) into thinking it sees 200 GB (with virtual LBA), but you only really allocate 15 GB. Sophisticated management software sets thresholds, alarms, and (here it comes) automated re–provisioning of the storage as real utilization grows.

DAS, SAN, and NAS
The three fundamental types of storage attachment, terms we are all familiar with to a greater or lesser degree.

• DAS is the most familiar, and goes back the farthest in Open Systems. The storage is internally attached to the host. The application talks to the file system, which talks to the storage, all in one server. The storage isn’t shared to any other server’s applications. NAS and SAN were introduced as ways to share access.
• NAS storage introduces a network between the application, and the file system and storage. Access to the storage is over IP, through a CIFS (Windows Share) or NFS. The application or end user doesn’t know or care where the file system is: virtualization. This is a common form of shared storage, best suited to file serving (documents, spreadsheets, etc.)
• With SANs, the network is moved to sit between the storage, and the file system and application. The networking is typically done with Fibre Channel (just another variation of SCSI), but also can be done over IP or iSCSI: a way to move SCSI commands over a network that happens to be IP rather than FC. See the illustration below. Again, the storage is virtualized so that it might be more easily managed. 

Storage Area Networking (SAN): it enables logical storage devices to be shared and further virtualized to add new capabilities to help support new data center strategies: (you guessed it) Server Virtualization.

Virtualize My Storage
Where do we do this highest layer of virtualization, the one that allows us to transparently move storage around, grow it and re–assign it? Three places: the host, the array, and the network.
The host: we’ve discussed this. This is normally associated with Logical Volume Managers. With the advent of server virtualization, however, we now can virtualize storage on the host machine (ESX host) for the many virtual machines running there. These virtual hard drives or VMDK, can be local, or on shared storage, more on that in a moment. Also, the FC Host Bus Adapter connecting the ESX host to the SAN contains a vital virtualization piece called NPIV, which allows many virtual machines to use one FC HBA.   

The network: this is probably the most complex place to manage the storage, but can provide true heterogeneous storage virtualization and a single management interface for all virtualized storage.
The storage array: this is the most common way today to virtualize storage, both for the “traditional” ways we have discussed and for the cutting edge capabilities we need. Array–based virtualization is known as proprietary, or “lock–in,” but there is a new generation of storage controllers that allow you to take the advanced virtualization techniques of replication, thin provisioning and snapshots and allow you to use them across arrays of different vendors.

That’s right. Different vendors. We’ll get to that. Now, let’s talk about the special storage needs of virtual servers.

Virtual Servers and Shared Storage
Server virtualization solutions offer functionality that relies on, and requires, underlying networked storage. The virtual machines can be located on the ESX host, but the virtual disks can be on shared storage. You can’t have one without the other. Well, you can, but you miss out on a lot of really cool stuff:

• Live migration of virtual servers
• Dynamic resource sharing and allocation
• Automated failover
• Disaster recovery
• Bare–metal restores  

Deploying server virtualization and migrating physical servers to virtual machines creates the perfect opportunity to consolidate storage, centralize management, increase capacity utilization, improve availability, enhance data protection, and reduce backup windows.

How Should I Plan My Virtual Storage?
Server virtualization is a strategic technology, making your data center more flexible and efficient. You can consolidate multiple application workloads onto a fewer number of physical servers, thereby increasing server utilization while reducing hardware, power, and cooling costs. But server virtualization and consolidation introduce some key storage planning challenges.

• Selecting the best shared storage approach for a virtualized server deployment
• Preventing poor storage utilization on consolidated servers by preventing over allocation of capacity.

The Best Storage for My Virtual Servers
Many storage vendors, such as EqualLogic and Hitachi Data Systems, have aggressively certified their technologies with server virtualization products. Not all storage options offer an equivalent set of capabilities in virtualized server environments. Data center and storage managers need to consider more than just the potential fit of their existing storage.

• What is the best shared–storage approach for the applications hosted on a physical server, SAN or NAS? High I/O requirements would lead you to SAN.
• How will shared storage for virtualized servers be managed? Can the organization use the same management tools it uses for physical servers?

Thin Provisioning
Back to this…
How do you contend with reduced storage utilization on servers running multiple virtual workloads? Ironically, while server virtualization is an excellent method for increasing CPU utilization, it can drive down storage utilization.

• Major virtualization platforms such as ESX Server favor large, upfront allocations of capacity to a volume—up to double the needed capacity per volume per virtual machine (VM)—in the event that the VM must be suspended and its state written to disk.
• In a highly consolidated environment, this means that storage administrators will end up allocating a lot of storage for a relatively small number of VMs.
• The tendency toward over–allocation of storage capacity creates both low storage utilization and high costs, both of which drastically reduce the benefits of a virtualized server deployment.

To address this challenge, storage managers should consider thin provisioning. Though thin provisioning presents a logical volume of any size to VMware, it commits physical storage capacity only when data is written or used on a volume. By using thin provisioning in combination with server virtualization, an organization can optimize both server and storage utilization rates. Not all storage vendors currently offer this capability, but arrays from Hitachi Data Systems and EqualLogic (among others) do offer thin provisioning.

What are the capabilities each offer?

If I am using network storage (NAS or SAN via Fibre Channel or iSCSI), I gain the ability to do VMotion (that live migration of VMs mentioned above) and those advanced capabilities that sit on top of it (i.e., High Availability and DRS). If I am using a SAN that is either Fibre Channel or iSCSI, I also gain the ability to do things like snapshots and cloning. If I am running on a Fibre Channel SAN, I also gain the ability to do clustering at the VM level. If I use thin provisioning, I can reap the benefits of all of this, without wasting excessive amounts of storage. If I choose the right vendor, I can manage all of my storage arrays as one virtual pool and using one management interface. 

Best of Breed Solutions. GP Recommendation: Hitachi
The Hitachi Data Systems Universal Storage Platform is a storage controller technology that can virtualize external storage, logically partition that storage, and do thin provisioning and universal replication in flexible configurations. The USP–VM is the modular version of this, and probably one that many of you could fit into your modular storage environment.  

The USP–VM can scale up to 32 Petabytes! It provides a common interface for all of your external storage: EMC, HP, 3Par, whatever. With Hitachi software, such as Universal Volume manger, Virtual Partition manager, and Tiered Storage Manager, you can seamlessly migrate data off of older or less reliable arrays, create Virtual Storage Machines (think of dedicating cache, ports and disk to your ESX hosts to eliminate I/O contention) and many other otherwise arduous storage management tasks. It slices, it dices, and it makes your life easier.