What Are the New Storage Connectivity Options?

• 8 Gb Fibre Channel: The high-performance high-end storage protocol that sends SCSI commands and block data serially to tape and disk targets


• iSCSI – sending those same SCSI commands and block data over Ethernet, in TCP/IP packets


• FCoE: Fibre Channel over Ethernet. What is Enhanced Ethernet and what does it have to do with Fibre Channel?


• 10 Gigabit Ethernet. iSCSI over 10 GbE? What about NAS using 10 GbE?

Here is a concise protocol summary from the Fibre Channel Industry Association that we can refer to as the review proceeds.

Figure 1: Protocol Summary

Why Do We Need Faster Storage Connectivity?

A number of factors are driving adoption and the eventual ubiquity of 8 Gbps Fibre Channel and 10 Gbps Ethernet:

• Multi-socket, multi-core server technology supports higher workload levels which, in turn, demand greater network throughput.


• Server virtualization enables workload consolidation, which contributes additionally to network throughput demands.


• Increasing use of network storage requires higher bandwidth between servers and storage.

FCoE & Enhanced Ethernet

FCoE is an emerging standard for Ethernet, based on 10 GbE, due in final form mid-2009 from the T11 Standards body of INCITS (www.t11.org/index.html).

FCoE uses an Enhanced Ethernet specification that is Lossless (like FC), to deliver the standard FC Payload. This allows mapping of FC payload onto a special Ethernet Frame type, thus leveraging Ethernet cabling and switching. It also retains some of the most valued and familiar features of FC: zoning and name server database. FCoE looks like native FC to the OS, applications and management tools. Discovery, WWN, zoning—all the same.

Ethernet requires three primary modifications to deliver the capabilities of Fibre Channel over Ethernet networks:

1. Encapsulation of native Fibre Channel frames into Ethernet Frames.


2. Extensions to the Ethernet protocol itself to enable an Ethernet fabric in which frames are not routinely lost during periods of congestion.


3. Mapping between Fibre Channel N_port IDs (aka FCIDs) and Ethernet MAC addresses.

Computer servers using FCoE are initially expected to use converged network adapters (CNAs), which contain both Fibre Channel Host Bus Adapter (HBA) and Ethernet network interface card (NIC) functionality on the same adapter card in combination with one or more physical Ethernet ports.

FCoE Advantages

FCoE enables I/O consolidation and offers these enhancements to the datacenter:

• Fewer Network Interface Cards per server


• Reduction in server power budget and reduced cooling requirements


• Significant reduction in the amount of cables to manage


• Seamless connectivity with existing Storage Area Networks and Ethernet Networks—you can continue to leverage existing FC tools, security model and infrastructure

Products are already on the market: Cisco (Nexus 5200), Brocade (DCS Backbone), Emulex and QLogic (Converged Network Adapters) are all releasing products to support this. Hitachi and EMC are set to update their controllers to support this.

Cisco and the Nexus

The new Cisco Nexus 5010 28-port switch is a one rack unit (RU) switch supporting 10 Gigabit Ethernet, Cisco Data Center Ethernet (DCE), Fibre Channel over Ethernet (FCoE), and Fibre Channel, able to consolidate traffic from local area networks, storage area networks, and server clusters onto a single unified fabric. Winner of VMworld 2008 Best of Show among other awards.

With a unified fabric, IT organizations can simplify cabling infrastructure, reduce the number of required adapters, lower costs, and reduce power consumption and carbon footprint.

QLogic Converged Network Adapter

QLE8042: Dual Port 10Gbps Ethernet to PCIe Converged Network Adapter (CNA), with 10 GbE full duplex line rate, Jumbo Frame, Stateless Offload, Enhance Ethernet Compliance, 150,000 IOPS per port and Windows and Linux support.

FCoE Disadvantages

• Do you want to be an early adopter? This one will be hot…or not?


• Not routable, at least at first. This is initially intended to be a datacenter protocol


• A new chipset is required, so new 10 Gb Ethernet switches are required for Enhanced Ethernet and FCoE

  - If you haven’t deployed 10 GbE yet, this isn’t a problem.

• You need to understand FC storage management principles of zoning, discovery, and World Wide Names

  - If you are already an FC shop, this isn’t a problem.

iSCSI

iSCSI has become the storage connectivity of choice for small business, with applications such as VMware VI3, Exchange, and MS SQL certified on it. It runs on standard Ethernet networks. All the major vendors are onboard now, such as:

• Hitachi
• Cisco
• Dell EqualLogic
• EMC
• NetApp
• QLogic
• HP

Figure 2: iSCSI Protocol Stack

10 Gig iSCSI

10 GbE iSCSI: the speed of 10 GB Ethernet with the iSCSI protocol. I think the FCoE will appeal to the FC crowd, and for those that have always harbored a secret grudge with FC and the vendors that pushed it, this protocol is for them!

Seriously, those new storage administrators that adopted iSCSI and are comfortable with its features and implementation will be able to justify and implement truly enterprise level shared storage systems with this speed.

Also, this protocol will remain routable, allowing storage to be shared outside the datacenter.

But…(there is always a “but”)

• Tools & management applications are not equal to FC


• 10 Gb iSCSI targets are further down the road


• There are no 10 GbE iSCSI HBAs yet

NetApp is already out there blogging on the advantages of running their filers using NFS over 10 GbE: tested average throughput using NFS over 10 GbE is 14 Gbs, followed by 12.5 Gbs for 10 GbE iSCSI. Hitachi is there, too, with powerful NAS from BlueArc, with two 10 GbE port per controller.

10 GbE iSCSI will have some technical hurdles to overcome for error-handling with collisions at high speed. Will we really need 10 GbE iSCSI for SMB applications? You can decide…

8 Gb Fibre Channel

IDC outlines the Minimum Business Attributes of the Enterprise Data Center, all of which are affected positively by the adoption of 8 Gb FC:

• Manageability
• Scalability
• Availability
• Security
• Virtualization
• Power
• Cooling
• Storage
• Floor space

Virtualization aggregates multiple applications and OSes, which jacks up I/O traffic significantly. VMware and other server virtualization vendors are driving this. 8 Gb Technology would enable higher levels of aggregation at the adapter level.

Figure 3: Virtual Server I/O

Brocade and Hitachi Data Systems

Brocade and Hitachi Data Systems: a solid partnership and tested solutions. Brocade is out in front on the Converged Enhanced Ethernet with the DCX Backbone, Brocade 815/825 8 Gb HBAs, and the Brocade switches 300, 5100, 5300 and blades for the 48000 Director.

Tested and certified for 8 Gb FC and iSCSI is the new AMS2000 Series, with up to 8 FC Host Ports per controller.

Figure 4: Brocade and HDS Fabric

InfiniBand: What Is It?

Like Fibre Channel, PCI Express, and Serial ATA, InfiniBand is a point-to-point bidirectional serial link connecting processors with high speed disk. InfiniBand also provides RDMA capabilities for low CPU overhead. It uses a switched topology.

InfiniBand’s data rate is at 2.5 gigabit per second (Gbit/s) in each direction per connection. With double (DDR) and quad data (QDR) speeds, for 5 Gbit/s or 10 Gbit/s, a quad-rate 12X link carries 120 Gbit/s raw, or 96 Gbit/s of useful data.

Figure 5: IB Efficiency

InfiniBand’s use is expanding in commercial cluster computing, where interconnect performance and price performance is the driving factor. IDC projects a rather healthy future for InfiniBand (IB) adoption over the next five years. The growth of multicore processors, server virtualization, and I/O consolidation is going to drive InfiniBand adoption in the enterprise.

Besides raw performance, InfiniBand also includes quality-of-service (QoS) features that enable multiple types of traffic to be safely managed over a single pipe.

Take Home? Here’s the Scorecard:

• InfiniBand: still one of the best storage connectivity methods around, paired up with RDMA and channel-based architecture, but often relegated to the HPC niche.


• 10 GbE & iSCSI: it will take a while to work out the iSCSI implementation working at 10 Gbps speeds, and iSCSI arrays also have some engineering work to do to handle it. Meanwhile, what about NAS?


• FCoE: the hot new interconnect, but will “converged” and “unified” networks hit speed bumps?


• 8 Gb FC: it is here, it works, no problem. You just need an FC-trained storage administrator and a few bucks.

What are the other kids doing? Here are some surveys…

• A recent ESG research survey indicates that 15% of companies with 100 to 999 employees and more than 100 servers report that FCoE is the most important storage-related initiative for their organization over the next 24 months.


• In its most recent study, TIP found that approximately 16% of Global 2000 organizations have installed iSCSI storage.


• Analysts at the IDC project flat growth for Fibre Channel fabrics through 2010: they predict more than a 60 percent increase in iSCSI adoption within the same timeframe.

Figure 6: Protocol Adoption Rate

I hope this has helped clarify some of the questions that we routinely hear from our customers across the country. If you’d like to view the webcast version of this article, you can use this link: Building a Data Superhighway. And if you’d like to talk about how these solutions will fit in your particular environment, contact your GreenPages Account Manager.

‹ ‹ ‹ Back to Newsletter