As we know, nowadays the network speed has reached 400G at data centers. And, most servers are coming with at least 25G or 50G network adaptors now as a default configuration. This fact brings challenging questions from our customers and we are facing iSCSI vs Fibre Channel storage network comparisons. Of course, priority is being competitive at the business and catching the future trends.
Before continuing, we assume you are familiar with block storage technologies. However, if you want a brief history of these protocols you can check one of the best technical deep-dive webcast about Fibre Channel and iSCSI comparison by SNIA.
Let’s focus on our main subject here; performance and cost!
First of all, we prepared a testbed to compare apples to apples. This is a very important point to highlight our decision. Because we see lots of non-comparable test scenarios on the internet, I mean apples to oranges. For example, comparing 1G iSCSI vs 8G Fibre Channel. Also, we know Fibre Channel networks are dedicated over SAN switches but somehow iSCSI tests are conducted on a shared backbone or edge switches. Similarly comparing 40G iSCSI vs 16G Fibre Channel is nonsense at all.
So what we did here is to create a dedicated TCP/IP network with 100G ethernet switches and a 32G Fibre Channel SAN network. And we started to test respectively 40G ethernet vs 32G FC then 100G ethernet vs 64G FC. We chose this because they are dominant current average connectivities at the market, and can be compared as apples to apples both in terms of price and performance.
First of all, let’s look at the economics;
* Costs are calculated from the online list prices of these components.
While the most important thing is availability we should create our storage infrastructure redundantly. So we need at least 2 components from each to find the minimal cost per server to create redundant storage fabric. Below table shows costs per connectivity type only for a single server. However, you can calculate your own costs depending on your numbers easily.
According to cost tables, creating an FC SAN storage network is ~67% more expensive than an iSCSI network. But before deciding on your future infrastructure, we need performance numbers.
Test Environment Details
To avoid any storage bottlenecks we used the NGX Storage NS200 SCM system.
Storage: NGX Storage NS200 SCM all-flash
- 2TB DRAM
- 24 Intel® Optane™ SSD DC P5800X 400GB
- 4 x 32G Fibre Channel
- 4 x 100GbE QSFP
Server : Dell R740 / OS Ubuntu 20.04
- 256GB Memory
- Intel(R) Xeon(R) Gold 6142M CPU @ 2.60GHz
- 2 x 32G Fibre Channel
- 1 x 40G (Mellanox Connect-X 4)
- 1 x 100G (Mellanox Connect-X 5)