The Impact of Block Sizes in a Data Center

Pete Koehler is an Engineer for PernixData.

Guesswork is often the enemy of those responsible for data center design, operations, and optimization. Unknown variables lead to speculation, which inhibits predictability and often compromises success. In the world of storage, many mysteries still remain, unfortunately, with block sizes being one of the most prominent. While the concept of a block size is fairly simple, its impact on both storage performance and cost is profound. Yet, surprisingly, many enterprises lack the proper tools for measuring block sizes, let alone understanding them and using this information to optimize data center design.

Let’s look this topic in more detail to better understand what a block is and why it is so important to your storage and application environment.

What is Block Size?

Without diving deeper than necessary, a block is simply a chunk of data. In the context of storage I/O, it would be a unit in a data stream; a read or a write from a single I/O operation. Block size refers to the payload size of a single unit. We can blame a bit of this confusion on what a block is by a bit of overlap in industry nomenclature. Commonly used terms like blocks sizes, cluster sizes, pages, latency, etc. may be used in disparate conversations, but what is being referred to, how it is measured, and by whom may often vary. Within the context of discussing file systems, storage media characteristics, hypervisors, or operating systems, these terms are used interchangeably, but do not have universal meaning.

Most who are responsible for Data Center design and operation know the term as an asterisk on a performance specification sheet of a storage system, or a configuration setting in a synthetic I/O generator. Performance specifications on a storage system are often the result of a synthetic test using the most favorable block size (often 4K or smaller) for an array to maximize the number of IOPS that an array can service. Synthetic I/O generators typically allow one to set this, but users often have no idea what the distribution of block sizes are across their workloads, or if it is even possibly to simulate that with synthetic I/O. The reality is that many applications draw a unique mix of block sizes at any given time, depending on the activity.

The difficulty with understanding the impact of block sizes always comes back to one key issue - the lack of ability to view them, and interpret their impact. This is quite surprising considering how many performance issues related to storage are ultimately tied to block sizes. Understanding such an important element of storage shouldn’t be so difficult. .

Why Does Block Size Matter?

As mentioned prior, a block is how much storage payload is sent in a single unit. The physics of it become obvious when you think about the size of a 4KB payload versus a 256KB payload (or even a 512KB payload). Since we refer to them as a block, let’s use a square to represent their relative capacities.

Throughput is the result of IOPS, and the block size for each I/O being sent or received. Since a 256KB block has 64 times the amount of data as a 4K block, size impacts throughput. In addition, the size and quantity of blocks impacts bandwidth on the fabric and the amount of processing required on the servers, network and storage environments. All of these items have a big impact on application performance.

This variability in performance is more prominent with Flash than traditional spinning disk, and thus should be carefully observed when procuring an All Flash Array or other device using solid state storage. Reads are relatively easy for flash, but the methods used for writing to NAND Flash can inhibit the same performance results from reads, especially with writes using large blocks. A very small number of writes using large blocks can trigger all sorts of activity on the flash devices that obstructs the effective performance from behaving as it does with smaller block I/O. This volatility in performance is a surprise to just about everyone when they first see it.

Block size can impact storage performance regardless of the type of storage architecture used. Whether it is a traditional SAN infrastructure, or a distributed storage solution used in a hyper-converged environment, the same factors, and challenges remain. Storage systems may be optimized for different block size that may not necessarily align with your workloads. This could be the result of design assumptions of the storage system, or limits of their architecture. The abilities of storage solutions to cope with certain workload patterns vary greatly as well. The difference between a good storage system and a poor one often comes down to the abilities of it to handle large block I/O. Insight into this information should be a part of the procurement, design and operation of any environment.

The Applications that Generate Blocks

What makes the topic of block sizes so interesting are the operating systems, the applications, and the workloads that generate them. The block sizes are often dictated by the processes of the OS and the applications that are running in them.

Unlike what many might think, there is often a wide mix of block sizes that are being used at any given time on a single VM, and it can change dramatically by the second. These changes have profound impact on the ability for the VM and the infrastructure it lives on to deliver the I/O in a timely manner. It’s not enough to know that perhaps 30 percent of the blocks are 64KB in size. One must understand how they are distributed over time, and how latencies or other attributes of those blocks of various sizes relate to each other.

Traditional Methods Lack Visibility

The traditional methods for viewing block sizes have been limited. They provide an incomplete picture of their impact – whether it be across the data center, or against a single workload. Below is a breakdown of some common methods for measuring block sizes, and a description as to why they are lacking:

The result, especially at scale, is a very labor intensive exercise that is an incomplete solution. It is extremely rare that an Administrator runs through this exercise on even a single VM to understand their I/O characteristics.

The hypervisor is an ideal control plane to analyze the data. It focuses on the results of the VMs without being dependent on nuances of in-guest metrics or a feature of a storage solution. It is inherently the ideal position in the data center for proper, holistic understanding of your environment.

The Absence of Block Size in Data Center Design Exercises

The flaw with many design exercises is we assume we know what our assumptions are. Let’s consider typical inputs when it comes to storage design. This includes factors such as

Most who have designed or managed environments have gone through some variation of this exercise, followed by a little math to come up with the correct blend of disks, RAID levels, and fabric to support the desired performance. Known figures are used when they are available, and the others might be filled in with assumptions. But yet, block sizes, and everything they impact are nowhere to be found. Why? Lack of visibility, and understanding.

An infrastructure only exists because of the need to run services and applications on it. Let those applications and workloads help tell you what type of storage fits your environment best. Not the other way around.

Summary

Proper visual acuity and understanding of the distribution of block sizes across an environment pays dividends throughout the entire lifecycle of an environment. Understanding and accommodating for block sizes in the design, operation and optimization phases of the VM lifecycle leads to more predictable application delivery in your environment, with possibly a more affordable price tag.

Industry Perspectives is a content channel at Data Center Knowledge highlighting thought leadership in the data center arena. See our guidelines and submission process for information on participating. View previously published Industry Perspectives in our Knowledge Library.