Pairing for Scalability: In-Memory Data Grids and the Cloud

William Bain is CEO of ScaleOut Software.

William-Bain-tn

Cloud computing is rapidly being adopted by companies across a wide range of industries. Among its many benefits, the ability to offer on-demand elastic computing enables enterprises to maintain fast application performance in the face of burgeoning workloads. Cloud infrastructure must easily scale to add more application instances, and the environment must offer platform services that enable applications to effectively and transparently scale performance over those instances. This is where cloud-based in-memory data grids (IMDGs) enter the picture. By dramatically simplifying and enhancing the deployment of scalable applications within cloud infrastructures, in-memory data grids play a key role in delivering on the promise of cloud computing.

Even better, today’s advanced IMDGs can work across cloud and on-premise environments and some even integrate data parallel computational engines that can perform real-time analytics on cloud-based data, while the data is rapidly changing.

Elastic, Memory-Based Storage in the Cloud

Scalable applications hosted in the cloud need to eliminate performance bottlenecks so that they can take full advantage of the cloud's elastic resources. The use of an in-memory data grid gives applications a scalable storage repository for fast-changing application data, eliminating disk bottlenecks and minimizing use of cloud-based persistent storage. It also enables transparent sharing of application data across a pool of application instances, which simplifies design and reduces development time.

To be effective, the IMDG needs to natively support elastic scaling of the data grid to meet application requirements. As servers are added, the grid should automatically scale its storage capacity throughput and load, permitting applications to benefit from consistently fast access times. When no longer needed, grid servers can be removed, and stored data automatically compacts into the remaining servers. The in-memory data grid's natural elasticity helps applications enjoy the full benefits of running in the cloud.

Transparent Data Migration

In-memory data grids can also significantly reduce the complexity of migrating applications to the cloud, which helps both developers and IT managers seamlessly take advantage of cloud resources. They do this by integrating data grids at multiple sites into a single logical, and coherent in-memory data grid. It can be viewed as forming a "bridge" to the cloud, automatically migrating data between on-premise and cloud environments as needed.

By making data seamlessly available regardless of location, you can avoid the need for applications to manually re-stage grid data into a separate cloud-based store. Applications benefit from immediate, transparent access to fast-changing data worldwide.

For example, consider a premise-hosted ecommerce Web farm that needs to scale into the cloud to handle high seasonal demand. To accomplish this, the Web site’s administrator reconfigures the IP load-balancer to distribute Web requests across both on-premise and cloud-based Web servers. By using an in-memory data grid that supports data integration, all Web servers transparently and coherently share session data within a single, virtualized Web server farm spanning both sites. The migration of session data is automatically managed as it flows into and out of the cloud without the need for explicit re-staging by IT administrators.

Data Analysis

In addition to automatic scalability and transparent data migration, in-memory data grids open the door to performing powerful data analysis within the cloud. Advanced IMDGs provide an integrated computational platform for data-parallel analysis in the cloud. By integrating a highly efficient "map/reduce" style execution engine with a scalable, in-memory data grid, real-time analysis can be performed on live operational data held in the IMDG and can be run continuously while the data is being updated. The result: businesses can spot opportunities and issues when they arise and take immediate action. There are a multitude of use cases that can benefit from real-time analysis, like financial risk analysis, fraud detection, recommendation engines, reservation systems, and many more.

In summary, the cloud and IMDGs are highly complementary to each other. Together they offer an environment where elastic computing can be employed to provide a range of benefits to help enterprises meet today’s computing challenges and to optimize business results.

Industry Perspectives is a content channel at Data Center Knowledge highlighting thought leadership in the data center arena. See our guidelines and submission process for information on participating. View previously published Industry Perspectives in our Knowledge Library.