Yahoo Japan Deploys Brocade's Ethernet Fabric to Support Hadoop Infrastructure

Japan’s largest Internet company Yahoo Japan Corporation has deployed Brocade's Ethernet fabric solution as the network foundation for an enterprise-wide Hadoop-based Big Data project.

Big Data is a key strategic initiative for Yahoo Japan. Hadoop is used in analyzing vast amounts of information from its portal to help improve overall quality of its services, as well as in the creation of new services. Hadoop is also used to support e-commerce and generate personalized and relevant content recommendations for users.

The new automated network infrastructure connects hundreds of servers supporting thousands of nodes dedicated to supporting Hadoop analysts across the enterprise.

The Brocade VDX Ethernet Fabric modular and fixed port switches replace a legacy Spanning Tree protocol-based network. Several Ethernet fabric solutions were tested and Brocade was selected because of performance, low latency, deep buffers as well as automated deployment and operational simplicity. It could handle the Big Data and eliminated manual configuration.

The legacy network wasn’t suitable for the new Hadoop design because of data throughput limitations and inability to easily add capacity. Along with increasing operational complexities, it meant application performance and availability was put at risk.

The new Hadoop environment is able to scale out with additional nodes quickly and cost-effectively and supports “bursty” high-speed communications between nodes, according to Yahoo Japan.

"In addition to delivering the high-performance needed to support enormous traffic processing for our new Hadoop environment, the Brocade VCS Fabric has allowed us to reduce the time spent on network deployment and operation by more than half, which has significantly lowered operational costs," said Kenya Murakoshi, Yahoo Japan’s manager of data center network, infrastructure engineering department, site operation division. "Compared to other fabric technologies, the simplicity of Brocade VCS fabrics is incredible, allowing us to quickly scale our network with tremendous flexibility."

High-density Brocade VDX 8770 modular chassis switches are used as aggregation switches that integrate a host of racks to create fabrics between the switches. The VDX chassis are connected to Brocade MLXe core routers using the switches' high bandwidth multi-chassis trunking (MCT) feature. The company has also deployed fixed-port Brocade VDX switches to build fabrics for smaller Hadoop infrastructures used for other projects at Yahoo Japan group companies.