The Challenges of Implementing Microservices

Wayne Gibbins is CCO at Wercker.

The emerging combination of micro-service architectures, Docker containers, programmable infrastructure, cloud, and modern Continuous Delivery (CD) techniques have enabled a true paradigm shift for delivering business value through software development.

Gartner sees the same shifts, and in its "2016 Hype Cycle for Application Architecture," the research group suggested that the micro-service architectural style is at the “peak of inflated expectations.” Accordingly, there may be some difficult times ahead for organizations that choose to adopt microservices, but we believe that one of the primary first steps towards success with this new architectural style is the codification and validation of functional and nonfunctional requirements within a CD pipeline.

Look Before You Leap: Visibility and the Codification of Requirements

The adoption of agile and lean software development methodologies has been a reaction to the need for increased speed within the business. For organizations still struggling to adopt these approaches, techniques like Value Stream Mapping and Wardley Mapping can prove invaluable. In reality, many organizations don’t have a holistic picture of what it takes to get an initial business idea or functionality improvement through to the end user.

It can also be tempting to blame one part of the organization for slowing delivery without really understanding the journey of a work item through the entire delivery system. One area that often gets blamed is Quality Assurance (QA), as the industry has attempted to validate this after software has been designed and implemented (leaving QA to be the "gatekeepers" of quality), to focus on ineffective manual efforts, or to sacrifice this part of delivery entirely.

We are strong believers in making work visible. A strong corollary of this need for visibility is the need to codify requirements as this facilitates automation, which in turn enables speed. The desire of CEOs and CIOs to deliver value through software more rapidly, and with increased safety, must be built upon automation. Many companies are codifying and validating requirements using custom steps within their CD pipelines and workflows. Behavior-Driven Development (BDD) is very useful for forming and codifying business hypothesis and behavior. Automated performance and security testing - with specified ranges of acceptability and codified into steps - should be considered table-stakes in the world of modern software development.

Microservices: Macro Impact with People, Architecture and Ops

Much has been said about the impact of Conway’s Law on the development of modern software, and indeed, the micro-service architectural style emerged out of organizational re-aligned at the unicorn organizations like Netflix, Gilt Group, and Spotify. This process has been further refined into the “Inverse Conway Maneuver”, where an organization deliberately attempts to manipulate structure and team alignment to promote an intended architecture within software i.e. forming small, cross-functional teams with well-defined business goals. Automated platform deployment and operational support should result in the creation of a microservice-based system.

Delivering a single microservice is not inherently difficult. The bigger issues arrive with integration. This is particularly challenging for enterprises that are naturally tackling complex problems that require many components to work correctly together to deliver the promised value. During the first iteration of service-oriented architecture, classical SOA, the solution was to specify service functionality with Web Service Definition Language (WSDL), effectively providing a contract in which to test and validate integrations. Although a great idea, the reality of WSDL was that it could be overly restrictive, and there were compatibility issues across platforms.

As the micro-service style has evolved, early proponents clearly took note of the integration challenges with SOA. Methodologies like Consumer-Driven Contracts (CDC) have emerged for the design and specification of services, with some even calling this “Test-Driven Design (TDD) for modern architecture”; the evolution of service virtualization to promote isolated testing of services; and run-time validation is frequently performed in staging (or production) with canary deployments and synthetic transactions continually testing happy path functionality and critical nonfunctional requirements.

All of these methodologies can, and should, be integrated within your build pipeline.  As software consultant Kevlin Henney famously suggested: The reason automobiles have brakes is so that they can travel faster; and in the world of implementing microservices to deliver rapid value, the integration of automated validation into your continuous delivery pipeline is directly analogous - if something goes wrong, the system automatically hits the brakes on the deployment.

Summary

The micro-service architectural style provides many benefits. Gartner cautions that we are at the “peak of inflated expectations.” The hype around this approach should not be ignored, but the risks can be mitigated with a sensible and structured approach to implementing Continuous Delivery (CD) within your software delivery process. The successful implementation of microservices does not occur purely at the architectural level, and attention must be paid to foster the correct organizational structure and develop a culture of hypothesis-driven development to yield the full benefits.

Finally, validating the functional and nonfunctional requirements of a micro-service system is essential, both at the individual service and also (arguably most importantly) at the integrated system level. We strongly believe in the power of building effective continuous delivery pipelines and view this as a core component for organizations looking to embrace microservices and the modern "cloud-native" approach to delivering valuable software.

Opinions expressed in the article above do not necessarily reflect the opinions of Data Center Knowledge and Penton.

Industry Perspectives is a content channel at Data Center Knowledge highlighting thought leadership in the data center arena. See our guidelines and submission process for information on participating. View previously published Industry Perspectives in our Knowledge Library.