Practical Learnings on Microservices

In a recent project, we delivered over 60 Microservices deployed on multiple Tomcat servers each in Production.

Below are some notes on real-life learnings.

Designing the Microservices:

•Business focus
•Small services with CRUD operations on single business function or domain 

•Design
•Use Lightweight REST based communication (client-to-service and service-to-service)
        •Keep Loosely Coupled
       •Ensure Services are Stateless
•Appropriate design patterns such as Aggregator, Proxy & Branch Patterns are commonly used
•In unavoidable cases - there will be Distributed Transactions and need to design for them

•Resilience
•Must design for Failure – e.g. Delays, Errors or Unavailability of another service or 3rd party system.
•Provide default functionality in case of failures from a service
•Rely on Input Validation - (client-to-service and service-to-service)

•Observability
•Centralized Logging and Monitoring is a must across distributed microservices

•Log events for timeouts and shut downs

•Logging to include the level, hostname (instance name), message

•Log events can be used for capacity planning and scaling e.g. which services need higher instances

•Business data related metrics such as no. of Bookings, Time taken to fill out the form 

•Log events which can be used for capacity planning and scaling

•Automation
•Use Testing tools for integration of services
•Quick feedback on check-ins and failures in the CI/CD pipeline

Benefits:

§Independent Development teams

•Business domain-driven design

•Responsive to business changes

§Independent Deployments

•Minimal impact and regression testing for small fixes

•Able to quickly take down and redeploy one service without impact to the overall system or other functional services.

§Independent Scalability

•Slots well into on-demand hosting and cloud services for scalability 

§Reusability for other Enterprise projects

Constraints:

§Higher Complexity for developers initially

§Deal with error handling, timeouts, retry mechanisms

§Service versioning can get confusing and needs to be managed.

§Managing Distributed Transactions requires Design rework.

§Requires increased number of VMs to support environments adding to infrastructure costs
§Need clear deployment strategy at the start to identify which services will be deployed on which VMs

§Increased overhead for Configuration management and Rundeck/UDeploy deployments during project life cycle and in production as a result of 60+ separate code builds and deployments

§Across environments, there are 240+ Tomcat Servers in Production and 480+  Servers in lower environments