This is achieved through fault tolerance and failover strategies, which together improve availability, reliability, and system resilience.
What Is Fault Tolerance?
Fault Tolerance refers to a system's ability to continue functioning even when individual components fail.Instead of crashing or exposing errors to customers, fault-tolerant systems mask failures using redundancy, replication, retries, and recovery methods.
When done properly, the user never learns that something went wrong—the system absorbs failures internally and gracefully.
Example: Checkout Resiliency
Consider a global E-commerce checkout system with these services:1. Inventory Service
2. Payment Gateway
3. Order Service
4. Shipping Service
A customer adds a laptop to the cart and proceeds to buy it. Now imagine that at the exact moment the user clicks "Place Order," the Inventory Service node in one availability zone crashes.
A non-fault-tolerant system will return an error like "Inventory unavailable" and lose the transaction.
A fault-tolerant system does not fail the transaction—it automatically shifts to a healthy replica and continues execution.
Failover Strategies
While fault tolerance enables a system to continue operating despite failures, failover is the mechanism that automatically redirects requests from a failed component to a healthy one.Effective failover strategies minimize downtime, isolate failures, and prevent localized issues from escalating into system-wide outages.
Modern distributed systems combine multiple failover techniques to improve availability, reliability, and resilience. The most commonly used strategies include:
1. Active-Active & Active-Passive Failover
Multiple service instances are deployed across different servers, availability zones, or regions to eliminate single points of failure.In an Active-Active architecture, all instances serve production traffic simultaneously.
A load balancer distributes requests across healthy instances, and if one instance becomes unavailable, traffic is automatically routed to the remaining healthy instances with little or no interruption.
In an Active-Passive architecture, only the primary instance serves requests while one or more standby instances remain idle. When the primary fails, a standby instance is promoted to become the new primary.
Example: Checkout Service
An e-commerce platform deploys three Checkout Service instances across different availability zones.If one instance crashes, the load balancer automatically routes new requests to the remaining healthy instances without affecting customers.
CheckoutService service = loadBalancer.nextHealthyInstance();
return service.checkout(order);2. Circuit Breakers
A Circuit Breaker prevents cascading failures by temporarily stopping requests to a service that is repeatedly failing or responding slowly.Instead of allowing every request to wait for a timeout, the circuit breaker immediately returns a fallback response or an error.
After a configurable recovery period, it allows a small number of requests to verify whether the service has recovered before resuming normal traffic.
Example: Payment Gateway
Suppose the primary payment provider becomes slow due to high traffic. Without a circuit breaker, every checkout request waits several seconds before timing out, exhausting application threads.With a circuit breaker, new requests fail fast and are redirected to a backup gateway.
try {
return primaryGateway.charge(order);
} catch (Exception e) {
circuitBreaker.open();
return backupGateway.charge(order);
}3. Graceful Degradation
Graceful degradation allows an application to continue providing its core functionality even when non-critical services become unavailable.Rather than failing the entire request, the system temporarily disables optional features until the dependent service recovers.
Example: Product Recommendations
During an outage, the recommendation engine becomes unavailable.Instead of failing the product page, the application simply hides the "Recommended Products" section while allowing customers to browse products, add items to the cart, and complete purchases.
Recommendations recommendations;
try {
recommendations = recommendationService.get(productId);
} catch (Exception ex) {
recommendations = Recommendations.empty();
}
return ProductPage.of(product, recommendations);4. Bulkhead Isolation
Bulkhead Isolation separates resources such as thread pools, connection pools, or compute resources so that failures in one service cannot consume resources needed by others.This prevents localized failures from spreading across the entire application.
Example: Notification Service
Suppose an email service becomes slow. If all application requests share the same thread pool, email requests may consume every available thread and prevent checkout requests from executing.By assigning dedicated thread pools to each service, checkout continues operating normally even if email processing becomes overloaded.
ExecutorService checkoutPool =
Executors.newFixedThreadPool(50);
ExecutorService notificationPool =
Executors.newFixedThreadPool(10);5. Automated Retries with Exponential Backoff
Temporary failures such as network interruptions, leader elections, or brief service overloads often resolve themselves within a short period.Instead of failing immediately, applications retry the request after progressively increasing delays. This exponential backoff reduces unnecessary load on recovering services while increasing the likelihood of success.
Example: Inventory Service
The Inventory Service experiences a temporary timeout during a database failover.Instead of returning an error immediately, the application retries after waiting 1 second, then 2 seconds, then 4 seconds before ultimately giving up if the request still fails.
for (int retry = 0; retry < 3; retry++) {
try {
return inventoryService.reserve(itemId);
} catch (TimeoutException ex) {
Thread.sleep((long) Math.pow(2, retry) * 1000);
}
}
throw new ServiceUnavailableException();6. Health Checks & Auto-Healing
Modern orchestration platforms continuously monitor the health of application instances using health checks.Instances that fail these checks are automatically removed from service, and replacement instances are created to maintain the desired capacity.
This process, often called auto-healing, allows applications to recover from failures with minimal manual intervention.
Example: Kubernetes Deployment
A Checkout Service pod crashes because of an out-of-memory error.Kubernetes detects that the health check is failing, removes the unhealthy pod from the load balancer, starts a replacement pod, and begins routing traffic to it once it becomes healthy.
livenessProbe:
httpGet:
path: /actuator/health
port: 8080
readinessProbe:
httpGet:
path: /actuator/health
port: 8080Conclusion
Failures are inevitable in distributed systems, so modern architectures are designed to tolerate them rather than avoid them.By combining fault tolerance with effective failover strategies, systems can continue operating even when individual components, servers, or entire regions become unavailable.
These techniques help maintain availability, preserve data integrity, and minimize service disruption, ensuring that users can continue interacting with the application even during infrastructure failures.
