Service Discovery & API Gateway
Service Discovery — Why Hardcoded URLs Don't Work at Scale
Solving auto-scaling node IP shifts and tracking healthy application nodes.
Interview: Service resolution architectures. Expect questions comparing server-side vs client-side service discovery models, DNS caching traps in the JVM, and handling node IP shifts in cloud environments.
Introduction
In monolithic applications, services call each other through direct in-memory method invocations. In microservices architectures, however, services communicate over the network. If you hardcode target physical URLs (like http://10.0.1.25:8080) in configuration files, any node restart or container rescheduling will break communication.
Cloud environments are highly dynamic. Pods spin up, scale down, and shift IP addresses constantly. **Service Discovery** solves this by maintaining a dynamic database of active, healthy instances. Services query this database to resolve locations at runtime, avoiding hardcoded dependencies.
Why It Matters
Automating service resolution prevents network breakage during rolling updates, enables client-side load balancing, and ensures that traffic is only routed to active, healthy application instances.
Server-Side vs. Client-Side Service Discovery
Organizations choose between two primary service discovery topologies:
• Server-Side Discovery (e.g., AWS ALB, Kubernetes Services): The client makes a call to a central load balancer (proxy). The load balancer queries the registry and routes the traffic. Simple for the client, but introduces a network hop and a potential bottleneck.
• Client-Side Discovery (e.g., Netflix Eureka, HashiCorp Consul): The client queries the service registry directly to fetch the list of healthy IPs. It caches this list locally and load-balances the calls itself using a client library. This eliminates the middleman proxy hop.
The JVM DNS Caching Trap
A common trap when using traditional DNS names for dynamic microservices is JVM caching. By default, the Java security manager caches DNS resolutions **forever** (or until JVM restart).
If a target service scales up or moves to new IP addresses, the JVM will continue sending traffic to the cached old IPs, overloading them or failing if those nodes are deleted. To prevent this, tune the security TTL settings:
networkaddress.cache.ttl=30 (force resolution refreshes every 30 seconds).
Quick Quiz
Q1: What is the primary operational disadvantage of relying on default JVM DNS caching configurations in dynamic container clusters?
A) It increases network packet sizes.
B) By caching DNS entries indefinitely, the JVM fails to discover new IPs of auto-scaled or relocated instances, sending traffic to stale endpoints.
C) It automatically disables garbage collection.
D) It forces the application to run as root.
Answer: B — Tuning the JVM DNS TTL is critical to ensure that client applications frequently refresh IP records and route traffic to healthy, active targets.
Scenario-Based Challenge
Production Scenario:
Your company deploys microservices behind an auto-scaling group. When the target service scales from 2 instances to 10 under load, the client application experiences timeouts because it continues routing all traffic to the original 2 instances. You confirm DNS records show all 10 IPs. How do you resolve this?
View Solution
To ensure the client discovers all auto-scaled instances:
1. **Configure JVM DNS TTL:** Set the network cache timeout in your JVM startup flags or inside the security configuration:
-Dsun.net.inetaddr.ttl=10 (refresh DNS resolution every 10 seconds).
2. **Migrate to Client-Side Discovery:** Implement a dynamic registry (like Eureka) and use a client-side load balancer (like Spring Cloud LoadBalancer). This bypasses DNS caching entirely by fetching updated IP lists directly from the registry.
Interview Questions
1. Explain the difference between client-side and server-side service discovery.
In client-side discovery, the client application queries the service registry, caches the active IP addresses locally, and routes calls using a local load balancer (no proxy middleman). In server-side discovery, the client calls a central load balancer (like AWS ALB), which queries the registry internally and routes the request, adding a network hop but keeping client code simple.
Production Considerations
For Kubernetes deployments, prefer native Kubernetes Services and DNS resolution over custom Eureka setups. Kubernetes manages endpoints internally, simplifying operations and reducing dependencies.