In a microservices architecture, exposing dozens of backend services directly to clients is neither secure nor maintainable. The API gateway is the single controlled entry point — the architectural boundary between internal system complexity and external client simplicity. A well-designed API gateway handles authentication, authorization, rate limiting, request routing, protocol translation, caching, and observability. Designed poorly, it becomes a bottleneck and single point of failure. Designed well, it's the most impactful architectural component in your platform.
Scope Note
This article covers API gateway design for B2B SaaS platforms, mobile-backend architectures, and third-party developer portals. We examine self-hosted options (Kong, Traefik, Envoy-based) and managed solutions (AWS API Gateway, Azure APIM, Google Cloud Apigee).
Gateway Responsibilities: A Layered Architecture Model
The gateway's responsibilities must be organized by concern, not functionality. Each layer has a distinct owner and failure mode:
API Gateway Layer Architecture
| Layer | Components | Responsibility |
|---|---|---|
| Edge Layer | CDN, WAF, DDoS | TLS termination, IP allowlisting, geo-routing, DDoS protection |
| Security Layer | JWT validation, API keys, OAuth2 | Authentication, authorization, tenant resolution |
| Traffic Management | Rate limiting, circuit breaker | Throttling, retries, backpressure |
| Routing & Transform | Path routing, Kustomize | Request/response transformation, protocol bridging |
| Upstream Services | Microservices, legacy APIs | Business logic execution |
Rate Limiting: Algorithms and Architectural Placement
Rate limiting is more complex than it appears. There are four primary algorithms, each with distinct characteristics that make them appropriate for different scenarios.
Rate Limiting Algorithm Comparison
| Algorithm | Burst Handling | Implementation | Best For |
|---|---|---|---|
| Fixed Window | Poor (boundary bursts) | Redis INCR + EXPIRE |
Simple APIs, low-risk endpoints |
| Sliding Window | Good | Redis sorted sets |
Most REST APIs |
| Token Bucket | Excellent (allows bursts) | Redis + Lua script (atomic) |
Developer APIs, bursty clients |
| Leaky Bucket | None (smooths all) | Queue-based processing |
Downstream service protection |
Rate Limiting Implementation Rules
- Always enforce rate limits in distributed Redis — never in-memory (multi-node gateways)
- Use Lua scripts for atomic check-and-increment to prevent race conditions
- Scope limits at multiple levels: per-IP, per-API-key, per-tenant, per-endpoint
- Always return
X-RateLimit-Limit,X-RateLimit-Remaining,X-RateLimit-Resetheaders - Expensive endpoints (reports, exports) get tighter limits than simple reads
API Versioning: Designing for Change Without Breaking Clients
API versioning is a contractual commitment. Once an API is public, removing or changing existing fields breaks clients — often in production. The three versioning strategies each have trade-offs:
Versioning Strategy Comparison
| Strategy | Example | Cacheability | Discoverability | Recommendation |
|---|---|---|---|---|
| URL Versioning | /api/v2/orders |
Excellent | High | Recommended for most teams |
| Header Versioning | Accept: application/vnd.v2+json |
Poor | Low | Complex clients |
| Query Parameter | /api/orders?version=2 |
Good | Medium | Legacy compatibility |
Design principle: Versions are created only for breaking changes. Adding fields, adding optional parameters, and adding new endpoints are non-breaking and require no version bump. Maintain a deprecation policy — no version removed without at least 12 months notice and a migration guide.
Authentication Federation: JWT, OAuth2, and mTLS Patterns
The gateway is the right place to centralize authentication so individual backend services never handle credentials directly. For end-user APIs, OAuth2 with PKCE flows and JWT access tokens is the standard.
Authentication Patterns by Use Case
| Client Type | Auth Pattern | Token Format | Gateway Action |
|---|---|---|---|
| End user (web/mobile) | OAuth2 + PKCE | JWT (short-lived) | Validate signature via JWKS |
| Third-party developer | API Key | Opaque key (hashed in DB) | Key lookup + rate limit check |
| Internal service | mTLS / SPIFFE | X.509 certificate | Certificate validation via mesh |
| Machine-to-machine | OAuth2 Client Credentials | JWT (service scope) | Scope enforcement |
Critical Security Pattern
Never use long-lived API keys without rotation mechanisms. Implement key rotation APIs, log all key usage with timestamps and IPs, alert on anomalous usage patterns (unexpected geographic origins, volume spikes), and provide key revocation with graceful grace periods.
Self-Hosted vs. Managed API Gateways: The Decision Framework
| Gateway | Type | Best For | Key Trade-off |
|---|---|---|---|
| Kong | Self-hosted on K8s | High volume, full control, plugin ecosystem | Requires platform engineering team |
| AWS API Gateway | Fully managed | AWS-native, Lambda integration, low ops overhead | Per-request pricing at scale |
| Azure APIM | Fully managed | Enterprise Azure ecosystems, built-in developer portal | Complex configuration |
| Apigee (GCP) | Fully managed | API product management, monetization, analytics | High cost, enterprise-focused |
| Traefik / Envoy | Self-hosted | Custom routing, sidecar proxy patterns | Minimal API management features |
Work With Us
Overseas IT Solution designs and implements production-grade API gateways, developer portals, and backend architectures for SaaS products across the globe.
Visit overseasitsolution.com© 2026 Overseas IT Solution · overseasitsolution.com · Ahmedabad, Gujarat, India
