The system should generate unique short URLs, support billions of URLs, provide fast redirection, and remain highly available.
Requirements
Functional Requirements
The URL Shortener system should satisfy the following functional requirements.1. Generate a unique short URL for a given long URL.
2. Redirect a short URL to the original URL.
3. Support custom aliases (optional).
4. Support URL expiration (optional).
5. Track basic analytics such as click count (optional).
Non-Functional Requirements
In addition to the functional requirements, the URL Shortener system should satisfy the following non-functional requirements.1. Low latency.
2. High availability.
3. Horizontal scalability.
4. High read throughput.
5. Fault tolerance.
High-Level Architecture
The following diagram illustrates the high-level architecture of a URL Shortener. Client requests are routed through a Load Balancer to the URL Service, which handles URL creation and redirection requests.The service uses Redis for low-latency lookups, a Bloom Filter to reduce unnecessary database queries, a database for persistent URL storage, and Kafka to process click analytics asynchronously.
API Design
The URL Shortener system can expose the following REST APIs to create, retrieve, and manage shortened URLs.Create Short URL
Request:POST /api/v1/urls
{
"url": "https://www.example.com/articles/system-design"
}
{
"shortUrl": "https://tiny.ly/aB12Cd"
}
HTTP Redirect
The URL Service does not return the original URL in the response body.Instead, it responds with an HTTP redirect, allowing the client (browser or application) to automatically navigate to the original URL.
The most commonly used redirect status codes are 301 Moved Permanently and 302 Found.
A 301 redirect indicates that the resource has permanently moved and can be cached by browsers and search engines, making it suitable for permanent URL mappings.
A 302 redirect indicates a temporary redirect and is generally preferred for URL shorteners because the destination URL may change over time, and click analytics can be recorded on every request without aggressive client-side caching.
HTTP/1.1 302 Found
Location: https://www.example.com/articles/system-designChoosing the Database
A URL shortener performs a very high number of read operations compared to writes. The database should provide fast lookups by short code while supporting horizontal scaling as the number of URLs grows.Relational Database
A SQL database provides strong consistency and guarantees uniqueness using PRIMARY KEY or UNIQUE constraints.It is a good choice for small and medium-sized deployments where transactional consistency is important.
Database Design
URL
----------------------------------------------------------
id | short_code | long_url | created_at | expires_at
NoSQL Database
A NoSQL database is preferred for internet-scale deployments because it supports horizontal partitioning and can store billions of URLs across multiple servers.Since URL lookup is typically performed using the short code, the short code naturally becomes the partition key.
Database Design
The((short_code)) syntax explicitly indicates that short_code is the partition key. A clustering key is not required because each short code maps to exactly one URL.
Therefore, every request is a simple key-value lookup:
short_code → URL, which is the access pattern Cassandra is optimized for.
CREATE TABLE urls (
short_code TEXT,
long_url TEXT,
created_at TIMESTAMP,
expires_at TIMESTAMP,
PRIMARY KEY ((short_code))
);
For example, a dedicated url_click_events table stores click events for each short URL, while the primary urls table remains optimized for fast lookups and redirects.
CREATE TABLE url_click_events (
short_code TEXT,
event_time TIMESTAMP,
country TEXT,
device TEXT,
browser TEXT,
PRIMARY KEY ((short_code), event_time)
) WITH CLUSTERING ORDER BY (event_time DESC);
Generating Short URLs
Several approaches can be used to generate unique short URLs. The choice depends on the required scalability, collision tolerance, and deployment architecture.Auto Increment + Base62 Encoding
In this approach, the system first generates a unique numeric ID using a database auto-increment column or a distributed ID generator.The numeric ID is then converted to a Base62 string using the following character set:
0-9 A-Z a-z
125 --> cb
1000000 --> 4C92
This approach is simple to implement and guarantees unique short codes. However, if IDs are generated by a single database, it can become a bottleneck under heavy load.
Additionally, because IDs are sequential, the generated short URLs are predictable, making it easier to infer the rate of URL creation or enumerate existing URLs.
Random String Generation
In this approach, the system generates a random Base62 string and checks whether it already exists in the database.If the generated short code is already in use, a new random string is generated and the process is repeated until a unique value is found.
For SQL databases, a
UNIQUE constraint on the short code guarantees that duplicate values cannot be inserted, even when multiple requests are processed concurrently.
For NoSQL databases such as Cassandra, uniqueness is typically achieved by using the short code as the primary (partition) key, along with conditional writes when necessary to prevent duplicate inserts.
INSERT INTO urls (
short_code,
long_url,
created_at,
expires_at
)
VALUES (
'aB12Cd',
'https://www.example.com/articles/system-design',
toTimestamp(now()),
null
)
IF NOT EXISTS;
However, collisions are possible because random generation can occasionally produce a short code that already exists.
In such cases, the database rejects the duplicate insertion, and the application generates another random short code and retries until a unique value is found.
Distributed ID Generator
In distributed deployments, relying on a single database for ID generation can become a bottleneck.Instead, use a distributed ID generator such as Snowflake, where each application instance generates globally unique IDs independently.
These IDs are then encoded using Base62 to produce compact short URLs without requiring centralized coordination.
A typical Snowflake ID consists of:
+-----------+-----------+--------------+
| Timestamp | MachineId | Sequence No. |
+-----------+-----------+--------------+
public class SnowflakeGenerator {
private static final long EPOCH = 1704067200000L; // Custom epoch
private final long machineId;
private long lastTimestamp = -1;
private long sequence = 0;
public SnowflakeGenerator(long machineId) {
this.machineId = machineId;
}
public synchronized long nextId() {
long timestamp = System.currentTimeMillis();
if (timestamp == lastTimestamp) {
sequence++;
} else {
sequence = 0;
lastTimestamp = timestamp;
}
return ((timestamp - EPOCH) << 22)
| (machineId << 12)
| sequence;
}
}
Snowflake ID → Base62 Encoding → Short URL.
987654321012345678 → 8M0kXyP → /8M0kXyP
Request Flow
Create URL
The client submits a long URL to the URL Service.The service generates a unique ID, encodes it using Base62 to create a short code, stores the mapping between the short code and the original URL in the database, and returns the shortened URL to the client.
SELECT short_code
FROM urls
WHERE long_url = 'https://example.com/article';
Redirect
When a client accesses a short URL, the URL Service first checks Redis. If the mapping is found, the original URL is returned immediately.Otherwise, the service retrieves it from the database, stores it in Redis for future requests, and returns an HTTP redirect to the client.
Since redirect requests greatly outnumber URL creation requests, an effective caching layer can offload the vast majority of database reads.
Supporting Custom Aliases
Instead of generating a short code automatically, users can optionally provide a custom alias such as https://tiny.ly/summer-sale or https://tiny.ly/openai.Before creating the short URL, the service validates that the alias follows the allowed format and is not already in use.
A UNIQUE constraint (or the short code as the primary key in NoSQL databases) ensures that duplicate aliases cannot be created.
INSERT INTO urls (short_code, long_url)
VALUES ('summer-sale', 'https://example.com')
IF NOT EXISTS;
Reserved aliases such as admin, login, api, and help should be blocked to prevent conflicts with application routes. Alias length and allowed characters should also be validated to ensure URL safety.
Long URL
https://www.example.com/summer-sale
Custom Alias
summer-sale
Short URL
https://tiny.ly/summer-sale
Supporting URL Expiration
Some shortened URLs are temporary and should become inaccessible after a specified expiration time. The service stores an expires_at timestamp along with each URL.Whenever a redirect request is received, the URL Service checks whether the URL has expired.
If the current time is greater than expires_at, the request is rejected with an appropriate response such as 404 Not Found or 410 Gone.
Expired URLs should be removed periodically using scheduled cleanup jobs.
In Redis, the cache entry can be assigned a TTL (Time To Live) based on the URL's expiration time, ensuring it is automatically evicted when the URL expires.
Collecting Click Analytics
Every redirect request can generate an analytics event containing information such as the short URL, timestamp, IP address, country, device type, and browser.Instead of updating analytics synchronously, the URL Service publishes each click event to a message broker such as Kafka.
Background consumers process these events asynchronously and update reporting databases or dashboards without increasing redirect latency.
{
"shortCode": "aB12Cd",
"timestamp": "2026-08-02T10:15:00Z",
"country": "India",
"device": "Mobile",
"browser": "Chrome"
}
High Availability
The URL shortener should remain available even if individual servers or infrastructure components fail. Multiple URL Service instances running behind a Load Balancer eliminate a single point of failure.Critical components such as Redis and the database should be deployed with replication and automatic failover.
If the primary node fails, a replica can take over, allowing the system to continue serving requests with minimal downtime.
For globally distributed deployments, services can be deployed across multiple regions.
Traffic is routed to the nearest healthy region, improving both availability and response time while providing disaster recovery in case an entire region becomes unavailable.
Scalability
The application layer scales horizontally by adding more URL Service instances behind the Load Balancer. Since the service is stateless, any instance can process any request.Frequently accessed URLs are served from Redis, significantly reducing database traffic and improving redirect latency.
For relational databases, read replicas can handle redirect requests while the primary database processes writes, improving read throughput.
In Cassandra, data is automatically replicated across multiple nodes based on the configured replication factor, allowing any replica to serve read requests without a dedicated primary-replica architecture.
As the number of URLs grows into the billions, the database can be sharded using the short code as the shard key.
Each shard stores a subset of the URL mappings, enabling storage capacity and throughput to scale horizontally across multiple database servers.
Handling Hot URLs & Cache Stampede
Some short URLs may suddenly become extremely popular due to social media, news articles, or marketing campaigns, creating traffic hotspots.Since the same URL is requested repeatedly, serving it from Redis avoids overwhelming the database.
Another challenge is the cache stampede. If many cached URLs expire at the same time, thousands of concurrent requests may simultaneously miss the cache and query the database, causing a sudden spike in database load.
A common solution is to use TTL jitter, where a small random offset is added to each cache entry's expiration time so that keys expire gradually instead of all at once.
For example, instead of assigning every entry a fixed TTL of 3600 seconds, the application can use:
ttl = 3600 + random(0, 300); // 60-65 minutes
redis.set(shortCode, longUrl, ttl);
Additionally, cache warming can proactively refresh frequently accessed URLs before they expire, while request coalescing (or distributed locking) ensures that only one request reloads an expired entry from the database and the remaining requests wait for the cache to be refreshed.
For cache warming, a background job periodically refreshes the most frequently accessed URLs before they expire, ensuring they remain in the cache.
Top 1000 URLs → Background Scheduler → Refresh Redis TTL / Reload Data
Request Coalescing (Distributed Locking)
Consider a popular short URL whose Redis cache entry has just expired. Hundreds of client requests may arrive simultaneously.Since all of them observe a cache miss, they would normally issue database queries at the same time, creating a cache stampede.
To prevent this, every request first attempts to acquire a distributed lock in Redis. The first request successfully executes:
SET lock:aB12Cd "locked" NX EX 5
OK. This request becomes responsible for loading the URL from the database.
SELECT long_url
FROM urls
WHERE short_code = 'aB12Cd';
SET aB12Cd "https://www.example.com/articles/system-design" EX 3600
DEL lock:aB12Cd
EX 5), preventing the lock from remaining indefinitely.
Meanwhile, all other concurrent requests execute the same command:
SET lock:aB12Cd "locked" NX EX 5
nil. These requests do not query the database. Instead, they wait briefly (for example, 50–100 ms) and retry the Redis lookup.
The corresponding Java implementation is shown below.
String url = redis.get(shortCode);
if (url != null) {
return url;
}
if (acquireLock(shortCode)) {
try {
url = database.get(shortCode);
redis.set(shortCode, url, ttl);
return url;
} finally {
releaseLock(shortCode);
}
} else {
Thread.sleep(50); // or exponential backoff
return redis.get(shortCode);
}
Rate Limiting
A URL shortener is vulnerable to abuse through automated bots, brute-force enumeration of short URLs, and denial-of-service attacks.To protect the system, requests can be rate limited based on the client's IP address, API key, or user account.
A common approach is the Token Bucket algorithm, where each client is allocated a fixed number of tokens that are replenished over time.
Every request consumes one token. Once all tokens are exhausted, additional requests are rejected with HTTP 429 Too Many Requests until more tokens become available.
A distributed rate limiter is typically implemented using Redis, where atomic operations ensure consistent request counts across multiple application instances.
Bloom Filter
Many requests may reference short URLs that do not exist. Without optimization, every invalid request eventually results in a database lookup, unnecessarily increasing database load.A Bloom Filter is a probabilistic data structure that quickly determines whether a short code might exist or definitely does not exist.
On a cache miss, the URL Service checks the Bloom Filter before querying the database.
String url = redis.get(shortCode);
if (url != null) {
return url;
}
if (!bloomFilter.mightContain(shortCode)) {
return Response.status(404).build();
}
url = database.get(shortCode);
However, false positives are possible, meaning the filter may indicate that a URL exists when it does not. In such cases, the URL Service simply performs the normal database lookup before returning 404 Not Found.
Even with occasional false positives, Bloom Filters significantly reduce unnecessary database queries caused by invalid requests.
In production, the Bloom Filter is typically stored in a shared service such as RedisBloom or rebuilt from the database during application startup.
This ensures that application restarts do not lose the previously inserted short codes.
Conclusion
A URL Shortener may appear simple, but designing one at internet scale requires careful consideration of unique ID generation, low-latency redirects, efficient caching, database partitioning, high availability, and fault tolerance.By combining techniques such as Base62 encoding, Redis caching, horizontal scaling, database sharding, and asynchronous analytics, the system can efficiently serve billions of URLs while maintaining fast response times and high reliability. :contentReference[oaicite:0]{index=0}
These techniques enable the system to support billions of URLs while delivering low-latency redirects, high availability, and horizontal scalability.
