The API Was Slow. Redis Was Not the Fix

An API was slow, and my first solution was familiar:

• add Redis
• add EC2 auto scaling
• spread the load

The tools were reasonable.

The diagnosis was not.

After tracing one request, I found that the service loaded and parsed the same account rules several times inside the same request.

The system did not need shared caching or more servers yet.

It needed to stop repeating the same work.

A request-local cache removed the bottleneck without adding a network dependency, credentials, invalidation rules, deployment changes, or another component to monitor.

Trace the Request Before Changing the Architecture

The slow path looked like this:

def handle_request(items: list[Item]) -> list[Price]:
    results = []

    for item in items:
        rules = load_and_parse_rules(item.account_id)
        results.append(apply_rules(item, rules))

    return results

Several items belonged to the same account. The rules were stable during the request, but the service reloaded and reparsed them for every item.

The measured problem was:

Repeated work inside one request.

Redis solves a different problem:

Shared cached state across processes or machines.

Auto scaling solves another:

Remaining load exceeds the capacity of current instances.

Neither matched the actual bottleneck.

Fix the Smallest Boundary

The smallest sufficient fix was request-local reuse:

def handle_request(items: list[Item]) -> list[Price]:
    rules_by_account: dict[str, Rules] = {}
    results = []

    for item in items:
        rules = rules_by_account.get(item.account_id)

        if rules is None:
            rules = load_and_parse_rules(item.account_id)
            rules_by_account[item.account_id] = rules

        results.append(apply_rules(item, rules))

    return results

The change had one narrow effect:

Stable rules are loaded once per account per request.

The cache disappeared at the end of the request. That gave it useful properties:

• no stale state across requests
• no invalidation mechanism
• no shared credentials
• no network failure
• no extra deployment
• no memory that grows indefinitely
• no new on-call dependency

It solved the measured waste at the boundary where the waste existed.

Infrastructure Has an Operating Cost

Each caching boundary buys something and creates obligations:

Adding Redis early would have widened the system before proving that shared state was necessary.

Adding more instances would have multiplied the inefficient request path across more machines.

Separate Efficiency From Capacity

These problems are often confused:

• Efficiency: one request performs unnecessary work
• Capacity: the optimized workload still exceeds available compute

The correct sequence is:

measure
  -> remove redundant work
  -> measure again
  -> add shared infrastructure only if required
  -> add capacity only if optimized load exceeds capacity

Scaling before improving efficiency can hide the root cause while increasing cost.

Know When the Cache Must Move Outward

A request-local cache is not always enough.

Move to a longer-lived boundary only when measurement proves the need.

An in-process cache may be justified when:

• the same stable data is repeatedly loaded across requests
• temporary staleness is acceptable
• one process can own the cache

A shared cache may be justified when:

• many processes need the same expensive result
• recomputation cost remains material after local fixes
• the application can tolerate or survive cache failure
• invalidation and access rules are explicit

At each step, ask what new failure mode the cache introduces:

• What happens when it is wrong?
• What happens when it is empty?
• What happens when it is full?
• What happens when it is unavailable?
• How stale may the value become?
• Who can read or modify it?

The wider the cache boundary, the higher the operational burden.

The Useful Rule

Do not begin with the infrastructure associated with a class of problem.

Begin with the measured constraint inside the actual request path.

For this API, the fix was not “make it scalable.”

It was:

Stop recomputing stable data inside one request.

That smaller statement produced the smaller and more reliable solution.

Performance Checklist

Before adding infrastructure to a slow system, ask:

1. Which operation is actually slow?
2. Is the work repeated unnecessarily?
3. Does the result need to outlive one request?
4. Must cached state be shared across processes?
5. How stale may it become?
6. What invalidates it?
7. Can the application continue when the cache fails?
8. Is the remaining problem efficiency or capacity?
9. What new operational dependency will the proposed fix create?

Remove waste first.

Add infrastructure when the remaining problem genuinely requires infrastructure.