Surviving the Cache Battle between Next.js and Cloudflare

Picture this: You’ve just finished building a beautiful new feature for your serverless blog. You hit "Publish", check your shiny DynamoDB console, and boom—the new record is there. Validation passed, the Lambda fired correctly, all systems green.

You open your browser, refresh localhost:3000, expecting to see your new story sitting proudly at the top of your "Latest Stories" grid.

But it's not there. It still shows 5 posts instead of 6.

You refresh again. Hard refresh. Nothing. You check the API directly via curl:

curl -s https://cloud-record.dev/api/posts?limit=100 | jq '.items | length'

Output: 5.

What is going on? You literally just saw the item in the database! Are there gremlins in the network? Did my AWS Lambda decide to take the day off? I spent almost an hour questioning my sanity, restarting servers, and tearing apart my Next.js App Router hooks.

Here is the story of how the very tools designed to make our websites blazing fast Next.js and Cloudflare were silently conspiring against my API.

The Culprit: Aggressive Edge Caching

In modern web development, we are obsessed with speed. To achieve it, we often layer multiple caching strategies. In my setup, I have:

1. Next.js (App Router) trying to aggressively cache fetch() calls by default.

2. AWS CloudFront acting as an origin proxy.

3. Cloudflare sitting at the very edge, protecting my domain and caching global requests.

Because my local environment was pointing to my production API (NEXT_PUBLIC_API_URL=https://cloud-record.dev/api), I didn't realize that Cloudflare was hijacking my local dev requests.

Cloudflare saw that my local machine was requesting the exact same URL (/api/posts?limit=6) that hundreds of other requests had asked for recently. Since my API Gateway wasn't explicitly broadcasting strict Cache-Control headers, Cloudflare simply served a stale copy of the data from its Edge node in my city. It never bothered to ask AWS if there were new posts!

The "Aha!"

After pulling my hair out, I realized something: What if I just ask the API for something unique?

I ran the curl command again, but this time I appended a completely meaningless parameter to the URL:

curl -s "https://cloud-record.dev/api/posts?limit=100×tamp=123123123" | jq '.items | length'

Output: 6.

Boom. There it was. The moment I forcefully bypassed Cloudflare's Edge node by requesting a unique URL string, the request successfully reached AWS Lambda, read my DynamoDB table, and returned the actual live data.

The Server vs. Client Fix

Now I knew why it was happening, but how do I fix it elegantly in Next.js?

Since my blog relies on a Static Export (output: "export"), I need Next.js to respect the cache during the npm run build phase so it can bake the HTML. But I must bypass the cache when the user's browser (or my local npm run dev session) is fetching fresh data.

To solve this, I wrote a smart condition in my custom usePosts Hook to differentiate between the Node.js Server Environment and the Web Browser:

// app/hooks/usePosts.ts
export function usePosts(limit: number = 6) {
  // ... state logic ...

  const url = new URL(`${baseUrl}/posts`);
  url.searchParams.append('limit', limit.toString());

  // Are we running on the Server (Next.js Build) or the Browser?
  const isServer = typeof window === 'undefined';

  if (!isServer) {
    // We are on the client! Bust the Cloudflare and Browser caches 
    // by appending a fresh timestamp to the request.
    url.searchParams.append('_t', Date.now().toString());
  }

  const response = await fetch(url.toString(), {
    method: 'GET',
    headers: { 'Content-Type': 'application/json' },
    // Use 'force-cache' so the SSG build doesn't crash, 
    // but use 'no-store' in the browser for live data.
    cache: isServer ? 'force-cache' : 'no-store', 
  });
  
  // ...
}

The Ironclad Backend Defense (Optional but Recommended)

To prevent this globally and permanently, I also added explicit Cache-Control headers straight from the source. I updated my AWS Lambda utils.py file so that whenever an API response is sent out, it explicitly orders any proxy on the internet to never cache the database payload:

def build_response(status_code: int, body: Any) -> Dict[str, Any]:
    return {
        "statusCode": status_code,
        "headers": {
            "Content-Type": "application/json",
            "Cache-Control": "no-cache, no-store, must-revalidate",
            "Pragma": "no-cache"
        },
        "body": json.dumps(body, cls=DecimalEncoder)
    }

Conclusion

Caching is a double-edged sword. It's the reason our serverless architectures are incredibly fast and cost mere pennies to run, but it can also be a developer's worst nightmare when layers step on each other's toes.

Always check your Edge Network responses, and when in doubt, toss a random query parameter into your URL to see if you're talking to your database or just talking to a ghost in the CDN!