Why is my simple dict cache leaking data between api calls?

[MichaelKM]

I’m trying to implement a simple cache in my Python API to avoid hitting an external service too often. I went with a basic dictionary at first, but I’m running into issues where the data seems to persist between requests in a way I didn’t expect, almost like it’s becoming a shared global state. I’m not sure if my approach is fundamentally flawed or if I’m just missing something about how the application scope works.

[Hannah88]

I had a module level dict too. It lingered across requests as long as the process stayed alive, which surprised me when I restarted the server.

[LarryL]

We added a tiny TTL on the stored items and logged hits versus misses, and we saw the data only cleared when the process died. So it’s per process, not shared across all workers.

[Larry.M]

Are you sure the problem isn't that you're hitting the same endpoint with the same params every time and the result ends up being reused?

[Addison.D]

Maybe the real issue is that you run multiple worker processes and each has its own memory. A distributed store would be needed if you want true sharing.