Description
Hello,
I have a multiproces/thread scenario where my cached function can be called at the same time.
I end up falling into the scenario of
dogpile.cache/dogpile/cache/region.py
Lines 954 to 958 in de85b10
So what I have is an expired value in the cache.
Thread 1 retrieves it, notices it's expired, starts creating a new value, and will return the new value.
Meanwhile; Thread 2 retrieves it, notices it's expired, notices thread 1 is busy creating and returns the old value.
But the old value is entirely useless for my case. It's expired for a reason. Is there a way to configure the cache region to never return expired values and just block/wait for creation instead. Just like a cache miss?
(simplified) Code to reproduce my issue:
import os
import time
from multiprocessing import Process
from dogpile.cache import make_region
cache_region = make_region()
cache_region.configure(
"dogpile.cache.dbm",
expiration_time=1,
arguments={
"filename": "/tmp/cache.dbm",
},
)
@cache_region.cache_on_arguments()
def get_value():
print(os.getpid(), "Making new value...")
time.sleep(2)
return f"Got value {time.time():.0f}"
def run():
print(os.getpid(), get_value())
print("3 sync calls in a row (value should be made once):")
run()
run()
run()
processes = []
for i in range(2):
process = Process(target=run)
processes.append(process)
time.sleep(2)
print()
print("2 async calls after cache expired (value should be made once) and be different from the previous:")
for process in processes:
process.start()
for process in processes:
process.join()Example output:
3 sync calls in a row (value should be made once):
103954 Making new value...
103954 Got value 1740579012
103954 Got value 1740579012
103954 Got value 1740579012
2 async calls after cache expired (value should be made once) and be different from the previous:
103978 Got value 1740579012
103977 Making new value...
103977 Got value 1740579016
In the above output, the line 103978 Got value 1740579012 is bad for me. That should never return an expired value.