simplecache retains written files indefinitely

[tharwan]

Hi, we ran into a problem where processes would ran out of disk space because SimpleCacheFileSystem keeps the LocalTempFiles forever.

It is also not quite obvious how to clear the cache because clear_cache() has the potential to remove currently open files and also might need a call to _mkcache() afterwards.

There is this comment in LocalTempFile that says

# we do not delete the local copy, it's still in the cache.

but it does not really provide a solution.

Happy to provide a PR if there is a clear direction how the project would like to solve this (if it wants to).

[martindurant]

The idea of SimpleCache was not to have to do any checks at all - unlike the others that maintain some kind of metadata. The simplest answer I can think of it to extend clear_cache() to take a timestamp or relative time ("7 days ago") and remove all files in the cache that were created before then.

[tharwan]

Would you be open to have the LocalTempFile to be a pluggable class in SimpleCacheFileSystem so that it would be easier to inject a class that cleans up after uploading?