是否为python建立了已建立的备忘录磁盘装饰器? [英] Is there an established memoize on-disk decorator for python?

问题描述

我一直在寻找python模块，该模块提供了具有以下功能的便笺装饰器:

• 将缓存存储在磁盘上，以便在以后的程序运行中重复使用.
• 适用于任何可腌制的参数，最重要的是numpy数组.
• (奖金)检查参数是否在函数调用中发生了突变.

我找到了一些用于此任务的小代码段，并且可能自己实现了一个代码段，但是我希望为该任务建立一个已确定的程序包.我还发现了 incpy ，但这似乎不适用于标准的python解释器.

理想情况下，我想使用类似 functools.lru_cache 之类的东西以及缓存存储在磁盘上.有人可以为此指出一个合适的包裹吗?

解决方案

我意识到这是一个已有2年历史的问题了，这个问题不会算作老套"的装饰器，但是……

这很简单，您真的不必担心仅使用已建立的代码.模块的 docs 链接到此答案中，我显示了所需的工作量.或者，您可以在GitHub上获得完整的工作版本(以自定义或按原样使用) /a>.

还有其他方法可以扩展它-在cache_info中放入一些与保存相关的统计信息(上次保存时间，自上次保存以来的命中率，…)，复制缓存并将其保存在后台线程中，而不是保存在后台线程中内联保存等等.但是我想不出任何值得做的事情，这并不容易.

I have been searching a bit for a python module that offers a memoize decorator with the following capabilities:

• Stores cache on disk to be reused among subsequent program runs.
• Works for any pickle-able arguments, most importantly numpy arrays.
• (Bonus) checks whether arguments are mutated in function calls.

I found a few small code snippets for this task and could probably implement one myself, but I would prefer having an established package for this task. I also found incpy, but that does not seem to work with the standard python interpreter.

Ideally, I would like to have something like functools.lru_cache plus cache storage on disk. Can someone point me to a suitable package for this?

解决方案

I realize this is a 2-year-old question, and that this wouldn't count as an "established" decorator, but…

This is simple enough that you really don't need to worry about only using established code. The module's docs link to the source because, in addition to being useful in its own right, it works as sample code.

So, what do you need to add? Add a filename parameter. At run time, pickle.load the filename into the cache, using {} if it fails. Add a cache_save function that just pickle.saves the cache to the file under the lock. Attach that function to wrapper the same as the existing ones (cache_info, etc.).

If you want to save the cache automatically, instead of leaving it up to the caller, that's easy; it's just a matter of when to do so. Any option you come up with—atexit.register, adding a save_every argument so it saves every save_every misses, …—is trivial to implement. In this answer I showed how little work it takes. Or you can get a complete working version (to customize, or to use as-is) on GitHub.

There are other ways you could extend it—put some save-related statistics (last save time, hits and misses since last save, …) in the cache_info, copy the cache and save it in a background thread instead of saving it inline, etc. But I can't think of anything that would be worth doing that wouldn't be easy.

这篇关于是否为python建立了已建立的备忘录磁盘装饰器?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！