caffeine

This was a good attempt but flawed. 1. You should use JMH to handle warmup, jit, timings, averaging of runs, etc. You might enjoy the following talks on the subject, though I am sure there are other gems out there. - Performance Anxiety: https://wiki.jvmlangsummit.com/images/1/1d/PerformanceAnxiet... - Benchmarking for Good: https://www.youtube.com/watch?v=SKPdqgD1I2U - (The Art of) (Java) Benchmarking: https://shipilev.net/talks/j1-Oct2011-21682-benchmarking.pdf - A Crash Course in Modern Hardware: https://www.youtube.com/watch?v=OFgxAFdxYAQ - How NOT to Write a Microbenchmark https://www.slideshare.net/slideshow/2002-microbenchmarks/28... - Anatomy of a flawed microbenchmark https://web.archive.org/web/20110513090823/http://www.ibm.co... 2. An uncontended lock is basically free, as you noticed. 3. Guava does implement the Map interface

That's kind of the idea of Caffeine, it has admission buffers, and it adapts automatically between LRU and LFU. The original algorithm is called Windiw TinyLFU (design https://github.com/ben-manes/caffeine/wiki/Design), see it in action e.g. here: https://github.com/ben-manes/caffeine/wiki/Efficiency

You might be interested in this thread [1] where I described an idea for how to incorporate the latency penalty into the eviction decision. A developer even hacked a prototype that showed promise. The problem is that there is not enough variety in the available trace data to be confident that a design isn't overly fit to a particular workload and doesn't generalize. As more data sets become available it will become possible to experiment with ideas and fix unexpected issues until a correct, simple, elegant design emerges. [1] https://github.com/ben-manes/caffeine/discussions/1744

As the article mentions, Caffeine's approach is to monitor the workload and adapt to these phase changes. This stress test [1] demonstrates shifting back and forth between LRU and MRU request patterns, and the cache reconfiguring itself to maximize the hit rate. Unfortunately most policies are not adaptive or do it poorly. Thankfully most workloads are a relatively consistent pattern, so it is an atypical worry. The algorithm designers usually have a target scenario, like cdn or database, so they generally skip reporting the low performing workloads. That may work for a research paper, but when providing a library we cannot know what our users workloads are nor should we expect engineers to invest in selecting the optimal algorithm. Caffeine's adaptivity removes this burden and broaden its applicability, and other language ecosystems have

I tried to reimplement Linux’s algorithm in [1], but I cannot be sure about correctness. They adjust the fixed sizes at construction based on device’s total memory, so it varies if a phone or server. This fast trace simulation in the CI [2] may be informative (see DClock). Segmentation is very common, where algorithms differ by how they promote and how/if they adapt the sizes. [1] https://github.com/ben-manes/caffeine/blob/master/simulator/... [2] https://github.com/ben-manes/caffeine/actions/runs/130865965...

I went through some of the issues to see how aggressively they close them and found this gem: https://github.com/ben-manes/caffeine/issues/1824#issuecomme...

One optimization for background refresh is coalescing the individual reloads into a batch operation based on a time/space window. Here is how we do it in the Java world. [1] [1] https://github.com/ben-manes/caffeine/tree/master/examples/c...

If anyone is interested in space & time efficient caching and working set tracking check out some of my favorites, gil einziger https://scholar.google.co.il/citations?user=kWivlnsAAAAJ and the ex coho data guys https://www.usenix.net/system/files/conference/osdi14/osdi14.... Ben Manes[1] worked with Gil Einziger on an implementation of TinyLFU and actually contributed to an updated version of the TinyLFU paper, IIRC. Gil has a ton of very relevant work[2][3] around cache admission, control, and information density. Coho data was working very large storage arrays, specifically optimizing data placement across different media/hosts/etc. lots of similar problems where even the metadata of the working set was prohibitively large. So they had space & time efficient functions to track access patterns in realtime. Their business didnt work out (

That could work efficiently in a sampled caching policy. For example, Hyperbolic [1] maintains the frequency and insertion time on each entry and evicts the entry in the sample with the least utility value of frequency / (currentTime - insertionTime). For yours, you could easily add it to the SampledPolicy in this simulator [2] and run it against a few traces to see how it compares. [1] https://www.usenix.org/system/files/conference/atc17/atc17-b... [2] https://github.com/ben-manes/caffeine/wiki/Simulator

/u/someplaceguy, Those LIRS traces, along with many others, are available at this page [1]. I did a cursory review using their traces with Caffeine's and the author's simulators to avoid bias or a mistaken implementation. In their target workloads Caffeine was on par or better [2]. I have not seen anything novel in this or their previous works and find their claims to be easily disproven, so I have not implement this policy in Caffeine’s simulator yet. [1]: https://github.com/ben-manes/caffeine/wiki/Simulator [2]: https://github.com/1a1a11a/libCacheSim/discussions/20

My implementation is reviewed and linked from the official S3-FIFO page. https://s3fifo.com A benchmark is as follows. https://github.com/ben-manes/caffeine/wiki/Efficiency I can't deal with you any longer. Over.

That, and also when caffeine came out it replaced one of the major uses (caching) of guava. https://github.com/ben-manes/caffeine

not to distract from your valid points but, when used properly, Caffeine + Reactor can work together really nicely [1]. [1]: https://github.com/ben-manes/caffeine/blob/master/examples/c...

Yes, I think that is my main concern in that often research papers do not disclose the weaknesses of their approaches and the opposing tradeoffs. There is no silver bullet. The stress workload that I use is to chain corda-large [1], 5x loop [2], corda-large at a cache size of 512 entries (6M requests). This shifts from a strongly LRU-biased pattern to an MRU one, and then back again. My solution to this was to use hill climbing by sampling the hit rate to adaptively size of the admission window (aka your FIFO) to reconfigure the cache region sizes. You already have similar code in your CACHEUS implementation which built on that idea to apply it to a multi-agent policy. Caffeine adjusts the frequency comparison for admission slightly to allow ~1% of losing warm candidates to enter the main region. This is to protect against hash flooding a

I wonder why all these papers ignore comparison against W-TinyLFU. https://github.com/ben-manes/caffeine/wiki/Efficiency Shows that it really outperforms ARC as well and they also have an optimal oracle version that they evaluate against to show how much room there is left (admittedly the oracle version itself implies you’re picking some global criterion to optimize but that’s trickier when in reality there are multiple axes along which to optimize and you can’t simultaneously do well across all of them). Also the lack of evaluation of the cache strategy against shifting workloads is also problematic.