Commit Graph

23 Commits

Author SHA1 Message Date
452030de5e use more efficient way to determine needed bytes for variable byte
encoding

I experimented with a few branch-free variants, but they were slower
than just using ternary operators and readable code.
2024-04-01 12:44:30 +02:00
bc520f97ed move DateIndexExtension to DataStore 2021-05-12 19:00:25 +02:00
10155f9cdb use special enum for DateBucket units
Preparation step for having custom intervals.
2020-09-27 17:06:27 +02:00
b8f77dc9a6 replace custom timezones with UTC
we are only using UTC
2020-09-27 08:21:24 +02:00
50f555d23c add interval splitting for bar charts 2020-04-05 08:14:09 +02:00
75391f21ff extract code from DateIndexExtension to LongToDateBucket
Making it possible to reuse the code to sort timestamps
into date based buckets.
2020-04-03 19:46:08 +02:00
07ad62ddd9 use Junit5 instead of TestNG
We want to be able to use @SpringBootTest tests that fully initialize
the Spring application. This is much easier done with Junit than TestNG.
Gradle does not support (at least not easily) to run Junit and TestNG
tests. Therefore we switch to Junit with all tests.
The original reason for using TestNG was that Junit didn't support
data providers. But that finally changed in Junit5 with
ParameterizedTest.
2019-12-13 14:33:20 +01:00
e931856041 merge projects file-utils, byte-utils and pdb-utils
It turned out that most projects needed at least
two of the utils projects. file-utils and byte-utils
had only one class. Merging them made sense.
2019-12-08 18:47:54 +01:00
06b379494f apply new code formatter and save action 2019-11-24 10:20:43 +01:00
6eaf4e10fc add maxSize parameter to HotEntryCache 2019-08-24 19:24:20 +02:00
00c20dae6b use long instead of Instant for time
Working with longs is faster and requires less
cache. The space in L123 caches is precious.
2019-08-19 18:58:24 +02:00
feda901f6d remove event types
We only have removal events. The additional complexity
of having a generic interface for many different event
types does not pay off.
2019-08-18 20:30:25 +02:00
4d9ea6d2a8 switch back to my own HotEntryCache implementation
Guava's cache does not evict elements reliably by
time. Configure a cache to have a lifetime of n
seconds, then you cannot expect that an element is
actually evicted after n seconds with Guava.
2019-08-18 20:14:14 +02:00
0dc908c79c show the removal listener of HotEntryCache is not called on expire 2019-08-18 09:27:18 +02:00
117ef4ea34 use guava's cache as implementation for the HotEntryCache
My own implementation was faster, but was not able to
implement a size limitation.
2019-02-16 10:23:52 +01:00
e537e94d39 HotEntryCache will update Instants only once per second
Calling Instant.now() several hundred thousand times per
second can be expensive. In my measurements >10% of the
time spend when loading new data was spend calling
Instant.now().
Fixed this by storing an Instant as static member and
updating it periodically in a separate thread.
2018-12-21 19:16:55 +01:00
73ad27ab96 remove lastAccessMap
In the last commit I added a lastAccessMap to the HotEntryCache.
This map made it much more efficient to evict entries. But it
also made and put and get operation much more expensive. Overall
that change lead to a 65% decrease in ingestion performance of
the PerformanceDB.
Fixed by removing the map again. Eviction has to look at all
elements again.
2018-12-21 10:28:34 +01:00
afba3b6f77 elements not evicted if new elements are added 2018-12-20 16:13:55 +01:00
9889252205 use only one thread for evictions
Instead of spawning a new thread for every cache, we use a single thread
that will evict entries from all caches.
The thread keeps a weak reference to the caches, so that they can be
garbage collected.
2018-11-24 08:32:05 +01:00
f78f69328b add cache for docId to Doc mapping
A Doc does not change once it is created, so it is easy to cache.
Speedup was from 1ms per Doc to 3ms for 444 Docs (0.00675ms/Doc).
2018-11-22 19:51:07 +01:00
5343c0d427 reduce memory usage
Reduce memory usage by storing the filename as string instead of
individual tags.
2018-03-19 19:21:57 +01:00
ahr
64db4c48a2 add plots for percentiles 2017-11-06 16:57:22 +01:00
d4fd25dc4c replace LinkedHashMap with a more memory efficient implementation
This saves approximately 50MB of heap space.
2017-09-30 17:51:02 +02:00