- The DiskStorage uses only one file instead of millions.
Also the block size is only 512 byte instead of 4kb, which
helps to reduce the memory usage for short sequences.
- Update primitiveCollections to get the new LongList.range
and LongList.rangeClosed methods.
- BSFile now stores Time&Value sequences and knows how to
encode the time values with delta encoding.
- Doc had to do some magic tricks to save memory. The path
was initialized lazy and stored as byte array. This is no
longer necessary. The patch was replaced by the
rootBlockNumber of the BSFile.
- Had to temporarily disable the 'in' queries.
- The stored values are now processed as stream of LongLists
instead of Entry. The overhead for creating Entries is
gone, so is the memory overhead, because Entry was an
object and had a reference to the tags, which is
unnecessary.
It can store multiple streams of integers in a single
file. It uses blocks of 512 byte, which is only 1/8th
of the block size the file based data-store uses. This
reduces the overhead and waste of memory for short
integer streams significantly. Storing data in one big
file, instead of many small files, makes backups much
more efficient.
The new datetimepicker can be used to specify date ranges. We no longer
need to define a start date and a range. This simplifies the code
for zooming and shifting considerably.
ParallelRequestsAggregator generates a line plot that shows the number
of parallel requests among the plotted events.
This plot has two issues:
1. It only considers events that are plotted. Events that occur later,
but were started within the plotted time frame are not considered.
2. For performance reasons we are only plotting points when a value
changed. This leads to diagonal lines.
Unfortunately the datetime picker does not support seconds. But it is
one of the few that support date and time and are flexible enough to
be used with VueJS.
1. yRange is set to 0-120 minutes. This makes the gallery plots easier
to compare.
2. Set groupBy to pod/method/build. That is what I use the most and I
think this will help in most cases.
Ingestion speed dropped drastically with the old implementation.
In some situations to 7 entries per second over a 10 second period
(sic!). When using the already opened RandomAccessFile the speed
is back to previous values of 40k-50k entries per second on my 10 year
old machine on an encrypted spinning disk.
The old implementation opened a new buffered reader everytime
getPathByOffset was called. This took 1/20th of a second or
longer. For queries that visited thousands of files this could
take a long time.
We are now using a RandomAccessFile, that is opened once. The
average time spend in getPathByOffset is now down to 0.11ms.
The path in Doc is not optional. This reduces memory consumption,
because we only have to store a long (the offset in the listing file).
This assumes, that only a small percentage of Docs is requested.
- split the 'sortby' select field into two fields
- sort by average
- legend shows plotted and total values in the date range
- removed InlineDataSeries, because it was not used anymore
In logarithmic plots it is not easy to spot which event are longer than
60 seconds. A thin grey line help with this.
Also: fixed the marker lines (zoom area) which broke when y-ranges were
introduced. The lines were not drawn correctly, when the y-axis offset
was greater than 0.
Gnuplot supports differen coordinate systems. 'graph' is relative to the
area within the axes, 0,0 is bottom left and 1,1 is top right. And
'first' is based on the values of the x1 axis.
By using "graph 0.25,0" we say that the starting point is 25% of the
x-axis and 0% of the y-axis without having to compute the exact values.
Sometimes it is useful to specify the certain y-axis range. For example
when you are only interested in the values that take longer than a
threshold. Or when you want to exclude some outliers. When you want to
compare plots in a gallery, it is very handy when all plots have the
same data-area.
Eventually we want to only support what is now called aggregate, but
not have to implement different plot types. So instead of supporting
percentile plots for dashboards I removed them. You can still get
percentile plots together with the scatter plot.
Scaling big plots to small thumbnails results in bad images that barely
show any details.
We solve this by calling gnuplot a second time to generate the
thumbnails. They don't have any labels and are rendered in the required
size, so that not scaling is necessary.
Thumbnails have to be requested explicitly, because it can be expensive
to compute them.
