andi/perfdb
1
0
Fork 0
Code Issues 1 Pull Requests Packages Projects Releases Activity
86 Commits 2 Branches 0 Tags
8baf05962f1c06d30ba516b975a7bc1a47108126
Commit Graph
86 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
andi 8baf05962f group by multiple fields 
Before we could only group by a single field. But it is acutally
very useful to group by multiple fields. For example to see the
graph for a small set of methods grouped by host and project.
 2017-04-12 19:16:19 +02:00
andi 6cc6e679a4 sort fields in 'group by' 2017-04-11 18:39:12 +02:00
andi ee15594070 remove TODOs 
They don't make sense anymore.
E.g. the Tags class is used by classes outside of
org.lucares.performance.db.
 2017-04-11 18:09:29 +02:00
andi cc7e461ebf replace PdbRepository with @Bean annotated method 2017-04-11 18:06:33 +02:00
andi b8b4a6d760 remove deprecated constructor and getter 2017-04-10 20:15:22 +02:00
andi ac8ad8d30f close open files when no new entries are received 
If for 10 seconds no new entry is received, then all open 
files are flushed and closed.
We do this to make sure, that we do not loose data, when
we kill the process.
There is still a risk of data loss if we kill the process
while entries are received.
 2017-04-10 20:13:10 +02:00
andi 24259d7d72 replace log4j2.properties with log4j2.xml 
The properties support for log4j2 is not well documented and
possibly buggy. I couldn't figure out how to change the log
level of a logger.
 2017-04-10 19:50:49 +02:00
andi 72436e9c8c extract utility method 
a method to send json over tcp can be used by several tests
 2017-04-08 08:21:50 +02:00
andi 81b7cfa7bd add default size so that we don't have to specify it in tests 2017-04-08 08:20:59 +02:00
andi d72d6df0f4 update third-party libraries 2017-04-08 08:18:39 +02:00
andi ed17d84da4 remove obsolete and disabled test 2017-04-08 08:18:17 +02:00
andi 7b92a306f3 remove warning by using the logger 2017-04-02 11:15:41 +02:00
andi 2d78a70883 duration for inserts was wrong 
The bug was, that we computed the difference between millis and nanos.
Also log duration for flushes.
 2017-04-02 11:15:24 +02:00
andi cd6b71d35a use shorter folder names 
reduces the risk of too long file names
 2017-04-02 11:13:08 +02:00
andi 8ddae957f5 make location of log4j config file configurable 2017-04-02 11:12:22 +02:00
andi 4ec91a4d17 update console log layout pattern 
Use pattern similar to spring boot's default.
 2017-04-02 11:11:49 +02:00
andi 5fb285a4b4 gnuplot needs forward slashes even on windows 2017-04-01 19:15:28 +02:00
andi 46ce445010 return information about the plotted sequences 
this information can be used for tests
 2017-04-01 18:27:43 +02:00
andi 2875237272 replace the dateTo with an interval 
I hope an interval is easier to handle than having 
to change two date fields
 2017-03-27 20:07:32 +02:00
andi 726258020f make it possible to specify the time in the range 2017-03-27 19:38:02 +02:00
andi 364997e611 show better error message, when no data points are found 2017-03-26 17:30:50 +02:00
andi ee00ecb4b5 remove obsolete class 2017-03-20 19:02:01 +01:00
andi 7cf638400c ignore application properties for the 'dev' profile 2017-03-20 18:55:59 +01:00
andi c245a1f7d4 fix typo that broke the gnuplot integration 2017-03-20 18:52:48 +01:00
andi cc71a32d27 make location of gnuplot configurable 2017-03-19 20:04:07 +01:00
andi ea905c2315 replace sysout with logger 2017-03-19 09:20:00 +01:00
andi 9ab5d76d93 better exception logging 2017-03-19 09:08:41 +01:00
andi aadc9cbd21 move TcpIngestor to pdb-ui 
and start it in the web application.
Also use the spring way of handling property files.
 2017-03-19 08:00:18 +01:00
andi 16f9c92d13 add special logger for throughput metrics 
so that we can enable/disable it individually
 2017-03-18 10:20:14 +01:00
andi a01c8b3907 fix flaky test and improve error handling 
just ignore invalid entries
 2017-03-18 10:14:41 +01:00
andi a221259417 remove project pdb-keyword-db 
it was only created to test how fast a keyword db implementation would
be that works with integer arrays
 2017-03-17 16:26:00 +01:00
andi 513c256352 update third party libraries 2017-03-17 16:23:21 +01:00
andi ade8463e46 update to jquery 3.2.0 2017-03-17 16:12:18 +01:00
andi 182a907466 replace flex boxes with new grids 2017-03-17 13:13:12 +01:00
andi 3456177291 add date range filter 2017-03-17 11:17:57 +01:00
andi 8cc42916a4 scroll in autocomplete proposals 2017-02-12 19:43:35 +01:00
andi f178603bc1 add time to x-axis labels only if range is less than two weeks 2017-02-12 19:08:42 +01:00
andi 5aee6f5e4d use label '<none>' to for values that have not value for groupBy field 2017-02-12 18:56:37 +01:00
andi e0a2c90cfe make active row in autocomplete proposals better visible 2017-02-12 18:23:52 +01:00
andi 141e090129 ignore user properties 2017-02-12 18:20:50 +01:00
andi a0f67c383c remove user config 2017-02-12 18:20:38 +01:00
andi d137f7c9cb limit the number of plots 2017-02-12 18:17:36 +01:00
andi e47519e6d9 change button 'search' to 'plot' 
add label to 'groupBy' option
 2017-02-12 12:34:18 +01:00
andi becd04ed13 set terminal mode for gnuplot to noenhanced 
otherwise labels are parsed as LaTeX
 2017-02-12 11:15:37 +01:00
andi 562dadb692 group plots by field 2017-02-12 09:59:14 +01:00
andi b238849d65 use text input for filtering, again 2017-02-12 09:32:46 +01:00
andi c2e42ea5fb test didn't stop correctly 
because I used offer() on a blocking queue of length 1 (chances are
high, that the element won't be added).
 2017-02-05 11:21:09 +01:00
andi 0c9195011a use log4j in pdb-ui 2017-02-05 11:20:00 +01:00
andi 3722ba02b1 add slf4j via log4j 2 logging 2017-02-05 09:53:25 +01:00
andi 175a866c90 update third-party libraries 2017-02-05 08:54:49 +01:00
andi 30b92459a7 update gradle to 3.3 2017-02-05 08:32:15 +01:00
andi 9c86f8136b you can now make runnable distributions via './gradlew distZip' 2017-02-05 08:29:59 +01:00
andi ba2076cbb5 check performance with primitive ints instead of strings as doc id 
Queries can be done in less than a millisecond even for hundreds of
thousands of documents.
 2017-02-04 10:11:09 +01:00
andi 4f77515bbd test for keywords db performance 2017-01-07 09:10:42 +01:00
andi c283568757 group plots by a single field 2016-12-30 18:45:01 +01:00
andi 62437f384f minor unimportant changes 2016-12-30 13:16:30 +01:00
andi 58bb64c80a save 12ms in when checking if cached writer can be used 2016-12-29 19:33:45 +01:00
andi f520f18e13 leverage the cached pdbwriters 
this increased performance from 500 entries per second to 4000.
 2016-12-29 19:24:16 +01:00
andi de241ceb6d finalize refactoring 2016-12-29 18:27:15 +01:00
andi 68ac1dd631 reuse pdb writers 2016-12-28 08:39:20 +01:00
andi db0b3d6d24 new file format 
Store values in sequences of variable length. Instead of using 8 bytes
per entry we are now using between 2 and 20 bytes. But we are also able
to store every non-negative long value.
 2016-12-27 10:24:56 +01:00
andi c5f0e8514c remove debug output 2016-12-23 19:28:11 +01:00
andi 580733d267 only store the tag specific base folder in the database 
before that we added each file (one per day and tag combination) to the
db
 2016-12-23 19:12:30 +01:00
andi 6969c8ce46 all storage files for the same tags use the same storage folder 
- added an additional data folder as first level
 2016-12-23 16:35:00 +01:00
andi 85eaee940e change directory structure 
- the tags come first, then the date, 
  e.g. "mykey=myvalue_<uuid>/2016/01/01/<uuid>"
- We do this, so that we don't have to tag each file, 
  but only the root folder. This should speed up searches
 2016-12-23 15:07:08 +01:00
andi 5efab12063 test which verifies the dates in each file are monotonically increasing 2016-12-23 13:04:05 +01:00
andi 470f3c730d add UT for testing multiple files for different days 2016-12-23 12:48:26 +01:00
andi 61de6695f9 use better contrast colors for autocomplete 2016-12-23 10:44:43 +01:00
andi 24ee0fa60a scroll selected element into view 2016-12-23 10:42:39 +01:00
andi 95e34831d3 simple auto-completion for the search box 2016-12-23 10:32:51 +01:00
andi bc5d1b0b7b show rendered image 
added loading icon
image is scaled to available space
 2016-12-21 20:06:11 +01:00
andi d1e39513f3 create web application 2016-12-21 17:48:36 +01:00
andi 35054b00b8 check what starts faster json, ludb or mapdb 2016-12-17 10:54:54 +01:00
andi d4c694dea3 group results by a single field 2016-12-14 19:36:38 +01:00
andi b25060a5d2 add first most simple result object 2016-12-14 17:59:04 +01:00
andi 2fd7c39ef9 remove obsolete classes 2016-12-13 18:44:00 +01:00
andi a4d09ee4b3 remove svak and grok 2016-12-13 18:41:46 +01:00
andi fa4921fcc9 use custom csv writer for performance 2016-12-13 18:41:19 +01:00
andi 876520eb4c do not create a new ObjectMapper per entry 
also read value with MappingIterator.
This made reading 20-30 times faster. 
We can now read and index 100k-500k per second.
The varianz might be due to LuDB slowness.
 2016-12-12 18:45:02 +01:00
andi 89fbaf2d06 TcpIngestor that receives a stream of json objects and stores them 2016-12-11 18:40:44 +01:00
andi e936df6f7e render plot with a single dataseriew 2016-12-10 18:50:29 +01:00
andi 81b39c5675 small enhancements 2016-12-10 15:36:06 +01:00
andi 4376f8f783 log4j does not guarantee monotonically increasing date values 2016-12-10 15:35:29 +01:00
andi 34ee64fff1 insert entries for different tags in one stream 2016-12-10 14:10:41 +01:00
andi a409c4c5d0 use java.time for time 2016-12-10 08:16:55 +01:00
andi 256b278428 inital commit 2016-12-04 10:41:37 +01:00