andi/perfdb
1
0
Fork 0
Code Issues 1 Pull Requests Packages Projects Releases Activity
283 Commits 2 Branches 0 Tags
a2e63cca447a8d29029e62f9faf4e73d39b49203
Commit Graph

145 Commits

Author SHA1 Message Date
andi 44f30aafee add a new facade in front of DataStore 
This is done in preparation for the proposal API.
In order to compute proposals we need to consume the
API of the DataStore, but the code does not need to
be in the DataStore. 
Extracting the API allows us to separate these concerns.
 2017-04-16 10:11:46 +02:00
andi ac1ee20046 replace ludb with data-store 
LuDB has a few disadvantages. 
  1. Most notably disk space. H2 wastes a lot of valuable disk space.
     For my test data set with 44 million entries it is 14 MB 
     (sometimes a lot more; depends on H2 internal cleanup). With 
     data-store it is 15 KB.
     Overall I could reduce the disk space from 231 MB to 200 MB (13.4 %
     in this example). That is an average of 4.6 bytes per entry.
  2. Speed:
     a) Liquibase is slow. The first time it takes approx. three seconds
     b) Query and insertion. with data-store we can insert entries 
        up to 1.6 times faster.

Data-store uses a few tricks to save disk space:
  1. We encode the tags into the file names.
  2. To keep them short we translate the key/value of the tag into 
     shorter numbers. For example "foo" -> 12 and "bar" to 47. So the
     tag "foo"/"bar" would be 12/47. 
     We then translate this number into a numeral system of base 62
     (a-zA-Z0-9), so it can be used for file names and it is shorter.
     That way we only have to store the mapping of string to int.
  3. We do that in a simple tab separated file.
 2017-04-16 09:07:28 +02:00
andi 1163c1ca22 ignore 'test-output' directory 
This is a TestNG output directory.
 2017-04-13 20:12:55 +02:00
andi 58f8606cd3 use special logger for insertion metrics 
This allows us to enable/disable metric logging without having to log 
other stuff.
 2017-04-13 20:12:00 +02:00
andi ce44c3d8d6 add logarithmic scaling for the y-axis 
Often we have a few very high values and a lot
low values. With a linearly scaled y-axis the
plot is mostly useless.
 2017-04-12 19:59:35 +02:00
andi 8baf05962f group by multiple fields 
Before we could only group by a single field. But it is acutally
very useful to group by multiple fields. For example to see the
graph for a small set of methods grouped by host and project.
 2017-04-12 19:16:19 +02:00
andi 6cc6e679a4 sort fields in 'group by' 2017-04-11 18:39:12 +02:00
andi ee15594070 remove TODOs 
They don't make sense anymore.
E.g. the Tags class is used by classes outside of
org.lucares.performance.db.
 2017-04-11 18:09:29 +02:00
andi cc7e461ebf replace PdbRepository with @Bean annotated method 2017-04-11 18:06:33 +02:00
andi 24259d7d72 replace log4j2.properties with log4j2.xml 
The properties support for log4j2 is not well documented and
possibly buggy. I couldn't figure out how to change the log
level of a logger.
 2017-04-10 19:50:49 +02:00
andi 72436e9c8c extract utility method 
a method to send json over tcp can be used by several tests
 2017-04-08 08:21:50 +02:00
andi 81b7cfa7bd add default size so that we don't have to specify it in tests 2017-04-08 08:20:59 +02:00
andi ed17d84da4 remove obsolete and disabled test 2017-04-08 08:18:17 +02:00
andi 8ddae957f5 make location of log4j config file configurable 2017-04-02 11:12:22 +02:00
andi 4ec91a4d17 update console log layout pattern 
Use pattern similar to spring boot's default.
 2017-04-02 11:11:49 +02:00
andi 46ce445010 return information about the plotted sequences 
this information can be used for tests
 2017-04-01 18:27:43 +02:00
andi 2875237272 replace the dateTo with an interval 
I hope an interval is easier to handle than having 
to change two date fields
 2017-03-27 20:07:32 +02:00
andi 726258020f make it possible to specify the time in the range 2017-03-27 19:38:02 +02:00
andi 364997e611 show better error message, when no data points are found 2017-03-26 17:30:50 +02:00
andi 7cf638400c ignore application properties for the 'dev' profile 2017-03-20 18:55:59 +01:00
andi ea905c2315 replace sysout with logger 2017-03-19 09:20:00 +01:00
andi 9ab5d76d93 better exception logging 2017-03-19 09:08:41 +01:00
andi aadc9cbd21 move TcpIngestor to pdb-ui 
and start it in the web application.
Also use the spring way of handling property files.
 2017-03-19 08:00:18 +01:00
andi 513c256352 update third party libraries 2017-03-17 16:23:21 +01:00
andi ade8463e46 update to jquery 3.2.0 2017-03-17 16:12:18 +01:00
andi 182a907466 replace flex boxes with new grids 2017-03-17 13:13:12 +01:00
andi 3456177291 add date range filter 2017-03-17 11:17:57 +01:00
andi 8cc42916a4 scroll in autocomplete proposals 2017-02-12 19:43:35 +01:00
andi e0a2c90cfe make active row in autocomplete proposals better visible 2017-02-12 18:23:52 +01:00
andi 141e090129 ignore user properties 2017-02-12 18:20:50 +01:00
andi a0f67c383c remove user config 2017-02-12 18:20:38 +01:00
andi d137f7c9cb limit the number of plots 2017-02-12 18:17:36 +01:00
andi e47519e6d9 change button 'search' to 'plot' 
add label to 'groupBy' option
 2017-02-12 12:34:18 +01:00
andi 562dadb692 group plots by field 2017-02-12 09:59:14 +01:00
andi b238849d65 use text input for filtering, again 2017-02-12 09:32:46 +01:00
andi 0c9195011a use log4j in pdb-ui 2017-02-05 11:20:00 +01:00
andi 175a866c90 update third-party libraries 2017-02-05 08:54:49 +01:00
andi 9c86f8136b you can now make runnable distributions via './gradlew distZip' 2017-02-05 08:29:59 +01:00
andi 4f77515bbd test for keywords db performance 2017-01-07 09:10:42 +01:00
andi c283568757 group plots by a single field 2016-12-30 18:45:01 +01:00
andi 61de6695f9 use better contrast colors for autocomplete 2016-12-23 10:44:43 +01:00
andi 24ee0fa60a scroll selected element into view 2016-12-23 10:42:39 +01:00
andi 95e34831d3 simple auto-completion for the search box 2016-12-23 10:32:51 +01:00
andi bc5d1b0b7b show rendered image 
added loading icon
image is scaled to available space
 2016-12-21 20:06:11 +01:00
andi d1e39513f3 create web application 2016-12-21 17:48:36 +01:00