October 2, 2005
Performance with Compressible Data Better than Expected
We recently added a performance tester to the examples directory of the trial and deployment downloads: InfinityDBPerformanceTest.java. The results were pleasantly surprising. When tested against a common form of compressible data, speed and disk space efficiency for all operations improved dramatically. We have been quoting 30K inserts/sec/GHz, but with this rather typical compressible data,we are getting 120K insertions/sec/GHz, in a warm cache (with no disk I/O, i.e. all data in the cache). The cold cache performance (which requires allocating cache blocks during warmup) is only slightly slower.
The performance for pure random data is roughly what we expected, with next() being a bit slower than expected, and insert()/delete() somewhat faster.
Data compression was also dramatic: 20MB of compressible data was stored in 1.4MB. Data compression for purely random data is only 28% worse than the theoretical minimum, with the 20MB stored in 25637K bytes. A recent third-party potential client found that the database generated by the SleepyCat (Berkely DB) B-Tree took 140MB for similar purely random data.
This compressible data uses repeating 10-byte prefixes and repeating 10-byte suffixes. The former is common in many uses of the engine (see ItemSpaceDataStructures.htm). The Latter simulates attribute values with small numbers of distinct values. Also, the ZLib compressor got going with its Huffman coding because we used only 40 values per byte, simulating common English text.
Please examine the tester itself and try changing various parameters. Access the download link from our home page. Here is the actual output:
Performance test of InfinityDB B-Tree database engine.
File=c:\temp\InfinityDBPerformanceTest.idb
Entries=1000000 bytes/entry=20
RANDOM COMPRESSIBLE DATA
insert with cold cache, compressible data time=3.6259999999999994 iterations/sec=275785.9900717044
compressible db size=1420075
next with compressible data time=3.5159999999999996 iterations/sec=284414.1069397042
delete with compressible data time=3.6409999999999996 iterations/sec=274649.8214776161
insert warm cache, compressible data time=3.3129999999999997 iterations/sec=301841.2315122246
RANDOM UNCOMPRESSIBLE DATA
insert cold cache, uncompressible data time=14.219 iterations/sec=70328.43378577959
uncompressible db size=25637958
next with uncompressible data time=7.561999999999999 iterations/sec=132240.1481089659
delete with uncompressible data time=9.577 iterations/sec=104416.83199331732
insert warm cache, uncompressible data time=9.39 iterations/sec=106496.27263045793
Performance test done.
Posted 5 years, 4 months ago on October 2, 2005
The trackback url for this post is http://boilerbay.com/infinitydb/forum/bblog/trackback.php/28/
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comment pending moderation
Comments have now been turned off for this post