Simple Access Using Maps

  1. Introduction
  2. Creating and opening a database
  3. Using a Map or Set to Access Data
  4. Nesting Maps for Extensibility
  5. Primitive Data Types
  6. Long Objects
Previous Next

Introduction

InfinityDB is a Java embedded database with high reliability, transactionality, extremely fast multi-core concurrency, in-memory and on-disk compression, a simple top-level Map-based interface, optional direct low-level access, and many other features.

The new top-level InfinityDB API is based on nestable persistent Maps and Sets. It is easy to build complete complex applications using these. Coding is quick, safe, and easy. Debugging is easy because the internal state of the database is well- defined as nothing more than a single mutable tuple set, so it is limited in complexity, and there is no mysterious internal logic or other state. Access is also provided as before to the underlying 'engine' for even higher performance.

There is currently only one example for the new version 4.0 top-level Map access view, which is installation-root/src/com/infinitydb/map/db/MapHelloWorld.java, which has much more than just the typical 'hello' world printer, such as a simple recursive JSON printer. The complete source code for the Map access layer is in installation-root/src/com/infinitydb/map/**/*.java.. This code is simple, and it demonstrates the low-level access as well. The Map interface is actually the java.util.concurrent.ConcurrentNavigableMap, which is the most general Map available, a subinterface of both java.util.concurrent.ConcurrentMap and java.util.NavigableMap. ConcurrentMap has many important extensions. NavigableMap is a subinterface of java.util.SortedMap, providing more ordered access capabilities. NavigableMap includes higher(), lower(), ceiling(), and floor(), on both keys and Entries, as well as new inclusivity parameters on subMap(), headMap() and tailMap(), and and descending Maps. Similar improvements exist for NavigableSet.

The lower-level direct engine-level access was the only technique provided before version 4.0, so all of the documentation and examples currently refer to this level, which is the 'ItemSpace'. The ItemSpace concept is very simple, but it can be ignored if desired, and applications can stay entirely at the Map level with no mention of the ItemSpace. Just be warned that the ItemSpace comes up in all of the rest of the documentation, even including everything after the start of this manual, so a brief familiarity will be helpful.

ItemSpace-based example code launch scripts are in installation-root/examples and source is in installation-root/src/com/infinitydb/examples. Start with HelloWorld.java, Also, ItemSpaceDataStructures will be interesting as it explains some possible uses of the ItemSpace API.  

Creating and opening a database

An InfinityDB database resides in a single file. To create or open a database only one method is needed:
import com.infinitydb.InfinityDB;
    ..
    // create a db
    String fileName = "c:/temp/testdb.idb";
    InfinityDB db = InfinityDB.create(fileName, true/*overwrite if exists*/);
    ...    
    // or, if a db file already exists:
    ...
    InfinityDB db = InfinityDB.open(fileName, true/*allow updates*/);
InfinityDB is AutoClosable. If desired, the default cache size of 2.5MB can be overridden on in the create() or open() as well. The cache contains copies in memory of disk blocks from the database file that are in frequent use or which have just been created. The cache grows as needed but will not exceed the specified size. A bigger cache improves performance at the expense of memory space.

Later, the code will just refer to db, meaning a database opened or created this way.  

Using a Map or Set to Access Data

All database update and retrieval operations at the Map access level require an InfinityDBMap or InfinityDBSet. The typical code looks like:
    NavigableSet set = new InfinityDBSet(db);
    set.add("hello world!");
    System.out.println("set: " + set.first());
or
    Map map = new InfinityDBMap(db);
    map.put("key", "hello world!");
    System.out.println("map: " + map.get("key"));
 

Nesting Maps for Extensibility

Arbitrarily complex structures can be created by nesting the Maps. To nest a map called "mymap" under the root, do this:
    InfinityDBMap map = new InfinityDBMap(db);
    InfinityDBMap nestedMap = map.getMap("mymap");
    nestedMap.put("key", "hello world!");
    System.out.println("nestedMap: " + nestedMap.get("key"));
The creation of a nested Map using getMap(Object key) does not itself store anything. Instead, when an Entry is put into the nested Map, a 'tuple' (which at the lower 'engine' level is called an 'Item') is stored in the database. One is actually working only with the logical tuples, not Map objects. The nestedMap.put("key","hello world!") actually puts a tuple like ("mymap", "key", "hello world!") into the database. Each Map has a fixed prefix that is prepended to all tuples to which it relates, and it can nest on any other Map. Maps are thread-safe.

Then retrieval operations like iterating the keySet() can be used:

    for (String s : nestedMap.keySet())
        System.out.println(s);
And to print all of the submaps:
    for (String s : map.keySet())
        System.out.println(s);

Standard Java Object serialization is not used at all. (This should be a big relief!) Instead, the database is only a set of these logical tuples, which are serialized sequences of primitives. The tuples have a maximum size based on the serialized representations of about 1660 chars. When a tuple longer than that size is put(), or added(), a CursorLengthException is thrown, and nothing changes. Therefore, one does not nest Maps indiscriminately. The maximum nesting level is almost never a limitation in practice, because if needed, there are other ways to break large data structures down into sets of tuples.  

Primitive Data Types

Data types for short data elements including most Java primitives and some other relatively small elements are provided.

The serialized forms of the data types are not necessary to take into account except for their lengths if desired. They are stored in prefix-compressed 'tuples' ('Items' elsewhere) with variable length in memory, and then Zip-compressed and UTF-8 compressed on disk. As Items in memory, they are always kept in a char[] array, serially, each with an individual type indicator followed by the element data, which is self- delimiting. Longs are optimized. A byte[] is not used for speed in memory.

.
Java typeSerialized sizestorage description
boolean or Boolean1 char 'true' or 'false'
long or Long1 char for numbers from -9 to 91, otherwise 5 chars A positive or negative 64 bit integer. The other integral types like byte, short, and int or their primitive wrappers are stored as longs. This preserves database extensibility and future-proofing. The encoding compresses small numbers.
float or Float3 chars stored in byte-packed chars, with a special re-encoding to provide proper comparison as raw chars.
double or Double5 chars stored in byte-packed chars, with a special re-encoding to provide proper comparison as raw chars
Date5 chars stored as fixed-length packed bytes. Can be used with long Map.getTime(Object K, long defaultTime) and Map.putTime(Object K, long time)
String2 chars plus String contents. One type char plus a zero terminator. Internal zeros or ones are quoted into two chars. This must be kept short enough to allow the tuple to fit in 1660 chars. Hence use as a key or short field is common. There is no char data type - they should either be stored as Strings or longs depending on the desired semantics. (It is possible to use chars at the lower 'engine level' for special raw access purposes).
char[]1 type char, 1 length char, then raw chars from the array, with no zero quoting This is logically different from Strings. Sorted by length. This must be kept short enough to allow the tuple to fit in 1660 chars. These are used by CharacterLongObjects, which are created by getWriter() and getReader(). CLOB arrays are never 1026 chars total.
byte[]1 type char, 1 length char, then bytes, one byte per char, with no zero quoting Sorted by its length. This must be kept short enough to allow the tuple to fit in 1660 chars. These are used by getOutputStream() and getInputStream(), in which case they are never longer than 1026 chars total.
ByteString1 type char plus packed bytes in chars, somewhat more than two bytes per char. Sorted by initial bytes like a String, with shorter ones earlier. Any bytes are allowed. Every byte string of a particular length takes exactly the same storage because there is no quoting. The packing algorithm is efficient but complex. Slower than byte[].
EntityClass1 type char, then chars encoded as for Long aboveA special 'EntityClass' class used for 'EAV' structures as described elsewhere.It has an internal identifier that is structured like a long.
Attribute1 type char, then chars encoded as for Long above A special 'Attribute' class used for 'EAV' structures as described elsewhere. It has an internal identifier that is structured like a long.
Index1 type char, then chars encoded as for Long aboveA special 'Index' class used for 'huge arrays' as described elsewhere. It has an internal long that represents the position in the huge array. Also used to make a huge array of char[] or byte[] elements of length up to 1024 representing a CLOB or BLOB.
 

Long Objects

A CLOB or BLOB ("Character Long OBject" or "Binary Long OBject") can be stored easily without size limitations:
    Writer writer = new InfinityDBMap(db).getWriter(KEY);
    PrintWriter printer = new PrintWriter(writer);
    printer.println("hello world!");
    writer.close();
    Reader reader = new InfinityDBMap(db).getReader(KEY);
    BufferedReader br = new BufferedReader(reader);
    System.out.print("Character Long Object: " + br.readLine());

Further information in this manual is all about the database in general, but also much of it assumes understanding of the ItemSpace. The most information is about Transactions, because it is a deep subject, but InfinityDB provides a rich set of solutions. Also see boilerbay.com and boilerbay.com/infinitydb/. This manual is at manual For info about the data structures you can build at the lower engine level, is helpful to read ItemSpaceDataStructures

Previous Next


Copyright © 1997-2017 Boiler Bay.