InfinityDB Encrypted is identical to InfinityDB Embedded but it also has the vital but simple security features shown here. Using encryption can be as simple as providing a password when the database file is created, and then supplying it again when the database file is re-opened.
The password can be changed at any time easily and securely as well, by opening using the current password and invoking a single method. Changeability of passwords is vital for security, but must be baked into the original design: the passwords cannot be simply stored in the file! We use an approved standard technology called ‘key encryption keys’ to provide this. To ‘shred’ a file instantly and securely, you can change the key to a long random number that you then ‘forget’.
The many advanced features are also simple to use, to provide even more security. All of them are explained below.
// Copyright (C) 2014-2019 Roger L. Deran. All Rights Reserved.
//
// Mar 24, 2019 Roger L. Deran
//
// THIS SOFTWARE CONTAINS CONFIDENTIAL INFORMATION AND TRADE SECRETS
// OF Roger L Deran. USE, DISCLOSURE, OR REPRODUCTION IS PROHIBITED
// WITHOUT THE PRIOR EXPRESS WRITTEN PERMISSION OF Roger L Deran.
//
// Roger L Deran. MAKES NO REPRESENTATIONS OR WARRANTIES ABOUT
// THE SUITABILITY OF THE SOFTWARE, EITHER EXPRESS OR IMPLIED,
// INCLUDING BUT NOT LIMITED TO THE IMPLIED WARRANTIES OF
// MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE, OR
// NON-INFRINGEMENT. Roger L Deran. SHALL NOT BE LIABLE FOR
// ANY DAMAGES SUFFERED BY LICENSEE AS A RESULT OF USING,
// MODIFYING OR DISTRIBUTING THIS SOFTWARE OR ITS DERIVATIVES.
package com.infinitydb.examples;
import java.io.File;
import java.io.IOException;
import java.security.GeneralSecurityException;
import java.security.KeyPair;
import java.security.KeyPairGenerator;
import java.security.Principal;
import java.security.PrivateKey;
import java.security.PublicKey;
import java.security.SignatureException;
import java.security.cert.CertPathValidatorException;
import java.security.cert.TrustAnchor;
import java.security.cert.X509Certificate;
import java.util.Arrays;
import java.util.HashSet;
import java.util.Random;
import java.util.Set;
import com.infinitydb.Cu;
import com.infinitydb.InfinityDB;
import com.infinitydb.ItemSpace;
import com.infinitydb.security.IncorrectPassWordException;
import com.infinitydb.security.SignatureHashAlgorithm;
import com.infinitydb.security.SignatureInfo;
import com.infinitydb.security.SignatureInfoSet;
import com.infinitydb.security.X509CertificatePath;
/**
*
* An example of using the InfinityDB encryption feature of version 5. Just
* provide a password to create or open an encrypted file.
*
* There is virtually no performance hit. Compression is just as effective: 1 to
* 10x or more. Unencrypted version 4 files can be used without change by the
* new version and will remain unencrypted and compatible. Version 4 will throw
* an IOException when an encrypted file is attempted to be opened.
*
* Databases can also be hashed, signed or the signatures verified at high
* speed. The file contains a set of signature definitions i.e. 'SignatureInfos'
* each with an X509 certificate path or a 'bare public key', and each with a
* selectable hash algorithm. SignatureInfos can be signed in subsets sharing
* the data hash computation. Each SignatureInfo can stay in signed or unsigned
* state after close(). The signing state persists until the database content
* changes.
*
* Certificate paths in the SignatureInfos can be validated based on a set of
* trusted certificates. External storage or availability of signing
* certificates after they are put in the file is not necessary, only private
* keys for signing and trusted public keys or trusted certificates for possible
* validation. Verification can use client-implemented strategies like 'any
* signature based on this public key is enough' or 'any N signatures is
* enough', or 'any validated signatures with selected certificates based on the
* distinguished name is enough'. Signatures can be verified and the hash
* computed without the password.
*
* The implementation uses an underlying 'shim' called EncryptedRandomAccessFile
* that provides its overlying InfinityDB with a logical
* GeneralizedRandomAccessFile, while physically storing the data as encrypted
* blocks in a normal RandomAccessFile. The InfinityDB-specific
* GeneralizedRandomAccessFile is necessary instead of a subclass of
* RandomAccessFile, because the latter cannot be subclassed (this is considered
* an original mistake in Java - InputStream and OutputStream are OK though).
*
* The EncryptedRandomAccessFile also contains a 'header' before the encrypted
* blocks that describes the file state, and which contains structure for future
* extensions, signature information and eventually information for
* 'enveloping'. The header itself is variable-length but has a limited fixed
* space at the front of a particular file - if too much data is attempted to be
* written in that space, an IOException is thrown, but the file is still usable
* in its previous state. Currently the size is fixed at 100K but later it will
* be settable on create(). This should be plenty. The header can change without
* the hash being changed.
*/
public class EncryptionExample {
static final long CACHE_SIZE = 100_000_000;
// Passwords are char[] so you can zero them out to minimize time in memory.
// A String can't be zeroed.
static final char[] PASS_WORD = new char[] { 'a', 'b', 'c' };
// We change the password to this later on
static final char[] NEW_PASS_WORD = new char[] { 'd', 'e', 'f' };
/*
* 0 is for regular 128-bit AES strength with no export issues, 1 is for
* strong 256-bit AES.
*
* This is not required on open, but set by create permanently. In the far
* future there will be more if these two become obsolete.
*
* Note that a database created with strong encryption can only be opened by
* a JVM with strong encryption enabled. Some countries control the use or
* distribution of strong encryption. However, it can normally be enabled
* with simple changes to files in $JAVA_HOME/jre/lib/security.
*/
static final int ENCRYPTION_PARAMETERS_NUMBER = 0;
// To generate some example content
static final int ITEM_COUNT = 100_000;
static final Random random = new Random();
// The database path.
static final String PATH = getPath();
public static void main(String... args) {
System.out.println("Database path=" + PATH);
demoEncryptedMode();
demoHash();
demoSigning();
demoCertificateValidation();
}
/**
* The db is encrypted just because the password and encryption params are
* given. That's the only required API change for encryption and integrity
* checking.
*/
static void demoEncryptedMode() {
try (Cu cu = Cu.alloc()) {
InfinityDB db = InfinityDB.create(PATH, true, CACHE_SIZE,
PASS_WORD, ENCRYPTION_PARAMETERS_NUMBER);
random.setSeed(0);
for (int i = 0; i < ITEM_COUNT; i++) {
// create items like "hello" 391
cu.clear().append("hello").append(random.nextLong());
db.insert(cu);
}
System.out.println(
"count of Items (should be " + ITEM_COUNT + ")="
+ countItems(db));
// Won't affect Item count - just makes data durable.
db.commit();
/*
* This countItems() will read every block, and a side-effect, the
* hMac authenticity check based on the password will be able to
* detect externally-caused corruption. Or for more speed use
* getPlainTextBlockHash() for that.
*
* However, there is an elaborate 'backup attack' which in principle
* can substitute corresponding blocks between two backups to affect
* the contents and yield a usable db. To detect it, use signatures
* or keep hashes and compare them later.
*/
System.out.println(
"count of Items after commit (should be " + ITEM_COUNT
+ ")="
+ countItems(db));
db.close();
// Try to open with no password.
// The encryption parameters number is not supplied - it is
// fixed after creation.
try {
db = InfinityDB.open(PATH, true);
} catch (Exception e) {
// expected - password not provided
System.out.println("no password: " + e.getMessage());
}
// try to open with wrong password
try {
db = InfinityDB.open(PATH, true,
new char[] { 'w', 'r', 'o', 'n', 'g' });
} catch (IncorrectPassWordException e) {
// expected - password wrong
System.out.println("wrong password: " + e.getMessage());
}
// Opens correctly with correct password
db = InfinityDB.open(PATH, true, PASS_WORD);
System.out.println(
"count of Items after re-open (should be " + ITEM_COUNT
+ ")=" + countItems(db));
/*
* New feature in 5.1: passwords can be changed.
*/
db.changePassWord(NEW_PASS_WORD);
db.close();
// Opens correctly with new password
db = InfinityDB.open(PATH, true, NEW_PASS_WORD);
// Same seed so same Items as were inserted.
random.setSeed(0);
// Delete all Items randomly
for (int i = 0; i < ITEM_COUNT; i++) {
// create Items like "hello" 391
cu.clear().append("hello").append(random.nextLong());
db.delete(cu);
}
System.out.println(
"count of Items after deletion (should be 0)="
+ countItems(db));
db.commit();
System.out.println(
"count of Items after deletion and commit (should be 0)="
+ countItems(db));
} catch (Throwable e) {
e.printStackTrace();
}
}
/**
* Compute the hash of the database contents, i.e. the set of Items.
*
* There is a very fast hash of the encrypted data and a different hash of
* the unencrypted .i.e 'plain' data that as a side-effect checks the
* integrity of each block.
*
* This hash will be different for different DBs created separately, even if
* the Item set is the same. However, if a given DB is not changed, its hash
* will stay the same even after close()/open() or reads. So an
* externally-caused corruption i.e. modification or truncation of the
* encrypted data blocks will generate a different hash.
*
* The hashing feature is not available on a legacy unencrypted version 4 or
* earlier db, throwing an Exception. It is used in signing encrypted dbs.
*
* It is based on SHA-256, so the length of the hash is currently 32 bytes.
* The hash is very fast, but it must read all of the file. Some day it will
* even be multi-threaded.
*
* Future versions of InfinityDB may change the hash algorithm or
* implementation, for example if the security of SHA256 is compromised or
* performance can be improved. The hash algorithm used then will be client
* selectable or automatically adapted to the particular open file.
*
* This actually hashes the unencrypted or encrypted data blocks and logical
* file length of the EncryptedRandomAccessFile 'shim' underlying the normal
* InfinityDB. There is a part of the EncryptedRandomAccessFile layer that
* can be changed without changing the hash: there is a 'header' that
* contains signatures and so on, that can be altered or signed.
*/
static void demoHash() {
try (Cu cu = Cu.alloc()) {
InfinityDB db = InfinityDB.create(PATH, true, CACHE_SIZE,
PASS_WORD, ENCRYPTION_PARAMETERS_NUMBER);
// Hash both the encrypted and the plain i.e. unencrypted data
System.out.println("initial hashes");
byte[] hashEncrypted =
db.getHashOfEncryptedBlocksAndLogicalLength();
System.out
.println("hashEncrypted=" + Arrays.toString(hashEncrypted));
// A side-effect of this hash is that all blocks are (HMac)
// integrity checked
byte[] hashPlain = db.getHashOfPlainTextBlocks();
System.out.println("hashPlain=" + Arrays.toString(hashPlain));
// Put in anything to change the hashes
cu.append("hello");
db.insert(cu);
cu.clear().append("world");
db.insert(cu);
// DB cannot be dirty to do the hash
db.commit();
// The hashes change
System.out.println("hashes change");
hashEncrypted = db.getHashOfEncryptedBlocksAndLogicalLength();
System.out
.println("hashEncrypted=" + Arrays.toString(hashEncrypted));
hashPlain = db.getHashOfPlainTextBlocks();
System.out.println("hashPlain=" + Arrays.toString(hashPlain));
db.close();
db = InfinityDB.open(PATH, true, PASS_WORD);
System.out.println("hashes are unchanged");
hashEncrypted = db.getHashOfEncryptedBlocksAndLogicalLength();
System.out
.println("hashEncrypted=" + Arrays.toString(hashEncrypted));
hashPlain = db.getHashOfPlainTextBlocks();
System.out.println("hashPlain=" + Arrays.toString(hashPlain));
db.close();
// You can get the encrypted hash without the password.
System.out.println("hashEncrypted is unchanged");
hashEncrypted =
InfinityDB.getHashOfEncryptedBlocksAndLogicalLength(PATH);
System.out
.println("hashEncrypted=" + Arrays.toString(hashEncrypted));
} catch (Throwable e) {
e.printStackTrace();
}
}
/**
* You can sign an encrypted db with one or more certificates or bare public
* keys.
*
* A SignatureInfo is stored in the file, and it associates a cert or public
* key with a signing algorithm like "SHA256", which can also be specified
* as SigningHashAlgorithm.SHA256. A fully-signed db can be checked so that
* it is known to contain the same Items as when it was signed. The public
* keys or certificates used to sign it can be queried, and its signing
* state can be read. Modifying the db changes it to unsigned state. Also,
* accidental or malicious corruption of the database data after signing
* will be detected by validating the signatures.
*/
static void demoSigning() {
try (Cu cu = Cu.alloc()) {
InfinityDB db = InfinityDB.create(PATH, true, CACHE_SIZE,
PASS_WORD, ENCRYPTION_PARAMETERS_NUMBER);
// Add an arbitrary unnecessary Item
db.insert(cu.append("hello").append("world"));
// Can't sign a dirty db. isDirty() is true
db.commit();
// Now isDirty() is false.
// We can sign without an actual certificate: just a bare public key
// will work.
// Generate an RSA key pair.
KeyPairGenerator keyPairGenerator =
KeyPairGenerator.getInstance("RSA");
// number of key bits. Historically 1024, now 2048 is OK, 4096 best
// but slow and large.
keyPairGenerator.initialize(2048);
KeyPair keyPair = keyPairGenerator.generateKeyPair();
PrivateKey privateKey = keyPair.getPrivate();
PublicKey publicKey = keyPair.getPublic();
/*
* SignatureHashAlgorithm.SHA256 just maps to "SHA256". You specify
* it as well as the cert or key and the total algorithm name like
* "SHA256withRSA" is generated internally. (There is a special case
* for ECDSA that is handled for you.) You can add text after that
* explicitly for algorithms that have a suffix. This creates a set
* of one SignatureInfo. You can add or remove them at any time.
* SignatureHashAlgorithm.SHA256 is recommended.
*/
SignatureInfoSet signatureInfoSet = new SignatureInfoSet(
SignatureHashAlgorithm.SHA256, publicKey);
// Put the signature configuration definition into the header of the
// EncryptedRandomAccessFile underneath.
db.setSignatureInfoSet(signatureInfoSet);
try {
db.verifySignatures();
System.out.println(
"missing expected signature verification failure exception");
} catch (SignatureException e) {
// expected - nothing is signed yet.
System.out.println(
"got expected signature verification failure exception");
}
/*
* Provide the private key so we can sign. All of the signatures
* need private keys eventually. If there are multiple
* SignatureInfos, the right ones are automatically located and
* associated based on their publicKeys.
*/
int matchCount = db.setMatchingPrivateKey(privateKey);
System.out.println("matchCount=" + matchCount
+ " should be 1 because one publicKey/privateKey assocation was made");
System.out.println("current matched private keys="
+ db.getMatchedPrivateKeysCount() + " should be 1");
System.out.println(
"currently signed=" + db.getSignedSignatureInfosCount()
+ " should be 0");
/*
* Compute the hash of the data blocks, put that in the header, and
* sign the header. The SignatureInfos having currently associated
* privateKeys are signed, and any already-signed SignatureInfos
* stay signed. This scans the entire db data, but it is fast.
*/
db.sign();
System.out.println(
"currently signed count="
+ db.getSignedSignatureInfosCount()
+ " should be 1");
System.out.println("current matched private keys="
+ db.getMatchedPrivateKeysCount() + " should be 1");
/*
* This is not necessary, but it removes private keys as if we had
* not just signed. Then we can destroy the privateKeys to minimize
* their time in memory. Also db.destroyAllPrivateKeys() can do it
* and then nulls their references. They should be destroyed quickly
* so a memory dump or debug session won't show them. (The same is
* true of passwords, which should be char[] so they can be zeroed
* quickly.)
*/
db.nullAllPrivateKeys();
System.out.println("current matched private keys="
+ db.getMatchedPrivateKeysCount() + " should be 0");
System.out.println(
"currently signed=" + db.getSignedSignatureInfosCount()
+ " should be 1");
System.out.println(
"isFullySigned=" + db.isFullySigned() + " should be true");
/*
* If the db became dirty or we commit(), then the SignatureInfos go
* to unsigned state.
*/
db.close();
/*
* If the header was corrupted by external modification or
* truncation of the file, we cannot open, getting IOException. If
* the db is signed, those signatures apply to the header, and we
* verify that they are valid or IOException results. Even without
* signatures, the header is hMac-protected based on the password.
*/
db = InfinityDB.open(PATH, true, PASS_WORD);
System.out.println(
"isFullySigned=" + db.isFullySigned() + " should be true");
/*
* This computes the hash of the encrypted data blocks and logical
* length and compares with that in the header at the encryption
* layer. If the computed hash and the header hash are different, a
* SignatureException results. The db must not be dirty. All of the
* signatures must be in signed state or an Exception results.
* However there is no guarantee as to the number of SignatureInfos
* in the db - even 0! This takes time to scan all of the data, but
* at high speed.
*/
db.verifySignatures();
/*
* No Exception, so the signatures are fully signed and verified.
* Now for more security make sure the signature we just verified
* has the proper signatory, in this case a bare public Key. This
* could be a certificate too. This also makes sure that there is
* indeed a signature there. If there is a cert with that public key
* rather than a bare public key, it is recognized too.
*/
boolean isRecognizedPublicKey =
db.getSignatureInfoSet().isContains(publicKey);
System.out.println(
"is recognized public key=" + isRecognizedPublicKey
+ " should be true");
/*
* If you just want to make sure some signature is there, use
* getSignedSignatureInfosCount(). You could accept the signing if
* there is any particular nonzero number of signatures even less
* than the value of getSignatureInfosCount() - maybe just one.
*/
db.close();
/*
* You can verify without opening with the password. However, only
* the signatures are used for checking the validity of the header,
* since the hMac can't be done without the password. This is still
* secure if there is a SignatureInfo. You can do some other
* read-only things without the password, like get a copy of the
* SignatureInfoSet or get the hash and more later on.
*/
InfinityDB.verifySignatures(PATH);
/*
* Now we use multiple signatures.
*
* We can actually use the same publicKey (or cert) twice, but with
* different signing algorithms!
*
* A SignatureInfo is equal to another if the signing algorithms are
* equal and the certificates are equal. They are also equal if the
* signing algorithms are equal and they have equal 'bare' public
* keys instead of certs.
*/
db = InfinityDB.open(PATH, true, PASS_WORD);
KeyPair keyPair2 = keyPairGenerator.generateKeyPair();
PrivateKey privateKey2 = keyPair2.getPrivate();
PublicKey publicKey2 = keyPair2.getPublic();
// We use three signatures. We add two more. The existing one is
// kept. This retrieves a copy of the internal one in the header.
SignatureInfoSet signatureInfoSet2 = db.getSignatureInfoSet();
/*
* These two SignatureInfos differ only in the algorithm, sharing
* the public key!
*/
signatureInfoSet2.add(SignatureHashAlgorithm.MD5, publicKey2);
signatureInfoSet2.add(SignatureHashAlgorithm.SHA512, publicKey2);
// This clears any signatures - the SignatureInfos all go to
// unsigned state
db.setSignatureInfoSet(signatureInfoSet2);
// There are three now
System.out.println("signatureInfosSet2=" + signatureInfoSet2);
// This matches both of the occurrences of publicKey2 with different
// algorithms, and they both are signed together.
db.setMatchingPrivateKey(privateKey2);
// sign the SignatureInfo that was already there too.
db.setMatchingPrivateKey(privateKey);
// Three signatures are computed and stored in the header
db.sign();
System.out.println(
"isFully signed=" + db.isFullySigned() + " should be true");
System.out.println(
"signed count=" + db.getSignedSignatureInfosCount()
+ " should be 3");
db.verifySignatures();
// No SignatureException happened
/*
* Remove the private keys for signing. Now if we sign again, the
* already signed SignatureInfos do not go to unsigned state. We can
* close the file, come back later, open it, sign some more
* SignatureInfos, and so on until all of them are signed and we can
* verifySignatures().
*/
db.nullAllPrivateKeys();
// Stays in fully signed state
db.sign();
// Any SignatureInfos being in unsigned state will cause
// SignatureException to be thrown
db.verifySignatures();
// No SignatureException happened
System.out.println("signed count="
+ db.getSignedSignatureInfosCount() + " should be 3");
db.close();
} catch (Throwable e) {
e.printStackTrace();
}
}
/**
* Show how to validate the certificate chains of the SignatureInfos in an
* InfinityDB database based on a set of one or more trusted certs.
*
* This allows for much flexibility: for example, the SignatureInfos can
* contain various end-entity ('leaf') certs that are not individually
* recognized by the recipient or verifier of the db, but which are signed
* by a given expected trusted root cert. So there can be a group of various
* possible signers in in different places, each signing with its own leaf
* cert and corresponding private key. If the db is signed by anyone in the
* group, it is considered OK.
*
* We have no actual certificates in this test, because we can't generate
* them on-the-fly without the BouncyCastle JCA Provider library.
*
* SignatureInfos contain X509CertificatePaths (an InfinityDB specific
* class) which can be read and written to PEM (a standard base-64 text
* format).
*/
static void demoCertificateValidation() {
try {
KeyPairGenerator keyPairGenerator =
KeyPairGenerator.getInstance("RSA");
KeyPair endEntityKeyPair = keyPairGenerator.generateKeyPair();
// PrivateKey endEntityPrivateKey = endEntityKeyPair.getPrivate();
PublicKey endEntityPublicKey = endEntityKeyPair.getPublic();
KeyPair rootKeyPair = keyPairGenerator.generateKeyPair();
// PrivateKey rootPrivateKey = rootKeyPair.getPrivate();
PublicKey rootPublicKey = rootKeyPair.getPublic();
/*
* We will just use a bare public key instead of an end-entity cert,
* so this SignatureInfo will be ignored, and this test is not
* definitive. This creates a set with one SignatureInfo.
*/
SignatureInfoSet signatureInfoSet =
new SignatureInfoSet(SignatureHashAlgorithm.MD5,
endEntityPublicKey);
/*
* Make a set of root certs to validate against. This can be a
* keyStore instead, such as a PKCS#12 or java JKS keystore.
*/
Set<TrustAnchor> trustAnchors = new HashSet<>();
TrustAnchor trustAnchor = new TrustAnchor("CN=MyRootCA",
rootPublicKey, null/* nameConstraints */);
trustAnchors.add(trustAnchor);
// Make sure all the SignatureInfos' X509CertificatePaths are
// validated.
signatureInfoSet.validate(trustAnchors);
// No CertPathValidatorException so the db is safe.
// Or do the validation and checking in a more detailed way.
// Prints "false"
System.out.println("isTrusted=" + isTrusted(signatureInfoSet,
"OU=OurGroup", trustAnchors));
} catch (Throwable e) {
e.printStackTrace();
}
}
/**
* You can also validate and check each SignatureInfo one-at-a-time, to
* recognize or ignore some, for example. Again, there are no certs in this
* demo, only public keys, so this doesn't test anything (we don't assume
* the BouncyCastle library is available to create certs). You could use
* db.getSignedSignatureInfos() in order to scan over only the signed ones.
*/
static boolean isTrusted(SignatureInfoSet signatureInfoSet,
String trustedName, Set<TrustAnchor> trustAnchors)
throws GeneralSecurityException {
for (SignatureInfo signatureInfo : signatureInfoSet) {
X509CertificatePath x509CertificatePath =
signatureInfo.getX509CertificatePath();
if (x509CertificatePath != null) {
// Not a bare public key
try {
x509CertificatePath.validate(trustAnchors);
/*
* Do something with the valid leaf i.e. end-entity cert,
* like filter based on the distinguished name.
*/
X509Certificate x509Certificate =
x509CertificatePath.getCertificate(0);
Principal principal =
x509Certificate.getSubjectDN();
// This might be like "CN=Jennifer, OU=OurGroup".
String distinguishedName = principal.getName();
// If that is a sufficient cert to supply trust, we are
// done.
if (distinguishedName.contains(trustedName))
return true;
} catch (CertPathValidatorException e) {
// Ignore invalid certs
}
}
}
return false;
}
/**
* Create a temporary file that will delete itself on exit.
*/
static String getPath() {
try {
File tempFile =
File.createTempFile("testInfinityDBEncryption", ".infdb");
tempFile.deleteOnExit();
String path = tempFile.getAbsolutePath();
return path;
} catch (IOException e) {
e.printStackTrace();
System.exit(1);
return null;
}
}
static int countItems(ItemSpace itemSpace) throws IOException {
try (Cu cu = Cu.alloc()) {
int i = 0;
while (itemSpace.next(cu))
i++;
return i;
}
}
}