InfinityDB Encrypted is identical to InfinityDB Embedded but it also has the vital but simple security features shown here. Using encryption can be as simple as providing a password when the database file is created, and then supplying it again when the database file is re-opened.
The password can be changed at any time easily and securely as well, by opening using the current password and invoking a single method. Changeability of passwords is vital for security, but must be baked into the original design: the passwords cannot be simply stored in the file! We use an approved standard technology called ‘key encryption keys’ to provide this. To ‘shred’ a file instantly and securely, you can change the key to a long random number that you then ‘forget’.
The many advanced features are also simple to use, to provide even more security. All of them are explained below.
// Copyright (C) 2014-2019 Roger L. Deran. All Rights Reserved. // // Mar 24, 2019 Roger L. Deran // // THIS SOFTWARE CONTAINS CONFIDENTIAL INFORMATION AND TRADE SECRETS // OF Roger L Deran. USE, DISCLOSURE, OR REPRODUCTION IS PROHIBITED // WITHOUT THE PRIOR EXPRESS WRITTEN PERMISSION OF Roger L Deran. // // Roger L Deran. MAKES NO REPRESENTATIONS OR WARRANTIES ABOUT // THE SUITABILITY OF THE SOFTWARE, EITHER EXPRESS OR IMPLIED, // INCLUDING BUT NOT LIMITED TO THE IMPLIED WARRANTIES OF // MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE, OR // NON-INFRINGEMENT. Roger L Deran. SHALL NOT BE LIABLE FOR // ANY DAMAGES SUFFERED BY LICENSEE AS A RESULT OF USING, // MODIFYING OR DISTRIBUTING THIS SOFTWARE OR ITS DERIVATIVES. package com.infinitydb.examples; import java.io.File; import java.io.IOException; import java.security.GeneralSecurityException; import java.security.KeyPair; import java.security.KeyPairGenerator; import java.security.Principal; import java.security.PrivateKey; import java.security.PublicKey; import java.security.SignatureException; import java.security.cert.CertPathValidatorException; import java.security.cert.TrustAnchor; import java.security.cert.X509Certificate; import java.util.Arrays; import java.util.HashSet; import java.util.Random; import java.util.Set; import com.infinitydb.Cu; import com.infinitydb.InfinityDB; import com.infinitydb.ItemSpace; import com.infinitydb.security.IncorrectPassWordException; import com.infinitydb.security.SignatureHashAlgorithm; import com.infinitydb.security.SignatureInfo; import com.infinitydb.security.SignatureInfoSet; import com.infinitydb.security.X509CertificatePath; /** * * An example of using the InfinityDB encryption feature of version 5. Just * provide a password to create or open an encrypted file. * * There is virtually no performance hit. Compression is just as effective: 1 to * 10x or more. Unencrypted version 4 files can be used without change by the * new version and will remain unencrypted and compatible. Version 4 will throw * an IOException when an encrypted file is attempted to be opened. * * Databases can also be hashed, signed or the signatures verified at high * speed. The file contains a set of signature definitions i.e. 'SignatureInfos' * each with an X509 certificate path or a 'bare public key', and each with a * selectable hash algorithm. SignatureInfos can be signed in subsets sharing * the data hash computation. Each SignatureInfo can stay in signed or unsigned * state after close(). The signing state persists until the database content * changes. * * Certificate paths in the SignatureInfos can be validated based on a set of * trusted certificates. External storage or availability of signing * certificates after they are put in the file is not necessary, only private * keys for signing and trusted public keys or trusted certificates for possible * validation. Verification can use client-implemented strategies like 'any * signature based on this public key is enough' or 'any N signatures is * enough', or 'any validated signatures with selected certificates based on the * distinguished name is enough'. Signatures can be verified and the hash * computed without the password. * * The implementation uses an underlying 'shim' called EncryptedRandomAccessFile * that provides its overlying InfinityDB with a logical * GeneralizedRandomAccessFile, while physically storing the data as encrypted * blocks in a normal RandomAccessFile. The InfinityDB-specific * GeneralizedRandomAccessFile is necessary instead of a subclass of * RandomAccessFile, because the latter cannot be subclassed (this is considered * an original mistake in Java - InputStream and OutputStream are OK though). * * The EncryptedRandomAccessFile also contains a 'header' before the encrypted * blocks that describes the file state, and which contains structure for future * extensions, signature information and eventually information for * 'enveloping'. The header itself is variable-length but has a limited fixed * space at the front of a particular file - if too much data is attempted to be * written in that space, an IOException is thrown, but the file is still usable * in its previous state. Currently the size is fixed at 100K but later it will * be settable on create(). This should be plenty. The header can change without * the hash being changed. */ public class EncryptionExample { static final long CACHE_SIZE = 100_000_000; // Passwords are char[] so you can zero them out to minimize time in memory. // A String can't be zeroed. static final char[] PASS_WORD = new char[] { 'a', 'b', 'c' }; // We change the password to this later on static final char[] NEW_PASS_WORD = new char[] { 'd', 'e', 'f' }; /* * 0 is for regular 128-bit AES strength with no export issues, 1 is for * strong 256-bit AES. * * This is not required on open, but set by create permanently. In the far * future there will be more if these two become obsolete. * * Note that a database created with strong encryption can only be opened by * a JVM with strong encryption enabled. Some countries control the use or * distribution of strong encryption. However, it can normally be enabled * with simple changes to files in $JAVA_HOME/jre/lib/security. */ static final int ENCRYPTION_PARAMETERS_NUMBER = 0; // To generate some example content static final int ITEM_COUNT = 100_000; static final Random random = new Random(); // The database path. static final String PATH = getPath(); public static void main(String... args) { System.out.println("Database path=" + PATH); demoEncryptedMode(); demoHash(); demoSigning(); demoCertificateValidation(); } /** * The db is encrypted just because the password and encryption params are * given. That's the only required API change for encryption and integrity * checking. */ static void demoEncryptedMode() { try (Cu cu = Cu.alloc()) { InfinityDB db = InfinityDB.create(PATH, true, CACHE_SIZE, PASS_WORD, ENCRYPTION_PARAMETERS_NUMBER); random.setSeed(0); for (int i = 0; i < ITEM_COUNT; i++) { // create items like "hello" 391 cu.clear().append("hello").append(random.nextLong()); db.insert(cu); } System.out.println( "count of Items (should be " + ITEM_COUNT + ")=" + countItems(db)); // Won't affect Item count - just makes data durable. db.commit(); /* * This countItems() will read every block, and a side-effect, the * hMac authenticity check based on the password will be able to * detect externally-caused corruption. Or for more speed use * getPlainTextBlockHash() for that. * * However, there is an elaborate 'backup attack' which in principle * can substitute corresponding blocks between two backups to affect * the contents and yield a usable db. To detect it, use signatures * or keep hashes and compare them later. */ System.out.println( "count of Items after commit (should be " + ITEM_COUNT + ")=" + countItems(db)); db.close(); // Try to open with no password. // The encryption parameters number is not supplied - it is // fixed after creation. try { db = InfinityDB.open(PATH, true); } catch (Exception e) { // expected - password not provided System.out.println("no password: " + e.getMessage()); } // try to open with wrong password try { db = InfinityDB.open(PATH, true, new char[] { 'w', 'r', 'o', 'n', 'g' }); } catch (IncorrectPassWordException e) { // expected - password wrong System.out.println("wrong password: " + e.getMessage()); } // Opens correctly with correct password db = InfinityDB.open(PATH, true, PASS_WORD); System.out.println( "count of Items after re-open (should be " + ITEM_COUNT + ")=" + countItems(db)); /* * New feature in 5.1: passwords can be changed. */ db.changePassWord(NEW_PASS_WORD); db.close(); // Opens correctly with new password db = InfinityDB.open(PATH, true, NEW_PASS_WORD); // Same seed so same Items as were inserted. random.setSeed(0); // Delete all Items randomly for (int i = 0; i < ITEM_COUNT; i++) { // create Items like "hello" 391 cu.clear().append("hello").append(random.nextLong()); db.delete(cu); } System.out.println( "count of Items after deletion (should be 0)=" + countItems(db)); db.commit(); System.out.println( "count of Items after deletion and commit (should be 0)=" + countItems(db)); } catch (Throwable e) { e.printStackTrace(); } } /** * Compute the hash of the database contents, i.e. the set of Items. * * There is a very fast hash of the encrypted data and a different hash of * the unencrypted .i.e 'plain' data that as a side-effect checks the * integrity of each block. * * This hash will be different for different DBs created separately, even if * the Item set is the same. However, if a given DB is not changed, its hash * will stay the same even after close()/open() or reads. So an * externally-caused corruption i.e. modification or truncation of the * encrypted data blocks will generate a different hash. * * The hashing feature is not available on a legacy unencrypted version 4 or * earlier db, throwing an Exception. It is used in signing encrypted dbs. * * It is based on SHA-256, so the length of the hash is currently 32 bytes. * The hash is very fast, but it must read all of the file. Some day it will * even be multi-threaded. * * Future versions of InfinityDB may change the hash algorithm or * implementation, for example if the security of SHA256 is compromised or * performance can be improved. The hash algorithm used then will be client * selectable or automatically adapted to the particular open file. * * This actually hashes the unencrypted or encrypted data blocks and logical * file length of the EncryptedRandomAccessFile 'shim' underlying the normal * InfinityDB. There is a part of the EncryptedRandomAccessFile layer that * can be changed without changing the hash: there is a 'header' that * contains signatures and so on, that can be altered or signed. */ static void demoHash() { try (Cu cu = Cu.alloc()) { InfinityDB db = InfinityDB.create(PATH, true, CACHE_SIZE, PASS_WORD, ENCRYPTION_PARAMETERS_NUMBER); // Hash both the encrypted and the plain i.e. unencrypted data System.out.println("initial hashes"); byte[] hashEncrypted = db.getHashOfEncryptedBlocksAndLogicalLength(); System.out .println("hashEncrypted=" + Arrays.toString(hashEncrypted)); // A side-effect of this hash is that all blocks are (HMac) // integrity checked byte[] hashPlain = db.getHashOfPlainTextBlocks(); System.out.println("hashPlain=" + Arrays.toString(hashPlain)); // Put in anything to change the hashes cu.append("hello"); db.insert(cu); cu.clear().append("world"); db.insert(cu); // DB cannot be dirty to do the hash db.commit(); // The hashes change System.out.println("hashes change"); hashEncrypted = db.getHashOfEncryptedBlocksAndLogicalLength(); System.out .println("hashEncrypted=" + Arrays.toString(hashEncrypted)); hashPlain = db.getHashOfPlainTextBlocks(); System.out.println("hashPlain=" + Arrays.toString(hashPlain)); db.close(); db = InfinityDB.open(PATH, true, PASS_WORD); System.out.println("hashes are unchanged"); hashEncrypted = db.getHashOfEncryptedBlocksAndLogicalLength(); System.out .println("hashEncrypted=" + Arrays.toString(hashEncrypted)); hashPlain = db.getHashOfPlainTextBlocks(); System.out.println("hashPlain=" + Arrays.toString(hashPlain)); db.close(); // You can get the encrypted hash without the password. System.out.println("hashEncrypted is unchanged"); hashEncrypted = InfinityDB.getHashOfEncryptedBlocksAndLogicalLength(PATH); System.out .println("hashEncrypted=" + Arrays.toString(hashEncrypted)); } catch (Throwable e) { e.printStackTrace(); } } /** * You can sign an encrypted db with one or more certificates or bare public * keys. * * A SignatureInfo is stored in the file, and it associates a cert or public * key with a signing algorithm like "SHA256", which can also be specified * as SigningHashAlgorithm.SHA256. A fully-signed db can be checked so that * it is known to contain the same Items as when it was signed. The public * keys or certificates used to sign it can be queried, and its signing * state can be read. Modifying the db changes it to unsigned state. Also, * accidental or malicious corruption of the database data after signing * will be detected by validating the signatures. */ static void demoSigning() { try (Cu cu = Cu.alloc()) { InfinityDB db = InfinityDB.create(PATH, true, CACHE_SIZE, PASS_WORD, ENCRYPTION_PARAMETERS_NUMBER); // Add an arbitrary unnecessary Item db.insert(cu.append("hello").append("world")); // Can't sign a dirty db. isDirty() is true db.commit(); // Now isDirty() is false. // We can sign without an actual certificate: just a bare public key // will work. // Generate an RSA key pair. KeyPairGenerator keyPairGenerator = KeyPairGenerator.getInstance("RSA"); // number of key bits. Historically 1024, now 2048 is OK, 4096 best // but slow and large. keyPairGenerator.initialize(2048); KeyPair keyPair = keyPairGenerator.generateKeyPair(); PrivateKey privateKey = keyPair.getPrivate(); PublicKey publicKey = keyPair.getPublic(); /* * SignatureHashAlgorithm.SHA256 just maps to "SHA256". You specify * it as well as the cert or key and the total algorithm name like * "SHA256withRSA" is generated internally. (There is a special case * for ECDSA that is handled for you.) You can add text after that * explicitly for algorithms that have a suffix. This creates a set * of one SignatureInfo. You can add or remove them at any time. * SignatureHashAlgorithm.SHA256 is recommended. */ SignatureInfoSet signatureInfoSet = new SignatureInfoSet( SignatureHashAlgorithm.SHA256, publicKey); // Put the signature configuration definition into the header of the // EncryptedRandomAccessFile underneath. db.setSignatureInfoSet(signatureInfoSet); try { db.verifySignatures(); System.out.println( "missing expected signature verification failure exception"); } catch (SignatureException e) { // expected - nothing is signed yet. System.out.println( "got expected signature verification failure exception"); } /* * Provide the private key so we can sign. All of the signatures * need private keys eventually. If there are multiple * SignatureInfos, the right ones are automatically located and * associated based on their publicKeys. */ int matchCount = db.setMatchingPrivateKey(privateKey); System.out.println("matchCount=" + matchCount + " should be 1 because one publicKey/privateKey assocation was made"); System.out.println("current matched private keys=" + db.getMatchedPrivateKeysCount() + " should be 1"); System.out.println( "currently signed=" + db.getSignedSignatureInfosCount() + " should be 0"); /* * Compute the hash of the data blocks, put that in the header, and * sign the header. The SignatureInfos having currently associated * privateKeys are signed, and any already-signed SignatureInfos * stay signed. This scans the entire db data, but it is fast. */ db.sign(); System.out.println( "currently signed count=" + db.getSignedSignatureInfosCount() + " should be 1"); System.out.println("current matched private keys=" + db.getMatchedPrivateKeysCount() + " should be 1"); /* * This is not necessary, but it removes private keys as if we had * not just signed. Then we can destroy the privateKeys to minimize * their time in memory. Also db.destroyAllPrivateKeys() can do it * and then nulls their references. They should be destroyed quickly * so a memory dump or debug session won't show them. (The same is * true of passwords, which should be char[] so they can be zeroed * quickly.) */ db.nullAllPrivateKeys(); System.out.println("current matched private keys=" + db.getMatchedPrivateKeysCount() + " should be 0"); System.out.println( "currently signed=" + db.getSignedSignatureInfosCount() + " should be 1"); System.out.println( "isFullySigned=" + db.isFullySigned() + " should be true"); /* * If the db became dirty or we commit(), then the SignatureInfos go * to unsigned state. */ db.close(); /* * If the header was corrupted by external modification or * truncation of the file, we cannot open, getting IOException. If * the db is signed, those signatures apply to the header, and we * verify that they are valid or IOException results. Even without * signatures, the header is hMac-protected based on the password. */ db = InfinityDB.open(PATH, true, PASS_WORD); System.out.println( "isFullySigned=" + db.isFullySigned() + " should be true"); /* * This computes the hash of the encrypted data blocks and logical * length and compares with that in the header at the encryption * layer. If the computed hash and the header hash are different, a * SignatureException results. The db must not be dirty. All of the * signatures must be in signed state or an Exception results. * However there is no guarantee as to the number of SignatureInfos * in the db - even 0! This takes time to scan all of the data, but * at high speed. */ db.verifySignatures(); /* * No Exception, so the signatures are fully signed and verified. * Now for more security make sure the signature we just verified * has the proper signatory, in this case a bare public Key. This * could be a certificate too. This also makes sure that there is * indeed a signature there. If there is a cert with that public key * rather than a bare public key, it is recognized too. */ boolean isRecognizedPublicKey = db.getSignatureInfoSet().isContains(publicKey); System.out.println( "is recognized public key=" + isRecognizedPublicKey + " should be true"); /* * If you just want to make sure some signature is there, use * getSignedSignatureInfosCount(). You could accept the signing if * there is any particular nonzero number of signatures even less * than the value of getSignatureInfosCount() - maybe just one. */ db.close(); /* * You can verify without opening with the password. However, only * the signatures are used for checking the validity of the header, * since the hMac can't be done without the password. This is still * secure if there is a SignatureInfo. You can do some other * read-only things without the password, like get a copy of the * SignatureInfoSet or get the hash and more later on. */ InfinityDB.verifySignatures(PATH); /* * Now we use multiple signatures. * * We can actually use the same publicKey (or cert) twice, but with * different signing algorithms! * * A SignatureInfo is equal to another if the signing algorithms are * equal and the certificates are equal. They are also equal if the * signing algorithms are equal and they have equal 'bare' public * keys instead of certs. */ db = InfinityDB.open(PATH, true, PASS_WORD); KeyPair keyPair2 = keyPairGenerator.generateKeyPair(); PrivateKey privateKey2 = keyPair2.getPrivate(); PublicKey publicKey2 = keyPair2.getPublic(); // We use three signatures. We add two more. The existing one is // kept. This retrieves a copy of the internal one in the header. SignatureInfoSet signatureInfoSet2 = db.getSignatureInfoSet(); /* * These two SignatureInfos differ only in the algorithm, sharing * the public key! */ signatureInfoSet2.add(SignatureHashAlgorithm.MD5, publicKey2); signatureInfoSet2.add(SignatureHashAlgorithm.SHA512, publicKey2); // This clears any signatures - the SignatureInfos all go to // unsigned state db.setSignatureInfoSet(signatureInfoSet2); // There are three now System.out.println("signatureInfosSet2=" + signatureInfoSet2); // This matches both of the occurrences of publicKey2 with different // algorithms, and they both are signed together. db.setMatchingPrivateKey(privateKey2); // sign the SignatureInfo that was already there too. db.setMatchingPrivateKey(privateKey); // Three signatures are computed and stored in the header db.sign(); System.out.println( "isFully signed=" + db.isFullySigned() + " should be true"); System.out.println( "signed count=" + db.getSignedSignatureInfosCount() + " should be 3"); db.verifySignatures(); // No SignatureException happened /* * Remove the private keys for signing. Now if we sign again, the * already signed SignatureInfos do not go to unsigned state. We can * close the file, come back later, open it, sign some more * SignatureInfos, and so on until all of them are signed and we can * verifySignatures(). */ db.nullAllPrivateKeys(); // Stays in fully signed state db.sign(); // Any SignatureInfos being in unsigned state will cause // SignatureException to be thrown db.verifySignatures(); // No SignatureException happened System.out.println("signed count=" + db.getSignedSignatureInfosCount() + " should be 3"); db.close(); } catch (Throwable e) { e.printStackTrace(); } } /** * Show how to validate the certificate chains of the SignatureInfos in an * InfinityDB database based on a set of one or more trusted certs. * * This allows for much flexibility: for example, the SignatureInfos can * contain various end-entity ('leaf') certs that are not individually * recognized by the recipient or verifier of the db, but which are signed * by a given expected trusted root cert. So there can be a group of various * possible signers in in different places, each signing with its own leaf * cert and corresponding private key. If the db is signed by anyone in the * group, it is considered OK. * * We have no actual certificates in this test, because we can't generate * them on-the-fly without the BouncyCastle JCA Provider library. * * SignatureInfos contain X509CertificatePaths (an InfinityDB specific * class) which can be read and written to PEM (a standard base-64 text * format). */ static void demoCertificateValidation() { try { KeyPairGenerator keyPairGenerator = KeyPairGenerator.getInstance("RSA"); KeyPair endEntityKeyPair = keyPairGenerator.generateKeyPair(); // PrivateKey endEntityPrivateKey = endEntityKeyPair.getPrivate(); PublicKey endEntityPublicKey = endEntityKeyPair.getPublic(); KeyPair rootKeyPair = keyPairGenerator.generateKeyPair(); // PrivateKey rootPrivateKey = rootKeyPair.getPrivate(); PublicKey rootPublicKey = rootKeyPair.getPublic(); /* * We will just use a bare public key instead of an end-entity cert, * so this SignatureInfo will be ignored, and this test is not * definitive. This creates a set with one SignatureInfo. */ SignatureInfoSet signatureInfoSet = new SignatureInfoSet(SignatureHashAlgorithm.MD5, endEntityPublicKey); /* * Make a set of root certs to validate against. This can be a * keyStore instead, such as a PKCS#12 or java JKS keystore. */ Set<TrustAnchor> trustAnchors = new HashSet<>(); TrustAnchor trustAnchor = new TrustAnchor("CN=MyRootCA", rootPublicKey, null/* nameConstraints */); trustAnchors.add(trustAnchor); // Make sure all the SignatureInfos' X509CertificatePaths are // validated. signatureInfoSet.validate(trustAnchors); // No CertPathValidatorException so the db is safe. // Or do the validation and checking in a more detailed way. // Prints "false" System.out.println("isTrusted=" + isTrusted(signatureInfoSet, "OU=OurGroup", trustAnchors)); } catch (Throwable e) { e.printStackTrace(); } } /** * You can also validate and check each SignatureInfo one-at-a-time, to * recognize or ignore some, for example. Again, there are no certs in this * demo, only public keys, so this doesn't test anything (we don't assume * the BouncyCastle library is available to create certs). You could use * db.getSignedSignatureInfos() in order to scan over only the signed ones. */ static boolean isTrusted(SignatureInfoSet signatureInfoSet, String trustedName, Set<TrustAnchor> trustAnchors) throws GeneralSecurityException { for (SignatureInfo signatureInfo : signatureInfoSet) { X509CertificatePath x509CertificatePath = signatureInfo.getX509CertificatePath(); if (x509CertificatePath != null) { // Not a bare public key try { x509CertificatePath.validate(trustAnchors); /* * Do something with the valid leaf i.e. end-entity cert, * like filter based on the distinguished name. */ X509Certificate x509Certificate = x509CertificatePath.getCertificate(0); Principal principal = x509Certificate.getSubjectDN(); // This might be like "CN=Jennifer, OU=OurGroup". String distinguishedName = principal.getName(); // If that is a sufficient cert to supply trust, we are // done. if (distinguishedName.contains(trustedName)) return true; } catch (CertPathValidatorException e) { // Ignore invalid certs } } } return false; } /** * Create a temporary file that will delete itself on exit. */ static String getPath() { try { File tempFile = File.createTempFile("testInfinityDBEncryption", ".infdb"); tempFile.deleteOnExit(); String path = tempFile.getAbsolutePath(); return path; } catch (IOException e) { e.printStackTrace(); System.exit(1); return null; } } static int countItems(ItemSpace itemSpace) throws IOException { try (Cu cu = Cu.alloc()) { int i = 0; while (itemSpace.next(cu)) i++; return i; } } }