REST API’s with PatternQueries

REST is a very simple industry-standard request-response protocol supported by InfinityDB Server. Each REST access targets a specific URL, providing a request content in JSON and receiving back a JSON response content, somewhat like HTTP. Also Blobs like images can be transferred. In InfinityDB Server, each such URL is handled by a specific PatternQuery, so accesses are like remote procedure calls. PatternQueries can be bypassed for READ or WRITE permissions. Here is a simple example with the syntax of the database in this case being i code (all queries and other data can be specified as JSON as well, even blobs).

The support code for access via REST is at github.com/infinitydb/infinitydb.

Database Names

Above, the URL includes the database name ‘my/db’ which is always two words separated by a single slash, where the words are a letter followed by letters, digits, dot, dash, and underscore. The server can contain any number of databases, each being a single InfinityDB Embedded file or else a remoted database on another server. The server admin user can create and manage databases and remoting, as well as managing users, passwords, roles, grants and permissions. Not shown are the user name and password passed as base64 in the request (in the same way as an HTTP Basic Authentication, hence without hashing or encryption, but the server uses SSL for encryption and verification so this is not a problem).

Query Name

After the database name are two strings for the query name – the “interface name” and the “method name”. Queries are stored inside the database that they apply to, so a database describes its own API, thereby isolating client applications from the internal structure of the data. It is therefore easy for the database to be restructured, extended, simplified, combined, or secured without affecting client applications. API can be added by storing new queries, and new ‘parameters’ or ‘returned data’ can be added by adding a few Items to queries. To execute a query, its definition must be like Query "interface" "method name" query definition.

The first string in the query name is an ‘interface name’, which is like an internet domain name, and is generally kept constant for any group of queries. It is one or more dot-separated components, where each component is one or more lower-case letters, digits, or dash. It must start with a lower-case letter. One puts the components in the reverse order compared to internet domains, however, and by making them correspond to actual domains, it is possible for query authors to keep their interface names globally unique. By holding interface names constant, REST clients can hard-code them. Note that in a URL, the interface and method name are both double-quoted strings, and in some URLs these show up with the double-quotes becoming %22. Also, method names can contain any chars, often even spaces!

Permissions

The admin can set permissions for given roles based on interface names or prefixes of interface names.. Multiple interface permissions may be granted to any particular role for any particular database. The admin can also set permissions to require an ‘Option setter true’ or ‘Option getter true’ specified in the query for more control like the Item Query "my.domain" "my method" query Option getter true. These are independent of the REST ‘method’ such as GET, PUT, POST, or DELETE; queries use GET for getting blobs, otherwise POST. The admin may also grant permission to read and/or to write to a database, but these permissions are independent of query permissions. By default, queries can not be executed at all. See the PatternQuery Reference for more. An example permission might be read query:prefix=com.infinitydb.examples query:interface=com.infinitydb,getter. That role could do direct read access via REST or a user could browse the data but not modify it. Also, a REST or user could run queries with interface names starting with com.infinitydb.examples, and could run queries with exactly com.infinitydb but only those queries flagged as ‘getters’. A permission failure will return a 405 result code and a message.

Data

Content is either JSON or BLOBs or ‘Binary Long Objects’ according to the ‘Content-Type’ header parameter of the request or response, following the mime type naming rules of HTTP. For JSON the mimetype is ‘application/json’. For JSON, the ‘action’ URL parameter is ‘execute-query’, while for blobs, the action is “execute-get-blob-query” or “execute-put-blob-query”. Transfer rate is very high and efficient for blob data. The mimetype is stored associated with each blob in the database but not for JSON, since that is self-describing. JSON queries can receive data in the request content and return it in the response content for each call, but for putting blobs, only the binary blob data is in the content, so further ‘parameters’ must be in the URL as described below. For getting a blob, there is no way to also return some JSON along with it. (Currently we don’t support returning JSON containing one or more embedded blobs for Python clients because the blobs don’t get converted into long contiguous byte arrays, and transferring blobs as JSON is less efficient. So, you do multiple accesses, one per blob.)

Direct Access

If a user has read or write access to a database, then the database can also be accessed directly by a URL without any query being involved, and with no ‘action’ URL parameter. In that case, the part of the URL after the database name is translated directly into an Item prefix and all of the suffixes of that Item prefix are marshalled and transmitted in as the request content or out as the response content in JSON form. In case the Item suffixes in the database are formatted in a particular way, a URL can be used to get or put a BLOB also without a PatternQuery or by using the ‘action=get-blob’ or ‘action=put-blob’ URL query parameters.

An example URL to get some JSON is https://infinitydb.com/infinitydb/data/demo/readonly/Documentation (user name ‘testUser’ password ‘db’, database ‘demo/readonly’, Item prefix ‘Documentation’). For an example BLOB see https://infinitydb.com/infinitydb/data/demo/readonly/Pictures/%22pic2%22.

Blobs

InfinityDB stores BLOBs that are to be associated with a mimetype in this way (this is i-code but could be JSON, and actually this is tightly compressed binary inside the database file):

... com.infinitydb.blob {
    com.infinitydb.blob.data [
         Bytes(xx_xx_xx..._xx)
         Bytes(xx_xx_xx..._xx)
    ]
    com.infinitydb.blob.mimetype "text/plain"
} 

Any unlimited length series of bytes can be represented by a list of short byte array components as above under the attribute com.infinitydb.blob.data. The byte arrays are all of length 1024 except the last, which is one to 1024 as necessary. This format is compatible with JSON and i-code, so actually blobs can be embedded in those text formats because the Bytes(..) is a standard InfinityDB Item component token format representing a byte array. In the database browser web pages, displaying in i-code or tabular mode converts such structures into proper visible form, such as images or printable text, but while editing i-code or JSON, the format above shows up, so you can cut and paste blobs as text. The database browser has a single-blob and multi-file blob upload feature as well, and for downloading you can click on the blob to make it full screen and save it.

Request and Response MetaData

The ‘params’ URL Parameter

If a put-blob query or in fact any query wants parameters beyond just the blob data and mime type, they can be passed in the URL as the ‘params’ parameter, in URL-quoted JSON form, and the query uses a root symbol of kind “params url parameter” to match on it. So, query pattern =params some pattern matching suffixes with query Where 'params' kind 'params url parameter' will do it. The URL is of form https://domain/...?action=action&params=urlquotedjson. The ‘params’ parameter can be combined with any others that may also be needed. If the request is a GET, then the params are parsed from the query string part of the URL, or if it is a POST and the Content-Type is ‘application/x-www-form-urlencoded’ then the params are parsed from that.

Getting the Entire Set of URL Parameters

Also, the query can look at the entire set of URL parameters individually with the ‘request parameters’ symbol kind, which is a root. In that case, the parameters take the form query pattern =reqparams RequestParameter name value value. with query Where 'reqparams' kind 'request parameters' and the names and values are just strings, not JSON. If it is a GET, the parameters are in the query string part of the URL, but if it is a POST, then if the Content-Type is ‘application/x-www-form-urlencoded’ then the parameters are parsed from that.

Headers

The query can look into the request headers with query pattern =req Header name value =value_symbol. with query Where 'req' kind 'request header' and the names and values are just strings, not JSON. Also a response header can be set with query result =resp Header name value value. with query Where 'resp' kind 'response header' . You can ask for a header to be removed with query result =resp Header name remove true. The value of the ‘Authorization’ request header is always '(hidden)'.

Python infinitydb.access Module

Python has special optional support for REST. The infinitydb.access module provides many features, such as helpers for formatting JSON-communicated data as dicts. There are classes for EntityClass and Attribute and so on. Also, the database can be navigated bidirectionally by Item and modified, and queries can be invoked. All of this is possible without the library, however. To install the module (don’t forget to keep pip up-to-date too):

python3 -m pip install --upgrade infinitydb

There are module functions to convert between either ‘compacted tips’ or ‘uncompacted tips’ form of dicts. In the latter, all values are None but in ‘compacted’ mode there may be nulls or not. You can also use lists to avoid that problem.

There are functions to combine adjacent primitive components together into tuple keys and back, even allowing the tuples to have variable lengths. By ‘adjacent’ we mean one primitive component is a key and the other is a key or value inside the next level deeper.

The dicts may also include nested lists, but in case you need to see the Index components distinctly as keys, there are functions for that. Python dicts are not sorted (keys are hashed, and there is some kind of ‘ordered dict’ class but it doesn’t work properly) so lists are required for ordering, and magnitude comparisons of distinct types are not supported, unlike within InfinityDB. The dicts are automatically sorted once they reach InfinityDB.

You can also deal with the JSON directly with other routes.

Take a look at the free open-source code in https://github.com/infinitydb/infinitydb.git for the entire module.

Here is some code using the module:

from infinitydb.access import *

# port 443 is usual
infinitydb_url = 'https://infinitydb.com/infinitydb/data'
# True or None for a server with verifiable TLS certificate, 
# which means it has a certificate from a trusted certificate authority
# with a matching domain name.
verify = True
#infinitydb_url = 'https://localhost/infinitydb/data'
#verify = False 

""" Database names (URI components) """

# Databases have one slash separating two names each like
# [A-Za-z][A-Za-z0-9._-]*
# An infinitydb server by default has these available for the testUser
database_uri = 'demo/writeable' 
#database_uri = 'demo/readonly'

""" The User Name and Password """

# A public guest user for browsing and experimentation, in the
# 'guest' role. Contact us for your own experimentation login and db.
user = 'testUser'
password = 'db'

""" The connection to the database """

infdb = InfinityDBAccessor(infinitydb_url, db=database_uri, user=user, password=password, verify=verify)

infdb.head() # check that the connection is working
    
""" Get JSON given a prefix Item from the REST connection """

# This shows direct access to the db, but we prefer query-based access below 
# To see the documentation graphically in the demo/readonly database, go to:
# infinitydb.com/infinitydb/data/demo/readonly/Documentation?action=edit
# or without the action=edit to see it in JSON form.
# Here we read that JSON into content, with success being a boolean.
# The success only indicates that some data was read, not that there was an error.
# Real errors raise InfinityDBError
# The JSON is represented by nested dicts and lists.
# We use a path prefix of EntityClass('Documentation') which is an 
# Item with a single initial class component:
success, content, content_type = infdb.get_json([EntityClass('Documentation')])
print(content)

# Launch a query on the server. There is no request header or response header
# This just copies and restructures a small amount of data on the db 
def copy_aircraft():
    success, content, response_content_type = infdb.execute_query(
        ['examples','Aircraft to AircraftModel'])

# Some statistics calculated over the samples table
def get_summarize_samples():
    success, content, response_content_type = infdb.execute_query(
        ['examples','summarize samples2'])
    return success, content, response_content_type

# for blobs, we do a direct 'get blob' type of query. Very fast        
def get_image(pic):
    data = { Attribute('name'): pic }
    success, content, response_content_type = infdb.execute_get_blob_query(
        ['examples','Display Image'], data=data)
    return success, content, response_content_type

def get_people_country():
    success, content, response_content_type = infdb.execute_query(
        ['examples.person','get residence state'])
    return success, content, response_content_type
    

try:
    print('copy aircraft')
    copy_aircraft()
    print('summarize samples')
    success, content, type = get_summarize_samples()
    if success:
        print(' ',content)
    success, content, type = get_image('pic0')
    if success:
        print('retrieved image size=', len(content))
    success, content, type = get_people_country()
    print(content)
    
except InfinityDBError as e:
    print('Could not access infdb ', e)
except Exception as e:
    print('Could not access infdb ', e)
    

Direct REST withJSON

In case you access directly from JavaScript in nodejs or in AJAX in a web page or in bash’s curl command or elsewhere, there are a few things to keep in mind. Each Item is just considered a path into JSON nested Objects, one key per component, with the final component being a terminal value or null. Of course the Index type is encoded as a list, but empty lists are pruned away.

Underscore Quoting

The JSON we use has ‘underscore quoting’. This form is visible in the database browser in that display mode. The rules are simple. Since each JSON key must be a string, we ‘quote’ the 12 types by stuffing an underscore before each token. So to encode an attribute as a key, you use “_myattribute”. If you actually want a literal underscore, ‘stuff in’ an additional underscore there, like “__mystring”. This applies to the values as well except for doubles, longs, true/false and null.

We have to preserve the types because double, float, and long are stored distinctly. The parser will assume that all numbers are doubles, so the parser must be given ‘_5’ or ‘_5.0f’ to signal otherwise. The formatter does the inverse, but in JavaScript, all numbers are combined into one type. Hence we have to have the programmer signal the type from context to the parser.

Lists in InfinityDB are represented with special data types called ‘Index’, and these automatically convert into JSON lists and back, so there are no number conversion issues. An empty list cannot be represented, though, so you still have to check for a missing list.

Three Types of Numbers

InfinityDB numbers can be long (64-bits), IEEE-754 double (64 bits) or IEEE-754 float (32 bits). Those are separate in order to be compatible with all possible contexts such as Java, C, C++ and so on. The three are not perfectly interconvertible because double or float have exponents that are too large for a long and mantissas that are too short.

In the database, these can be stored mixed together, affecting the sorting because floats, doubles, and longs come out in that order when compared with each other (other DBMS cannot store dynamically typed data at all, let alone compare and store them as sets, as can InfinityDB with all types). If, for example, a terminal component always has only one value and is not effectively a ‘set’, there is no issue because different types will not be compared.

JavaScript numbers are all 64-bit IEEE-754 doubles so you cannot differentiate between a real number with an integer value and an actual integer. The only issue is that x.0 may become x so we define that all JavaScript numbers are doubles in the request content whether they have a decimal or not. Then you explicitly ‘underscore quote’ longs and floats as shown below. (JavaScript does have a BigInt feature with unlimited precision, but we do not use it, and such unlimited precision numbers are normally very expensive and used only when necessary.)

The number situation in Python is easier because it differentiates int and float, but Python has no 32-bit real, so InfinityDB 32-bit floats become Python 64-bit ‘floats’.

You probably want to be consistent in choosing one numeric type for storage, but they are freely convertible in expressions, and you can set the type of a symbol to ‘number’, ‘long’, ‘double’, or ‘float’ or any subset to control the static strong type checking. In fact the type checking is a ‘combination’ of static and dynamic, since you can be specific or general in the static typing, yet types are converted dynamically as needed. So if you have a symbol with type of { ‘double’; ‘float’; } then the expressions are ensured at compile time to be compatible with those, yet a float is dynamically promoted at runtime if combined with a double. Any set of types can be declared as statically allowable for a given symbol but must agree with the results of expressions and symbol references. A plain symbol (which is one that matches database data) declared with a subset of types will match only data having those types; therefore if new data arrives in the database with other types, it will not ‘break’ the query..

Nulls for Sets

Occasionally objects with null values will appear as the terminal values; for example, in cases where there are two or more Items that differ only in the terminal component. This is because there is no ‘set’ concept in JSON, so we use an Object with multiple keys for the elements of the set, with null values. This can be an issue because sometimes something you consider a set happens to have only one value in it or the reverse, so the regular value disappears and is replaced with an object with null values. You can use lists instead. Or, you can write queries that, for example, choose the first or last value or test for multiple values and error-out or limit the number of values or match only one value and so on.

For the general non-list case, you have to check for whether the terminal value you want is actually a regular value, a null or an Object and act appropriately. Within an ItemSpace, Items are just independent paths and there is no such issue. You can debug this kind of issue by just executing the query by hand with the database browser and looking at the request or response content, or look at the data in the ‘JSON Underscore Quoted’ view. It is possible for the database to contain Items that are prefixes of others, but these cannot be represented in JSON, and they normally do not matter, although other formats can work with them. Note that the com.infinitydb.remote.SimpleRest Java framework handles this situation gracefully. Here is an example, where k4 is ‘set valued’:

{
    "k1" : null,
    "k2" : "value",
    "k3" : null,
    "k4" : {
        "v" : null,
        "v2" : null
    }
}

Bash Curl Command

Access from the unix/linux bash command can use curl. The Item prefix in the example here contains standard Item component tokens with slashes between. The Item Prefix goes after https://domain/infinitydb/data/ or other port. In this example, the Documentation component is a class (starts with UC), the description token is an attribute (starts with LC) and the [0] token is a list index.

Direct

This example does direct read access, so it does GET and it needs READ permission, but to write you use POST and you need WRITE permission. For direct access to blobs, use ?action=get-blob or action=put-blob. For put-blob, add a Content-Type header. Actually, if you omit get-blob but the path lands on a blob, you will get a blob anyway. To be sure you always get JSON, use ?action=as-json, although you get weird JSON describing a blob. Blobs are stored in the database under the ‘magic’ attribute com.infinitydb.blob. If that attribute is not there immediately after the prefix you supplied, you get application/json.

Queries

For queries, add ?action=execute-query, ?action=execute-get-blob-query, or ?action=execute-put-blob-query. Queries use a two-part string interface name (like “com.infinitydb”) and string method name (possibly with spaces) instead of a full path, after https://domain/infinitydb/data/. Use the appropriate port and GET for execute-get-blob-query and POST otherwise. These require QUERY permission. QUERY permissions can allow access to a set of interfaces or their prefixes, plus control of setters/getters. Some put-blob queries look for information in the query string parameters, especially in a JSON parameter with the name ‘params’. Using queries is valuable in isolating the database structure and behavior from the API.

You can also do a HEAD just to test for connectivity. (Currently it returns text/plain, ignoring the path).

$ curl -u testUser:db -X GET 'https://infinitydb.com/infinitydb/data/demo/readonly/Documentation/"Basics"/description/\[0\]'
"This is a brief description of the concepts of an InfinityDB database."

JavaScript

For any JavaScript access, the main thing to take care of is the ‘underscore quoting’ that InfinityDB uses to store strongly-typed values inside strings. The strings can then be used as keys or values in an object. We provide the free open source example software to do the I/O and testing with nodejs using the axios and https modules which are CommonJS modules. There is also a small framework of classes to help. See the free open-source repository at https://github.com/infinitydb/infinitydb.git where you will find src/javascript/infinitydb_access.js.

That code also helps with parsing the Blob structure described above from JSON into raw bytes so you can embed images and other things into JSON. Below is the section that actually does the I/O for GET, and you can see how simple it is, but you will need IdbBlob from the full code. Below that is the HTML with AJAX, i.e. XHR interface. See the curl command above for more details on post and queries.

// This is part of main()...

	// Read all of the queries as application/json. This is good exercize as all types are used.
	const queryUri = idbEncodeUri(new IdbClass('Query'));
	// The entire documentation as application/json
	const documentationsUri = idbEncodeUri(new IdbClass('Documentation'));
	// One picture, as binary image/jpeg - 1.7MB, fast, compressed.
	const picturesPic0Uri = idbEncodeUri(new IdbClass('Pictures'),'pic0');

	const protocol = 'https';
//	const protocol = 'http';
	const host = 'infinitydb.com';
//	const host = 'localhost';
	const port = 443;
//	const db = '/demo/readonly';
	const db = '/demo/writeable';
	const uri = queryUri;

	const axios = require('axios');
	let https;
	let http;
	try {
		https = require('node:https');
		http = require('node:http');
	} catch (err) {
		console.error('https support is disabled!');
	}
	
	// Use the axios module for REST

	console.log(protocol + '://' + host + ':' + port + db + uri);
	try {
		axios.get(protocol + '://' + host + ':' + port + '/infinitydb/data' + db + uri, {
				auth: { username: 'testUser', password: 'db' },
				responseType: 'arraybuffer' 
			}
		).then(function(response) {
			const blob = new IdbBlob(response.data, response.headers['content-type']);
			console.log('From axios: ' + blob.contentType + ' blob length ' + blob.v.length);
			testReceivedData(blob);
		}).catch(function(error) {
			console.error(error);
		});
	} catch (e) {
		console.log(e);
	}

	// Use the https module	for REST

	const options = {
		hostname: host,
		path: '/infinitydb/data' + db + uri,
		port: port,
		method: 'GET',
		headers: {
			'Authorization': 'Basic ' + Buffer.from('testUser:db').toString('base64')
		}
	};

	// Remember to switch this to correspond to protocol above
	const req = https.request(options, (res) => {
//	const req = http.request(options, (res) => {
		// Return raw data.
		const dataChunks = [];

		res.on('data', (chunk) => {
			dataChunks.push(chunk);
		});
		res.on('end', () => {
			const fullData = Buffer.concat(dataChunks);
			const blob = new IdbBlob(fullData, res.headers['content-type']);
			console.log('From https: ' + blob.contentType + ' blob length ' + blob.v.length);
			testReceivedData(blob);
		});
	});
	req.on('error', (error) => {
		console.error(error);
	});
	req.end();

Here is how to do it in HTML. You can still use the quoters and other utilities from the nodejs version above.

<head></head>
<body>
	
<script>
function get() {
	var xhr = new XMLHttpRequest();
	var username = 'testUser';
	var password = 'db';
	
	var resultContainer = document.getElementById('result');
	xhr.onreadystatechange = function() {
	  if (xhr.readyState === 4) {
	    if (xhr.status === 200) {
	      // The request is complete and successful
	      var responseObject = JSON.parse(xhr.responseText);
	      // format with indention 2
	      var responseJson = JSON.stringify(responseObject, null, 2);
	      console.log(responseJson);
	      
	      var subtext = responseObject.Basics._description[0];
	      
	      // Set the content of the resultContainer element
//	      resultContainer.innerHTML = responseJson;
      	  resultContainer.innerHTML = subtext;
	    } else {
	      // Handle errors here
      	  resultContainer.innerHTML = 'Request failed with status:' + xhr.status;
	      console.error('Request failed with status:', xhr.status);
	    }
	  }
	};
	
	xhr.open('GET', 'https://infinitydb.com/infinitydb/data/demo/readonly/Documentation', true);
	
	// Add Basic Authentication header
	var credentials = username + ':' + password;
	var encodedCredentials = btoa(credentials);
	xhr.setRequestHeader('Authorization', 'Basic ' + encodedCredentials);
	
	xhr.send();
}
</script>

<form>
	<input type="button" onclick="get()" value="Load"></input></form>
	<pre>
		<div id="result" name="result"></div>
	</pre>
</form>	
</body>

Java

There are two cases for REST access in Java:

  1. You are running in a JVM that has access to infinitydb.jar, and you can use the com.infinitydb.remote.RestConnection class there to read and write REST JSON data into an ItemSpace. That is very convenient because you can leverage the rest of InfinityDB with ItemSpaces for many purposes. Also, the RemoteClientItemSpace can be used in this case to connect to servers using the ItemPacket protocol. Of course you can also use case 2 mixed together.
  2. You do not have InfinityDB in the JVM and you want the minimal code to plug into your application. As an example, below is the test code that runs in the Junit framework, and which contains invocations of the basic access functions within the test cases. The JsonElement, JsonObject, JsonList, and JsonValue classes provide a clean way to parse and format JSON, and then to work with it because they implement the familar java.util.Map interface. There are classes that represent the various data types, such as IdbClass, IdbAttribute and so on, plus a Blob class to hold a content type and byte array. You can do much of this directly without this helper code – just see the ‘command’ method. See above also, where we described the bash ‘curl’ command and how simple it is.

The free open-source code for this and the complete InfinityDBSimpleRestClient is in the repository at https://github.com/infinitydb/infinitydb.git .

// MIT License
//
// Copyright (c) 2023 Roger L. Deran
//
// Permission is hereby granted, free of charge, to any person obtaining a copy
// of this software and associated documentation files (the "Software"), to deal
// in the Software without restriction, including without limitation the rights
// to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
// copies of the Software, and to permit persons to whom the Software is
// furnished to do so, subject to the following conditions:
//
// The above copyright notice and this permission notice shall be included in
// all copies or substantial portions of the Software.
//
// THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
// IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
// FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
// AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
// LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
// OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
// THE SOFTWARE.

package com.infinitydb.simplerest;

import java.util.Date;
import java.util.List;

import org.junit.Assert;
import org.junit.Test;

/**
 * Test the InfinityDB REST access code for Java. See infinitydb.com,
 * and boilerbay.com.
 */
public class InfinityDBSimpleRestClientTest {
    
    static class Target {
        String host;
        String userName;
        String passWord;
        boolean isDisableSSLSecurity;
        Target(String host, String userName, String passWord, boolean isDisableSSLSecurity) {
            this.host = host;
            this.userName = userName;
            this.passWord = passWord;
            this.isDisableSSLSecurity = isDisableSSLSecurity;
        }
    }
    
    static final Target INFINITYDB = new Target(
            "https://infinitydb.com/infinitydb/data/demo/writeable", 
            "testUser", "db",
            false);
    static final Target LOCAL = new Target(
            "http://localhost/infinitydb/data/demo/writeable",
            "testUser", "db",
             false);
    static final Target TARGET = INFINITYDB;
    
    static InfinityDBSimpleRestClient idb;
    
    static InfinityDBSimpleRestClient getClient( ) {
        if (idb != null)
            return idb;
        
        System.out.println("host=" + TARGET.host);
        idb = new InfinityDBSimpleRestClient(TARGET.host);
        
        idb.setUserNameAndPassWord(TARGET.userName, TARGET.passWord);
        /*
         * Do this if you have an https server but the host name is not
         * verifiable due to lack of an installed X509 certificate there. See
         * the InfinityDB Server documentation at boilerbay.com. Installing a
         * certificate requires root access - look in
         * /data/infinitydb-home/key-store. Use certificate-tool.py to help you
         * either use letsEncrypt, self-signed, or external.
         * 
         * This is rather slow, because it breaks keep-alive. That should be
         * encouragement for getting the certificate in the server.
         */
        if (TARGET.isDisableSSLSecurity)
            idb.setDisableSSLSecurity(true);
        return idb;
    }
    
    // Dozens of queries return. They have many different component types
    @Test
    public void testGetAllQueries() throws Exception {
        testGet("get all queries", new IdbClass("Query"));
    }

    // A single string returns
    @Test
    public void testGetAStringFromTheDocumentation() throws Exception {
        testGet("get one line",
                new IdbClass("Documentation"),"Basics",
                new IdbAttribute("description"), new IdbIndex(0));
    }

    public void testGet(String msg, Object... item) throws Exception {
        InfinityDBSimpleRestClient idb = getClient();
        PrintTime t = new PrintTime();
        Blob response = null;
        for (int i = 0; i < 3; i++) {
            response = idb.get(item);
            t.printTime(msg, response.length());
        }
        String contentType = response.getContentType();
        Assert.assertEquals("application/json", contentType);
        String s = response.toString();
        // Throws if bad
        JsonElement root = new JsonParser(s).parse();
    }

    @Test
    public void testPutBlob() throws Exception {
        InfinityDBSimpleRestClient idb = getClient();
        PrintTime t = new PrintTime();

        for (int i = 0; i < 3; i++) {
            Blob request = new Blob("\"hello from Java\"", "application/json");
            idb.putBlob(request, new IdbClass("Trash"), new IdbClass("JavaDemo"),
                    new IdbClass("DemoPost"), new Date());
            t.printTime("put blob", request.length());
        }
    }

    @Test
    public void testExecute() throws Exception {
        testExecute("exec get image",
                new JsonObject(false, new IdbAttribute("name"), "pic0"),
                "examples", "Display Image", "application/json");
    }
    
    public void testExecute(String msg, JsonObject jsonObject, String ifc, String method,
            String expectedContentType) throws Exception {
        InfinityDBSimpleRestClient idb = getClient();

        PrintTime t = new PrintTime();
        Blob response = null;
        for (int i = 0; i < 3; i++) {
            Blob requestBlob = new Blob(jsonObject);
             response = idb.executeQuery(
                ifc, method, requestBlob, null);
            t.printTime(msg, response.length());
        }
        String contentType = response.getContentType();
        Assert.assertEquals(expectedContentType, contentType);
        checkJsonParsing(response);
    }
        
    @Test
    public void testExecuteGetBlobQueryImage() throws Exception {
        testExecuteGetBlobQuery("exec get blob image ",
                new Blob("{ \"_name\" : \"pic0\" }", "application/json"),
                "examples", "Display Image", "image/jpeg");
    }
    
    public void testExecuteGetBlobQuery(String msg, Blob requestBlob, String ifc, String method,
            String expectedContentType) throws Exception {
        InfinityDBSimpleRestClient idb = getClient();
        PrintTime t = new PrintTime();

        Blob response = null;
        for (int i = 0; i < 3; i++) {
             response = idb.executeGetBlobQuery(
                ifc, method, requestBlob, null);
            t.printTime(msg, response.length());
        }
        String contentType = response.getContentType();
        Assert.assertEquals(expectedContentType, contentType);
    }

    @Test
    public void testExecutePutBlobQuery() throws Exception {
        testExecutePutBlobQuery("exec put blob ",
                "examples", "Put blob by name",
                new Blob("I am a plain text blob"),
                new Blob("{ \"_name\" : \"my blob\" }", "application/json"),
                "application/json");
    }

    public void testExecutePutBlobQuery(String msg, 
            String ifc, String method,
            Blob requestBlob, 
            Blob paramsBlob,
            String expectedContentType) throws Exception {
        InfinityDBSimpleRestClient idb = getClient();
        PrintTime t = new PrintTime();

        Blob response = null;
        for (int i = 0; i < 3; i++) {
             response = idb.executePutBlobQuery(
                ifc, method, requestBlob, paramsBlob);
            t.printTime(msg, requestBlob.length());
        }
        String contentType = response.getContentType();
//        System.out.println("content type " + contentType);
        Assert.assertEquals(expectedContentType, contentType);
    }

    void checkJsonParsing(Blob response) {
        PrintTime t = new PrintTime();
        String s = response.toString();
        t.printTime("response.toString()", response.length());
        // System.out.println("Response: "+ s);

        JsonElement root = new JsonParser(s).parse();
        t.printTime("parse", response.length());

        String formatted = root.toString();
        //String formatted = root.toStringAsExtendedFormatIndented();
//        System.out.println("Formatted: " + formatted);
        t.printTime("root.toString()", formatted.length());

        // Do another round to see if it stays the same 
        JsonElement root2 = new JsonParser(formatted).parse();
//        String formatted2 = root2.toString();
//        System.out.println("Formatted2: " + formatted2);
        Assert.assertEquals(root, root2); 

        /*
         * You can print in the special 'extended JSON' format specific to
         * InfinityDB in which we avoid quoting keys as double-quoted strings. 
         * There is a display mode in the browser for it, and it is
         * quite relaxing. 
         */
        String formattedExtended = root.toStringAsExtendedFormat();
        t.printTime("toStringAsExtended", formattedExtended.length());
//            System.out.println("Extended Format: " + formattedExtended);
        JsonElement rootExtended = new JsonParser(formattedExtended, false).parse();
        t.printTime("parse ExtendedFormat", formattedExtended.length());
        
        // Do another round to see if it is consistent.
        String formattedExtended2 = rootExtended.toStringAsExtendedFormat();
//            System.out.println("Extended Format re-formatted: " + formattedExtended2);
        t.printTime("toStringAsExtended", formattedExtended2.length());

         Assert.assertEquals(root, rootExtended);
    }

    @Test
    public void testJson() {
        JsonElement e0 = new JsonObject(false, new IdbAttribute("att"), new Long(5), "hi");
        
        JsonElement o0 = new JsonObject();
        o0.put(new JsonValue("hi"), null);
        JsonElement o = new JsonObject();
        o.put(new JsonValue(new Long(5)), o0);
        JsonElement o1 = new JsonObject();
        o1.put(new JsonValue(new IdbAttribute("att")), o);
        Assert.assertEquals(e0, o1);
    }
    @Test
    public void testJsonCompacted() {
        JsonElement e0 = new JsonObject(true, new IdbAttribute("att"), new Long(5), "hi");

        JsonElement o = new JsonObject();
        o.put(new JsonValue(new Long(5)), new JsonValue("hi"));
        JsonElement o1 = new JsonObject();
        o1.put(new JsonValue(new IdbAttribute("att")), o);
        Assert.assertEquals(e0, o1);
    }
    @Test
    public void testJsonList() {
        JsonElement e0 = new JsonObject(false, new IdbAttribute("att"), new IdbIndex(0), "hi");
        
        JsonElement o0 = new JsonObject();
        o0.put(new JsonValue("hi"), null);
        JsonList o = new JsonList();
        o.add(o0);
        JsonElement o1 = new JsonObject();
        o1.put(new JsonValue(new IdbAttribute("att")), o);
        Assert.assertEquals(e0, o1);
    }
    @Test
    public void testJsonListCompacted() {
        JsonElement e0 = new JsonObject(true, new IdbAttribute("att"), new IdbIndex(0), "hi");

        JsonList o = new JsonList();
        o.add(new JsonValue("hi"));
        JsonElement o1 = new JsonObject();
        o1.put(new JsonValue(new IdbAttribute("att")), o);
        Assert.assertEquals(e0, o1);
        
    }
    
    @Test
    public void testParse() {
        JsonElement e0 = new JsonObject(true, new IdbAttribute("att"), "hi");
        e0.insert(true, new IdbClass("Ec"), new Long(5), new Double(5));
        JsonElement e1 = new JsonParser("{ \"_att\" : \"hi\", \"_Ec\" : { \"_5\" : \"_5.0\" }}").parse();
        Assert.assertEquals(e0, e1);
    }
    
    @Test
    public void testFlatten() throws Exception {
        JsonElement e0 = new JsonObject(true, new IdbAttribute("att"), "hi");
        e0.insert(true, new IdbClass("Ec"), new Long(5), new Double(5));
        List<List<Object>> items = e0.flattenToList();
        for (int i = 0; i < items.size(); i++)
//            System.out.println("items[i] " + items.get(i));
        Assert.assertEquals(items.size(), 2);
        JsonElement e1 = JsonElement.unflattenFromList(true, items);
        Assert.assertEquals(e0, e1);
    }

    static class PrintTime {
        long t = System.currentTimeMillis();
        
        void printTime(String msg, int len) {
            long t2 = System.currentTimeMillis();
            double time = (t2 - t) / 1e3;
            System.out.format("%30s time=%3.4f len=%6d speed=%5.3f KB/s\r\n", msg, time, len, len / time / 1e3);
            t = t2;
        }
    }
}