REST is a very simple industry-standard request-response protocol supported by InfinityDB Server. Each REST access targets a specific URL, providing a request content in JSON and receiving back a JSON response content, somewhat like HTTP. Also Blobs like images can be transferred. In InfinityDB Server, each such URL is normally handled by a specific PatternQuery, so accesses are like remote procedure calls. PatternQueries can be bypassed for READ or WRITE permissions. Here is a simple example with the syntax of the database in this case being i code (all queries and other data can be specified as JSON as well, even blobs):
Database Names
Above, the URL includes the database name ‘my/db’ which is always two words separated by a single slash, where the words are a letter followed by letters, digits, dot, dash, and underscore. The server can contain any number of databases, each being a single InfinityDB Embedded file or else a remoted database on another server. The server admin user can create and manage databases and remoting, as well as managing users, passwords, roles, grants and permissions. Not shown are the user name and password passed as base64 in the request (in the same way as an HTTP Basic Authentication, hence without hashing or encryption, but the server uses SSL for encryption and verification so this is not a problem).
Query Name
After the database name are two strings for the query name – the “interface name” and the “method name”. Queries are stored inside the database that they apply to, so a database describes its own API, thereby isolating client applications from the internal structure of the data. It is therefore easy for the database to be restructured, extended, simplified, combined, or secured without affecting client applications. API can be added by storing new queries, and new ‘parameters’ or ‘returned data’ can be added by adding a few Items to queries. To execute a query, its definition must be like Query "interface" "method name" query definition.
The first string in the query name is an ‘interface name’, which is like an internet domain name, and is generally kept constant for any group of queries. It is one or more dot-separated components, where each component is one or more lower-case letters, digits, or dash. It must start with a lower-case letter. One puts the components in the reverse order compared to internet domains, however, and by making them correspond to actual domains, it is possible for query authors to keep their interface names globally unique. By holding interface names constant, REST clients can hard-code them. Note that in a URL, the interface and method name are both double-quoted strings, and in some URLs these show up with the double-quotes becoming %22. Also, method names can contain any chars, often even spaces!
Permissions
The admin can set permissions for given roles based on interface names or prefixes of interface names.. Multiple interface permissions may be granted to any particular role for any particular database. The admin can also set permissions to require an ‘Option setter true’ or ‘Option getter true’ specified in the query for more control like the Item Query "my.domain" "my method" query Option getter true. These are independent of the REST ‘method’ such as GET, PUT, POST, or DELETE; queries use GET for getting blobs, otherwise POST. The admin may also grant permission to read and/or to write to a database, but these permissions are independent of query permissions. By default, queries can not be executed at all. See the PatternQuery Reference for more. An example permission might be read query:prefix=com.infinitydb.examples query:interface=com.infinitydb,getter. That role could do direct read access via REST or a user could browse the data but not modify it. Also, a REST or user could run queries with interface names starting with com.infinitydb.examples, and could run queries with exactly com.infinitydb but only those queries flagged as ‘getters’. A permission failure will return a 405 result code and a message.
Data
Content is either JSON or BLOBs or ‘Binary Long Objects’ according to the ‘Content-Type’ header parameter of the request or response, following the mime type naming rules of HTTP. For JSON the mimetype is ‘application/json’. For JSON, the ‘action’ URL parameter is ‘execute-query’, while for blobs, the action is “execute-get-blob-query” or “execute-put-blob-query”. Transfer rate is very high and efficient for blob data. The mimetype is stored associated with each blob in the database but not for JSON, since that is self-describing. JSON queries can receive data in the request content and return it in the response content for each call, but for putting blobs, only the binary blob data is in the content, so further ‘parameters’ must be in the URL as described below. For getting a blob, there is no way to also return some JSON along with it. (Currently we don’t support returning JSON containing one or more embedded blobs for Python clients because the blobs don’t get converted into long contiguous byte arrays, and transferring blobs as JSON is less efficient. So, you do multiple accesses, one per blob.)
Direct Access
If a user has read or write access to a database, then the database can also be accessed directly by a URL without any query being involved, and with no ‘action’ URL parameter. In that case, the part of the URL after the database name is translated directly into an Item prefix and all of the suffixes of that Item prefix are marshalled and transmitted in as the request content or out as the response content in JSON form. In case the Item suffixes in the database are formatted in a particular way, a URL can be used to get or put a BLOB also without a PatternQuery or by using the ‘action=get-blob’ or ‘action=put-blob’ URL query parameters.
An example URL to get some JSON is https://infinitydb.com:37411/infinitydb/data/demo/readonly/Documentation (user name ‘testUser’ password ‘db’, database ‘demo/readonly’, Item prefix ‘Documentation’). For an example BLOB see https://infinitydb.com:37411/infinitydb/data/demo/readonly/Pictures/%22pic2%22.
Blobs
InfinityDB stores BLOBs that are to be associated with a mimetype in this way (this is i-code but could be JSON, and actually this is tightly compressed binary inside the database file):
... com.infinitydb.blob {
com.infinitydb.blob.data [
Bytes(xx_xx_xx..._xx)
Bytes(xx_xx_xx..._xx)
]
com.infinitydb.blob.mimetype "text/plain"
}
Any unlimited length series of bytes can be represented by a list of short byte array components as above under the attribute com.infinitydb.blob.data. The byte arrays are all of length 1024 except the last, which is one to 1024 as necessary. This format is compatible with JSON and i-code, so actually blobs can be embedded in those text formats because the Bytes(..) is a standard InfinityDB Item component token format representing a byte array. In the database browser web pages, displaying in i-code or tabular mode converts such structures into proper visible form, such as images or printable text, but while editing i-code or JSON, the format above shows up, so you can cut and paste blobs as text. The database browser has a single-blob and multi-file blob upload feature as well, and for downloading you can click on the blob to make it full screen and save it.
Request and Response MetaData
The ‘params’ URL Parameter
If a put-blob query or in fact any query wants parameters beyond just the blob data and mime type, they can be passed in the URL as the ‘params’ parameter, in URL-quoted JSON form, and the query uses a root symbol of kind “params url parameter” to match on it. So, query pattern =params some pattern matching suffixes with query Where 'params' kind 'params url parameter' will do it. The URL is of form https://domain/...?action=action¶ms=urlquotedjson. The ‘params’ parameter can be combined with any others that may also be needed. If the request is a GET, then the params are parsed from the query string part of the URL, or if it is a POST and the Content-Type is ‘application/x-www-form-urlencoded’ then the params are parsed from that.
Getting the Entire Set of URL Parameters
Also, the query can look at the entire set of URL parameters individually with the ‘request parameters’ symbol kind, which is a root. In that case, the parameters take the form query pattern =reqparams RequestParameter name value value. with query Where 'reqparams' kind 'request parameters' and the names and values are just strings, not JSON. If it is a GET, the parameters are in the query string part of the URL, but if it is a POST, then if the Content-Type is ‘application/x-www-form-urlencoded’ then the parameters are parsed from that.
Headers
The query can look into the request headers with query pattern =req Header name value =value_symbol. with query Where 'req' kind 'request header' and the names and values are just strings, not JSON. Also a response header can be set with query result =resp Header name value value. with query Where 'resp' kind 'response header' . You can ask for a header to be removed with query result =resp Header name remove true. The value of the ‘Authorization’ request header is always '(hidden)'.
Python infinitydb.access Module
Python has special optional support for REST. The infinitydb.access module provides many features, such as helpers for formatting JSON-communicated data as dicts. There are classes for EntityClass and Attribute and so on. Also, the database can be navigated bidirectionally by Item and modified, and queries can be invoked. All of this is possible without the library, however. To install the module (don’t forget to keep pip up-to-date too):
python3 -m pip install --upgrade infinitydb
Here is some code using it:
from infinitydb.access import *
# port 37411 is usual
infinitydb_url = 'https://infinitydb.com:37411/infinitydb/data'
#infinitydb_url = 'http://localhost:37411/infinitydb/data'
""" Database names (URI components) """
# Databases have one slash separating two names each like
# [A-Za-z][A-Za-z0-9._-]*
# An infinitydb server by default has these available for the testUser
database_uri = 'demo/writeable'
#database_uri = 'demo/readonly'
""" The User Name and Password """
# A public guest user for browsing and experimentation, in the
# 'guest' role. Contact us for your own experimentation login and db.
user = 'testUser'
password = 'db'
""" The connection to the database """
infdb = InfinityDBAccessor(infinitydb_url, db=database_uri, user=user, password=password)
""" Get JSON given a prefix Item from the REST connection """
# This shows direct access to the db, but we prefer query-based access below
# To see the documentation graphically in the demo/readonly database, go to:
# https://infinitydb.com:37411/infinitydb/data/demo/readonly/Documentation?action=edit
# or without the action=edit to see it in JSON form.
# Here we read that JSON into content, with success being a boolean.
# The success only indicates that some data was read, not that there was an error.
# Real errors raise InfinityDBError
# The JSON is represented by nested dicts and lists.
# We use a path prefix of EntityClass('Documentation') which is an
# Item with a single initial class component:
success, content, content_type = infdb.get_json([EntityClass('Documentation')])
print(content)
# Launch a query on the server. There is no request header or response header
# This just copies and restructures a small amount of data on the db
def copy_aircraft():
success, content, response_content_type = infdb.execute_query(
['examples','Aircraft to AircraftModel'])
# The models are returned as a dict representing a set or None
# like { Model: {'100' : None, '200B' : None, ...} }
def get_aircraft_models(model):
data = { Attribute('aircraft'): model}
success, content, response_content_type = infdb.execute_query(
['examples','Get Aircraft Models'], data=data)
return success, content, response_content_type
# for blobs, we do a direct 'get blob' type of query. Very fast
def get_image(pic):
data = { Attribute('name'): pic }
success, content, response_content_type = infdb.execute_get_blob_query(
['examples','Display Image'], data=data)
return success, content, response_content_type
try:
print('copy aircraft')
copy_aircraft()
print('get aircraft models')
success, content, type = get_aircraft_models('747')
if success:
print(' ',content)
success, content, type = get_image('pic0')
if success:
print('retrieved image size=', len(content))
except InfinityDBError as e:
print('Could not access infdb ', e)
except Exception as e:
print('Could not access infdb ', e)
Direct REST withJSON
Each Item is just considered a path into JSON nested Objects, one key per component, with the final component being a terminal value or null. Of course the Index type is encoded as a list, but empty lists are pruned away.
Underscore Quoting
In case you access directly from JavaScript in nodejs or in AJAX in a web page or in bash’s curl command or elsewhere, note that the JSON has ‘underscore quoting’. This form is visible in the database browser in that display mode. The rules are simple. Since each JSON key must be a string, we ‘quote’ the 12 types by stuffing an underscore before each token. So to encode an attribute as a key, you use “_myattribute”. If you actually want a literal underscore, ‘stuff in’ an additional underscore there, like “__mystring”. This applies to the values as well except for doubles, longs, true/false and null.
We have to preserve the types because double, float, and long are stored distinctly. The parser will assume that all numbers are doubles, so the parser must be given ‘_5’ or ‘_5.0f’ to signal otherwise. The formatter does the inverse, but in JavaScript, all numbers are combined into one type. Hence we have to have the programmer signal the type from context to the parser.
Lists in InfinityDB are represented with special data types called ‘Index’, and these automatically convert into JSON lists and back, so there are no number conversion issues. An empty list cannot be represented, though, so you still have to check for a missing list.
Three Types of Numbers
InfinityDB numbers can be long (64-bits), IEEE-754 double (64 bits) or IEEE-754 float (32 bits). Those are separate in order to be compatible with all possible contexts such as Java, C, C++ and so on. The three are not perfectly interconvertible because double or float have exponents that are too large for a long and mantissas that are too short.
In the database, these can be stored mixed together, affecting the sorting because floats, doubles, and longs come out in that order when compared with each other (other DBMS cannot store dynamically typed data at all, let alone compare and store them as sets, as can InfinityDB with all types). If, for example, a terminal component always has only one value and is not effectively a ‘set’, there is no issue because different types will not be compared.
JavaScript numbers are all 64-bit IEEE-754 doubles so you cannot differentiate between a real number with an integer value and an actual integer. The only issue is that x.0 may become x so we define that all JavaScript numbers are doubles in the request content whether they have a decimal or not. Then you explicitly ‘underscore quote’ longs and floats as shown below. JavaScript does have a BigInt feature but it is not used.
You probably want to be consistent in choosing one numeric type for storage, but they are freely convertible in expressions, and you can set the type of a symbol to ‘number’, ‘long’, ‘double’, or ‘float’ or any subset to control the static strong type checking. In fact the type checking is a ‘combination’ of static and dynamic, since you can be specific or general in the static typing, yet types are converted dynamically as needed. So if you have a symbol with type of { ‘double’; ‘float’; } then the expressions are ensured at compile time to be compatible with those, yet a float is dynamically promoted at runtime if combined with a double. Any set of types can be declared as statically allowable for a given symbol but must agree with the values and parameters of expressions. A plain symbol (which is one that matches database data) declared with a subset of types will match only data having those types; therefore if new data arrives in the database with other types, it will not ‘break’ the query..
Nulls for Sets
Occasionally objects with null values will appear as the terminal values; for example, in cases where there are two or more Items that differ only in the terminal component. This is because there is no ‘set’ concept in JSON, so we use an Object with multiple keys for the elements of the set, with null values. This can be an issue because sometimes something you consider a set happens to have only one value in it or the reverse, so the regular value disappears and is replaced with an object with null values. You can use lists instead. Or, you can write queries that, for example, choose the first or last value or test for multiple values and error-out or limit the number of values or match only one value and so on.
For the general non-list case, you have to check for whether the terminal value you want is actually a regular value, a null or an Object and act appropriately. Within an ItemSpace, Items are just independent paths and there is no such issue. You can debug this kind of issue by just executing the query by hand with the database browser and looking at the request or response content, or look at the data in the ‘JSON Underscore Quoted’ view. It is possible for the database to contain Items that are prefixes of others, but these cannot be represented in JSON, and they normally do not matter, although other formats can work with them. Here is an example, where k4 is ‘set valued’:
{
"k1" : null,
"k2" : "value",
"k3" : null,
"k4" : {
"v" : null,
"v2" : null
}
}
The Python Module
The number situation in Python is easier because it differentiates int and float, but Python has no 32-bit real, so InfinityDB 32-bit floats become Python 64-bit ‘floats’. JSON is automatically converted to dicts.
There are module functions to convert between either ‘compacted tips’ or ‘uncompacted tips’ form of dicts. In the latter, all values are None and the nulls issue above does not arise, but in ‘compacted’ mode, the issue is still there. You can also use lists to avoid that problem.
There are functions to combine adjacent primitive components together into tuple keys and back, even allowing the tuples to have variable lengths. By ‘adjacent’ we mean one primitive component is a key and the other is a key or value inside the next level deeper.
The dicts may also include nested lists, but in case you need to see the Index components distinctly as keys, there are functions for that. Python dicts are not sorted (keys are hashed, and there is some kind of ‘ordered dict’ class but it doesn’t work properly) so lists are required for ordering, and magnitude comparisons of distinct types are not supported, unlike within InfinityDB. The dicts are automatically sorted once they reach InfinityDB. The module includes classes for EntityClass, Attribute, and Index which can be keys or values.
You can also deal with the JSON directly, as in JavaScript.
Bash Curl Command
Access from the unix/linux bash command can use curl. The Item prefix in the example here contains standard Item component tokens with slashes between. The Item Prefix goes after https://domain:37411/infinitydb/data/ or other port. In this example, the Documentation component is a class (starts with UC), the description token is an attribute (starts with LC) and the [0] token is a list index.
Direct
This example does direct read access, so it does GET and it needs READ permission, but to write you use POST and you need WRITE permission. For direct access to blobs, use ?action=get-blob or action=put-blob. For put-blob, add a Content-Type header. Actually, if you omit get-blob but the path lands on a blob, you will get a blob anyway. To be sure you always get JSON, use ?action=as-json, although you get weird JSON describing a blob. Blobs are stored in the database under the ‘magic’ attribute com.infinitydb.blob. If that attribute is not there immediately after the prefix you supplied, you get application/json.
Queries
For queries, add ?action=execute-query, ?action=execute-get-blob-query, or ?action=execute-put-blob-query. Queries use a two-part string interface name (like “com.infinitydb”) and string method name (possibly with spaces) instead of a full path, after https://domain:37411/infinitydb/data/. Use the appropriate port and GET for execute-get-blob-query and POST otherwise. These require QUERY permission. QUERY permissions can allow access to a set of interfaces or their prefixes, plus control of setters/getters. Some put-blob queries look for information in the query string parameters, especially in a JSON parameter with the name ‘params’. Using queries is valuable in isolating the database structure and behavior from the API.
You can also do a HEAD just to test for connectivity. (Currently it returns text/plain, ignoring the path).
$ curl -u testUser:db -X GET 'https://infinitydb.com:37411/infinitydb/data/demo/readonly/Documentation/"Basics"/description/\[0\]' "This is a brief description of the concepts of an InfinityDB database."
JavaScript Access
For any JavaScript access, the main thing to take care of is the ‘underscore quoting’ that InfinityDB uses to store strongly-typed values inside strings. The strings can then be used as keys or values in an object. Below is a quoter/unquoter to handle this. Here, numbers as values of an object are always real, even if the decimal is missing. To encode an int, use qInt(n) and for a float use qFloat(n) which output strings starting with underscore. Values can be booleans, strings, or numbers. Strings that are not intended to be underscore quoted can either be left as-is, or in the special case when they already start with underscore, an additional underscore is prepended. First is the nodejs version with the axios and https modules which are CommonJS modules, and below that is the HTML with AJAX, i.e. XHR interface. See the curl command above for more details on post and queries.
// MIT License
//
// Copyright (c) 2023 Roger L. Deran
//
// Permission is hereby granted, free of charge, to any person obtaining a copy
// of this software and associated documentation files (the "Software"), to deal
// in the Software without restriction, including without limitation the rights
// to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
// copies of the Software, and to permit persons to whom the Software is
// furnished to do so, subject to the following conditions:
//
// The above copyright notice and this permission notice shall be included in
// all copies or substantial portions of the Software.
//
// THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
// IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
// FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
// AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
// LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
// OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
// THE SOFTWARE.
// Instead of these, you can also write { _myAttribute : 5; }
class EntityClass {
constructor(name) {
this.name = name;
}
toString() {
return this.name;
}
}
class Attribute {
constructor(name) {
this.name = name;
}
toString() {
return this.name;
}
}
// Note we consider ArrayBuff to be UTF-8, not InfinityDB byte array!
// To get blobs, use action=get-blob, action=put-blob, action=execute-get-blob-query,
// or action=put-blob-query.
function qKey(o) {
if (o === null || o === undefined) {
throw new TypeError('keys must not be null');
} else if (typeof o === 'boolean') {
return o ? '_true' : '_false';
} else if (typeof o === 'number') {
// default is always double! You do not have to quote! You will forget this.
return qDouble(o);
} else if (typeof o === 'string') {
return o.charAt(0) !== '_' ? o : '_' + o;
} else if (o instanceof Date) {
return '_' + o.toISOString();
} else if (o instanceof EntityClass || o instanceof Attribute) {
return '_' + o.toString();
} else if (typeof o === 'object' && o.constructor === ArrayBuffer) {
// Convert ArrayBuffer to a string representation (works in both contexts)
const decoder = new TextDecoder('utf-8');
const decodedString = decoder.decode(o);
return decodedString.charAt(0) !== '_' ? decodedString : '_' + decodedString;
} else {
throw new TypeError('keys must be primitive or date');
}
}
// Only needed for keys, or just use qKey() everywhere
function qDouble(n) {
return '_' + toDoubleString(n);
}
function toDoubleString(n) {
return Number.isInteger(n) ? n + '.0' : n.toString();
}
function qFloat(n) {
return '_' + toFloatString(n);
}
function toFloatString(n) {
return Number.isInteger(n) ? n + '.0f' : n.toString() + 'f';
}
function qLong(n) {
return '_' + toLongString(n);
}
function toLongString(n) {
return '' + Math.floor(n);
}
// Note when the JSON is parsed at the InfinityDB end, all numbers
// are considered double, so you have to use qLong(n) and qFloat(n)
// instead to provide context and generate '_5' or '_5.0f'.
function qValue(o) {
if (o === null || o === undefined) {
return null;
} else if (typeof o === 'boolean' || typeof o === 'number') {
return o;
} else {
return qKey(o);
}
}
// Unquote an underscore-quoted value
// This works for keys or values
// Be careful with numbers: InfinityDB long and float become JS numbers
function uq(o) {
if (typeof o !== 'string')
return o;
if (o === '') {
return '';
} else if (o.charAt(0) !== '_') {
return o;
} else if (o.startsWith('__')) {
return o.slice(1);
}
// it is a string starting with a single _
o = o.slice(1);
if (o === 'null') {
return null;
} else if (o === 'true') {
return true;
} else if (o === 'false') {
return false;
} else if (!isNaN(Number(o))) {
return parseFloat(o);
} else if (o.endsWith('f') && !isNaN(Number(o.slice(0, -1)))) {
return parseFloat(o.slice(0, -1));
} else if (isValidIsoDate(o)) {
return new Date(o);
} else if (isValidEntityClass(o)) {
return new EntityClass(o);
} else if (isValidAttribute(o)) {
return new Attribute(o);
} else {
throw new TypeError('cannot underscore-unquote ' + o);
}
}
function isValidIsoDate(dateString) {
const isoDateRegex = /^(\d{4}-\d{2}-\d{2}T\d{2}:\d{2}:\d{2}(\.\d{1,3})?)(Z|[-+]\d{2}:\d{2})?$/;
return isoDateRegex.test(dateString);
}
function isValidEntityClass(s) {
const metaRegex = /^[A-Z][a-zA-Z\d\._]*$/;
return metaRegex.test(s);
}
function isValidAttribute(s) {
const metaRegex = /^[a-z][a-zA-Z\d\._]*$/;
return metaRegex.test(s);
}
function exampleCreateObject() {
const o = 'anything';
// any string at all
const s = '_s';
// classes start with UC, then any letters, digits, underscores or dots
// These are really optional - you can just add an underscore except for dots
const C = new EntityClass('C_a.b');
// classes start with LC, then any letters, digits, underscores or dots
const a = new Attribute('a_a.b');
const n = 5;
const object = {
// Quote any type, but all numbers become double!
[qKey(o)]: qValue(o),
// This forces purely ISO dates
[qKey(new Date())]: qValue(new Date()),
['_2023-09-17T21:24:35.653Z']: qValue(new Date('2023-09-17T21:24:35.653Z')),
['_2023-09-17T21:24:35.653-07:00']: qValue(new Date('2023-09-17T21:24:35.653-07:00')),
// Use qLong() to add the '_' to force long, which is a 64-bit integer
// on the InfinityDB side
_5: '_5',
[qLong(n)]: qLong(n),
// Use qFloat() to add the '_' and 'f' to force float on the InfinityDB side
_5f: '_5f',
'_5.0f': '_5.0f',
[qFloat(n)]: qFloat(n),
// Use the default double for numbers.
// You will forget that double is default, not long!!
'_5.0': 5,
'_5.1': 5.1,
'_5.1': '_5.1',
[qKey(n)]: n,
// Here x is a string on the InfinityDB side, not an attribute or class
'x': 'x',
// here _C is a class without the '_' on the InfinityDB side
'_C_.a': '_C_.a',
// Here _a is an attribute without the '_' on the InfinityDB side
'_a._a': '_a._a',
// here __s is a string starting with single quote '_' on the InfinityDB side
// We add an extra '_'
'__some string': '__some string',
// s is unknown string automatically underscore quoted if necessary
[qKey(s)]: qValue(s),
// Unknown class or attribute
[qKey(C)]: qValue(C),
[qKey(a)]: qValue(a),
[qKey(true)]: true,
[qKey(false)]: false
};
return object;
}
function exampleUnquoteObject(o) {
for (const key in o) {
const k = uq(key);
const v = uq(o[key]);
console.log(`Key: ${typeof k}:${k}, Value: ${typeof v}:${v}`);
}
}
const axios = require('axios');
let https;
try {
https = require('node:https');
} catch (err) {
console.error('https support is disabled!');
}
function main() {
// Demo the underscore-quoting functions
const object = exampleCreateObject();
console.log(JSON.stringify(object));
exampleUnquoteObject(object);
// Use the axios module
try {
axios.get('https://infinitydb.com:37411/infinitydb/data/demo/readonly/Documentation',
{ auth: { username: 'testUser', password: 'db' } }
).then(function(response) {
console.log('From axios: ' + response.data.Basics._description[0]);
}).catch(function(error) {
console.error(error);
});
} catch (e) {
console.log(e);
}
// Use the https module
const options = {
hostname: 'infinitydb.com',
path: '/infinitydb/data/demo/readonly/Documentation',
port: 37411,
method: 'GET',
headers: {
'Authorization': 'Basic ' + Buffer.from('testUser:db').toString('base64')
}
};
const req = https.request(options, (res) => {
let data = '';
res.on('data', (chunk) => {
data += chunk;
});
res.on('end', () => {
console.log('From https: ' + JSON.parse(data).Basics._description[0]);
});
});
req.on('error', (error) => {
console.error(error);
});
req.end();
}
main();
Here is how to do it in HTML. You should still use the quoters from the nodejs version above.
<head></head>
<body>
<script>
function get() {
var xhr = new XMLHttpRequest();
var username = 'testUser';
var password = 'db';
var resultContainer = document.getElementById('result');
xhr.onreadystatechange = function() {
if (xhr.readyState === 4) {
if (xhr.status === 200) {
// The request is complete and successful
var responseObject = JSON.parse(xhr.responseText);
// format with indention 2
var responseJson = JSON.stringify(responseObject, null, 2);
console.log(responseJson);
var subtext = responseObject.Basics._description[0];
// Set the content of the resultContainer element
// resultContainer.innerHTML = responseJson;
resultContainer.innerHTML = subtext;
} else {
// Handle errors here
resultContainer.innerHTML = 'Request failed with status:' + xhr.status;
console.error('Request failed with status:', xhr.status);
}
}
};
xhr.open('GET', 'https://infinitydb.com:37411/infinitydb/data/demo/readonly/Documentation', true);
// Add Basic Authentication header
var credentials = username + ':' + password;
var encodedCredentials = btoa(credentials);
xhr.setRequestHeader('Authorization', 'Basic ' + encodedCredentials);
xhr.send();
}
</script>
<form>
<input type="button" onclick="get()" value="Load"></input></form>
<pre>
<div id="result" name="result"></div>
</pre>
</form>
</body>
Java REST
There are two cases for REST access in Java:
- You are running in a JVM that has access to infinitydb.jar, and you can use the com.infinitydb.remote.RestConnection class there to read and write REST JSON data into an ItemSpace. That is very convenient because you can leverage the rest of InfinityDB with ItemSpaces for many purposes. Also, the RemoteClientItemSpace can be used in this case to connect to servers using the ItemPacket protocol.
- You do not have InfinityDB in the JVM and you want the minimal code to plug into your application. Below is a piece of code that does this. Below that is a demo of its use. It provides JSON parsing and formatting for the special ‘underscore quoted’ keys. The JsonElement, JsonObject, JsonList, and JsonValue classes provide a clean way to parse and format JSON, and then to work with it because they implement the familar java.util.Map interface. There are classes that represent the various data types, such as EntityClass, Attribute and so on, plus a Blob class to hold a content type and byte array. These names will conflict with those in the infinitydb.jar, so you have to watch the import statement. (Future: we will be putting this into a github repository.)
// MIT License
//
// Copyright (c) 2023 Roger L. Deran
//
// Permission is hereby granted, free of charge, to any person obtaining a copy
// of this software and associated documentation files (the "Software"), to deal
// in the Software without restriction, including without limitation the rights
// to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
// copies of the Software, and to permit persons to whom the Software is
// furnished to do so, subject to the following conditions:
//
// The above copyright notice and this permission notice shall be included in
// all copies or substantial portions of the Software.
//
// THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
// IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
// FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
// AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
// LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
// OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
// THE SOFTWARE.
package com.infinitydb.simplerest;
import java.io.CharArrayWriter;
import java.io.IOException;
import java.io.InputStream;
import java.io.OutputStream;
import java.io.UnsupportedEncodingException;
import java.io.Writer;
import java.net.HttpURLConnection;
import java.net.MalformedURLException;
import java.net.URL;
import java.net.URLEncoder;
import java.text.ParseException;
import java.text.SimpleDateFormat;
import java.util.AbstractMap.SimpleEntry;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.Base64;
import java.util.Collection;
import java.util.Date;
import java.util.HashMap;
import java.util.HashSet;
import java.util.Iterator;
import java.util.List;
import java.util.Map;
import java.util.Set;
import java.util.regex.Pattern;
public class InfinityDBSimpleRestClient {
// Not URL quoted, unlike the prefix.
final String host;
String userName;
String passWord;
public InfinityDBSimpleRestClient(String host) {
if (host.endsWith("/"))
host = host.substring(0, host.length() - 1);
this.host = host;
}
public void setUserNameAndPassWord(String userName, String passWord) {
this.userName = userName;
this.passWord = passWord;
}
/*
* Returns application/json if there is no Blob there, otherwise the Blob.
* The existence of the Blob is determined by whether there is a
* com.infinitydb.Blob attribute and its inner parts there.
*/
public Blob get(Object... prefix) throws IOException {
return command("GET", null, null, prefix);
}
/*
* Like get but insists on JSON, so a Blob becomes com.infinitydb.Blob etc.
* That allows you to get other things with suffixes
* on that prefix other than just the Blob's suffixes.
*/
public Blob getAsJson(Object... prefix) throws IOException {
return command("GET", "as-json", null, prefix);
}
public Blob post(Blob Blob, Object... prefix) throws IOException {
return command("POST", null, Blob, prefix);
}
public Blob executeQuery(String interfaceName, String methodName, Blob Blob)
throws IOException {
return command("POST", "execute-query", Blob,
new EntityClass("Query"), interfaceName, methodName);
}
public Blob executeGetBlobQuery(String interfaceName, String methodName)
throws IOException {
return command("GET", "execute-get-Blob-query", null,
new EntityClass("Query"), interfaceName, methodName);
}
public Blob executePutBlobQuery(String interfaceName, String methodName,
Blob Blob) throws IOException {
return command("POST", "execute-put-Blob-query", Blob,
new EntityClass("Query"), interfaceName, methodName);
}
private Blob command(String method, String action, Blob blob,
Object... prefix) throws IOException {
if (method.equalsIgnoreCase("GET") && blob != null)
throw new RuntimeException(
"Method GET is not compatible with sending a Blob");
String urlString = getQuotedUrl(host, prefix);
if (action != null)
urlString = urlString + "?action=" + action;
URL url = new URL(urlString);
HttpURLConnection urlConnection =
(HttpURLConnection)url.openConnection();
urlConnection.setRequestMethod(method);
urlConnection.setDoInput(true);
if (userName != null) {
String credentials = userName + ":" + passWord;
String encodedCredentials =
Base64.getEncoder().encodeToString(credentials.getBytes());
urlConnection.setRequestProperty("Authorization",
"Basic " + encodedCredentials);
}
if (method.equalsIgnoreCase("POST") && blob != null) {
urlConnection.setDoOutput(true);
}
urlConnection.connect();
// System.out.println("Connection: " +
// urlConnection.getHeaderField("Connection"));
if (method.equalsIgnoreCase("POST") && blob != null) {
OutputStream out = urlConnection.getOutputStream();
out.write(blob.data);
out.flush();
}
InputStream in = urlConnection.getInputStream();
long contentLength = urlConnection.getContentLengthLong();
String contentType = urlConnection.getContentType();
Blob readBlob = new Blob((int)contentLength, contentType);
readBlob.readFully(in);
in.close();
urlConnection.disconnect();
return readBlob;
}
/**
* This can quote any URL path, even if it has components like new
* EntityClass("Documentation"),"Basics",new Attribute("description"), new
* Index(0) which becomes
* (host)/demo/readonly/Documentation/%22Basics%22/description/%5B0%5D. It
* does the components separately, then concatenates with / properly.
*
* @param host
* can have any slashes which are not quoted, but the final slash
* is optional.
* @param components
* a varargs series of either Object or Object[]. Typically these
* will be EntityClass, Attribute, Index, numbers, booleans etc.
* for the 12 types.
* @throws UnsupportedEncodingException
* @throws MalformedURLException
*/
public static String getQuotedUrl(String host, Object... components)
throws UnsupportedEncodingException, MalformedURLException {
StringBuffer sb = new StringBuffer();
for (Object o : components) {
if (o instanceof Object[]) {
for (Object c : ((Object[])o)) {
if (c instanceof String)
c = JsonParser.convertToJsonString((String)c);
String component = c.toString();
String quoted = URLEncoder.encode(component, "UTF-8");
sb.append("/").append(quoted);
}
} else {
if (o instanceof String)
o = JsonParser.convertToJsonString((String)o);
String quoted = URLEncoder.encode(o.toString(), "UTF-8");
sb.append("/").append(quoted);
}
}
return host + sb.toString();
}
}
/**
* Data read and written to the REST API is in blob format, whether it is
* JSON or images or other binary.
*/
class Blob {
final byte[] data;
final String contentType;
Blob(byte[] data, String contentType) {
this.data = data;
this.contentType = contentType;
}
Blob(int dataLength, String contentType) {
this.data = new byte[dataLength];
this.contentType = contentType;
}
Blob(String data) throws UnsupportedEncodingException {
this.data = data.getBytes("UTF-8");
this.contentType = "text/plain";
}
Blob(String data, String contentType) throws UnsupportedEncodingException {
this.data = data.getBytes("UTF-8");
this.contentType = contentType;
}
Blob(String data, String contentType, String charSet) throws UnsupportedEncodingException {
this.data = data.getBytes(charSet);
this.contentType = contentType;
}
int length() {
return data.length;
}
String getContentType() {
return contentType;
}
void readFully(InputStream in) throws IOException {
int off = 0;
while (true) {
int bytesRead = in.read(data, off, data.length - off);
if (bytesRead == -1)
return;
off += bytesRead;
}
}
@Override
public String toString() {
try {
return new String(data, "UTF-8");
} catch (IOException e) {
throw new RuntimeException("Cannot parse UTF-8 string in a Blob buffer ",e);
}
}
}
/**
* The 12 data types have wrappers to make them easy to deal with. If you want
* to work with an attribute you can construct one and give it its name, then
* refer to it elsewhere.
*
* If you are running in a JVM that has infinitydb.jar available, then there are
* standard classes like these already available, so these are redundant, and
* you probably want to use the RestConnection class to parse into ItemSpaces
* anyway. The primitive wrappers like Long, String etc are used as-is.
*/
class Meta {
final String name;
Meta(String name) {
this.name = name;
}
@Override
public String toString() {
return name;
}
@Override
public boolean equals(Object other) {
if (!(other instanceof Meta))
return false;
return ((Meta)other).name.equals(name);
}
@Override
public int hashCode() {
return name.hashCode();
}
}
class EntityClass extends Meta {
static final Pattern CLASS_PATTERN =
Pattern.compile("^[A-Z][a-zA-Z0-9._-]*$");
EntityClass(String name) {
super(name);
if (!isValidName(name))
throw new RuntimeException(
"EntityClass must be UC, then letters,"
+ " digits, dot, dash, and underscore.");
}
static boolean isValidName(String s) {
return CLASS_PATTERN.matcher(s).matches();
}
}
class Attribute extends Meta {
static final Pattern ATTRIBUTE_PATTERN =
Pattern.compile("^[a-z][a-zA-Z0-9._-]*$");
Attribute(String name) {
super(name);
if (!isValidName(name))
throw new RuntimeException("Attribute must be LC, then letters,"
+ " digits, dot, dash, and underscore.");
}
static boolean isValidName(String s) {
return ATTRIBUTE_PATTERN.matcher(s).matches();
}
}
/**
* These are special Long's that show up in Items at the location
* where the JSON should have a list. For example, to index into
* a list with the URL path, you place an Index(...) into the path.
*/
class Index {
long index;
Index(long index) {
this.index = index;
}
long getIndex() {
return index;
}
Index(String s) {
if (!s.startsWith("[") || !s.endsWith("]"))
throw new RuntimeException(
"Cannot parse index component: " + s);
s = s.substring(1, s.length() - 1);
index = Long.parseLong(s);
}
// In an Item, list indexes look like this.
@Override
public String toString() {
return "[" + index + "]";
}
@Override
public boolean equals(Object other) {
if (!(other instanceof Index))
return false;
return ((Index)other).index == index;
}
@Override
public int hashCode() {
return Long.hashCode(index);
}
}
class ByteArray {
static final String HEX = "0123456789ABCDEF";
byte[] bytes;
ByteArray(byte[] bytes) {
this.bytes = bytes;
}
// Parse
ByteArray(String s) {
if (!s.startsWith("Bytes(") || !s.endsWith(")"))
throw new RuntimeException(
"Cannot parse ByteArray component: " + s);
s = s.substring("Bytes(".length(), s.length() - 1);
bytes = fromHexString(s);
}
@Override
public String toString() {
return "Bytes(" + toHexString(bytes) + ")";
}
public static byte[] fromHexString(String s) {
byte[] bytes = new byte[(s.length() + 1) / 3];
int j = 0;
for (int i = 0; i < s.length();) {
char top = s.charAt(i++);
char bottom = s.charAt(i++);
// Skip underscore
i++;
int iTop = hexDigitParse(top);
int iBottom = hexDigitParse(bottom);
byte b = (byte)((iTop << 4) + (iBottom & 0xf));
bytes[j++] = b;
}
return bytes;
}
private static int hexDigitParse(char c) {
return c >= '0' && c <= '9' ? c - '0'
: c >= 'A' && c <= 'Z' ? c - 'A' : '_';
}
public static String toHexString(byte[] bytes) {
StringBuffer sb = new StringBuffer();
boolean isFirst = true;
for (int i = 0; i < bytes.length; i++) {
if (!isFirst)
sb.append("_");
isFirst = false;
char top = HEX.charAt((bytes[i] >> 4) & 0xf);
char bottom = HEX.charAt(bytes[i] & 0xf);
sb.append(top).append(bottom);
}
return sb.toString();
}
@Override
public boolean equals(Object other) {
if (!(other instanceof ByteArray))
return false;
return Arrays.equals(((ByteArray)other).bytes, bytes);
}
@Override
public int hashCode() {
return Arrays.hashCode(bytes);
}
}
// Rare. This is a ByteArray that sorts like a string.
class ByteString {
private byte[] bytes;
ByteString(byte[] bytes) {
this.bytes = bytes;
}
ByteString(String s) {
if (!s.startsWith("ByteString(") || !s.endsWith(")"))
throw new RuntimeException(
"Cannot parse ByteString component: " + s);
s = s.substring("ByteString(".length(), s.length() - 1);
bytes = ByteArray.fromHexString(s);
}
@Override
public String toString() {
return "ByteString(" + ByteArray.toHexString(bytes) + ")";
}
@Override
public boolean equals(Object other) {
if (!(other instanceof ByteString))
return false;
return Arrays.equals(((ByteString)other).bytes, bytes);
}
@Override
public int hashCode() {
return Arrays.hashCode(bytes);
}
}
// Not used for Blobs, but Clobs, which are rare
class CharArray {
final char[] chars;
CharArray(char[] chars) {
this.chars = chars;
}
// double-quotes are optional on the ends of the chars string.
// We will take off the underscore first in the JSONParser.
CharArray(String chars) {
if (!chars.startsWith("Chars(") || !chars.endsWith(")"))
throw new RuntimeException(
"Cannot parse Chars component: " + chars);
chars = chars.substring("Chars(".length(), chars.length() - 1);
String unescaped = JsonParser.unescapeJsonString(chars);
this.chars = unescaped.toCharArray();
}
@Override
public String toString() {
return "Chars(" + JsonParser.convertToJsonString(chars) + ")";
}
@Override
public boolean equals(Object other) {
if (!(other instanceof CharArray))
return false;
return Arrays.equals(((CharArray)other).chars, chars);
}
@Override
public int hashCode() {
return Arrays.hashCode(chars);
}
}
/**
* This a special parser for InfinityDB 'underscore quoted' JSON. It creates a
* tree of JsonElements with JsonObject, JsonList, and JsonValue subclasses,
* which are easy to work with because they implement java.util.Map.
*
* We need to encapsulate the 12 data types into JSON keys, which are always
* strings, so we use a trick: an initial underscore says we have a non-string
* type. If you actually want a string after all, it stays unchanged except that
* if it already has an underscore, we 'stuff' an additional one at front, and
* that is removed later during parsing.
*
* To make a nicer-looking format, we can also turn off the underscore quoting
* and leave all of the data types in plain form as keys and values, but it is
* of course not parseable as standard JSON.
*/
class JsonParser {
final List<String> jsonTokens;
int pos;
private static final SimpleDateFormat ISO_SIMPLE_DATE_FORMAT =
new SimpleDateFormat("yyyy-MM-dd'T'HH:mm:ss.SSSXXX");
private static final Pattern ISO_DATE_PATTERN = Pattern.compile(
"\\d{4}-\\d{2}-\\d{2}T\\d{2}:\\d{2}:\\d{2}(\\.\\d+)?(Z|[+-]\\d{2}:\\d{2})");
static class JsonTokenizer {
final String s;
int pos;
boolean isUnderscoreQuoting;
JsonTokenizer(String s) {
this.s = s;
}
// Turn off the default underscoreQuoting to get the
// special 'extended' JSON format that is simpler to read.
// It does not go on the wire though.
JsonTokenizer(String s, boolean isUnderscoreQuoting) {
this.s = s;
this.isUnderscoreQuoting = isUnderscoreQuoting;
}
// An initial pass that locates the tokens
List<String> tokenize() {
List<String> tokens = new ArrayList<>();
whiteSpaces();
int beforePos = 0;
while (jsonToken()) {
tokens.add(s.substring(beforePos, pos));
whiteSpaces();
beforePos = pos;
}
return tokens;
}
/*
* The isoDate is for extended JSON in InfinityDB as a key. Match it
* first, because looks like it is separate tokens due to the contained
* ':'.
*/
boolean jsonToken() {
return isoDate()
|| stringToken()
|| charSet("{}[],:")
|| longToken();
}
boolean any() {
if (pos < s.length()) {
pos++;
return true;
}
return false;
}
boolean longToken() {
if (!longTokenChar())
return false;
while (longTokenChar()) {
}
return true;
}
// The fact that we consider ':' a separator is tricky in the case of
// keys being dates in the Extended InfinityDB JSON format.
boolean longTokenChar() {
return !lookaheadWhiteSpace() && !lookaheadCharSet(",:[]{}") && any();
}
// Match at least one
boolean whiteSpaces() {
if (!lookaheadWhiteSpace())
return false;
while (lookaheadWhiteSpace()) {
if (!any())
return true;
}
return true;
}
// Doesn't move
boolean lookaheadWhiteSpace() {
return pos < s.length() && s.charAt(pos) <= ' ';
}
boolean lookaheadCharSet(String charSet) {
int beforePos = pos;
boolean matched = charSet(charSet);
pos = beforePos;
return matched;
}
boolean charSet(String charSet) {
if (pos >= s.length())
return false;
for (int i = 0; i < charSet.length(); i++) {
if (charSet.charAt(i) == s.charAt(pos)) {
pos++;
return true;
}
}
return false;
}
boolean digit() {
if (pos < s.length() && s.charAt(pos) >= '0' && s.charAt(pos) <= '9') {
pos++;
return true;
}
return false;
}
// Move forwards only if at least n digits are matched
boolean digits(int n) {
int beforePos = pos;
if (!digit())
return false;
// start at 1
for (int i = 1; i < n; i++) {
if (!digit()) {
pos = beforePos;
return false;
}
}
return true;
}
boolean match(char c) {
if (pos < s.length() && s.charAt(pos) == c) {
pos++;
return true;
}
return false;
}
boolean match(String s) {
if (this.s.substring(pos, this.s.length()).startsWith(s)) {
pos += s.length();
return true;
}
return false;
}
// like match but don't move
boolean lookahead(char c) {
return pos < s.length() && s.charAt(pos) == c;
}
// like match but don't move
boolean lookahead(String s) {
return this.s.substring(pos, this.s.length()).startsWith(s);
}
// We don't use regex because that is probably slow, but
// also it wants to have a $ so it matches completely,
// yet we are not giving it a pre-terminated string.
boolean isoDate() {
int beforePos = pos;
if (digits(4) && match('-')
&& digits(2) && match('-')
&& digits(2) && match('T')
&& digits(2) && match(':')
&& digits(2) && match(':')
&& digits(2)
&& millis() && timeZone())
return true;
pos = beforePos;
return false;
}
boolean millis() {
if (match('.'))
return digits(3);
return true;
}
boolean timeZone() {
if (match('-') || match('+'))
return digits(2) && match(':') && digits(2);
return match('Z');
}
boolean stringToken() {
if (!match('"'))
return false;
while (!match('"')) {
if (!quotedChar())
return false;
}
return true;
}
boolean quotedChar() {
if (match('\\'))
any();
return any();
}
}
boolean isUnderscoreQuoting = true;
public JsonParser(String json) {
this.jsonTokens = new JsonTokenizer(json).tokenize();
}
public JsonParser(String json, boolean isUnderscoreQuoting) {
this.isUnderscoreQuoting = isUnderscoreQuoting;
this.jsonTokens = new JsonTokenizer(json, isUnderscoreQuoting).tokenize();
}
/*
* Parsing is easy once we have tokenized it. We create a tree of
* JsonElement, the superclass of JsonObject, JsonList, and JsonValue. They
* all implement java.util.Map, so they are easy to deal with.
*/
JsonElement parse() {
if (pos >= jsonTokens.size())
throw new RuntimeException(
"JSON string terminated without complete parse");
if (match("{")) {
JsonObject jsonObject = new JsonObject();
// Remove the '_' from the start of the keys and convert to the
// 12 types
while (true) {
Object key = jsonTokens.get(pos);
key = unQuote(key, isUnderscoreQuoting);
if (key == null)
throw new RuntimeException("Unparseable token in JSON: "
+ jsonTokens.get(pos));
pos++;
if (!match(":"))
throw new RuntimeException("Expected ':' in JSON");
JsonElement value = parse();
jsonObject.put(new JsonValue(key), value);
if (match(","))
continue;
if (match("}"))
break;
throw new RuntimeException("Missing } in JSON");
}
return jsonObject;
} else if (match("[")) {
JsonList jsonList = new JsonList();
while (true) {
JsonElement o = parse();
jsonList.add(o);
if (match(","))
continue;
if (match("]"))
break;
throw new RuntimeException("Missing ] in JSON");
}
return jsonList;
} else {
String t = jsonTokens.get(pos);
pos++;
Object o = unQuote(t, isUnderscoreQuoting);
return new JsonValue(o);
}
}
boolean match(String s) {
if (pos < jsonTokens.size() && jsonTokens.get(pos).equals(s)) {
pos++;
return true;
}
return false;
}
public static String unescapeJsonString(String jsonString) {
// Remove surrounding double quotes
if (jsonString.startsWith("\"") && jsonString.endsWith("\"")) {
jsonString = jsonString.substring(1, jsonString.length() - 1);
}
StringBuilder unescapedString = new StringBuilder();
boolean isEscaped = false;
for (int i = 0; i < jsonString.length(); i++) {
char c = jsonString.charAt(i);
if (isEscaped) {
switch (c) {
case '\"' :
unescapedString.append('\"');
break;
case '\\' :
unescapedString.append('\\');
break;
case '/' :
unescapedString.append('/');
break;
case 'b' :
unescapedString.append('\b');
break;
case 'f' :
unescapedString.append('\f');
break;
case 'n' :
unescapedString.append('\n');
break;
case 'r' :
unescapedString.append('\r');
break;
case 't' :
unescapedString.append('\t');
break;
case 'u' :
// Unicode escape sequence
if (i + 4 < jsonString.length()) {
String unicodeHex =
jsonString.substring(i + 1, i + 5);
try {
int unicodeValue =
Integer.parseInt(unicodeHex, 16);
unescapedString.append((char)unicodeValue);
i += 4; // Skip the 4 hexadecimal digits
} catch (NumberFormatException e) {
// Invalid Unicode escape sequence; treat as a
// literal 'u'
unescapedString.append('u');
}
} else {
// Not enough characters for a valid Unicode escape;
// treat as a literal 'u'
unescapedString.append('u');
}
break;
default :
// Invalid escape sequence; treat as a literal character
unescapedString.append('\\').append(c);
break;
}
isEscaped = false;
} else if (c == '\\') {
isEscaped = true;
} else {
unescapedString.append(c);
}
}
return unescapedString.toString();
}
public static String convertToJsonString(String s) {
return convertToJsonString(s.toCharArray());
}
public static String convertToJsonString(char[] chars) {
StringBuilder jsonValue = new StringBuilder();
for (char c : chars) {
switch (c) {
case '\"' :
jsonValue.append("\\\"");
break;
case '\\' :
jsonValue.append("\\\\");
break;
case '\b' :
jsonValue.append("\\b");
break;
case '\f' :
jsonValue.append("\\f");
break;
case '\n' :
jsonValue.append("\\n");
break;
case '\r' :
jsonValue.append("\\r");
break;
case '\t' :
jsonValue.append("\\t");
break;
default :
if (c < ' ' || c > '~') {
// Unicode escape sequence
String unicodeHex = String.format("\\u%04x", (int)c);
jsonValue.append(unicodeHex);
} else {
jsonValue.append(c);
}
break;
}
}
return "\"" + jsonValue.toString() + "\"";
}
/*
* For quoting a JSON object's key. We convert all types into a string with
* initial '_', stuffing another if needed for strings. However, for
* the special 'extended' non-standard JSON format, we can omit the
* quotes around keys.
*/
public static String qKey(Object o, boolean isUnderscoreQuoting) {
String q = isUnderscoreQuoting ? qKeyUnderscoreQuoting(o) : qKeyExtended(o);
return q;
}
// Standard JSON string keys, but with underlines at front.
public static String qKeyUnderscoreQuoting(Object o) {
if (o == null) {
throw new RuntimeException(
"Cannot have a null JSON key in underscore-quoting");
} else if (o instanceof Long) {
return "_" + o;
} else if (o instanceof Boolean) {
return o == Boolean.TRUE ? "_true" : "_false";
} else if (o instanceof Double) {
String n = o.toString();
return "_" + (n.contains(".") ? n : n + ".0");
} else if (o instanceof Float) {
String n = o.toString();
// We add the 'f' but Java does not.
return "_" + (n.contains(".") ? n : n + ".0") + "f";
} else if (o instanceof String) {
String s = (String)o;
// 'Stuff' an extra '_' if one is already there
return s.startsWith("_") ? "_" + s : s;
} else if (o instanceof Date) {
return "_" + ISO_SIMPLE_DATE_FORMAT.format((Date)o);
} else if (o instanceof byte[]) {
return new ByteArray((byte[])o).toString();
} else if (o instanceof char[]) {
return new CharArray((char[])o).toString();
} else if (o instanceof EntityClass
|| o instanceof Attribute
|| o instanceof Index
|| o instanceof ByteArray
|| o instanceof CharArray
|| o instanceof ByteString) {
return "_" + o;
}
throw new RuntimeException("Unrecognized type: cannot "
+ "convert to JSON key or value: " + o);
}
// Our special 'extended' JSON format which minimizes quotes in keys
public static String qKeyExtended(Object o) {
if (o == null) {
throw new RuntimeException(
"Cannot have a null JSON key in underscore-quoting");
} else if (o instanceof Long) {
return o.toString();
} else if (o instanceof Float) {
String n = o.toString();
// We use the decimal point and trailing 'f' to indicate floats
// We add the 'f' but Java does not.
return "" + (n.contains(".") ? n : n + ".0") + "f";
} else if (o instanceof Double) {
// We use the decimal point to indicate doubles
String n = o.toString();
return "" + (n.contains(".") ? n : n + ".0");
} else if (o instanceof String) {
return JsonParser.convertToJsonString((String)o);
} else if (o instanceof Date) {
return ISO_SIMPLE_DATE_FORMAT.format((Date)o);
} else if (o instanceof byte[]) {
return new ByteArray((byte[])o).toString();
} else if (o instanceof char[]) {
return new CharArray((char[])o).toString();
} else {
return o.toString();
}
}
// For quoting a JSON object's value.
public static Object qValue(Object o, boolean isUnderscoreQuoting) {
if (o == null || o instanceof Double || o instanceof Boolean) {
return o;
} else {
return qKey(o, isUnderscoreQuoting);
}
}
// This works for key or value.
public static Object unQuote(Object o, boolean isUnderscoreQuoting) {
try {
if (!(o instanceof String))
return o;
String s = (String)o;
if (s.length() == 0)
return "";
else if (s.equals("null"))
return null;
else if (s.equals("true"))
return true;
else if (s.equals("false"))
return false;
if (!isUnderscoreQuoting) {
// The special 'extended' format where keys may not need double quotes.
if (s.startsWith("\"")) {
s = unescapeJsonString(s);
return s;
}
} else {
s = unescapeJsonString(s);
if (s.length() == 0)
return "\"\"";
if (s.charAt(0) != '_') {
// NOTE a key should never be null
return s.equals("null") ? null : s;
} else if (s.startsWith("__")) {
return s.substring(1);
}
// it is a string starting with a single _
s = s.substring(1);
if (s.length() == 0) {
throw new RuntimeException(
"Cannot underscore-unquote a lone underscore");
}
}
if (s.equals("null")) {
return null;
} else if (s.equals("true")) {
return true;
} else if (s.equals("false")) {
return false;
} else if (s.charAt(0) >= '0' && s.charAt(0) <= '9') {
if (ISO_DATE_PATTERN.matcher(s).matches()) {
return ISO_SIMPLE_DATE_FORMAT.parse(s);
} else if (!s.contains(".")) {
return Long.parseLong(s);
} else if (s.endsWith("f")) {
return Float.parseFloat(s.substring(0, s.length() - 1));
} else {
return Double.parseDouble(s);
}
} else if (EntityClass.isValidName(s)) {
return new EntityClass(s);
} else if (Attribute.isValidName(s)) {
return new Attribute(s);
} else if (s.startsWith("Bytes(")) {
return new ByteArray(s);
} else if (s.startsWith("ByteString(")) {
return new ByteString(s);
} else if (s.startsWith("Chars(")) {
return new CharArray(s);
}
// Can't handle byte[], char[]
throw new RuntimeException(
"Cannot underscore-unquote string: " + s);
} catch (ParseException e) {
throw new RuntimeException(e);
}
}
}
/**
* The output of parsing is a tree of these. There are three subclasses, for
* objects, lists, and values. They are easy to scan and construct because they
* implement Map, and therefore have many features.
*/
abstract class JsonElement implements
Map<JsonElement, JsonElement>, Iterable<JsonElement> {
public abstract Object value();
public abstract boolean isList();
public abstract boolean isObject();
public abstract boolean isValue();
// Can't iterate as a list by default. You can iterate a JsonList as
// if
// it were a JsonObject though, because the keys are Longs!
public Iterator<JsonElement> listIterator() {
return new Iterator<JsonElement>() {
@Override
public boolean hasNext() {
return false;
}
@Override
public JsonElement next() {
return null;
}
};
}
// Return underscore-quoted JSON, which is parseable as standard JSON
@Override
public String toString() {
try {
CharArrayWriter writer = new CharArrayWriter();
writeJson(writer, true, 0);
return writer.toString();
} catch (IOException e) {
throw new RuntimeException(
"Cannot do toString() on JsonElement", e);
}
}
public String toStringAsExtendedFormat() {
try {
CharArrayWriter writer = new CharArrayWriter();
writeJson(writer, false, 0);
return writer.toString();
} catch (IOException e) {
throw new RuntimeException(
"Cannot do toString() on JsonElement", e);
}
}
// Write as JSON underscore-quoted or not
public void writeJson(Writer writer, boolean isUnderscoreQuoting)
throws IOException {
writeJson(writer, isUnderscoreQuoting, 0);
}
// This loops over both objects and lists the same, because they are
// both Maps. For the lists, the key is the index.
void writeJson(Writer writer, boolean useUnderscoreQuoting, int depth) throws IOException {
if (isValue()) {
writer.write(underscoreQuote(useUnderscoreQuoting));
return;
}
boolean isFirst = true;
writer.write(isList() ? "[\r\n" : "{\r\n");
for (JsonElement v : this) {
JsonElement e = get(v);
if (!isFirst)
writer.write(",\r\n");
isFirst = false;
writer.write(indent(depth + 1));
if (isObject())
writer.write(v.underscoreQuote(useUnderscoreQuoting) + ": ");
if (e != null)
e.writeJson(writer, useUnderscoreQuoting, depth + 1);
}
writer.write("\r\n" + indent(depth) + (isList() ? "]" : "}"));
}
public static String indent(int depth) {
StringBuffer sb = new StringBuffer();
for (int i = 0; i < depth; i++)
sb.append(" ");
return sb.toString();
}
String underscoreQuote(boolean isUnderscoreQuoting) {
Object q = value();
if (q == null)
return "null";
q = isValue() ? JsonParser.qValue(value(), isUnderscoreQuoting)
: JsonParser.qKey(value(), isUnderscoreQuoting);
if (isUnderscoreQuoting && q instanceof String)
q = JsonParser.convertToJsonString((String)q);
return q.toString();
}
}
class JsonObject extends JsonElement {
private final Map<JsonElement, JsonElement> map = new HashMap<>();
@Override
public Set<JsonElement> keySet() {
return map.keySet();
}
@Override
public Iterator<JsonElement> iterator() {
return map.keySet().iterator();
}
@Override
public JsonElement get(Object key) {
return map.get(key);
}
/**
* The keys are JsonValues holding classes like EntityClass,
* Attribute, Long, String
*
* @param key
* any JsonValue
* @param value
* any JsonElement
*/
public JsonObject put(JsonValue key, JsonElement value) {
map.put(key, value);
return this;
}
public JsonObject putClass(String key, JsonElement value) {
map.put(new JsonValue(new EntityClass(key)), value);
return this;
}
public JsonObject putAttribute(String key, JsonElement value) {
map.put(new JsonValue(new Attribute(key)), value);
return this;
}
@Override
public Object value() {
if (map.size() == 0) {
return null;
}
if (map.size() == 1) {
for (Object o : map.keySet()) {
return o;
}
}
throw new RuntimeException("get value of JSON object: " + map);
}
@Override
public boolean isList() {
return false;
}
@Override
public boolean isObject() {
return true;
}
@Override
public boolean isValue() {
return false;
}
@Override
public boolean equals(Object other) {
if (!(other instanceof JsonObject))
return false;
return map.equals(((JsonObject)other).map);
}
@Override
public int hashCode() {
return map.hashCode();
}
@Override
public Set<Entry<JsonElement, JsonElement>> entrySet() {
return map.entrySet();
}
@Override
public int size() {
// TODO Auto-generated method stub
return 0;
}
@Override
public boolean isEmpty() {
return map.isEmpty();
}
@Override
public boolean containsKey(Object key) {
return map.containsKey(key);
}
@Override
public boolean containsValue(Object value) {
return map.containsValue(value);
}
@Override
public JsonElement put(JsonElement key, JsonElement value) {
return map.put(key, value);
}
@Override
public JsonElement remove(Object key) {
return map.remove(key);
}
@Override
public void putAll(
Map< ? extends JsonElement, ? extends JsonElement> m) {
map.putAll(m);
}
@Override
public void clear() {
map.clear();
}
@Override
public Collection<JsonElement> values() {
return map.values();
}
}
/*
* A JsonList does not implement java.util.List. It is a simulated map. You
* can't implement both Map and List. You can use the the Maps' get(Object)
* instead of the List get(int) so you won't even notice. The iterator returns
* a JsonValue(Index), which can be unwrapped to a long if needed or used directly with
* get().
*/
class JsonList extends JsonElement {
private final List<JsonElement> list = new ArrayList<>();
/*
* Implement this as if it were a set Note this is a bit expensive: we
* construct a new HashSet. Hopefully, this is rare, and you use the
* iterator and get(), which make the key an index into the list.
*/
@Override
public Set<JsonElement> keySet() {
return new HashSet<JsonElement>(list);
}
// Iterate it like a pseudo-map
@Override
public Iterator<JsonElement> iterator() {
return new Iterator<JsonElement>() {
int pos;
@Override
public boolean hasNext() {
return pos < list.size();
}
@Override
public JsonElement next() {
return new JsonValue(new Index(pos++));
}
};
}
// NOTE we round double and float, expecting them to in fact be integer.
@Override
public JsonElement get(Object key) {
if (key instanceof JsonValue)
key = ((JsonValue)key).value();
if (key instanceof Index) {
long index = ((Index)key).getIndex();
return list.get((int)index);
} else if (key instanceof Number) {
int index = ((Number)key).intValue();
return list.get(index);
}
throw new RuntimeException("JsonList: bad type for get(): " + key);
}
/**
* The element can be a JsonValue with a class like String, Long,
* EntityClass, Attribute or similar, or else it can be any other
* JsonElement for a possibly nested arrangement. Also, we
* wrap a 'bare' object like EntityClass or Long in a JsonValue to add it.
*/
public JsonList add(JsonElement e) {
if (e instanceof JsonElement)
list.add(e);
else
list.add(new JsonValue(e));
return this;
}
@Override
public Object value() {
return null;
}
@Override
public boolean isList() {
return true;
}
@Override
public boolean isObject() {
return false;
}
@Override
public boolean isValue() {
return false;
}
@Override
public boolean equals(Object other) {
if (!(other instanceof JsonList))
return false;
return list.equals(((JsonList)other).list);
}
@Override
public int hashCode() {
return list.hashCode();
}
// We might not even want this.
@Override
public Set<Entry<JsonElement, JsonElement>> entrySet() {
Set<Entry<JsonElement, JsonElement>> set = new HashSet<>();
for (int i = 0; i < list.size(); i++)
set.add(new SimpleEntry(new JsonValue(new Long(i)),
list.get(i)));
return set;
}
@Override
public int size() {
return list.size();
}
@Override
public boolean isEmpty() {
return list.isEmpty();
}
@Override
public boolean containsKey(Object key) {
if (key instanceof JsonValue)
key = ((JsonValue)key).value();
if (!(key instanceof Long))
return false;
return list.size() > ((Long)key).intValue();
}
@Override
public boolean containsValue(Object value) {
if (!(value instanceof JsonValue))
value = new JsonValue(value);
return list.contains(value);
}
@Override
public JsonElement put(JsonElement key, JsonElement value) {
throw new RuntimeException(
"cannot put a key into a JsonList: " + key);
}
@Override
public JsonElement remove(Object key) {
throw new RuntimeException(
"cannot remove a key from a JsonList: " + key);
}
@Override
public void putAll(
Map< ? extends JsonElement, ? extends JsonElement> m) {
throw new RuntimeException("cannot user putAll() of a map int a JsonList");
}
@Override
public void clear() {
list.clear();
}
@Override
public Collection<JsonElement> values() {
return list;
}
}
/**
* A single value such as an EntityClass, Attribute, ByteArray or the primitive
* wrappers like Long, Date and so on. Strings are escaped according to
* JavaScript rules. This can look like a Map with one entry, so
* while iterating, you don't need to worry about single values being
* handled differently. Everything looks like a Map, but these just
* have no 'contents' so get() returns null.
*/
class JsonValue extends JsonElement {
// Not a JsonElement such as a JsonValue. Tentatively nullable.
private final Object v;
/**
* @param v may be a JsonValue or any type of the 12 types.
*/
JsonValue(Object v) {
this.v = v instanceof JsonValue ? ((JsonValue)v).value() : v;
if (this.v instanceof JsonElement) {
throw new RuntimeException("A JsonValue cannot be constructed from a"
+ " value that is another JsonElement except for a JsonValue");
}
}
// Make this look like a singleton set. Contains JsonValues.
@Override
public Set<JsonElement> keySet() {
Set<JsonElement> set = new HashSet<>(1);
set.add(this);
return set;
}
// Iterate a single value as a single-element Map.
@Override
public Iterator<JsonElement> iterator() {
List<JsonElement> list = new ArrayList<>(1);
list.add(this);
return list.iterator();
}
// Iterate as a list with a single value.
@Override
public Iterator<JsonElement> listIterator() {
return iterator();
}
// We have nothing 'inside'.
@Override
public JsonElement get(Object o) {
return null;
}
// Not a JsonElement such as a JsonValue
@Override
public Object value() {
return v;
}
@Override
public boolean isList() {
return false;
}
@Override
public boolean isObject() {
return false;
}
@Override
public boolean isValue() {
return true;
}
@Override
public boolean equals(Object other) {
if (!(other instanceof JsonValue))
return false;
JsonValue otherJson = (JsonValue)other;
return v == null ? otherJson.v == null :
v.equals(otherJson.v);
}
@Override
public int hashCode() {
return v == null ? 42 : v.hashCode();
}
@Override
public Set<Entry<JsonElement, JsonElement>> entrySet() {
Set<Entry<JsonElement, JsonElement>> set = new HashSet<>(1);
set.add(new SimpleEntry(v, null));
return set;
}
@Override
public int size() {
return 1;
}
@Override
public boolean isEmpty() {
return false;
}
@Override
public boolean containsKey(Object key) {
return v == null ? key == null : v.equals(key);
}
@Override
public boolean containsValue(Object value) {
return false;
}
@Override
public JsonElement put(JsonElement key, JsonElement value) {
throw new RuntimeException("Cannot put a key/value into a JsonValue");
}
@Override
public JsonElement remove(Object key) {
throw new RuntimeException("Cannot remove a key/value from a JsonValue");
}
@Override
public void putAll(Map< ? extends JsonElement, ? extends JsonElement> m) {
throw new RuntimeException("Cannot putAll into a JsonValue");
}
@Override
public void clear() {
throw new RuntimeException("Cannot clear a JsonValue");
}
// We have no values, considering this in the Map sense.
@Override
public Collection<JsonElement> values() {
Set<JsonElement> set = new HashSet<>(1);
return set;
}
}
// MIT License
//
// Copyright (c) 2023 Roger L. Deran
//
// Permission is hereby granted, free of charge, to any person obtaining a copy
// of this software and associated documentation files (the "Software"), to deal
// in the Software without restriction, including without limitation the rights
// to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
// copies of the Software, and to permit persons to whom the Software is
// furnished to do so, subject to the following conditions:
//
// The above copyright notice and this permission notice shall be included in
// all copies or substantial portions of the Software.
//
// THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
// IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
// FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
// AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
// LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
// OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
// THE SOFTWARE.
package com.infinitydb.simplerest;
import java.io.CharArrayWriter;
/**
* This is in the boilerbay.com REST Access page for Direct Java.
*
* There is a JSON parser/formatter and a Json structure with subclasses that
* make access easy.
*/
public class InfinityDBSimpleRestClientDemo {
public static void main(String[] args) throws Exception {
try {
// String host =
// "https://infinitydb.com:37411/infinitydb/data/demo/readonly";
String host =
"http://localhost:37411/infinitydb/data/demo/writeable";
System.out.println("host=" + host);
InfinityDBSimpleRestClient idb = new InfinityDBSimpleRestClient(host);
idb.setUserNameAndPassWord("testUser", "db");
Blob response = null;
for (int i = 0; i < 10; i++) {
// response = idb.get(new EntityClass("Documentation"),"Basics",new
// Attribute("description"), new Index(0));
//response = idb.get(new EntityClass("Documentation"));
// response = idb.get(new EntityClass("Documentation"),"Basics");
// A binary Blob - very fast but only one at a time.
// response = idb.get(new EntityClass("Pictures"),"pic0");
// All pictures in JSON
// response = idb.get(new EntityClass("Pictures"));
// List of ByteArray
// response = idb.get(new EntityClass("Pictures"),"pic0",new
// Attribute("com.infinitydb.Blob"), new
// Attribute("com.infinitydb.Blob.bytes"));
// All queries
response = idb.get(new EntityClass("Query"));
// Read the text of the query - not executing it.
// response = idb.get(new EntityClass("Query"),"examples","Aircraft
// to AircraftModel");
// response = idb.get(new EntityClass("Query"),"examples","fish farm
// profit");
// The entire DB: TODO XXXX BROKEN - never returns due to
// transaction retrying!.
// response = idb.get();
printTime("IO", response.length());
}
String contentType = response.getContentType();
if (!contentType.equals("application/json"))
return;
String s = response.toString();
printTime("response.toString()", response.length());
// System.out.println("Response: "+ s);
JsonElement root = new JsonParser(s).parse();
printTime("parse", response.length());
String formatted = root.toString();
// System.out.println("Formatted: " + formatted);
printTime("root.toString()", formatted.length());
// Do another round to see if it stays the same
JsonElement root2 = new JsonParser(formatted).parse();
String formatted2 = root2.toString();
// System.out.println("Formatted2: " + formatted2);
// Test: should be equal.
System.out.println("Consistent format and parse back: " + root.equals(root2));
if (!root.equals(root2))
throw new RuntimeException("JsonElement trees differ.");
/*
* You can print in the special 'extended JSON' format specific to
* InfinityDB in which we avoid quoting keys as double-quoted strings.
* There is a display mode in the browser for it, and it is
* quite relaxing.
*/
String formattedExtended = root.toStringAsExtendedFormat();
printTime("toStringAsExtendedFormat()", formattedExtended.length());
// System.out.println("Extended Format: " + formattedExtended);
JsonElement rootExtended = new JsonParser(formattedExtended, false).parse();
printTime("parse ExtendedFormat", formattedExtended.length());
// Do another round to see if it is consistent.
String formattedExtended2 = rootExtended.toStringAsExtendedFormat();
// System.out.println("Extended Format re-formatted: " + formattedExtended2);
printTime("toStringAsExtendedFormat() again", formattedExtended2.length());
// Test: should be equal.
System.out.println("Consistent format and parse back in Extended format: "
+ root.equals(rootExtended));
if (!root.equals(rootExtended))
throw new RuntimeException("JsonElement trees differ in extended mode.");
} catch (Exception e) {
e.printStackTrace();
throw e;
}
}
static long t = System.currentTimeMillis();
static void printTime(String msg, int len) {
long t2 = System.currentTimeMillis();
double time = (t2 - t) / 1e3;
System.out.println(msg + " time=" + time
+ " len=" + len
+ " speed=" + len / time);
t = t2;
}
}