The REALITY X Database

Reality X is a database management system developed, owned and supplied by Northgate Information Solutions. Many Northgate software products run on Reality X database systems, including the Health applications. It is a version of the Reality operating system which runs as an application under UNIX or Windows NT operating systems.

Reality is a multi-dimensional Post-relational database. Most relational databases have tables consisting of columns and rows. Each row has a number of columns and each column in the row contains an item of data so the table is two-dimensional.

Reality has FILES instead of tables, ITEMS instead of rows and ATTRIBUTES (or LINES) instead of columns. In a Reality database an attribute can contain a number of pieces of data which can contain separate values, (called MULTIVALUES) each of which can contain SUBVALUES making it multi-dimensional. Unlike traditional databases where each column of each row has a fixed length and data type, Reality items are variable length and un-typed. There are no fixed lengths or types – different items in the same file can be different sizes; they can have different numbers of fields and the fields in the items can vary in length from item to item. This makes the Reality database uniquely flexible and also allows complex database applications to be created extremely quickly.

Structure of Reality databases

DATABASE ELEMENT/FILE	CONTAINS
SYSTEM	Pointers to location of each Account’s MD on disk Q-Pointer account definitions Pointers to accounts on other databases
ACCOUNT MD	Pointers to DICT sections, Commands, PROCs (especially Logon PROCs) Global Data Dictionary items Q-Pointers to files in other Accounts
DICT section	Pointer(s) to data section(s) Data dictionary items
Data section	Data and/or Programs

The structure of a REALITY database

In the above example, ACCOUNT1 has three files, FILE1, FILE2 and FILE3. ACCOUNT2 has two files, FILE1 and FILE2 but FILE2 has two data sections, FILE2,SECT1 and FILE2,SECT2.

Each user logs into a Reality database using a USER-ID and password. This can be controlled by the underlying UNIX or NT operating system or by the Reality database itself.

When a user logs on to a Reality system, they will, depending on how their user-id has been configured, either go straight into a default account or will be prompted for an account to log into. When a user has logged into an account they will be able to access all the files contained in that account. In addition, special links called Q-POINTERS can be set up in the MD of that account which will allow them to access files in other accounts or databases.

Q-Pointers can also be set up in the SYSTEM file to create what is known as a SYNONYM LOGIN which allows users to log in to an account under a different account name.

The MD of an account will often contain an item called a LOGIN PROC. This is a program which will be run whenever a user logs in usually it will give the user a menu which allows programs to be run.

If there is no login PROC or the user can break out of the program then they will be dropped to the Terminal Control Level known as TCL. TCL can be recognised by its : (colon) prompt. It is a command line interpreter which allows users to run any command contained in the account’s MD. These commands contain system commands (such as the editor and the ENGLISH enquiry language) and other programs written by users of the system.

A user can also disconnect cleanly from the database at this point by typing ‘OFF’. (Most menus will also contain the OFF command).

Reality Files

A file can contain data items.An item consists of a number of lines (attributes) of data. Unlike other databases where the size and data type of each field in a table has to be specified beforehand, Reality files and items can be any size limited only by available diskspace.

In a file, there can be any number of items, each of which can be any size. Each item has an ITEM-ID (sometimes known as the KEY) which is, essentially the name of the item. The item-id for each item ina file must be unique - you cannot have two items in one file with the same id. When an item in a file is displayed it will look something like this:-

(depending on the command used to display the item). The item in this display has an item-id of “402-20”. Attributes 6,7,8,9 and 11 are multivalued – they have multiple values separated by the multivalue marker – ASCII character 253 – visible as a “]” character.

File sizing and ‘Hashing’

The diskspace used by a database is separated into 1 Kbyte ‘chunks’ called frames. When a new file is created, a file size has to be specified, in frames. For instance the following command:-

CREATE-FILE TESTFILE 1 5

would create a file called TESTFILE with a Dictionary section which takes up 1 frame of diskspace and a Data section which takes up 5 frames of diskspace. These frames are contiguous - located next to each other on disk. The system maintains a table of which frames on the disk are not allocated to files. Each frame in the file contains pointers to the frames immediately preceding and following it. Sometimes, when a system crashes, these pointers or the free-space table can be corrupted resulting in what is called a Group Format Error or GFE.

When a new item is created in the file, the system uses a piece of software called the HASHING ALGORITHM to determine which frame of the file the item will be placed in. It does this by performing a calculation which uses the file size and the item-id of the item to be added. If an application needs to access that item later, the system will use the hashing algorithm to locate the item in the file without having to search the whole file, thus speeding up disk access times for the file.

The following example shows a 5 frame file where most of the items have been ‘hashed’ into the first frame:-

If a frame fills up with data, and the hashing algorithm decides that more data needs to be added to it, a new frame located elsewhere on the disk will be linked to it. This is called an overflow frame.

Every item which is hashed into frame 1 will be added to the end of the linked frame. If the linked frame fills up, another frame will be linked to it anddata will be placed in that. This means that a file can grow to fit the amount of data that needs to be stored in it.

However, files with lots of overflow are much slower to access than files with no overflow. This is because, when the system wants to access an item which has beenhashed into a frame with lots of overflow frames linked to it, it needs to load that frame and all the frames linked to it into memory. Also, as the overflow frames are not contiguous, they could be scattered across the disk and this will increase the amount of disk activity needed to read the file.

If a file does go into overflow it can be resized to the correct size.

Dictionary Attributes / ENGLISH enquiry language

The dictionary portions of files hold ‘Dictionary attributes’ which define the data held in the data portion of the file.  Dictionary attributes describe the fields of records in the data section.  Each attribute relates to a specific field and holds information concerning the format of the data in that field.  Dictionary attributes can also be used to process the data in field - performing calculations with it, converting it to different formats or even using the data as the item-id to read fields in other data items in the same or other files.

These dictionary attributes are used by the ENGLISH enquiry language, allowing the user to

            ENGLISH is similar to other query languages such as SQL in that complex reports can be generated with simple English-like commands.  For example, the following ENGLISH statement:-

Produces this report

Where NAME, ARR.DATE, ARR.TIME and CONS.ON.DUTY are all Dictionary Attributes which have been set up for the file ACC-VISITS.  Much more complicated reports can be created just as simply and output can be directed either to the screen or a print job.  Screen output is ‘paged’ – when a full screen of information has been displayed, the system will wait for a keypress before clearing the screen and displaying another page.

Database Programming Languages

There are two main programming languages, PROC and DATA/BASIC. PROC is a simple interpreted procedural language, often used as a pre-processor before running DATA/BASIC programs. A section of a typical PROC can be seen below:-

DATA/BASIC is a compiled programming language, originally derived from Dartmouth BASIC. Most applications for the REALITY database are written in DATA/BASIC. DATA/BASIC combines the ease of use of traditional BASIC with the data access facilities of the Reality database to allow the quick development of complex and sophisticated database applications. A segment of a typical DATA/BASIC program can be seen below.

Both PROC and DATA/BASIC can use embedded ENGLISH statements to select data from a file or produce reports.

There are a few other languages available for Reality - RPL (an extended version of PROC) and ALL which are used by some Northgate customers but most Reality applications are written in a combination of PROC and DATA/BASIC.

In addition to the traditional Reality-based languages, applications for the database can now be developed in C and Visual Basic, the latter allowing a Windows GUI front end for Reality programs rather than the traditional character based screens.

History of Reality

Reality was originally an operating system in its own right - the Reality system software controlled the computer and all associated devices (disks, tapes, comms connections etc.). The Reality computers (such as the Reality Royale, Sequoia and Series19) were all manufactured by Northgate Information Systems (as one of its previous incarnations - CMC, Microdata, McDonnell Douglas or MDIS).

Reality is one of a number of databases modelled on the PICK data model. In 1967, the American defence contractor TRW employed a number of Systems Architects to design an operating system for defence applications. This system was unique in that it was designed as a virtual machine with its own integral device and memory management routines before the hardware on which was to run has been determined. It came complete with its own database enquiry language - GIRLS Generalised Information Retrieval Language System - and was eventually implemented on an IBM mainframe.

One of the key System Architects, Richard (Dick) Pick later left TRW and began to develop the virtual machine idea with his own company, PICK systems Inc. A number of other companies soon brought out their own PICK-like systems including a British company, CMC - The Computer Machinery Company. Their system, called REALITY was developed in conjunction with Pick and was very similar to PICK with an extended version of GIRLS called ENGLISH.

In the late 1970s Dick Pick sued CMC, who had since changed their name to MICRODATA, in the U.S. courts for breach of intellectual copyright and unpaid royalties. After a long-running battle Pick won but was awarded the lowest amount possible in a case of this kind - $1 in royalties. The court awarded Microdata a full licence to develop and distribute the system.

Soon after this Microdata was bought out by the U.S. Aerospace manufacturer McDonnell Douglas who were intending to expand into computer manufacturing.

Since this time a number of other database systems closely resembling PICK have appeared on the market, including UniVerse, Unidata, Sequoia, AddsMentor, Advanced Revelation (ARev) jBASE and Advanced PICK (now being marketed as D3). These databases are all very similar to Reality and PICK, containing common features such as the ENGLISH query language (usually known as ACCESS), PROC and DATA/BASIC all contained within the traditional PICK file structure.

The McDonnell Douglas corporation ran into financial difficulties in the late 1980s and decided to sell its computer business which was eventually the subject of a management buyout in 1993. The new company, MDIS had its headquarters at Hemel Hempstead, like CMC and Microdata before it. Reality computers were manufactured at Hemel Hempstead up until 1993 but the company had already decided to develop a version of Reality which would be able to run on other operating systems - originally UNIX but now Windows NT as well. MDIS was renamed Northgate Information Systems in 2000.

The drive for open systems has led to greater integration of Reality with other operating systems, using standard tools such as SQL, etc.

Reality has a number of advantages over traditional relational databases:-

• Most other databases for large scale applications need very large, powerful systems - Reality will run on a much smaller (and consequently cheaper) system, generally with better performance.
• Applications can be developed very quickly compared to other DBMSs.
• Reports can be quickly generated using ENGLISH by users with minimal training.
• database design, is easier, quicker and more flexible due to the unique ‘multidimensional’ structure of Reality files.
• RealityX Data/Basic subroutines can now be called directly from Visual Basic programs, allowing sophisticated Windows or Web-Based applications to run with an underlying RealityX database