Appendix B : Concepts & Glossary

List of Concepts and Terms

Throughout this manual you will find the following symbol *. If you are reading this in the html Reader clicking on this item will take you immediately to the index.

* Section 1 : File

A file is the entity which includes a set of related data to be used by the DATAdesign engine. When we use the word file in this manual, we don't mean a file as in 'a file on disk', we mean a file which is used by the DATAdesign engine. The conventional usage for file won't be used a lot in this manual, and the conventional usage will be called a medium-file.

A file may have two distinct status. It may be either :

On disk
- disk-based
In memory
- memory-based

When a medium-file is not used by any job in memory (so there are no buffers using the file (see later)), then this medium-file will not be called a file. Files are referenced by (hopefully unique) filenames. These filenames are case-dependant.

* Section 2 : Buffer

A buffer is an entry-point to a file. It contains a copy of the current record. These buffers can be locked or given a read only status if required. If the buffer is not a read-only buffer then the record will be locked, that is, unavailable to all other buffers using this file.

* Section 3 : Record

Records are parts of a file which combine related data. If you go to a library and you want to find a book, you search the register, which is a database. In those libraries where you still have to find them manually, there will be a place where you can find a card for each book. Each of these cards is a record, and all the cards together are the file.

* Section 4 : Field

Fields are subdivisions of records. These subdivisions are available in all records. To use the example of the index at the library, the fields are the subdivisions of the cards, such as : author, title, publisher,.... Fields which are created in one record are then present in all other records in that particular file.

* Section 5 : Field Type

In the DATAdesign engine, fields are of well defined types. Five SuperBasic types are provided and these types should allow you to put any kind of data you want into a field.

It is up to the creator of the database to determine what a certain value in a field is supposed to mean.

* * 5.1 : raw data

type
: raw
description
: graphics / fonts .....
symbol
: none

These cannot be accessed in the DATAdesign main program

* 5.2 : char

type
: character
description
: text
symbol
: $ or none

Any field not given a leading symbol when added to a DATAdesign file is treated as a character field.

* 5.3 : short

type
: short
description
: short integers from -32768 to 32767 , selections and statuses
symbol
: %

* 5.4 : long

type
: long
description
: long integers from -2147483648 to 2147483647, dates etc
symbol
: &

* 5.5 : floating point

type
: floating point
description
: any numerical value, ieee doubles
symbol
: `

This is the field type that should be used if the number is of uncertain values or will contain a decimal point.

* Section 6 : Initialisation

Versions of the DATAdesign Engine prior to the release of ProWesS required that the engine be initiated with the ENGINE_INIT command. The current version is initiated as soon as it is loaded although the command has been retained in case the engine is removed.

If you remove the DATAdesign files job, then the engine and all the data will be removed. You should never do this. For memory protection reasons, when you delete the DATAdesign files job, all jobs which use the DATAdesign engine will also be removed. Should this happen the engine can be restored by ENGINE_INIT .

* Section 7: File Status

This is a special property which each file has. It indicates whether a file is disk-based or memory-based. By default all files are memory based when you create them. But that can be changed. There is no automatic switching between the two status.

* 7.1 : Disk Based

This file-status is included for two reasons.

First it will allow you to use very long files, even files which are much longer than memory will permit.

There is, however, always an index with references to the place which has to fit in memory (This shouldn't be a problem, every record only takes 18 bytes in this index).

Second this is the safest way you can work on a file. Even if a system-crash occurs for some reason, then a maximum of one record will be lost.

There are a few commands which are actually not that safe, but these will be mentioned later

When a file is disk based and a change to a record makes that record grow a bit, then it would have to be moved to the end of the file as it would not fit in the medium-file at the old place. To prevent this from happening too often, you can make sure that there is always a bit of empty space after each record. This space is called the inter-record-space. This is not relevant when a file is memory-based.

Even when a large inter-record-space is set (This is not advisable as it can waste a lot of disk space. We advice small values [e.g.10] are used) it may not be always be enough. That way large empty gaps will be created in the file. These gaps can be removed with garbage collection

* Section 8 : Inter-record-space

This defines the gap which is left between records on disk. This gap is used to overcome small variations in record size when you change something in a disk-based file. It is important that this is controlled since these small gaps may prevent some very large gaps from occurring in your medium-file. The Status of the inter-record-space is maintained by the use of Garbage Collection

Whitespace in a medium-file can be created because a record has grown a bit and doesn't fit in the file any more at the old place. The record will then be moved to the end of the file, thus leaving an empty gap. This empty gap will not be re-used unless the record which was just before it (In the physical medium file, not in the file as accessed with Next or Previous) grows a lot.

* 8.1Garbage Collection

This command should occasionally be used if your file is disk-based. It actually clears up all unwanted (ie. whitespace / inter-record-space) whitespace in the medium-file. If the inter-record-space is too small, you should collect garbage more often. Please take care, you only collect garbage after making a backup of your file. If a power failure should occur during garbage collection, then your file may be lost.

* Section 9 : Types of medium-files

DATAdesign always attempts to load or save three medium files. All these files should have the same name and be on the same device, the only difference is the extension (and the contents of course). The actual file (which has to be present) which contains all the records has the '_ddf' extension, and is the file which the engine always saves, loads or backs up. Then there is also a file with a '_ddm' extension which contains more information about the size of the window and especially which fields should be visible and which not. The programs also attempts to read an index file with the extension '_ddi'. This file contains all the records which are visible (filtered), and the order (sorted). If other programs have accessed the file they should also update this index file or you may have some missing records.

When you think that the index may not be up to date, you can rebuild it with a 'do' in Sort or Filter (whichever is valid) in the Commands Menu

* Section 10 : Filenames

The file name and device are normally stored separately in the DATAdesign engine. For this reason, we have to split the filename and device when loading. We do this by first removing any three letter extension that the name may include (if there is one), and then taking everything from the end till the last underscore ('_') as the filename, and the rest at the device. This does however cause problems in two cases. Filenames which are exactly three characters long should always be typed with an extension (e.g. '_ddf'). The other problem is that when the same file is accessed by another job which allows underscores in the filename will not be treated as the same file. This can cause problems with database integrity.

When there is another job which is accessing a file with the same name as the one you are trying to load, then this job will access the same file, which may not be the file you wanted. For this reason, it is vital that you, as a user, choose different names for all files. Do note that filenames are compared case dependant, so 'address' and 'Address' are treated as two different files.

* Section 11 : Mark Status

A mark status is a status which is file-specific (so it is the same for all buffers which have access to that file). It can have a value between zero and 255. If the mark status is zero we say it is cleared.

The mark status of a record can be used to make selections which can't be defined in filter. You could in fact consider it as an extra field which can contain at most one (small) number. The mark status is permanently visible in the Top Window after the F10 item and can be changed by a 'HIT' or 'DO' on that item. There is also a Clear All Mark command in the Commands Menu.

* Section 12 : RecordID

Each record in DATAdesign is given a unique number as soon as it is incorporated into the file and this RecordID can never change. The numbers are issued sequentially as the records are added and are vital to some of the workings of DATAdesign and its ancillary programs. If you delete a record from Datadesign the RecordID is removed from the list but they are never re-allocated so you can always be certain which record you are looking at.

The Application Programming Interface and the PFdata printing program both make extensive use of these RecordIDs

* Section 13 : 'Sorted' files

When a file has been 'Sorted' by issuing the Sort File command from the Commands Menu the 'Next Record' and 'Previous Record' icons in the Icon Bar will use the index rather than the recordIDs to select the files.

When you start putting records into a new file, the next record will always be an empty one until you have issued a sort command.

However, when you are modifying a sorted file, the next record will be the record which appears in the index just after the one you just entered.

In this case it may be more useful to 'HIT' the The New Record Icon or type [CONTROl-K] to get a blank record.


This document is mostly based on the original text for the DATAdesign Manual, revised and converted into html format by Roy Wood
PROGS, Professional & Graphical Software
last edited Saturday 24 August 1996