An awkward awker

The logical standard form of data handled by Awk: tables

Many applications (though not all) of the language Awk assume that the input data has such a logical structure as to be called a table. The concept of the table is the same as in the theory of "relational database". It is not accidental. The Awk language evolved together with the Unix operating system, and Unix uses relational databases for storing information necessary for its management.

A table is rectangular, in the sense that it consists of cells arranged as rows and columns, every column has the same number of rows, and every row has the same number of columns. It is not permitted to partially merge several rows into one. It may be a severe restriction. But, textbooks of relational database tell that re-ordering the information into a set of multiple tables with the "normalized" structure makes the data less prone to inconsistency, and often also saves the storage space. Actually I do not always normalize the data in that sense, but leave some columns containing redundant information. For example, If "station name" and "country" can be uniquely identified given "station ID number", database textbooks suggest that "station name" and "country" should be included not in the table of observational records but in the table of "list of stations" only. But I often include such logically redundant information in order to ease identification of data items. (I often inspect such a table visually, and I often find some outliers. Then I want to know the time and the place that each outlier value represents. It is helpful that the name of place is included in the same table rather than stored in another normalized table.)

The role of a row and that of a column is not symmetric. The order of columns is meaningful. It is assumed that the application that use the data gives a unique name to each column. The order of rows is (in principle) not meaningful. It is (in principle) the same data if the rows come in a different order. The rows are (in principle) uniquely identified by either the content of a certain row, or the combination of contents of a certain set of rows. This is a principle for tables in relational database, and not always followed by tables processed by Awk. But it is often better to follow the principle.


2004-Feb-21; revised 2004-Feb-27, 2004-Jul-03, 2015-Jun-13
MASUDA Kooiti

[To the index page of "an awkward awker"] [To the entrance page of K. Masuda in English]