diff -r a39066264509 -r 2868d772c27e relpipe-data/examples-awk-debugging.xml --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/relpipe-data/examples-awk-debugging.xml Tue May 28 21:18:20 2019 +0200 @@ -0,0 +1,129 @@ + + + Debugging AWK transformations + discovering variable mappings and transformation internals + 02200 + + + +

In most cases, AWK transformations should be quite straightforward, but sometimes we need to look inside the box.

+ +

Mapping attributes to variables

+ +

+ Relations have named attributes but in a language like AWK we work with named variables. + In most cases, the names will match 1:1. But not always. + The mapping is needed because not all valid attribute names are also valid variable names in particular language, thus sometimes some escaping or prefixing is necessary. + So there is --debug-variable-mapping option for printing the mappings between attributes and variables. +

+ + + +

This option prepends additional relation with these metadata to the stream:

+ +
+ +

If we are interested only in the mappings, we should use it in combination with --drop option:

+ + + +

which skips the actual data:

+ +
+ +

Because there were no collisions, variables have same names as attributes. But in this case:

+ + + +

mapping rules come in to the play:

+ +
+ +

in order to make variable names valid in AWK.

+ + +

Inspecting the internals of an AWK transformation

+ +

+ The relpipe-tr-awk calls AWK as a child process and passes data of given relation to it for actual processing. + Because it executes awk program found on $PATH, we can easily switch the AWK implementations. + In the source code repository, there is scripts/awk – a wrapper script. + We can modify the $PATH, so this wrapper will be called by relpipe-tr-awk. + This script captures CLI arguments, STDIN, STDOUT, STDERR and the exit code and saves them to files in the temp directory. + Using GNU Screen and the inotifywait we can build a kind of IDE and watch what happens inside during the transformation: +

+ + + +

+ So we can inspect the generated AWK code and the inputs and outputs of the AWK process. + Recommended usage is described in the scripts/awk script. +

+ + + +