diff -r a39066264509 -r 2868d772c27e relpipe-data/examples-awk-filtering.xml --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/relpipe-data/examples-awk-filtering.xml Tue May 28 21:18:20 2019 +0200 @@ -0,0 +1,108 @@ + + + Complex filtering with AWK + filtering records with AND, OR and functions + 02100 + + + +

+ If we need more complex filtering than relpipe-tr-grep can offer, we can write an AWK transformation. + Then we can use AND and OR operators and functions like regular expression matching or numerical formulas. +

+ +

+ The tool relpipe-tr-awk calls real AWK program (usually GNU AWK) installed on our system and passes data of given relation to it. + Thus we can use any AWK feature in our pipeline while processing relational data. + Relational attributes are mapped to AWK variables, so we can reference them by their names instead of mere field numbers. +

+ +

+ The --for-each option is used for both filtering (instead of --where) + and arbitrary code execution (for data modifications, adding records, computations or intentional side effects). + In AWK, filtering conditions are surrounded by (…) and actions by {…}. + Both can be combined together and multiple expressions can be separated by ; semicolon. + The record() function should be called instead of AWK print (which should never be used directly). + Calling record() is not necessary, when only filtering is done (and there are no data modifications). +

+ +

Filtering numbers

+ +

With AWK we can filter records using standard numeric operators like ==, <, >, >= etc.

+ + 2000)' \ + | relpipe-out-tabular]]> + +

and e.g. list files with certain sizes:

+ +

+ + +

Filtering strings

+ +

String values can be searched for certain regular expression:

+ + + +

e.g. fstab records having cdrom in the mount_point:

+ +

Case-insensitive search can be switched on by adding:

+ +

--define IGNORECASE integer 1

+ +

AND and OR

+ +

We can combine multiple conditions using || and && logical operators:

+ + + +

and build arbitrary complex filters

+ +

Nested (…) work as expected.

+ +

+ And AWK can do much more – it offers plenty of functions and language constructs that we can use in our transformations. + Comperhensive documentation can be found here: Gawk: Effective AWK Programming. +

+ + + +