relpipe-data/examples-awk-debugging.xml
author František Kučera <franta-hg@frantovo.cz>
Mon, 21 Feb 2022 00:43:11 +0100
branchv_0
changeset 329 5bc2bb8b7946
parent 258 2868d772c27e
permissions -rw-r--r--
Release v0.18
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
258
2868d772c27e Release v0.12 – AWK
František Kučera <franta-hg@frantovo.cz>
parents:
diff changeset
     1
<stránka
2868d772c27e Release v0.12 – AWK
František Kučera <franta-hg@frantovo.cz>
parents:
diff changeset
     2
	xmlns="https://trac.frantovo.cz/xml-web-generator/wiki/xmlns/strana"
2868d772c27e Release v0.12 – AWK
František Kučera <franta-hg@frantovo.cz>
parents:
diff changeset
     3
	xmlns:m="https://trac.frantovo.cz/xml-web-generator/wiki/xmlns/makro">
2868d772c27e Release v0.12 – AWK
František Kučera <franta-hg@frantovo.cz>
parents:
diff changeset
     4
	
2868d772c27e Release v0.12 – AWK
František Kučera <franta-hg@frantovo.cz>
parents:
diff changeset
     5
	<nadpis>Debugging AWK transformations</nadpis>
2868d772c27e Release v0.12 – AWK
František Kučera <franta-hg@frantovo.cz>
parents:
diff changeset
     6
	<perex>discovering variable mappings and transformation internals</perex>
2868d772c27e Release v0.12 – AWK
František Kučera <franta-hg@frantovo.cz>
parents:
diff changeset
     7
	<m:pořadí-příkladu>02200</m:pořadí-příkladu>
2868d772c27e Release v0.12 – AWK
František Kučera <franta-hg@frantovo.cz>
parents:
diff changeset
     8
2868d772c27e Release v0.12 – AWK
František Kučera <franta-hg@frantovo.cz>
parents:
diff changeset
     9
	<text xmlns="http://www.w3.org/1999/xhtml">
2868d772c27e Release v0.12 – AWK
František Kučera <franta-hg@frantovo.cz>
parents:
diff changeset
    10
		
2868d772c27e Release v0.12 – AWK
František Kučera <franta-hg@frantovo.cz>
parents:
diff changeset
    11
		<p>In most cases, AWK transformations should be quite straightforward, but sometimes we need to look inside the box.</p>
2868d772c27e Release v0.12 – AWK
František Kučera <franta-hg@frantovo.cz>
parents:
diff changeset
    12
		
2868d772c27e Release v0.12 – AWK
František Kučera <franta-hg@frantovo.cz>
parents:
diff changeset
    13
		<h2>Mapping attributes to variables</h2>
2868d772c27e Release v0.12 – AWK
František Kučera <franta-hg@frantovo.cz>
parents:
diff changeset
    14
		
2868d772c27e Release v0.12 – AWK
František Kučera <franta-hg@frantovo.cz>
parents:
diff changeset
    15
		<p>
2868d772c27e Release v0.12 – AWK
František Kučera <franta-hg@frantovo.cz>
parents:
diff changeset
    16
			Relations have named attributes but in a language like AWK we work with named variables.
2868d772c27e Release v0.12 – AWK
František Kučera <franta-hg@frantovo.cz>
parents:
diff changeset
    17
			In most cases, the names will match 1:1. But not always.
2868d772c27e Release v0.12 – AWK
František Kučera <franta-hg@frantovo.cz>
parents:
diff changeset
    18
			The mapping is needed because not all valid attribute names are also valid variable names in particular language, thus sometimes some escaping or prefixing is necessary. 
2868d772c27e Release v0.12 – AWK
František Kučera <franta-hg@frantovo.cz>
parents:
diff changeset
    19
			So there is <code>--debug-variable-mapping</code> option for printing the mappings between attributes and variables.
2868d772c27e Release v0.12 – AWK
František Kučera <franta-hg@frantovo.cz>
parents:
diff changeset
    20
		</p>
2868d772c27e Release v0.12 – AWK
František Kučera <franta-hg@frantovo.cz>
parents:
diff changeset
    21
		
2868d772c27e Release v0.12 – AWK
František Kučera <franta-hg@frantovo.cz>
parents:
diff changeset
    22
		<m:pre jazyk="bash"><![CDATA[relpipe-in-fstab \
2868d772c27e Release v0.12 – AWK
František Kučera <franta-hg@frantovo.cz>
parents:
diff changeset
    23
	| relpipe-tr-awk \
2868d772c27e Release v0.12 – AWK
František Kučera <franta-hg@frantovo.cz>
parents:
diff changeset
    24
		--relation '.*' \
2868d772c27e Release v0.12 – AWK
František Kučera <franta-hg@frantovo.cz>
parents:
diff changeset
    25
			--for-each '1' \
2868d772c27e Release v0.12 – AWK
František Kučera <franta-hg@frantovo.cz>
parents:
diff changeset
    26
			--debug-variable-mapping \
2868d772c27e Release v0.12 – AWK
František Kučera <franta-hg@frantovo.cz>
parents:
diff changeset
    27
	| relpipe-out-tabular]]></m:pre>
2868d772c27e Release v0.12 – AWK
František Kučera <franta-hg@frantovo.cz>
parents:
diff changeset
    28
2868d772c27e Release v0.12 – AWK
František Kučera <franta-hg@frantovo.cz>
parents:
diff changeset
    29
		<p>This option prepends additional relation with these metadata to the stream:</p>
2868d772c27e Release v0.12 – AWK
František Kučera <franta-hg@frantovo.cz>
parents:
diff changeset
    30
		
2868d772c27e Release v0.12 – AWK
František Kučera <franta-hg@frantovo.cz>
parents:
diff changeset
    31
		<pre><![CDATA[fstab.variableMapping:
2868d772c27e Release v0.12 – AWK
František Kučera <franta-hg@frantovo.cz>
parents:
diff changeset
    32
 ╭────────────────────┬───────────────────╮
2868d772c27e Release v0.12 – AWK
František Kučera <franta-hg@frantovo.cz>
parents:
diff changeset
    33
 │ attribute (string) │ variable (string) │
2868d772c27e Release v0.12 – AWK
František Kučera <franta-hg@frantovo.cz>
parents:
diff changeset
    34
 ├────────────────────┼───────────────────┤
2868d772c27e Release v0.12 – AWK
František Kučera <franta-hg@frantovo.cz>
parents:
diff changeset
    35
 │ device             │ device            │
2868d772c27e Release v0.12 – AWK
František Kučera <franta-hg@frantovo.cz>
parents:
diff changeset
    36
 │ dump               │ dump              │
2868d772c27e Release v0.12 – AWK
František Kučera <franta-hg@frantovo.cz>
parents:
diff changeset
    37
 │ mount_point        │ mount_point       │
2868d772c27e Release v0.12 – AWK
František Kučera <franta-hg@frantovo.cz>
parents:
diff changeset
    38
 │ options            │ options           │
2868d772c27e Release v0.12 – AWK
František Kučera <franta-hg@frantovo.cz>
parents:
diff changeset
    39
 │ pass               │ pass              │
2868d772c27e Release v0.12 – AWK
František Kučera <franta-hg@frantovo.cz>
parents:
diff changeset
    40
 │ scheme             │ scheme            │
2868d772c27e Release v0.12 – AWK
František Kučera <franta-hg@frantovo.cz>
parents:
diff changeset
    41
 │ type               │ type              │
2868d772c27e Release v0.12 – AWK
František Kučera <franta-hg@frantovo.cz>
parents:
diff changeset
    42
 ╰────────────────────┴───────────────────╯
2868d772c27e Release v0.12 – AWK
František Kučera <franta-hg@frantovo.cz>
parents:
diff changeset
    43
Record count: 7
2868d772c27e Release v0.12 – AWK
František Kučera <franta-hg@frantovo.cz>
parents:
diff changeset
    44
fstab:
2868d772c27e Release v0.12 – AWK
František Kučera <franta-hg@frantovo.cz>
parents:
diff changeset
    45
 ╭─────────────────┬──────────────────────────────────────┬──────────────────────┬───────────────┬───────────────────────────────────────┬────────────────┬────────────────╮
2868d772c27e Release v0.12 – AWK
František Kučera <franta-hg@frantovo.cz>
parents:
diff changeset
    46
 │ scheme (string) │ device                      (string) │ mount_point (string) │ type (string) │ options                      (string) │ dump (integer) │ pass (integer) │
2868d772c27e Release v0.12 – AWK
František Kučera <franta-hg@frantovo.cz>
parents:
diff changeset
    47
 ├─────────────────┼──────────────────────────────────────┼──────────────────────┼───────────────┼───────────────────────────────────────┼────────────────┼────────────────┤
2868d772c27e Release v0.12 – AWK
František Kučera <franta-hg@frantovo.cz>
parents:
diff changeset
    48
 │ UUID            │ 29758270-fd25-4a6c-a7bb-9a18302816af │ /                    │ ext4          │ relatime,user_xattr,errors=remount-ro │              0 │              1 │
2868d772c27e Release v0.12 – AWK
František Kučera <franta-hg@frantovo.cz>
parents:
diff changeset
    49
 │                 │ /dev/sr0                             │ /media/cdrom0        │ udf,iso9660   │ user,noauto                           │              0 │              0 │
2868d772c27e Release v0.12 – AWK
František Kučera <franta-hg@frantovo.cz>
parents:
diff changeset
    50
 │                 │ /dev/sde                             │ /mnt/data            │ ext4          │ relatime,user_xattr,errors=remount-ro │              0 │              2 │
2868d772c27e Release v0.12 – AWK
František Kučera <franta-hg@frantovo.cz>
parents:
diff changeset
    51
 │ UUID            │ a2b5f230-a795-4f6f-a39b-9b57686c86d5 │ /home                │ btrfs         │ relatime                              │              0 │              2 │
2868d772c27e Release v0.12 – AWK
František Kučera <franta-hg@frantovo.cz>
parents:
diff changeset
    52
 │                 │ /dev/mapper/sdf_crypt                │ /mnt/private         │ xfs           │ relatime                              │              0 │              2 │
2868d772c27e Release v0.12 – AWK
František Kučera <franta-hg@frantovo.cz>
parents:
diff changeset
    53
 ╰─────────────────┴──────────────────────────────────────┴──────────────────────┴───────────────┴───────────────────────────────────────┴────────────────┴────────────────╯
2868d772c27e Release v0.12 – AWK
František Kučera <franta-hg@frantovo.cz>
parents:
diff changeset
    54
Record count: 5]]></pre>
2868d772c27e Release v0.12 – AWK
František Kučera <franta-hg@frantovo.cz>
parents:
diff changeset
    55
2868d772c27e Release v0.12 – AWK
František Kučera <franta-hg@frantovo.cz>
parents:
diff changeset
    56
		<p>If we are interested only in the mappings, we should use it in combination with <code>--drop</code> option:</p>
2868d772c27e Release v0.12 – AWK
František Kučera <franta-hg@frantovo.cz>
parents:
diff changeset
    57
2868d772c27e Release v0.12 – AWK
František Kučera <franta-hg@frantovo.cz>
parents:
diff changeset
    58
		<m:pre jazyk="bash"><![CDATA[relpipe-in-fstab \
2868d772c27e Release v0.12 – AWK
František Kučera <franta-hg@frantovo.cz>
parents:
diff changeset
    59
	| relpipe-tr-awk \
2868d772c27e Release v0.12 – AWK
František Kučera <franta-hg@frantovo.cz>
parents:
diff changeset
    60
		--relation '.*' \
2868d772c27e Release v0.12 – AWK
František Kučera <franta-hg@frantovo.cz>
parents:
diff changeset
    61
			--for-each '1' \
2868d772c27e Release v0.12 – AWK
František Kučera <franta-hg@frantovo.cz>
parents:
diff changeset
    62
			--debug-variable-mapping \
2868d772c27e Release v0.12 – AWK
František Kučera <franta-hg@frantovo.cz>
parents:
diff changeset
    63
			--drop \
2868d772c27e Release v0.12 – AWK
František Kučera <franta-hg@frantovo.cz>
parents:
diff changeset
    64
	| relpipe-out-tabular]]></m:pre>
2868d772c27e Release v0.12 – AWK
František Kučera <franta-hg@frantovo.cz>
parents:
diff changeset
    65
2868d772c27e Release v0.12 – AWK
František Kučera <franta-hg@frantovo.cz>
parents:
diff changeset
    66
		<p>which skips the actual data:</p>
2868d772c27e Release v0.12 – AWK
František Kučera <franta-hg@frantovo.cz>
parents:
diff changeset
    67
		
2868d772c27e Release v0.12 – AWK
František Kučera <franta-hg@frantovo.cz>
parents:
diff changeset
    68
		<pre><![CDATA[fstab.variableMapping:
2868d772c27e Release v0.12 – AWK
František Kučera <franta-hg@frantovo.cz>
parents:
diff changeset
    69
 ╭────────────────────┬───────────────────╮
2868d772c27e Release v0.12 – AWK
František Kučera <franta-hg@frantovo.cz>
parents:
diff changeset
    70
 │ attribute (string) │ variable (string) │
2868d772c27e Release v0.12 – AWK
František Kučera <franta-hg@frantovo.cz>
parents:
diff changeset
    71
 ├────────────────────┼───────────────────┤
2868d772c27e Release v0.12 – AWK
František Kučera <franta-hg@frantovo.cz>
parents:
diff changeset
    72
 │ device             │ device            │
2868d772c27e Release v0.12 – AWK
František Kučera <franta-hg@frantovo.cz>
parents:
diff changeset
    73
 │ dump               │ dump              │
2868d772c27e Release v0.12 – AWK
František Kučera <franta-hg@frantovo.cz>
parents:
diff changeset
    74
 │ mount_point        │ mount_point       │
2868d772c27e Release v0.12 – AWK
František Kučera <franta-hg@frantovo.cz>
parents:
diff changeset
    75
 │ options            │ options           │
2868d772c27e Release v0.12 – AWK
František Kučera <franta-hg@frantovo.cz>
parents:
diff changeset
    76
 │ pass               │ pass              │
2868d772c27e Release v0.12 – AWK
František Kučera <franta-hg@frantovo.cz>
parents:
diff changeset
    77
 │ scheme             │ scheme            │
2868d772c27e Release v0.12 – AWK
František Kučera <franta-hg@frantovo.cz>
parents:
diff changeset
    78
 │ type               │ type              │
2868d772c27e Release v0.12 – AWK
František Kučera <franta-hg@frantovo.cz>
parents:
diff changeset
    79
 ╰────────────────────┴───────────────────╯
2868d772c27e Release v0.12 – AWK
František Kučera <franta-hg@frantovo.cz>
parents:
diff changeset
    80
Record count: 7]]></pre>
2868d772c27e Release v0.12 – AWK
František Kučera <franta-hg@frantovo.cz>
parents:
diff changeset
    81
2868d772c27e Release v0.12 – AWK
František Kučera <franta-hg@frantovo.cz>
parents:
diff changeset
    82
		<p>Because there were no collisions, variables have same names as attributes. But in this case:</p>
2868d772c27e Release v0.12 – AWK
František Kučera <franta-hg@frantovo.cz>
parents:
diff changeset
    83
				
2868d772c27e Release v0.12 – AWK
František Kučera <franta-hg@frantovo.cz>
parents:
diff changeset
    84
		<m:pre jazyk="bash"><![CDATA[relpipe-in-cli generate t 3 \
2868d772c27e Release v0.12 – AWK
František Kučera <franta-hg@frantovo.cz>
parents:
diff changeset
    85
		"if"      string \
2868d772c27e Release v0.12 – AWK
František Kučera <franta-hg@frantovo.cz>
parents:
diff changeset
    86
		"6pack"   string \
2868d772c27e Release v0.12 – AWK
František Kučera <franta-hg@frantovo.cz>
parents:
diff changeset
    87
		"spa ces" string \
2868d772c27e Release v0.12 – AWK
František Kučera <franta-hg@frantovo.cz>
parents:
diff changeset
    88
	| relpipe-tr-awk \
2868d772c27e Release v0.12 – AWK
František Kučera <franta-hg@frantovo.cz>
parents:
diff changeset
    89
		--relation t \
2868d772c27e Release v0.12 – AWK
František Kučera <franta-hg@frantovo.cz>
parents:
diff changeset
    90
			--debug-variable-mapping \
2868d772c27e Release v0.12 – AWK
František Kučera <franta-hg@frantovo.cz>
parents:
diff changeset
    91
			--drop \
2868d772c27e Release v0.12 – AWK
František Kučera <franta-hg@frantovo.cz>
parents:
diff changeset
    92
	| relpipe-out-tabular]]></m:pre>
2868d772c27e Release v0.12 – AWK
František Kučera <franta-hg@frantovo.cz>
parents:
diff changeset
    93
	
2868d772c27e Release v0.12 – AWK
František Kučera <franta-hg@frantovo.cz>
parents:
diff changeset
    94
		<p>mapping rules come in to the play:</p>
2868d772c27e Release v0.12 – AWK
František Kučera <franta-hg@frantovo.cz>
parents:
diff changeset
    95
		
2868d772c27e Release v0.12 – AWK
František Kučera <franta-hg@frantovo.cz>
parents:
diff changeset
    96
		<pre><![CDATA[t.variableMapping:
2868d772c27e Release v0.12 – AWK
František Kučera <franta-hg@frantovo.cz>
parents:
diff changeset
    97
 ╭────────────────────┬───────────────────╮
2868d772c27e Release v0.12 – AWK
František Kučera <franta-hg@frantovo.cz>
parents:
diff changeset
    98
 │ attribute (string) │ variable (string) │
2868d772c27e Release v0.12 – AWK
František Kučera <franta-hg@frantovo.cz>
parents:
diff changeset
    99
 ├────────────────────┼───────────────────┤
2868d772c27e Release v0.12 – AWK
František Kučera <franta-hg@frantovo.cz>
parents:
diff changeset
   100
 │ 6pack              │ _6pack            │
2868d772c27e Release v0.12 – AWK
František Kučera <franta-hg@frantovo.cz>
parents:
diff changeset
   101
 │ if                 │ _if               │
2868d772c27e Release v0.12 – AWK
František Kučera <franta-hg@frantovo.cz>
parents:
diff changeset
   102
 │ spa ces            │ spa_ces           │
2868d772c27e Release v0.12 – AWK
František Kučera <franta-hg@frantovo.cz>
parents:
diff changeset
   103
 ╰────────────────────┴───────────────────╯
2868d772c27e Release v0.12 – AWK
František Kučera <franta-hg@frantovo.cz>
parents:
diff changeset
   104
Record count: 3]]></pre>
2868d772c27e Release v0.12 – AWK
František Kučera <franta-hg@frantovo.cz>
parents:
diff changeset
   105
2868d772c27e Release v0.12 – AWK
František Kučera <franta-hg@frantovo.cz>
parents:
diff changeset
   106
		<p>in order to make variable names valid in AWK.</p>
2868d772c27e Release v0.12 – AWK
František Kučera <franta-hg@frantovo.cz>
parents:
diff changeset
   107
2868d772c27e Release v0.12 – AWK
František Kučera <franta-hg@frantovo.cz>
parents:
diff changeset
   108
2868d772c27e Release v0.12 – AWK
František Kučera <franta-hg@frantovo.cz>
parents:
diff changeset
   109
		<h2>Inspecting the internals of an AWK transformation</h2>
2868d772c27e Release v0.12 – AWK
František Kučera <franta-hg@frantovo.cz>
parents:
diff changeset
   110
		
2868d772c27e Release v0.12 – AWK
František Kučera <franta-hg@frantovo.cz>
parents:
diff changeset
   111
		<p>
2868d772c27e Release v0.12 – AWK
František Kučera <franta-hg@frantovo.cz>
parents:
diff changeset
   112
			The <code>relpipe-tr-awk</code> calls AWK as a child process and passes data of given relation to it for actual processing.
2868d772c27e Release v0.12 – AWK
František Kučera <franta-hg@frantovo.cz>
parents:
diff changeset
   113
			Because it executes <code>awk</code> program found on <code>$PATH</code>, we can easily switch the AWK implementations.
2868d772c27e Release v0.12 – AWK
František Kučera <franta-hg@frantovo.cz>
parents:
diff changeset
   114
			In the source code repository, there is <code>scripts/awk</code> – a wrapper script.
2868d772c27e Release v0.12 – AWK
František Kučera <franta-hg@frantovo.cz>
parents:
diff changeset
   115
			We can modify the <code>$PATH</code>, so this wrapper will be called by <code>relpipe-tr-awk</code>.
2868d772c27e Release v0.12 – AWK
František Kučera <franta-hg@frantovo.cz>
parents:
diff changeset
   116
			This script captures CLI arguments, STDIN, STDOUT, STDERR and the exit code and saves them to files in the temp directory.
2868d772c27e Release v0.12 – AWK
František Kučera <franta-hg@frantovo.cz>
parents:
diff changeset
   117
			Using GNU Screen and the <em>inotifywait</em> we can build a kind of IDE and watch what happens inside during the transformation:
2868d772c27e Release v0.12 – AWK
František Kučera <franta-hg@frantovo.cz>
parents:
diff changeset
   118
		</p>
2868d772c27e Release v0.12 – AWK
František Kučera <franta-hg@frantovo.cz>
parents:
diff changeset
   119
		
2868d772c27e Release v0.12 – AWK
František Kučera <franta-hg@frantovo.cz>
parents:
diff changeset
   120
		<m:img src="img/awk-wrapper-debug-1.png"/>
2868d772c27e Release v0.12 – AWK
František Kučera <franta-hg@frantovo.cz>
parents:
diff changeset
   121
		
2868d772c27e Release v0.12 – AWK
František Kučera <franta-hg@frantovo.cz>
parents:
diff changeset
   122
		<p>
2868d772c27e Release v0.12 – AWK
František Kučera <franta-hg@frantovo.cz>
parents:
diff changeset
   123
			So we can inspect the generated AWK code and the inputs and outputs of the AWK process.
2868d772c27e Release v0.12 – AWK
František Kučera <franta-hg@frantovo.cz>
parents:
diff changeset
   124
			Recommended usage is described in the <code>scripts/awk</code> script.
2868d772c27e Release v0.12 – AWK
František Kučera <franta-hg@frantovo.cz>
parents:
diff changeset
   125
		</p>
2868d772c27e Release v0.12 – AWK
František Kučera <franta-hg@frantovo.cz>
parents:
diff changeset
   126
2868d772c27e Release v0.12 – AWK
František Kučera <franta-hg@frantovo.cz>
parents:
diff changeset
   127
	</text>
2868d772c27e Release v0.12 – AWK
František Kučera <franta-hg@frantovo.cz>
parents:
diff changeset
   128
	
2868d772c27e Release v0.12 – AWK
František Kučera <franta-hg@frantovo.cz>
parents:
diff changeset
   129
</stránka>