author | František Kučera <franta-hg@frantovo.cz> |
Sat, 06 Jun 2020 13:22:57 +0200 | |
branch | v_0 |
changeset 300 | b9bd0f06b4a1 |
parent 288 | 5cf3a702f47d |
permissions | -rw-r--r-- |
288
5cf3a702f47d
examples: AWKing through a XML file
František Kučera <franta-hg@frantovo.cz>
parents:
diff
changeset
|
1 |
<stránka |
5cf3a702f47d
examples: AWKing through a XML file
František Kučera <franta-hg@frantovo.cz>
parents:
diff
changeset
|
2 |
xmlns="https://trac.frantovo.cz/xml-web-generator/wiki/xmlns/strana" |
5cf3a702f47d
examples: AWKing through a XML file
František Kučera <franta-hg@frantovo.cz>
parents:
diff
changeset
|
3 |
xmlns:m="https://trac.frantovo.cz/xml-web-generator/wiki/xmlns/makro"> |
5cf3a702f47d
examples: AWKing through a XML file
František Kučera <franta-hg@frantovo.cz>
parents:
diff
changeset
|
4 |
|
5cf3a702f47d
examples: AWKing through a XML file
František Kučera <franta-hg@frantovo.cz>
parents:
diff
changeset
|
5 |
<nadpis>AWKing through a XML file</nadpis> |
5cf3a702f47d
examples: AWKing through a XML file
František Kučera <franta-hg@frantovo.cz>
parents:
diff
changeset
|
6 |
<perex>use AWK to filter data from a D-Bus policy file</perex> |
5cf3a702f47d
examples: AWKing through a XML file
František Kučera <franta-hg@frantovo.cz>
parents:
diff
changeset
|
7 |
<m:pořadí-příkladu>03700</m:pořadí-příkladu> |
5cf3a702f47d
examples: AWKing through a XML file
František Kučera <franta-hg@frantovo.cz>
parents:
diff
changeset
|
8 |
|
5cf3a702f47d
examples: AWKing through a XML file
František Kučera <franta-hg@frantovo.cz>
parents:
diff
changeset
|
9 |
<text xmlns="http://www.w3.org/1999/xhtml"> |
5cf3a702f47d
examples: AWKing through a XML file
František Kučera <franta-hg@frantovo.cz>
parents:
diff
changeset
|
10 |
|
5cf3a702f47d
examples: AWKing through a XML file
František Kučera <franta-hg@frantovo.cz>
parents:
diff
changeset
|
11 |
<p> |
5cf3a702f47d
examples: AWKing through a XML file
František Kučera <franta-hg@frantovo.cz>
parents:
diff
changeset
|
12 |
The <a href="https://en.wikipedia.org/wiki/AWK">AWK</a> programming language and tool is designed for processing text |
5cf3a702f47d
examples: AWKing through a XML file
František Kučera <franta-hg@frantovo.cz>
parents:
diff
changeset
|
13 |
but in <m:name/> we can use all the power of AWK for processing relational data that came from various sources. |
5cf3a702f47d
examples: AWKing through a XML file
František Kučera <franta-hg@frantovo.cz>
parents:
diff
changeset
|
14 |
And „all the power“ means that we internally call the regular AWK implementation installed in our system (which is usually GNU AWK). |
5cf3a702f47d
examples: AWKing through a XML file
František Kučera <franta-hg@frantovo.cz>
parents:
diff
changeset
|
15 |
</p> |
5cf3a702f47d
examples: AWKing through a XML file
František Kučera <franta-hg@frantovo.cz>
parents:
diff
changeset
|
16 |
|
5cf3a702f47d
examples: AWKing through a XML file
František Kučera <franta-hg@frantovo.cz>
parents:
diff
changeset
|
17 |
<p> |
5cf3a702f47d
examples: AWKing through a XML file
František Kučera <franta-hg@frantovo.cz>
parents:
diff
changeset
|
18 |
In this example, we will read a D-Bus policy configuration file (an XML) |
5cf3a702f47d
examples: AWKing through a XML file
František Kučera <franta-hg@frantovo.cz>
parents:
diff
changeset
|
19 |
and filter it using a simple AWK expression. |
5cf3a702f47d
examples: AWKing through a XML file
František Kučera <franta-hg@frantovo.cz>
parents:
diff
changeset
|
20 |
The XML file looks like this: |
5cf3a702f47d
examples: AWKing through a XML file
František Kučera <franta-hg@frantovo.cz>
parents:
diff
changeset
|
21 |
</p> |
5cf3a702f47d
examples: AWKing through a XML file
František Kučera <franta-hg@frantovo.cz>
parents:
diff
changeset
|
22 |
|
5cf3a702f47d
examples: AWKing through a XML file
František Kučera <franta-hg@frantovo.cz>
parents:
diff
changeset
|
23 |
<m:pre jazyk="xml" src="examples/awk-through-xml.xml"/> |
5cf3a702f47d
examples: AWKing through a XML file
František Kučera <franta-hg@frantovo.cz>
parents:
diff
changeset
|
24 |
|
5cf3a702f47d
examples: AWKing through a XML file
František Kučera <franta-hg@frantovo.cz>
parents:
diff
changeset
|
25 |
<p> |
5cf3a702f47d
examples: AWKing through a XML file
František Kučera <franta-hg@frantovo.cz>
parents:
diff
changeset
|
26 |
We separate particular steps of our pipeline to shell functions for better readability and maintenance: |
5cf3a702f47d
examples: AWKing through a XML file
František Kučera <franta-hg@frantovo.cz>
parents:
diff
changeset
|
27 |
</p> |
5cf3a702f47d
examples: AWKing through a XML file
František Kučera <franta-hg@frantovo.cz>
parents:
diff
changeset
|
28 |
|
5cf3a702f47d
examples: AWKing through a XML file
František Kučera <franta-hg@frantovo.cz>
parents:
diff
changeset
|
29 |
<m:pre jazyk="bash" src="examples/awk-through-xml.sh"/> |
5cf3a702f47d
examples: AWKing through a XML file
František Kučera <franta-hg@frantovo.cz>
parents:
diff
changeset
|
30 |
|
5cf3a702f47d
examples: AWKing through a XML file
František Kučera <franta-hg@frantovo.cz>
parents:
diff
changeset
|
31 |
<p> |
5cf3a702f47d
examples: AWKing through a XML file
František Kučera <franta-hg@frantovo.cz>
parents:
diff
changeset
|
32 |
This simple sequence of steps will generate following result: |
5cf3a702f47d
examples: AWKing through a XML file
František Kučera <franta-hg@frantovo.cz>
parents:
diff
changeset
|
33 |
</p> |
5cf3a702f47d
examples: AWKing through a XML file
František Kučera <franta-hg@frantovo.cz>
parents:
diff
changeset
|
34 |
|
5cf3a702f47d
examples: AWKing through a XML file
František Kučera <franta-hg@frantovo.cz>
parents:
diff
changeset
|
35 |
<m:pre jazyk="text" src="examples/awk-through-xml.txt"/> |
5cf3a702f47d
examples: AWKing through a XML file
František Kučera <franta-hg@frantovo.cz>
parents:
diff
changeset
|
36 |
|
5cf3a702f47d
examples: AWKing through a XML file
František Kučera <franta-hg@frantovo.cz>
parents:
diff
changeset
|
37 |
<p> |
5cf3a702f47d
examples: AWKing through a XML file
František Kučera <franta-hg@frantovo.cz>
parents:
diff
changeset
|
38 |
We have processed two relations. |
5cf3a702f47d
examples: AWKing through a XML file
František Kučera <franta-hg@frantovo.cz>
parents:
diff
changeset
|
39 |
One (<code>limit</code>) was just passed through |
5cf3a702f47d
examples: AWKing through a XML file
František Kučera <franta-hg@frantovo.cz>
parents:
diff
changeset
|
40 |
while another (<code>policy</code>) was transformed. |
5cf3a702f47d
examples: AWKing through a XML file
František Kučera <franta-hg@frantovo.cz>
parents:
diff
changeset
|
41 |
In this case, we did just restriction (filtered certain records), but we can do also projections or other relational operations in our pipelines. |
5cf3a702f47d
examples: AWKing through a XML file
František Kučera <franta-hg@frantovo.cz>
parents:
diff
changeset
|
42 |
</p> |
5cf3a702f47d
examples: AWKing through a XML file
František Kučera <franta-hg@frantovo.cz>
parents:
diff
changeset
|
43 |
|
5cf3a702f47d
examples: AWKing through a XML file
František Kučera <franta-hg@frantovo.cz>
parents:
diff
changeset
|
44 |
<p> |
5cf3a702f47d
examples: AWKing through a XML file
František Kučera <franta-hg@frantovo.cz>
parents:
diff
changeset
|
45 |
This example also shows how we can structure our scripts – compose the pipeline from reusable fragments (shell functions). |
5cf3a702f47d
examples: AWKing through a XML file
František Kučera <franta-hg@frantovo.cz>
parents:
diff
changeset
|
46 |
Once we write the <em>parse</em> function for an input format, we can use it many times, with various filters/transformations and output formatters. |
5cf3a702f47d
examples: AWKing through a XML file
František Kučera <franta-hg@frantovo.cz>
parents:
diff
changeset
|
47 |
</p> |
5cf3a702f47d
examples: AWKing through a XML file
František Kučera <franta-hg@frantovo.cz>
parents:
diff
changeset
|
48 |
<p> |
5cf3a702f47d
examples: AWKing through a XML file
František Kučera <franta-hg@frantovo.cz>
parents:
diff
changeset
|
49 |
Instead of converting to an output format, we can also <m:a href="examples-out-bash">execute arbitrary system commands</m:a> for each record |
5cf3a702f47d
examples: AWKing through a XML file
František Kučera <franta-hg@frantovo.cz>
parents:
diff
changeset
|
50 |
(while having particular attribute values reliably separated in shell variables). |
5cf3a702f47d
examples: AWKing through a XML file
František Kučera <franta-hg@frantovo.cz>
parents:
diff
changeset
|
51 |
</p> |
5cf3a702f47d
examples: AWKing through a XML file
František Kučera <franta-hg@frantovo.cz>
parents:
diff
changeset
|
52 |
|
5cf3a702f47d
examples: AWKing through a XML file
František Kučera <franta-hg@frantovo.cz>
parents:
diff
changeset
|
53 |
</text> |
5cf3a702f47d
examples: AWKing through a XML file
František Kučera <franta-hg@frantovo.cz>
parents:
diff
changeset
|
54 |
|
5cf3a702f47d
examples: AWKing through a XML file
František Kučera <franta-hg@frantovo.cz>
parents:
diff
changeset
|
55 |
</stránka> |