author | František Kučera <franta-hg@frantovo.cz> |
Fri, 11 Jan 2019 23:08:47 +0100 | |
branch | v_0 |
changeset 234 | 2fc56dd7f003 |
parent 233 | a8029bd1c87a |
child 237 | 2adb6465eadd |
permissions | -rw-r--r-- |
23
0d2729ed16ed
zkouška interního odkazu
František Kučera <franta-hg@frantovo.cz>
parents:
18
diff
changeset
|
1 |
<stránka |
0d2729ed16ed
zkouška interního odkazu
František Kučera <franta-hg@frantovo.cz>
parents:
18
diff
changeset
|
2 |
xmlns="https://trac.frantovo.cz/xml-web-generator/wiki/xmlns/strana" |
0d2729ed16ed
zkouška interního odkazu
František Kučera <franta-hg@frantovo.cz>
parents:
18
diff
changeset
|
3 |
xmlns:m="https://trac.frantovo.cz/xml-web-generator/wiki/xmlns/makro"> |
0d2729ed16ed
zkouška interního odkazu
František Kučera <franta-hg@frantovo.cz>
parents:
18
diff
changeset
|
4 |
|
140
1d8a3f7ec393
create empty pages
František Kučera <franta-hg@frantovo.cz>
parents:
139
diff
changeset
|
5 |
<nadpis>Examples</nadpis> |
1d8a3f7ec393
create empty pages
František Kučera <franta-hg@frantovo.cz>
parents:
139
diff
changeset
|
6 |
<perex>Usage examples of Relational pipes tools</perex> |
1d8a3f7ec393
create empty pages
František Kučera <franta-hg@frantovo.cz>
parents:
139
diff
changeset
|
7 |
<pořadí>40</pořadí> |
4
1bb39595a51c
genrování hlavní nabídky #1
František Kučera <franta-hg@frantovo.cz>
parents:
2
diff
changeset
|
8 |
|
2
ab9099ff88fa
vkládání zápatí, jmenné prostory, saxon
František Kučera <franta-hg@frantovo.cz>
parents:
1
diff
changeset
|
9 |
<text xmlns="http://www.w3.org/1999/xhtml"> |
176
46042297e0d6
relpipe-tr-validator: specification and examples
František Kučera <franta-hg@frantovo.cz>
parents:
140
diff
changeset
|
10 |
|
46042297e0d6
relpipe-tr-validator: specification and examples
František Kučera <franta-hg@frantovo.cz>
parents:
140
diff
changeset
|
11 |
|
198
48a3c1c8c411
examples: they were tested in GNU Bash
František Kučera <franta-hg@frantovo.cz>
parents:
196
diff
changeset
|
12 |
<p> |
48a3c1c8c411
examples: they were tested in GNU Bash
František Kučera <franta-hg@frantovo.cz>
parents:
196
diff
changeset
|
13 |
All examples were tested in <a href="https://www.gnu.org/software/bash/">GNU Bash</a>. |
48a3c1c8c411
examples: they were tested in GNU Bash
František Kučera <franta-hg@frantovo.cz>
parents:
196
diff
changeset
|
14 |
But they should also work in other shells. |
48a3c1c8c411
examples: they were tested in GNU Bash
František Kučera <franta-hg@frantovo.cz>
parents:
196
diff
changeset
|
15 |
</p> |
48a3c1c8c411
examples: they were tested in GNU Bash
František Kučera <franta-hg@frantovo.cz>
parents:
196
diff
changeset
|
16 |
|
233
a8029bd1c87a
examples: fix header levels
František Kučera <franta-hg@frantovo.cz>
parents:
232
diff
changeset
|
17 |
<h2>relpipe-in-cli: Hello Wordl!</h2> |
185
6cd2e54d90ea
examples: relpipe-in-cli
František Kučera <franta-hg@frantovo.cz>
parents:
184
diff
changeset
|
18 |
|
6cd2e54d90ea
examples: relpipe-in-cli
František Kučera <franta-hg@frantovo.cz>
parents:
184
diff
changeset
|
19 |
<p> |
195
65138c5d5a7c
examples: fstab - another check
František Kučera <franta-hg@frantovo.cz>
parents:
187
diff
changeset
|
20 |
Let's start with an obligatory Hello World example. |
185
6cd2e54d90ea
examples: relpipe-in-cli
František Kučera <franta-hg@frantovo.cz>
parents:
184
diff
changeset
|
21 |
</p> |
6cd2e54d90ea
examples: relpipe-in-cli
František Kučera <franta-hg@frantovo.cz>
parents:
184
diff
changeset
|
22 |
|
6cd2e54d90ea
examples: relpipe-in-cli
František Kučera <franta-hg@frantovo.cz>
parents:
184
diff
changeset
|
23 |
<m:pre jazyk="bash"><![CDATA[relpipe-in-cli generate "relation_from_cli" 3 \ |
6cd2e54d90ea
examples: relpipe-in-cli
František Kučera <franta-hg@frantovo.cz>
parents:
184
diff
changeset
|
24 |
"a" "integer" \ |
6cd2e54d90ea
examples: relpipe-in-cli
František Kučera <franta-hg@frantovo.cz>
parents:
184
diff
changeset
|
25 |
"b" "string" \ |
6cd2e54d90ea
examples: relpipe-in-cli
František Kučera <franta-hg@frantovo.cz>
parents:
184
diff
changeset
|
26 |
"c" "boolean" \ |
6cd2e54d90ea
examples: relpipe-in-cli
František Kučera <franta-hg@frantovo.cz>
parents:
184
diff
changeset
|
27 |
"1" "Hello" "true" \ |
6cd2e54d90ea
examples: relpipe-in-cli
František Kučera <franta-hg@frantovo.cz>
parents:
184
diff
changeset
|
28 |
"2" "World!" "false"]]></m:pre> |
6cd2e54d90ea
examples: relpipe-in-cli
František Kučera <franta-hg@frantovo.cz>
parents:
184
diff
changeset
|
29 |
|
6cd2e54d90ea
examples: relpipe-in-cli
František Kučera <franta-hg@frantovo.cz>
parents:
184
diff
changeset
|
30 |
<p> |
6cd2e54d90ea
examples: relpipe-in-cli
František Kučera <franta-hg@frantovo.cz>
parents:
184
diff
changeset
|
31 |
This command generates relational data. |
6cd2e54d90ea
examples: relpipe-in-cli
František Kučera <franta-hg@frantovo.cz>
parents:
184
diff
changeset
|
32 |
In order to see them, we need to convert them to some other format. |
6cd2e54d90ea
examples: relpipe-in-cli
František Kučera <franta-hg@frantovo.cz>
parents:
184
diff
changeset
|
33 |
For now, we will use the "tabular" format and pipe relational data to the <code>relpipe-out-tabular</code>. |
6cd2e54d90ea
examples: relpipe-in-cli
František Kučera <franta-hg@frantovo.cz>
parents:
184
diff
changeset
|
34 |
</p> |
6cd2e54d90ea
examples: relpipe-in-cli
František Kučera <franta-hg@frantovo.cz>
parents:
184
diff
changeset
|
35 |
|
6cd2e54d90ea
examples: relpipe-in-cli
František Kučera <franta-hg@frantovo.cz>
parents:
184
diff
changeset
|
36 |
<m:pre jazyk="bash"><![CDATA[relpipe-in-cli generate "relation_from_cli" 3 \ |
6cd2e54d90ea
examples: relpipe-in-cli
František Kučera <franta-hg@frantovo.cz>
parents:
184
diff
changeset
|
37 |
"a" "integer" \ |
6cd2e54d90ea
examples: relpipe-in-cli
František Kučera <franta-hg@frantovo.cz>
parents:
184
diff
changeset
|
38 |
"b" "string" \ |
6cd2e54d90ea
examples: relpipe-in-cli
František Kučera <franta-hg@frantovo.cz>
parents:
184
diff
changeset
|
39 |
"c" "boolean" \ |
6cd2e54d90ea
examples: relpipe-in-cli
František Kučera <franta-hg@frantovo.cz>
parents:
184
diff
changeset
|
40 |
"1" "Hello" "true" \ |
6cd2e54d90ea
examples: relpipe-in-cli
František Kučera <franta-hg@frantovo.cz>
parents:
184
diff
changeset
|
41 |
"2" "World!" "false" \ |
6cd2e54d90ea
examples: relpipe-in-cli
František Kučera <franta-hg@frantovo.cz>
parents:
184
diff
changeset
|
42 |
| relpipe-out-tabular]]></m:pre> |
6cd2e54d90ea
examples: relpipe-in-cli
František Kučera <franta-hg@frantovo.cz>
parents:
184
diff
changeset
|
43 |
|
6cd2e54d90ea
examples: relpipe-in-cli
František Kučera <franta-hg@frantovo.cz>
parents:
184
diff
changeset
|
44 |
<p>Output:</p> |
6cd2e54d90ea
examples: relpipe-in-cli
František Kučera <franta-hg@frantovo.cz>
parents:
184
diff
changeset
|
45 |
|
6cd2e54d90ea
examples: relpipe-in-cli
František Kučera <franta-hg@frantovo.cz>
parents:
184
diff
changeset
|
46 |
<pre><![CDATA[relation_from_cli: |
6cd2e54d90ea
examples: relpipe-in-cli
František Kučera <franta-hg@frantovo.cz>
parents:
184
diff
changeset
|
47 |
╭─────────────┬────────────┬─────────────╮ |
6cd2e54d90ea
examples: relpipe-in-cli
František Kučera <franta-hg@frantovo.cz>
parents:
184
diff
changeset
|
48 |
│ a (integer) │ b (string) │ c (boolean) │ |
6cd2e54d90ea
examples: relpipe-in-cli
František Kučera <franta-hg@frantovo.cz>
parents:
184
diff
changeset
|
49 |
├─────────────┼────────────┼─────────────┤ |
6cd2e54d90ea
examples: relpipe-in-cli
František Kučera <franta-hg@frantovo.cz>
parents:
184
diff
changeset
|
50 |
│ 1 │ Hello │ true │ |
6cd2e54d90ea
examples: relpipe-in-cli
František Kučera <franta-hg@frantovo.cz>
parents:
184
diff
changeset
|
51 |
│ 2 │ World! │ false │ |
6cd2e54d90ea
examples: relpipe-in-cli
František Kučera <franta-hg@frantovo.cz>
parents:
184
diff
changeset
|
52 |
╰─────────────┴────────────┴─────────────╯ |
6cd2e54d90ea
examples: relpipe-in-cli
František Kučera <franta-hg@frantovo.cz>
parents:
184
diff
changeset
|
53 |
Record count: 2 |
6cd2e54d90ea
examples: relpipe-in-cli
František Kučera <franta-hg@frantovo.cz>
parents:
184
diff
changeset
|
54 |
]]></pre> |
6cd2e54d90ea
examples: relpipe-in-cli
František Kučera <franta-hg@frantovo.cz>
parents:
184
diff
changeset
|
55 |
|
6cd2e54d90ea
examples: relpipe-in-cli
František Kučera <franta-hg@frantovo.cz>
parents:
184
diff
changeset
|
56 |
<p> |
6cd2e54d90ea
examples: relpipe-in-cli
František Kučera <franta-hg@frantovo.cz>
parents:
184
diff
changeset
|
57 |
The syntax is simple as we see above. We specify the name of the relation, number of attributes, |
6cd2e54d90ea
examples: relpipe-in-cli
František Kučera <franta-hg@frantovo.cz>
parents:
184
diff
changeset
|
58 |
and then their definitions (names and types), |
6cd2e54d90ea
examples: relpipe-in-cli
František Kučera <franta-hg@frantovo.cz>
parents:
184
diff
changeset
|
59 |
followed by the data. |
6cd2e54d90ea
examples: relpipe-in-cli
František Kučera <franta-hg@frantovo.cz>
parents:
184
diff
changeset
|
60 |
</p> |
196
8dd5b7ecb736
examples: multiple relations in a single stream
František Kučera <franta-hg@frantovo.cz>
parents:
195
diff
changeset
|
61 |
|
8dd5b7ecb736
examples: multiple relations in a single stream
František Kučera <franta-hg@frantovo.cz>
parents:
195
diff
changeset
|
62 |
<p> |
8dd5b7ecb736
examples: multiple relations in a single stream
František Kučera <franta-hg@frantovo.cz>
parents:
195
diff
changeset
|
63 |
A single stream may contain multiple relations: |
8dd5b7ecb736
examples: multiple relations in a single stream
František Kučera <franta-hg@frantovo.cz>
parents:
195
diff
changeset
|
64 |
</p> |
8dd5b7ecb736
examples: multiple relations in a single stream
František Kučera <franta-hg@frantovo.cz>
parents:
195
diff
changeset
|
65 |
|
8dd5b7ecb736
examples: multiple relations in a single stream
František Kučera <franta-hg@frantovo.cz>
parents:
195
diff
changeset
|
66 |
<m:pre jazyk="bash"><![CDATA[(relpipe-in-cli generate a 1 x string hello; \ |
8dd5b7ecb736
examples: multiple relations in a single stream
František Kučera <franta-hg@frantovo.cz>
parents:
195
diff
changeset
|
67 |
relpipe-in-cli generate b 1 y string world) \ |
8dd5b7ecb736
examples: multiple relations in a single stream
František Kučera <franta-hg@frantovo.cz>
parents:
195
diff
changeset
|
68 |
| relpipe-out-tabular]]></m:pre> |
8dd5b7ecb736
examples: multiple relations in a single stream
František Kučera <franta-hg@frantovo.cz>
parents:
195
diff
changeset
|
69 |
|
8dd5b7ecb736
examples: multiple relations in a single stream
František Kučera <franta-hg@frantovo.cz>
parents:
195
diff
changeset
|
70 |
<p> |
8dd5b7ecb736
examples: multiple relations in a single stream
František Kučera <franta-hg@frantovo.cz>
parents:
195
diff
changeset
|
71 |
Thus we can combine various commands or files and pass the result to a single relational output filter (<code>relpipe-out-tabular</code> in this case) and get: |
8dd5b7ecb736
examples: multiple relations in a single stream
František Kučera <franta-hg@frantovo.cz>
parents:
195
diff
changeset
|
72 |
</p> |
8dd5b7ecb736
examples: multiple relations in a single stream
František Kučera <franta-hg@frantovo.cz>
parents:
195
diff
changeset
|
73 |
|
8dd5b7ecb736
examples: multiple relations in a single stream
František Kučera <franta-hg@frantovo.cz>
parents:
195
diff
changeset
|
74 |
<pre><![CDATA[a: |
8dd5b7ecb736
examples: multiple relations in a single stream
František Kučera <franta-hg@frantovo.cz>
parents:
195
diff
changeset
|
75 |
╭────────────╮ |
8dd5b7ecb736
examples: multiple relations in a single stream
František Kučera <franta-hg@frantovo.cz>
parents:
195
diff
changeset
|
76 |
│ x (string) │ |
8dd5b7ecb736
examples: multiple relations in a single stream
František Kučera <franta-hg@frantovo.cz>
parents:
195
diff
changeset
|
77 |
├────────────┤ |
8dd5b7ecb736
examples: multiple relations in a single stream
František Kučera <franta-hg@frantovo.cz>
parents:
195
diff
changeset
|
78 |
│ hello │ |
8dd5b7ecb736
examples: multiple relations in a single stream
František Kučera <franta-hg@frantovo.cz>
parents:
195
diff
changeset
|
79 |
╰────────────╯ |
8dd5b7ecb736
examples: multiple relations in a single stream
František Kučera <franta-hg@frantovo.cz>
parents:
195
diff
changeset
|
80 |
Record count: 1 |
8dd5b7ecb736
examples: multiple relations in a single stream
František Kučera <franta-hg@frantovo.cz>
parents:
195
diff
changeset
|
81 |
b: |
8dd5b7ecb736
examples: multiple relations in a single stream
František Kučera <franta-hg@frantovo.cz>
parents:
195
diff
changeset
|
82 |
╭────────────╮ |
8dd5b7ecb736
examples: multiple relations in a single stream
František Kučera <franta-hg@frantovo.cz>
parents:
195
diff
changeset
|
83 |
│ y (string) │ |
8dd5b7ecb736
examples: multiple relations in a single stream
František Kučera <franta-hg@frantovo.cz>
parents:
195
diff
changeset
|
84 |
├────────────┤ |
8dd5b7ecb736
examples: multiple relations in a single stream
František Kučera <franta-hg@frantovo.cz>
parents:
195
diff
changeset
|
85 |
│ world │ |
8dd5b7ecb736
examples: multiple relations in a single stream
František Kučera <franta-hg@frantovo.cz>
parents:
195
diff
changeset
|
86 |
╰────────────╯ |
8dd5b7ecb736
examples: multiple relations in a single stream
František Kučera <franta-hg@frantovo.cz>
parents:
195
diff
changeset
|
87 |
Record count: 1]]></pre> |
185
6cd2e54d90ea
examples: relpipe-in-cli
František Kučera <franta-hg@frantovo.cz>
parents:
184
diff
changeset
|
88 |
|
233
a8029bd1c87a
examples: fix header levels
František Kučera <franta-hg@frantovo.cz>
parents:
232
diff
changeset
|
89 |
<h2>relpipe-in-cli: STDIN</h2> |
185
6cd2e54d90ea
examples: relpipe-in-cli
František Kučera <franta-hg@frantovo.cz>
parents:
184
diff
changeset
|
90 |
|
6cd2e54d90ea
examples: relpipe-in-cli
František Kučera <franta-hg@frantovo.cz>
parents:
184
diff
changeset
|
91 |
<p> |
214
63f31bf2697f
examples: Writing an output filter in Bash
František Kučera <franta-hg@frantovo.cz>
parents:
213
diff
changeset
|
92 |
The number of <abbr title="Command-line interface">CLI</abbr> arguments is limited and they are passed at once to the process. |
185
6cd2e54d90ea
examples: relpipe-in-cli
František Kučera <franta-hg@frantovo.cz>
parents:
184
diff
changeset
|
93 |
So there is option to pass the values from STDIN instead of CLI arguments. |
6cd2e54d90ea
examples: relpipe-in-cli
František Kučera <franta-hg@frantovo.cz>
parents:
184
diff
changeset
|
94 |
Values on STDIN are expected to be separated by the null-byte. |
6cd2e54d90ea
examples: relpipe-in-cli
František Kučera <franta-hg@frantovo.cz>
parents:
184
diff
changeset
|
95 |
We can generate such data e.g. using <code>echo</code> and <code>tr</code> (or using <code>printf</code> or other commands): |
6cd2e54d90ea
examples: relpipe-in-cli
František Kučera <franta-hg@frantovo.cz>
parents:
184
diff
changeset
|
96 |
</p> |
6cd2e54d90ea
examples: relpipe-in-cli
František Kučera <franta-hg@frantovo.cz>
parents:
184
diff
changeset
|
97 |
|
6cd2e54d90ea
examples: relpipe-in-cli
František Kučera <franta-hg@frantovo.cz>
parents:
184
diff
changeset
|
98 |
<m:pre jazyk="bash"><![CDATA[echo -e "1\nHello\ntrue\n2\nWorld\nfalse" \ |
6cd2e54d90ea
examples: relpipe-in-cli
František Kučera <franta-hg@frantovo.cz>
parents:
184
diff
changeset
|
99 |
| tr \\n \\0 \ |
6cd2e54d90ea
examples: relpipe-in-cli
František Kučera <franta-hg@frantovo.cz>
parents:
184
diff
changeset
|
100 |
| relpipe-in-cli generate-from-stdin relation_from_stdin 3 \ |
6cd2e54d90ea
examples: relpipe-in-cli
František Kučera <franta-hg@frantovo.cz>
parents:
184
diff
changeset
|
101 |
a integer \ |
6cd2e54d90ea
examples: relpipe-in-cli
František Kučera <franta-hg@frantovo.cz>
parents:
184
diff
changeset
|
102 |
b string \ |
6cd2e54d90ea
examples: relpipe-in-cli
František Kučera <franta-hg@frantovo.cz>
parents:
184
diff
changeset
|
103 |
c boolean \ |
6cd2e54d90ea
examples: relpipe-in-cli
František Kučera <franta-hg@frantovo.cz>
parents:
184
diff
changeset
|
104 |
| relpipe-out-tabular]]></m:pre> |
6cd2e54d90ea
examples: relpipe-in-cli
František Kučera <franta-hg@frantovo.cz>
parents:
184
diff
changeset
|
105 |
|
6cd2e54d90ea
examples: relpipe-in-cli
František Kučera <franta-hg@frantovo.cz>
parents:
184
diff
changeset
|
106 |
<p> |
6cd2e54d90ea
examples: relpipe-in-cli
František Kučera <franta-hg@frantovo.cz>
parents:
184
diff
changeset
|
107 |
The output is same as above. |
6cd2e54d90ea
examples: relpipe-in-cli
František Kučera <franta-hg@frantovo.cz>
parents:
184
diff
changeset
|
108 |
We can use this approach to convert various formats to relational data. |
6cd2e54d90ea
examples: relpipe-in-cli
František Kučera <franta-hg@frantovo.cz>
parents:
184
diff
changeset
|
109 |
There are lot of data already in the form of null-separated values e.g. the process arguments: |
6cd2e54d90ea
examples: relpipe-in-cli
František Kučera <franta-hg@frantovo.cz>
parents:
184
diff
changeset
|
110 |
</p> |
6cd2e54d90ea
examples: relpipe-in-cli
František Kučera <franta-hg@frantovo.cz>
parents:
184
diff
changeset
|
111 |
|
6cd2e54d90ea
examples: relpipe-in-cli
František Kučera <franta-hg@frantovo.cz>
parents:
184
diff
changeset
|
112 |
<m:pre jazyk="bash"><![CDATA[cat /proc/$(pidof mc)/cmdline \ |
6cd2e54d90ea
examples: relpipe-in-cli
František Kučera <franta-hg@frantovo.cz>
parents:
184
diff
changeset
|
113 |
| relpipe-in-cli generate-from-stdin mc_args 1 a string \ |
6cd2e54d90ea
examples: relpipe-in-cli
František Kučera <franta-hg@frantovo.cz>
parents:
184
diff
changeset
|
114 |
| relpipe-out-tabular |
6cd2e54d90ea
examples: relpipe-in-cli
František Kučera <franta-hg@frantovo.cz>
parents:
184
diff
changeset
|
115 |
]]></m:pre> |
6cd2e54d90ea
examples: relpipe-in-cli
František Kučera <franta-hg@frantovo.cz>
parents:
184
diff
changeset
|
116 |
|
6cd2e54d90ea
examples: relpipe-in-cli
František Kučera <franta-hg@frantovo.cz>
parents:
184
diff
changeset
|
117 |
<p>If we have <code>mc /etc/ /tmp/</code> running in some other terminal, the output will be:</p> |
6cd2e54d90ea
examples: relpipe-in-cli
František Kučera <franta-hg@frantovo.cz>
parents:
184
diff
changeset
|
118 |
|
6cd2e54d90ea
examples: relpipe-in-cli
František Kučera <franta-hg@frantovo.cz>
parents:
184
diff
changeset
|
119 |
<pre><![CDATA[mc_args: |
6cd2e54d90ea
examples: relpipe-in-cli
František Kučera <franta-hg@frantovo.cz>
parents:
184
diff
changeset
|
120 |
╭────────────╮ |
6cd2e54d90ea
examples: relpipe-in-cli
František Kučera <franta-hg@frantovo.cz>
parents:
184
diff
changeset
|
121 |
│ a (string) │ |
6cd2e54d90ea
examples: relpipe-in-cli
František Kučera <franta-hg@frantovo.cz>
parents:
184
diff
changeset
|
122 |
├────────────┤ |
6cd2e54d90ea
examples: relpipe-in-cli
František Kučera <franta-hg@frantovo.cz>
parents:
184
diff
changeset
|
123 |
│ mc │ |
6cd2e54d90ea
examples: relpipe-in-cli
František Kučera <franta-hg@frantovo.cz>
parents:
184
diff
changeset
|
124 |
│ /etc/ │ |
6cd2e54d90ea
examples: relpipe-in-cli
František Kučera <franta-hg@frantovo.cz>
parents:
184
diff
changeset
|
125 |
│ /tmp/ │ |
6cd2e54d90ea
examples: relpipe-in-cli
František Kučera <franta-hg@frantovo.cz>
parents:
184
diff
changeset
|
126 |
╰────────────╯ |
6cd2e54d90ea
examples: relpipe-in-cli
František Kučera <franta-hg@frantovo.cz>
parents:
184
diff
changeset
|
127 |
Record count: 3]]></pre> |
6cd2e54d90ea
examples: relpipe-in-cli
František Kučera <franta-hg@frantovo.cz>
parents:
184
diff
changeset
|
128 |
|
6cd2e54d90ea
examples: relpipe-in-cli
František Kučera <franta-hg@frantovo.cz>
parents:
184
diff
changeset
|
129 |
<p> |
6cd2e54d90ea
examples: relpipe-in-cli
František Kučera <franta-hg@frantovo.cz>
parents:
184
diff
changeset
|
130 |
Also the <code>find</code> command can produce data separated by the null-byte: |
6cd2e54d90ea
examples: relpipe-in-cli
František Kučera <franta-hg@frantovo.cz>
parents:
184
diff
changeset
|
131 |
</p> |
6cd2e54d90ea
examples: relpipe-in-cli
František Kučera <franta-hg@frantovo.cz>
parents:
184
diff
changeset
|
132 |
|
6cd2e54d90ea
examples: relpipe-in-cli
František Kučera <franta-hg@frantovo.cz>
parents:
184
diff
changeset
|
133 |
<m:pre jazyk="bash"><![CDATA[find /etc/ -name '*ssh*_*' -print0 \ |
6cd2e54d90ea
examples: relpipe-in-cli
František Kučera <franta-hg@frantovo.cz>
parents:
184
diff
changeset
|
134 |
| relpipe-in-cli generate-from-stdin files 1 file_name string \ |
6cd2e54d90ea
examples: relpipe-in-cli
František Kučera <franta-hg@frantovo.cz>
parents:
184
diff
changeset
|
135 |
| relpipe-out-tabular]]></m:pre> |
6cd2e54d90ea
examples: relpipe-in-cli
František Kučera <franta-hg@frantovo.cz>
parents:
184
diff
changeset
|
136 |
|
6cd2e54d90ea
examples: relpipe-in-cli
František Kučera <franta-hg@frantovo.cz>
parents:
184
diff
changeset
|
137 |
<p>Will display something like this:</p> |
6cd2e54d90ea
examples: relpipe-in-cli
František Kučera <franta-hg@frantovo.cz>
parents:
184
diff
changeset
|
138 |
|
6cd2e54d90ea
examples: relpipe-in-cli
František Kučera <franta-hg@frantovo.cz>
parents:
184
diff
changeset
|
139 |
<pre><![CDATA[files: |
6cd2e54d90ea
examples: relpipe-in-cli
František Kučera <franta-hg@frantovo.cz>
parents:
184
diff
changeset
|
140 |
╭───────────────────────────────────╮ |
6cd2e54d90ea
examples: relpipe-in-cli
František Kučera <franta-hg@frantovo.cz>
parents:
184
diff
changeset
|
141 |
│ file_name (string) │ |
6cd2e54d90ea
examples: relpipe-in-cli
František Kučera <franta-hg@frantovo.cz>
parents:
184
diff
changeset
|
142 |
├───────────────────────────────────┤ |
6cd2e54d90ea
examples: relpipe-in-cli
František Kučera <franta-hg@frantovo.cz>
parents:
184
diff
changeset
|
143 |
│ /etc/ssh/ssh_host_ecdsa_key │ |
6cd2e54d90ea
examples: relpipe-in-cli
František Kučera <franta-hg@frantovo.cz>
parents:
184
diff
changeset
|
144 |
│ /etc/ssh/sshd_config │ |
6cd2e54d90ea
examples: relpipe-in-cli
František Kučera <franta-hg@frantovo.cz>
parents:
184
diff
changeset
|
145 |
│ /etc/ssh/ssh_host_ed25519_key.pub │ |
6cd2e54d90ea
examples: relpipe-in-cli
František Kučera <franta-hg@frantovo.cz>
parents:
184
diff
changeset
|
146 |
│ /etc/ssh/ssh_host_ecdsa_key.pub │ |
6cd2e54d90ea
examples: relpipe-in-cli
František Kučera <franta-hg@frantovo.cz>
parents:
184
diff
changeset
|
147 |
│ /etc/ssh/ssh_host_rsa_key │ |
6cd2e54d90ea
examples: relpipe-in-cli
František Kučera <franta-hg@frantovo.cz>
parents:
184
diff
changeset
|
148 |
│ /etc/ssh/ssh_config │ |
6cd2e54d90ea
examples: relpipe-in-cli
František Kučera <franta-hg@frantovo.cz>
parents:
184
diff
changeset
|
149 |
│ /etc/ssh/ssh_host_ed25519_key │ |
6cd2e54d90ea
examples: relpipe-in-cli
František Kučera <franta-hg@frantovo.cz>
parents:
184
diff
changeset
|
150 |
│ /etc/ssh/ssh_import_id │ |
6cd2e54d90ea
examples: relpipe-in-cli
František Kučera <franta-hg@frantovo.cz>
parents:
184
diff
changeset
|
151 |
│ /etc/ssh/ssh_host_rsa_key.pub │ |
6cd2e54d90ea
examples: relpipe-in-cli
František Kučera <franta-hg@frantovo.cz>
parents:
184
diff
changeset
|
152 |
╰───────────────────────────────────╯ |
6cd2e54d90ea
examples: relpipe-in-cli
František Kučera <franta-hg@frantovo.cz>
parents:
184
diff
changeset
|
153 |
Record count: 9]]></pre> |
6cd2e54d90ea
examples: relpipe-in-cli
František Kučera <franta-hg@frantovo.cz>
parents:
184
diff
changeset
|
154 |
|
6cd2e54d90ea
examples: relpipe-in-cli
František Kučera <franta-hg@frantovo.cz>
parents:
184
diff
changeset
|
155 |
|
233
a8029bd1c87a
examples: fix header levels
František Kučera <franta-hg@frantovo.cz>
parents:
232
diff
changeset
|
156 |
<h2>relpipe-in-fstab</h2> |
186
e33f13c91289
examples: relpipe-in-fstab
František Kučera <franta-hg@frantovo.cz>
parents:
185
diff
changeset
|
157 |
|
e33f13c91289
examples: relpipe-in-fstab
František Kučera <franta-hg@frantovo.cz>
parents:
185
diff
changeset
|
158 |
<p> |
e33f13c91289
examples: relpipe-in-fstab
František Kučera <franta-hg@frantovo.cz>
parents:
185
diff
changeset
|
159 |
Using command <code>relpipe-in-fstab</code> we can convert the <code>/etc/fstab</code> or <code>/etc/mtab</code> to relational data |
e33f13c91289
examples: relpipe-in-fstab
František Kučera <franta-hg@frantovo.cz>
parents:
185
diff
changeset
|
160 |
</p> |
e33f13c91289
examples: relpipe-in-fstab
František Kučera <franta-hg@frantovo.cz>
parents:
185
diff
changeset
|
161 |
|
e33f13c91289
examples: relpipe-in-fstab
František Kučera <franta-hg@frantovo.cz>
parents:
185
diff
changeset
|
162 |
<m:pre jazyk="bash"><![CDATA[relpipe-in-fstab | relpipe-out-tabular]]></m:pre> |
e33f13c91289
examples: relpipe-in-fstab
František Kučera <franta-hg@frantovo.cz>
parents:
185
diff
changeset
|
163 |
|
e33f13c91289
examples: relpipe-in-fstab
František Kučera <franta-hg@frantovo.cz>
parents:
185
diff
changeset
|
164 |
<p> |
e33f13c91289
examples: relpipe-in-fstab
František Kučera <franta-hg@frantovo.cz>
parents:
185
diff
changeset
|
165 |
and see them as a nice table: |
e33f13c91289
examples: relpipe-in-fstab
František Kučera <franta-hg@frantovo.cz>
parents:
185
diff
changeset
|
166 |
</p> |
e33f13c91289
examples: relpipe-in-fstab
František Kučera <franta-hg@frantovo.cz>
parents:
185
diff
changeset
|
167 |
|
e33f13c91289
examples: relpipe-in-fstab
František Kučera <franta-hg@frantovo.cz>
parents:
185
diff
changeset
|
168 |
<pre><![CDATA[fstab: |
e33f13c91289
examples: relpipe-in-fstab
František Kučera <franta-hg@frantovo.cz>
parents:
185
diff
changeset
|
169 |
╭─────────────────┬──────────────────────────────────────┬──────────────────────┬───────────────┬───────────────────────────────────────┬────────────────┬────────────────╮ |
e33f13c91289
examples: relpipe-in-fstab
František Kučera <franta-hg@frantovo.cz>
parents:
185
diff
changeset
|
170 |
│ scheme (string) │ device (string) │ mount_point (string) │ type (string) │ options (string) │ dump (integer) │ pass (integer) │ |
e33f13c91289
examples: relpipe-in-fstab
František Kučera <franta-hg@frantovo.cz>
parents:
185
diff
changeset
|
171 |
├─────────────────┼──────────────────────────────────────┼──────────────────────┼───────────────┼───────────────────────────────────────┼────────────────┼────────────────┤ |
e33f13c91289
examples: relpipe-in-fstab
František Kučera <franta-hg@frantovo.cz>
parents:
185
diff
changeset
|
172 |
│ UUID │ 29758270-fd25-4a6c-a7bb-9a18302816af │ / │ ext4 │ relatime,user_xattr,errors=remount-ro │ 0 │ 1 │ |
e33f13c91289
examples: relpipe-in-fstab
František Kučera <franta-hg@frantovo.cz>
parents:
185
diff
changeset
|
173 |
│ │ /dev/sr0 │ /media/cdrom0 │ udf,iso9660 │ user,noauto │ 0 │ 0 │ |
e33f13c91289
examples: relpipe-in-fstab
František Kučera <franta-hg@frantovo.cz>
parents:
185
diff
changeset
|
174 |
│ │ /dev/sde │ /mnt/data │ ext4 │ relatime,user_xattr,errors=remount-ro │ 0 │ 2 │ |
e33f13c91289
examples: relpipe-in-fstab
František Kučera <franta-hg@frantovo.cz>
parents:
185
diff
changeset
|
175 |
│ UUID │ a2b5f230-a795-4f6f-a39b-9b57686c86d5 │ /home │ btrfs │ relatime │ 0 │ 2 │ |
e33f13c91289
examples: relpipe-in-fstab
František Kučera <franta-hg@frantovo.cz>
parents:
185
diff
changeset
|
176 |
│ │ /dev/mapper/sdf_crypt │ /mnt/private │ xfs │ relatime │ 0 │ 2 │ |
e33f13c91289
examples: relpipe-in-fstab
František Kučera <franta-hg@frantovo.cz>
parents:
185
diff
changeset
|
177 |
╰─────────────────┴──────────────────────────────────────┴──────────────────────┴───────────────┴───────────────────────────────────────┴────────────────┴────────────────╯ |
e33f13c91289
examples: relpipe-in-fstab
František Kučera <franta-hg@frantovo.cz>
parents:
185
diff
changeset
|
178 |
Record count: 5]]></pre> |
e33f13c91289
examples: relpipe-in-fstab
František Kučera <franta-hg@frantovo.cz>
parents:
185
diff
changeset
|
179 |
|
e33f13c91289
examples: relpipe-in-fstab
František Kučera <franta-hg@frantovo.cz>
parents:
185
diff
changeset
|
180 |
<p>And we can do the same also with a remote <code>fstab</code> or <code>mtab</code>; just by adding <code>ssh</code> to the pipeline:</p> |
e33f13c91289
examples: relpipe-in-fstab
František Kučera <franta-hg@frantovo.cz>
parents:
185
diff
changeset
|
181 |
|
e33f13c91289
examples: relpipe-in-fstab
František Kučera <franta-hg@frantovo.cz>
parents:
185
diff
changeset
|
182 |
<m:pre jazyk="bash"><![CDATA[ssh example.com cat /etc/mtab | relpipe-in-fstab | relpipe-out-tabular]]></m:pre> |
e33f13c91289
examples: relpipe-in-fstab
František Kučera <franta-hg@frantovo.cz>
parents:
185
diff
changeset
|
183 |
|
e33f13c91289
examples: relpipe-in-fstab
František Kučera <franta-hg@frantovo.cz>
parents:
185
diff
changeset
|
184 |
<p> |
e33f13c91289
examples: relpipe-in-fstab
František Kučera <franta-hg@frantovo.cz>
parents:
185
diff
changeset
|
185 |
The <code>cat</code> runs remotely. The <code>relpipe-in-fstab</code> and <code>relpipe-out-tabular</code> run on our machine. |
e33f13c91289
examples: relpipe-in-fstab
František Kučera <franta-hg@frantovo.cz>
parents:
185
diff
changeset
|
186 |
</p> |
e33f13c91289
examples: relpipe-in-fstab
František Kučera <franta-hg@frantovo.cz>
parents:
185
diff
changeset
|
187 |
|
e33f13c91289
examples: relpipe-in-fstab
František Kučera <franta-hg@frantovo.cz>
parents:
185
diff
changeset
|
188 |
<p> |
e33f13c91289
examples: relpipe-in-fstab
František Kučera <franta-hg@frantovo.cz>
parents:
185
diff
changeset
|
189 |
n.b. the <code>relpipe-in-fstab</code> reads the <code>/etc/fstab</code> if executed on TTY. Otherwise, it reads the STDIN. |
e33f13c91289
examples: relpipe-in-fstab
František Kučera <franta-hg@frantovo.cz>
parents:
185
diff
changeset
|
190 |
</p> |
e33f13c91289
examples: relpipe-in-fstab
František Kučera <franta-hg@frantovo.cz>
parents:
185
diff
changeset
|
191 |
|
233
a8029bd1c87a
examples: fix header levels
František Kučera <franta-hg@frantovo.cz>
parents:
232
diff
changeset
|
192 |
<h2>relpipe-out-xml</h2> |
187
c952261978e8
examples: relpipe-out-xml
František Kučera <franta-hg@frantovo.cz>
parents:
186
diff
changeset
|
193 |
|
c952261978e8
examples: relpipe-out-xml
František Kučera <franta-hg@frantovo.cz>
parents:
186
diff
changeset
|
194 |
<p> |
c952261978e8
examples: relpipe-out-xml
František Kučera <franta-hg@frantovo.cz>
parents:
186
diff
changeset
|
195 |
Relational data can be converted to various formats and one of them is the XML. |
c952261978e8
examples: relpipe-out-xml
František Kučera <franta-hg@frantovo.cz>
parents:
186
diff
changeset
|
196 |
This is a good option for further processing e.g. using XSLT transformation or passing the XML data to some other tool. |
c952261978e8
examples: relpipe-out-xml
František Kučera <franta-hg@frantovo.cz>
parents:
186
diff
changeset
|
197 |
Just use <code>relpipe-out-xml</code> instead of <code>relpipe-out-tabular</code> and the rest of the pipeline remains unchanged: |
c952261978e8
examples: relpipe-out-xml
František Kučera <franta-hg@frantovo.cz>
parents:
186
diff
changeset
|
198 |
</p> |
c952261978e8
examples: relpipe-out-xml
František Kučera <franta-hg@frantovo.cz>
parents:
186
diff
changeset
|
199 |
|
c952261978e8
examples: relpipe-out-xml
František Kučera <franta-hg@frantovo.cz>
parents:
186
diff
changeset
|
200 |
<m:pre jazyk="bash"><![CDATA[ssh example.com cat /etc/mtab | relpipe-in-fstab | relpipe-out-xml]]></m:pre> |
c952261978e8
examples: relpipe-out-xml
František Kučera <franta-hg@frantovo.cz>
parents:
186
diff
changeset
|
201 |
|
c952261978e8
examples: relpipe-out-xml
František Kučera <franta-hg@frantovo.cz>
parents:
186
diff
changeset
|
202 |
<p> |
c952261978e8
examples: relpipe-out-xml
František Kučera <franta-hg@frantovo.cz>
parents:
186
diff
changeset
|
203 |
Will produce XML like this: |
c952261978e8
examples: relpipe-out-xml
František Kučera <franta-hg@frantovo.cz>
parents:
186
diff
changeset
|
204 |
</p> |
c952261978e8
examples: relpipe-out-xml
František Kučera <franta-hg@frantovo.cz>
parents:
186
diff
changeset
|
205 |
|
234
2fc56dd7f003
examples: update XML example (namespace, metadata) and move it to a separate file
František Kučera <franta-hg@frantovo.cz>
parents:
233
diff
changeset
|
206 |
<m:pre jazyk="xml" src="examples/relpipe-out-fstab.xml"/> |
187
c952261978e8
examples: relpipe-out-xml
František Kučera <franta-hg@frantovo.cz>
parents:
186
diff
changeset
|
207 |
|
c952261978e8
examples: relpipe-out-xml
František Kučera <franta-hg@frantovo.cz>
parents:
186
diff
changeset
|
208 |
<p> |
c952261978e8
examples: relpipe-out-xml
František Kučera <franta-hg@frantovo.cz>
parents:
186
diff
changeset
|
209 |
Thanks to XSLT, this XML can be easily converted e.g. to an XHTML table (<code>table|tr|td</code>) or other format. |
c952261978e8
examples: relpipe-out-xml
František Kučera <franta-hg@frantovo.cz>
parents:
186
diff
changeset
|
210 |
Someone can convert such data to a (La)TeX table. |
c952261978e8
examples: relpipe-out-xml
František Kučera <franta-hg@frantovo.cz>
parents:
186
diff
changeset
|
211 |
</p> |
c952261978e8
examples: relpipe-out-xml
František Kučera <franta-hg@frantovo.cz>
parents:
186
diff
changeset
|
212 |
|
c952261978e8
examples: relpipe-out-xml
František Kučera <franta-hg@frantovo.cz>
parents:
186
diff
changeset
|
213 |
<p> |
c952261978e8
examples: relpipe-out-xml
František Kučera <franta-hg@frantovo.cz>
parents:
186
diff
changeset
|
214 |
n.b. the format is not final and will change i future versions (XML namespace, more metadata etc.). |
c952261978e8
examples: relpipe-out-xml
František Kučera <franta-hg@frantovo.cz>
parents:
186
diff
changeset
|
215 |
</p> |
c952261978e8
examples: relpipe-out-xml
František Kučera <franta-hg@frantovo.cz>
parents:
186
diff
changeset
|
216 |
|
c952261978e8
examples: relpipe-out-xml
František Kučera <franta-hg@frantovo.cz>
parents:
186
diff
changeset
|
217 |
|
233
a8029bd1c87a
examples: fix header levels
František Kučera <franta-hg@frantovo.cz>
parents:
232
diff
changeset
|
218 |
<h2>relpipe-tr-validator</h2> |
176
46042297e0d6
relpipe-tr-validator: specification and examples
František Kučera <franta-hg@frantovo.cz>
parents:
140
diff
changeset
|
219 |
|
2
ab9099ff88fa
vkládání zápatí, jmenné prostory, saxon
František Kučera <franta-hg@frantovo.cz>
parents:
1
diff
changeset
|
220 |
<p> |
176
46042297e0d6
relpipe-tr-validator: specification and examples
František Kučera <franta-hg@frantovo.cz>
parents:
140
diff
changeset
|
221 |
Just a passthrough command, so these pipelines should produce the same hash: |
46042297e0d6
relpipe-tr-validator: specification and examples
František Kučera <franta-hg@frantovo.cz>
parents:
140
diff
changeset
|
222 |
</p> |
46042297e0d6
relpipe-tr-validator: specification and examples
František Kučera <franta-hg@frantovo.cz>
parents:
140
diff
changeset
|
223 |
|
46042297e0d6
relpipe-tr-validator: specification and examples
František Kučera <franta-hg@frantovo.cz>
parents:
140
diff
changeset
|
224 |
<m:pre jazyk="bash"><![CDATA[ |
46042297e0d6
relpipe-tr-validator: specification and examples
František Kučera <franta-hg@frantovo.cz>
parents:
140
diff
changeset
|
225 |
relpipe-in-fstab | relpipe-tr-validator | sha512sum |
46042297e0d6
relpipe-tr-validator: specification and examples
František Kučera <franta-hg@frantovo.cz>
parents:
140
diff
changeset
|
226 |
relpipe-in-fstab | sha512sum]]></m:pre> |
46042297e0d6
relpipe-tr-validator: specification and examples
František Kučera <franta-hg@frantovo.cz>
parents:
140
diff
changeset
|
227 |
|
46042297e0d6
relpipe-tr-validator: specification and examples
František Kučera <franta-hg@frantovo.cz>
parents:
140
diff
changeset
|
228 |
<p> |
46042297e0d6
relpipe-tr-validator: specification and examples
František Kučera <franta-hg@frantovo.cz>
parents:
140
diff
changeset
|
229 |
This tool can be used for testing whether a file contains valid relational data: |
87
25dec6931f18
Lepší odsazení, tabulátory.
František Kučera <franta-hg@frantovo.cz>
parents:
23
diff
changeset
|
230 |
</p> |
176
46042297e0d6
relpipe-tr-validator: specification and examples
František Kučera <franta-hg@frantovo.cz>
parents:
140
diff
changeset
|
231 |
|
46042297e0d6
relpipe-tr-validator: specification and examples
František Kučera <franta-hg@frantovo.cz>
parents:
140
diff
changeset
|
232 |
<m:pre jazyk="bash"><![CDATA[ |
46042297e0d6
relpipe-tr-validator: specification and examples
František Kučera <franta-hg@frantovo.cz>
parents:
140
diff
changeset
|
233 |
if relpipe-tr-validator < "some-file.rp" &> /dev/null; then |
46042297e0d6
relpipe-tr-validator: specification and examples
František Kučera <franta-hg@frantovo.cz>
parents:
140
diff
changeset
|
234 |
echo "valid relational data"; |
46042297e0d6
relpipe-tr-validator: specification and examples
František Kučera <franta-hg@frantovo.cz>
parents:
140
diff
changeset
|
235 |
else |
46042297e0d6
relpipe-tr-validator: specification and examples
František Kučera <franta-hg@frantovo.cz>
parents:
140
diff
changeset
|
236 |
echo "garbage"; |
46042297e0d6
relpipe-tr-validator: specification and examples
František Kučera <franta-hg@frantovo.cz>
parents:
140
diff
changeset
|
237 |
fi]]></m:pre> |
46042297e0d6
relpipe-tr-validator: specification and examples
František Kučera <franta-hg@frantovo.cz>
parents:
140
diff
changeset
|
238 |
|
46042297e0d6
relpipe-tr-validator: specification and examples
František Kučera <franta-hg@frantovo.cz>
parents:
140
diff
changeset
|
239 |
<p>or as a one-liner:</p> |
46042297e0d6
relpipe-tr-validator: specification and examples
František Kučera <franta-hg@frantovo.cz>
parents:
140
diff
changeset
|
240 |
|
46042297e0d6
relpipe-tr-validator: specification and examples
František Kučera <franta-hg@frantovo.cz>
parents:
140
diff
changeset
|
241 |
<m:pre jazyk="bash"><![CDATA[relpipe-tr-validator < "some-file.rp" &> /dev/null && echo "ok" || echo "error"]]></m:pre> |
46042297e0d6
relpipe-tr-validator: specification and examples
František Kučera <franta-hg@frantovo.cz>
parents:
140
diff
changeset
|
242 |
|
46042297e0d6
relpipe-tr-validator: specification and examples
František Kučera <franta-hg@frantovo.cz>
parents:
140
diff
changeset
|
243 |
<p> |
46042297e0d6
relpipe-tr-validator: specification and examples
František Kučera <franta-hg@frantovo.cz>
parents:
140
diff
changeset
|
244 |
If an error is found, it is reported on STDERR. So just omit the <code>&</code> in order to see the error message. |
46042297e0d6
relpipe-tr-validator: specification and examples
František Kučera <franta-hg@frantovo.cz>
parents:
140
diff
changeset
|
245 |
</p> |
46042297e0d6
relpipe-tr-validator: specification and examples
František Kučera <franta-hg@frantovo.cz>
parents:
140
diff
changeset
|
246 |
|
180
d342de2e09a4
examples: fstab formatting using -in-fstab, -out-nullbyte, xargs and Perl
František Kučera <franta-hg@frantovo.cz>
parents:
176
diff
changeset
|
247 |
|
233
a8029bd1c87a
examples: fix header levels
František Kučera <franta-hg@frantovo.cz>
parents:
232
diff
changeset
|
248 |
<h2>/etc/fstab formatting using -in-fstab, -out-nullbyte, xargs and Perl</h2> |
180
d342de2e09a4
examples: fstab formatting using -in-fstab, -out-nullbyte, xargs and Perl
František Kučera <franta-hg@frantovo.cz>
parents:
176
diff
changeset
|
249 |
|
d342de2e09a4
examples: fstab formatting using -in-fstab, -out-nullbyte, xargs and Perl
František Kučera <franta-hg@frantovo.cz>
parents:
176
diff
changeset
|
250 |
<p> |
d342de2e09a4
examples: fstab formatting using -in-fstab, -out-nullbyte, xargs and Perl
František Kučera <franta-hg@frantovo.cz>
parents:
176
diff
changeset
|
251 |
As we have seen before, we can convert <code>/etc/fstab</code> (or <code>mtab</code>) |
d342de2e09a4
examples: fstab formatting using -in-fstab, -out-nullbyte, xargs and Perl
František Kučera <franta-hg@frantovo.cz>
parents:
176
diff
changeset
|
252 |
to e.g. an XML or a nice and colorful table using <m:name/>. |
d342de2e09a4
examples: fstab formatting using -in-fstab, -out-nullbyte, xargs and Perl
František Kučera <franta-hg@frantovo.cz>
parents:
176
diff
changeset
|
253 |
But we can also convert these data back to the <code>fstab</code> format. And do it with proper indentation/padding. |
d342de2e09a4
examples: fstab formatting using -in-fstab, -out-nullbyte, xargs and Perl
František Kučera <franta-hg@frantovo.cz>
parents:
176
diff
changeset
|
254 |
Fstab has a simple format where values are separated by one or more whitespace characters. |
d342de2e09a4
examples: fstab formatting using -in-fstab, -out-nullbyte, xargs and Perl
František Kučera <franta-hg@frantovo.cz>
parents:
176
diff
changeset
|
255 |
But without proper indentation, these files look a bit obfuscated and hard to read (however, they are valid). |
d342de2e09a4
examples: fstab formatting using -in-fstab, -out-nullbyte, xargs and Perl
František Kučera <franta-hg@frantovo.cz>
parents:
176
diff
changeset
|
256 |
</p> |
d342de2e09a4
examples: fstab formatting using -in-fstab, -out-nullbyte, xargs and Perl
František Kučera <franta-hg@frantovo.cz>
parents:
176
diff
changeset
|
257 |
|
d342de2e09a4
examples: fstab formatting using -in-fstab, -out-nullbyte, xargs and Perl
František Kučera <franta-hg@frantovo.cz>
parents:
176
diff
changeset
|
258 |
<m:pre jazyk="text" src="examples/relpipe-out-fstab.txt"/> |
d342de2e09a4
examples: fstab formatting using -in-fstab, -out-nullbyte, xargs and Perl
František Kučera <franta-hg@frantovo.cz>
parents:
176
diff
changeset
|
259 |
|
d342de2e09a4
examples: fstab formatting using -in-fstab, -out-nullbyte, xargs and Perl
František Kučera <franta-hg@frantovo.cz>
parents:
176
diff
changeset
|
260 |
<p> |
d342de2e09a4
examples: fstab formatting using -in-fstab, -out-nullbyte, xargs and Perl
František Kučera <franta-hg@frantovo.cz>
parents:
176
diff
changeset
|
261 |
So let's build a pipeline that reformats the <code>fstab</code> and makes it more readable. |
d342de2e09a4
examples: fstab formatting using -in-fstab, -out-nullbyte, xargs and Perl
František Kučera <franta-hg@frantovo.cz>
parents:
176
diff
changeset
|
262 |
</p> |
d342de2e09a4
examples: fstab formatting using -in-fstab, -out-nullbyte, xargs and Perl
František Kučera <franta-hg@frantovo.cz>
parents:
176
diff
changeset
|
263 |
|
d342de2e09a4
examples: fstab formatting using -in-fstab, -out-nullbyte, xargs and Perl
František Kučera <franta-hg@frantovo.cz>
parents:
176
diff
changeset
|
264 |
<m:pre jazyk="bash">relpipe-in-fstab | relpipe-out-fstab > reformatted-fstab.txt</m:pre> |
d342de2e09a4
examples: fstab formatting using -in-fstab, -out-nullbyte, xargs and Perl
František Kučera <franta-hg@frantovo.cz>
parents:
176
diff
changeset
|
265 |
|
d342de2e09a4
examples: fstab formatting using -in-fstab, -out-nullbyte, xargs and Perl
František Kučera <franta-hg@frantovo.cz>
parents:
176
diff
changeset
|
266 |
<p> |
d342de2e09a4
examples: fstab formatting using -in-fstab, -out-nullbyte, xargs and Perl
František Kučera <franta-hg@frantovo.cz>
parents:
176
diff
changeset
|
267 |
We can hack together a script called <code>relpipe-out-fstab</code> that accepts relational data and produces <code>fstab</code> data. |
d342de2e09a4
examples: fstab formatting using -in-fstab, -out-nullbyte, xargs and Perl
František Kučera <franta-hg@frantovo.cz>
parents:
176
diff
changeset
|
268 |
Later this will be probably implemented as a regular tool, but for now, it is just an example of a ad-hoc shell script: |
d342de2e09a4
examples: fstab formatting using -in-fstab, -out-nullbyte, xargs and Perl
František Kučera <franta-hg@frantovo.cz>
parents:
176
diff
changeset
|
269 |
</p> |
d342de2e09a4
examples: fstab formatting using -in-fstab, -out-nullbyte, xargs and Perl
František Kučera <franta-hg@frantovo.cz>
parents:
176
diff
changeset
|
270 |
|
d342de2e09a4
examples: fstab formatting using -in-fstab, -out-nullbyte, xargs and Perl
František Kučera <franta-hg@frantovo.cz>
parents:
176
diff
changeset
|
271 |
<m:pre jazyk="bash" src="examples/relpipe-out-fstab.sh" odkaz="ano"/> |
d342de2e09a4
examples: fstab formatting using -in-fstab, -out-nullbyte, xargs and Perl
František Kučera <franta-hg@frantovo.cz>
parents:
176
diff
changeset
|
272 |
|
d342de2e09a4
examples: fstab formatting using -in-fstab, -out-nullbyte, xargs and Perl
František Kučera <franta-hg@frantovo.cz>
parents:
176
diff
changeset
|
273 |
<p> |
d342de2e09a4
examples: fstab formatting using -in-fstab, -out-nullbyte, xargs and Perl
František Kučera <franta-hg@frantovo.cz>
parents:
176
diff
changeset
|
274 |
In the first part, we prepend a single record (<code>relpipe-in-cli</code>) before the data coming from STDIN (<code>cat</code>). |
d342de2e09a4
examples: fstab formatting using -in-fstab, -out-nullbyte, xargs and Perl
František Kučera <franta-hg@frantovo.cz>
parents:
176
diff
changeset
|
275 |
Then, we use <code>relpipe-out-nullbyte</code> to convert relational data to values separated by a null-byte. |
184 | 276 |
This command processes only attribute values (skips relation and attribute names). |
180
d342de2e09a4
examples: fstab formatting using -in-fstab, -out-nullbyte, xargs and Perl
František Kučera <franta-hg@frantovo.cz>
parents:
176
diff
changeset
|
277 |
Then we used <code>xargs</code> to read the null-separated values and execute a Perl command for each record (pass to it a same number of arguments, as we have attributes: <code>--max-args=7</code>). |
d342de2e09a4
examples: fstab formatting using -in-fstab, -out-nullbyte, xargs and Perl
František Kučera <franta-hg@frantovo.cz>
parents:
176
diff
changeset
|
278 |
Perl does the actual formatting: adds padding and does some little tunning (merges two attributes and replaces empty values with <em>none</em>). |
d342de2e09a4
examples: fstab formatting using -in-fstab, -out-nullbyte, xargs and Perl
František Kučera <franta-hg@frantovo.cz>
parents:
176
diff
changeset
|
279 |
</p> |
d342de2e09a4
examples: fstab formatting using -in-fstab, -out-nullbyte, xargs and Perl
František Kučera <franta-hg@frantovo.cz>
parents:
176
diff
changeset
|
280 |
|
d342de2e09a4
examples: fstab formatting using -in-fstab, -out-nullbyte, xargs and Perl
František Kučera <franta-hg@frantovo.cz>
parents:
176
diff
changeset
|
281 |
<p>This is formatted version of the <code>fstab</code> above:</p> |
d342de2e09a4
examples: fstab formatting using -in-fstab, -out-nullbyte, xargs and Perl
František Kučera <franta-hg@frantovo.cz>
parents:
176
diff
changeset
|
282 |
|
d342de2e09a4
examples: fstab formatting using -in-fstab, -out-nullbyte, xargs and Perl
František Kučera <franta-hg@frantovo.cz>
parents:
176
diff
changeset
|
283 |
<m:pre jazyk="text" src="examples/relpipe-out-fstab.formatted.txt"/> |
d342de2e09a4
examples: fstab formatting using -in-fstab, -out-nullbyte, xargs and Perl
František Kučera <franta-hg@frantovo.cz>
parents:
176
diff
changeset
|
284 |
|
d342de2e09a4
examples: fstab formatting using -in-fstab, -out-nullbyte, xargs and Perl
František Kučera <franta-hg@frantovo.cz>
parents:
176
diff
changeset
|
285 |
<p> |
d342de2e09a4
examples: fstab formatting using -in-fstab, -out-nullbyte, xargs and Perl
František Kučera <franta-hg@frantovo.cz>
parents:
176
diff
changeset
|
286 |
And using following command we can verify, that the files differ only in comments and whitespace: |
d342de2e09a4
examples: fstab formatting using -in-fstab, -out-nullbyte, xargs and Perl
František Kučera <franta-hg@frantovo.cz>
parents:
176
diff
changeset
|
287 |
</p> |
d342de2e09a4
examples: fstab formatting using -in-fstab, -out-nullbyte, xargs and Perl
František Kučera <franta-hg@frantovo.cz>
parents:
176
diff
changeset
|
288 |
|
d342de2e09a4
examples: fstab formatting using -in-fstab, -out-nullbyte, xargs and Perl
František Kučera <franta-hg@frantovo.cz>
parents:
176
diff
changeset
|
289 |
<pre>relpipe-in-fstab | relpipe-out-fstab | diff -w /etc/fstab -</pre> |
195
65138c5d5a7c
examples: fstab - another check
František Kučera <franta-hg@frantovo.cz>
parents:
187
diff
changeset
|
290 |
|
65138c5d5a7c
examples: fstab - another check
František Kučera <franta-hg@frantovo.cz>
parents:
187
diff
changeset
|
291 |
<p> |
65138c5d5a7c
examples: fstab - another check
František Kučera <franta-hg@frantovo.cz>
parents:
187
diff
changeset
|
292 |
Another check (should print same hashes): |
65138c5d5a7c
examples: fstab - another check
František Kučera <franta-hg@frantovo.cz>
parents:
187
diff
changeset
|
293 |
</p> |
65138c5d5a7c
examples: fstab - another check
František Kučera <franta-hg@frantovo.cz>
parents:
187
diff
changeset
|
294 |
|
65138c5d5a7c
examples: fstab - another check
František Kučera <franta-hg@frantovo.cz>
parents:
187
diff
changeset
|
295 |
<pre><![CDATA[relpipe-in-fstab | sha512sum |
65138c5d5a7c
examples: fstab - another check
František Kučera <franta-hg@frantovo.cz>
parents:
187
diff
changeset
|
296 |
relpipe-in-fstab | relpipe-out-fstab | relpipe-in-fstab | sha512sum]]></pre> |
180
d342de2e09a4
examples: fstab formatting using -in-fstab, -out-nullbyte, xargs and Perl
František Kučera <franta-hg@frantovo.cz>
parents:
176
diff
changeset
|
297 |
|
d342de2e09a4
examples: fstab formatting using -in-fstab, -out-nullbyte, xargs and Perl
František Kučera <franta-hg@frantovo.cz>
parents:
176
diff
changeset
|
298 |
<p> |
d342de2e09a4
examples: fstab formatting using -in-fstab, -out-nullbyte, xargs and Perl
František Kučera <franta-hg@frantovo.cz>
parents:
176
diff
changeset
|
299 |
Regular implementation of <code>relpipe-out-fstab</code> will probably keep the comments |
d342de2e09a4
examples: fstab formatting using -in-fstab, -out-nullbyte, xargs and Perl
František Kučera <franta-hg@frantovo.cz>
parents:
176
diff
changeset
|
300 |
(it needs also one more attribute and small change in <code>relpipe-in-fstab</code>). |
d342de2e09a4
examples: fstab formatting using -in-fstab, -out-nullbyte, xargs and Perl
František Kučera <franta-hg@frantovo.cz>
parents:
176
diff
changeset
|
301 |
</p> |
d342de2e09a4
examples: fstab formatting using -in-fstab, -out-nullbyte, xargs and Perl
František Kučera <franta-hg@frantovo.cz>
parents:
176
diff
changeset
|
302 |
|
d342de2e09a4
examples: fstab formatting using -in-fstab, -out-nullbyte, xargs and Perl
František Kučera <franta-hg@frantovo.cz>
parents:
176
diff
changeset
|
303 |
<p> |
d342de2e09a4
examples: fstab formatting using -in-fstab, -out-nullbyte, xargs and Perl
František Kučera <franta-hg@frantovo.cz>
parents:
176
diff
changeset
|
304 |
For just mere <code>fstab</code> reformatting, this approach is a bit overengineering. |
184 | 305 |
We could skip the whole relational thing and do just something like this: |
180
d342de2e09a4
examples: fstab formatting using -in-fstab, -out-nullbyte, xargs and Perl
František Kučera <franta-hg@frantovo.cz>
parents:
176
diff
changeset
|
306 |
</p> |
d342de2e09a4
examples: fstab formatting using -in-fstab, -out-nullbyte, xargs and Perl
František Kučera <franta-hg@frantovo.cz>
parents:
176
diff
changeset
|
307 |
|
d342de2e09a4
examples: fstab formatting using -in-fstab, -out-nullbyte, xargs and Perl
František Kučera <franta-hg@frantovo.cz>
parents:
176
diff
changeset
|
308 |
<m:pre jazyk="bash">cat /etc/fstab | grep -v '^#' | sed -E 's/\s+/\n/g' | tr \\n \\0 | xargs -0 -n7 ...</m:pre> |
d342de2e09a4
examples: fstab formatting using -in-fstab, -out-nullbyte, xargs and Perl
František Kučera <franta-hg@frantovo.cz>
parents:
176
diff
changeset
|
309 |
|
d342de2e09a4
examples: fstab formatting using -in-fstab, -out-nullbyte, xargs and Perl
František Kučera <franta-hg@frantovo.cz>
parents:
176
diff
changeset
|
310 |
<p> |
d342de2e09a4
examples: fstab formatting using -in-fstab, -out-nullbyte, xargs and Perl
František Kučera <franta-hg@frantovo.cz>
parents:
176
diff
changeset
|
311 |
plus prepend the comment (or do everything in Perl). |
d342de2e09a4
examples: fstab formatting using -in-fstab, -out-nullbyte, xargs and Perl
František Kučera <franta-hg@frantovo.cz>
parents:
176
diff
changeset
|
312 |
But this example is intended as a demostration, how we can |
d342de2e09a4
examples: fstab formatting using -in-fstab, -out-nullbyte, xargs and Perl
František Kučera <franta-hg@frantovo.cz>
parents:
176
diff
changeset
|
313 |
1) prepend some additional data before the data from STDIN |
d342de2e09a4
examples: fstab formatting using -in-fstab, -out-nullbyte, xargs and Perl
František Kučera <franta-hg@frantovo.cz>
parents:
176
diff
changeset
|
314 |
2) use <m:name/> and traditional tools like <code>xargs</code> or <code>perl</code> together. |
d342de2e09a4
examples: fstab formatting using -in-fstab, -out-nullbyte, xargs and Perl
František Kučera <franta-hg@frantovo.cz>
parents:
176
diff
changeset
|
315 |
And BTW we have implemented a (simple but working) <em>relpipe output filter</em> – and did it without any serious programming, just put some existing commands together :-) |
d342de2e09a4
examples: fstab formatting using -in-fstab, -out-nullbyte, xargs and Perl
František Kučera <franta-hg@frantovo.cz>
parents:
176
diff
changeset
|
316 |
</p> |
d342de2e09a4
examples: fstab formatting using -in-fstab, -out-nullbyte, xargs and Perl
František Kučera <franta-hg@frantovo.cz>
parents:
176
diff
changeset
|
317 |
|
d342de2e09a4
examples: fstab formatting using -in-fstab, -out-nullbyte, xargs and Perl
František Kučera <franta-hg@frantovo.cz>
parents:
176
diff
changeset
|
318 |
<blockquote> |
d342de2e09a4
examples: fstab formatting using -in-fstab, -out-nullbyte, xargs and Perl
František Kučera <franta-hg@frantovo.cz>
parents:
176
diff
changeset
|
319 |
<p> |
d342de2e09a4
examples: fstab formatting using -in-fstab, -out-nullbyte, xargs and Perl
František Kučera <franta-hg@frantovo.cz>
parents:
176
diff
changeset
|
320 |
There is more Unix-nature in one line of shell script than there is in ten thousand lines of C. |
d342de2e09a4
examples: fstab formatting using -in-fstab, -out-nullbyte, xargs and Perl
František Kučera <franta-hg@frantovo.cz>
parents:
176
diff
changeset
|
321 |
<m:podČarou>see <a href="http://www.catb.org/~esr/writings/unix-koans/ten-thousand.html">Master Foo and the Ten Thousand Lines</a></m:podČarou> |
d342de2e09a4
examples: fstab formatting using -in-fstab, -out-nullbyte, xargs and Perl
František Kučera <franta-hg@frantovo.cz>
parents:
176
diff
changeset
|
322 |
</p> |
d342de2e09a4
examples: fstab formatting using -in-fstab, -out-nullbyte, xargs and Perl
František Kučera <franta-hg@frantovo.cz>
parents:
176
diff
changeset
|
323 |
</blockquote> |
d342de2e09a4
examples: fstab formatting using -in-fstab, -out-nullbyte, xargs and Perl
František Kučera <franta-hg@frantovo.cz>
parents:
176
diff
changeset
|
324 |
|
214
63f31bf2697f
examples: Writing an output filter in Bash
František Kučera <franta-hg@frantovo.cz>
parents:
213
diff
changeset
|
325 |
<h2>Writing an output filter in Bash</h2> |
63f31bf2697f
examples: Writing an output filter in Bash
František Kučera <franta-hg@frantovo.cz>
parents:
213
diff
changeset
|
326 |
|
63f31bf2697f
examples: Writing an output filter in Bash
František Kučera <franta-hg@frantovo.cz>
parents:
213
diff
changeset
|
327 |
<p> |
216 | 328 |
In previous example we created an output filter in Perl. |
329 |
We converted a relation to values separated by <code>\0</code> and then passed it through <code>xargs</code> to a perl <em>one-liner</em> (or a <em>multi-liner</em> in this case). |
|
214
63f31bf2697f
examples: Writing an output filter in Bash
František Kučera <franta-hg@frantovo.cz>
parents:
213
diff
changeset
|
330 |
But we can write such output filter in pure Bash without <code>xargs</code> and <code>perl</code>. |
63f31bf2697f
examples: Writing an output filter in Bash
František Kučera <franta-hg@frantovo.cz>
parents:
213
diff
changeset
|
331 |
Of course, it is still limited to a single relation (or it can process multiple relations of same type and do something like implicit <code>UNION ALL</code>). |
63f31bf2697f
examples: Writing an output filter in Bash
František Kučera <franta-hg@frantovo.cz>
parents:
213
diff
changeset
|
332 |
</p> |
63f31bf2697f
examples: Writing an output filter in Bash
František Kučera <franta-hg@frantovo.cz>
parents:
213
diff
changeset
|
333 |
|
63f31bf2697f
examples: Writing an output filter in Bash
František Kučera <franta-hg@frantovo.cz>
parents:
213
diff
changeset
|
334 |
<p> |
63f31bf2697f
examples: Writing an output filter in Bash
František Kučera <franta-hg@frantovo.cz>
parents:
213
diff
changeset
|
335 |
We will define a function that will help us with reading the <code>\0</code>-separated values and putting them into shell variables: |
63f31bf2697f
examples: Writing an output filter in Bash
František Kučera <franta-hg@frantovo.cz>
parents:
213
diff
changeset
|
336 |
</p> |
63f31bf2697f
examples: Writing an output filter in Bash
František Kučera <franta-hg@frantovo.cz>
parents:
213
diff
changeset
|
337 |
|
63f31bf2697f
examples: Writing an output filter in Bash
František Kučera <franta-hg@frantovo.cz>
parents:
213
diff
changeset
|
338 |
<m:pre jazyk="bash"><![CDATA[read_nullbyte() { for v in "$@"; do export "$v"; read -r -d '' "$v"; done }]]></m:pre> |
63f31bf2697f
examples: Writing an output filter in Bash
František Kučera <franta-hg@frantovo.cz>
parents:
213
diff
changeset
|
339 |
|
63f31bf2697f
examples: Writing an output filter in Bash
František Kučera <franta-hg@frantovo.cz>
parents:
213
diff
changeset
|
340 |
<!-- |
63f31bf2697f
examples: Writing an output filter in Bash
František Kučera <franta-hg@frantovo.cz>
parents:
213
diff
changeset
|
341 |
This version will not require the last \0: |
63f31bf2697f
examples: Writing an output filter in Bash
František Kučera <franta-hg@frantovo.cz>
parents:
213
diff
changeset
|
342 |
read_zero() { for v in "$@"; do export "$v"; read -r -d '' "$v" || [ ! -z "${!v}" ]; done } |
63f31bf2697f
examples: Writing an output filter in Bash
František Kučera <franta-hg@frantovo.cz>
parents:
213
diff
changeset
|
343 |
at least in case when the last value is not missing. |
63f31bf2697f
examples: Writing an output filter in Bash
František Kučera <franta-hg@frantovo.cz>
parents:
213
diff
changeset
|
344 |
Other values might be null/missing: \0\0 is OK. |
63f31bf2697f
examples: Writing an output filter in Bash
František Kučera <franta-hg@frantovo.cz>
parents:
213
diff
changeset
|
345 |
--> |
63f31bf2697f
examples: Writing an output filter in Bash
František Kučera <franta-hg@frantovo.cz>
parents:
213
diff
changeset
|
346 |
|
63f31bf2697f
examples: Writing an output filter in Bash
František Kučera <franta-hg@frantovo.cz>
parents:
213
diff
changeset
|
347 |
<p> |
63f31bf2697f
examples: Writing an output filter in Bash
František Kučera <franta-hg@frantovo.cz>
parents:
213
diff
changeset
|
348 |
Currently, there is no known way how to do this without a custom function (just with <code>read</code> built-in command of Bash and its parameters). |
63f31bf2697f
examples: Writing an output filter in Bash
František Kučera <franta-hg@frantovo.cz>
parents:
213
diff
changeset
|
349 |
But it is just a single line function, so not a big deal. |
63f31bf2697f
examples: Writing an output filter in Bash
František Kučera <franta-hg@frantovo.cz>
parents:
213
diff
changeset
|
350 |
</p> |
63f31bf2697f
examples: Writing an output filter in Bash
František Kučera <franta-hg@frantovo.cz>
parents:
213
diff
changeset
|
351 |
|
63f31bf2697f
examples: Writing an output filter in Bash
František Kučera <franta-hg@frantovo.cz>
parents:
213
diff
changeset
|
352 |
<p> |
63f31bf2697f
examples: Writing an output filter in Bash
František Kučera <franta-hg@frantovo.cz>
parents:
213
diff
changeset
|
353 |
And then we just read the values, put them in shell variables and process them in a cycle in a shell block of code: |
63f31bf2697f
examples: Writing an output filter in Bash
František Kučera <franta-hg@frantovo.cz>
parents:
213
diff
changeset
|
354 |
</p> |
63f31bf2697f
examples: Writing an output filter in Bash
František Kučera <franta-hg@frantovo.cz>
parents:
213
diff
changeset
|
355 |
|
63f31bf2697f
examples: Writing an output filter in Bash
František Kučera <franta-hg@frantovo.cz>
parents:
213
diff
changeset
|
356 |
<m:pre jazyk="bash"><![CDATA[relpipe-in-fstab \ |
63f31bf2697f
examples: Writing an output filter in Bash
František Kučera <franta-hg@frantovo.cz>
parents:
213
diff
changeset
|
357 |
| relpipe-out-nullbyte \ |
63f31bf2697f
examples: Writing an output filter in Bash
František Kučera <franta-hg@frantovo.cz>
parents:
213
diff
changeset
|
358 |
| while read_nullbyte scheme device mount_point fs_type options dump pass; do |
63f31bf2697f
examples: Writing an output filter in Bash
František Kučera <franta-hg@frantovo.cz>
parents:
213
diff
changeset
|
359 |
echo "Device ${scheme:+$scheme=}$device is mounted" \ |
63f31bf2697f
examples: Writing an output filter in Bash
František Kučera <franta-hg@frantovo.cz>
parents:
213
diff
changeset
|
360 |
"at $mount_point and contains $fs_type."; |
63f31bf2697f
examples: Writing an output filter in Bash
František Kučera <franta-hg@frantovo.cz>
parents:
213
diff
changeset
|
361 |
done]]></m:pre> |
63f31bf2697f
examples: Writing an output filter in Bash
František Kučera <franta-hg@frantovo.cz>
parents:
213
diff
changeset
|
362 |
|
63f31bf2697f
examples: Writing an output filter in Bash
František Kučera <franta-hg@frantovo.cz>
parents:
213
diff
changeset
|
363 |
<p> |
63f31bf2697f
examples: Writing an output filter in Bash
František Kučera <franta-hg@frantovo.cz>
parents:
213
diff
changeset
|
364 |
Which will print: |
63f31bf2697f
examples: Writing an output filter in Bash
František Kučera <franta-hg@frantovo.cz>
parents:
213
diff
changeset
|
365 |
</p> |
63f31bf2697f
examples: Writing an output filter in Bash
František Kučera <franta-hg@frantovo.cz>
parents:
213
diff
changeset
|
366 |
|
63f31bf2697f
examples: Writing an output filter in Bash
František Kučera <franta-hg@frantovo.cz>
parents:
213
diff
changeset
|
367 |
<pre><![CDATA[Device UUID=29758270-fd25-4a6c-a7bb-9a18302816af is mounted at / and contains ext4. |
63f31bf2697f
examples: Writing an output filter in Bash
František Kučera <franta-hg@frantovo.cz>
parents:
213
diff
changeset
|
368 |
Device /dev/sr0 is mounted at /media/cdrom0 and contains udf,iso9660. |
63f31bf2697f
examples: Writing an output filter in Bash
František Kučera <franta-hg@frantovo.cz>
parents:
213
diff
changeset
|
369 |
Device /dev/sde is mounted at /mnt/data and contains ext4. |
63f31bf2697f
examples: Writing an output filter in Bash
František Kučera <franta-hg@frantovo.cz>
parents:
213
diff
changeset
|
370 |
Device UUID=a2b5f230-a795-4f6f-a39b-9b57686c86d5 is mounted at /home and contains btrfs. |
63f31bf2697f
examples: Writing an output filter in Bash
František Kučera <franta-hg@frantovo.cz>
parents:
213
diff
changeset
|
371 |
Device /dev/mapper/sdf_crypt is mounted at /mnt/private and contains xfs.]]></pre> |
63f31bf2697f
examples: Writing an output filter in Bash
František Kučera <franta-hg@frantovo.cz>
parents:
213
diff
changeset
|
372 |
|
63f31bf2697f
examples: Writing an output filter in Bash
František Kučera <franta-hg@frantovo.cz>
parents:
213
diff
changeset
|
373 |
<p> |
215
c7e88edaedc5
examples: Writing an output filter in Bash: comment
František Kučera <franta-hg@frantovo.cz>
parents:
214
diff
changeset
|
374 |
Using this method, we can convert any single relation to any format (preferably some text one, but <code>printf</code> can produce also binary data). |
c7e88edaedc5
examples: Writing an output filter in Bash: comment
František Kučera <franta-hg@frantovo.cz>
parents:
214
diff
changeset
|
375 |
This is good for ad-hoc conversions and single-relation data. |
c7e88edaedc5
examples: Writing an output filter in Bash: comment
František Kučera <franta-hg@frantovo.cz>
parents:
214
diff
changeset
|
376 |
More powerful tools can be written in C++ and other languages like Java, Python, Guile etc. (when particular libraries are available). |
214
63f31bf2697f
examples: Writing an output filter in Bash
František Kučera <franta-hg@frantovo.cz>
parents:
213
diff
changeset
|
377 |
</p> |
63f31bf2697f
examples: Writing an output filter in Bash
František Kučera <franta-hg@frantovo.cz>
parents:
213
diff
changeset
|
378 |
|
207
4e45b9a177c2
examples: relpipe-tr-sed
František Kučera <franta-hg@frantovo.cz>
parents:
198
diff
changeset
|
379 |
<h2>Rename VG in /etc/fstab using relpipe-tr-sed</h2> |
4e45b9a177c2
examples: relpipe-tr-sed
František Kučera <franta-hg@frantovo.cz>
parents:
198
diff
changeset
|
380 |
|
4e45b9a177c2
examples: relpipe-tr-sed
František Kučera <franta-hg@frantovo.cz>
parents:
198
diff
changeset
|
381 |
<p> |
210
f0a2916368e2
small fixes and improvements
František Kučera <franta-hg@frantovo.cz>
parents:
209
diff
changeset
|
382 |
Assume that we have an <code>/etc/fstab</code> with many lines defining the mount-points (directories) of particular devices (disks) and we are using LVM. |
207
4e45b9a177c2
examples: relpipe-tr-sed
František Kučera <franta-hg@frantovo.cz>
parents:
198
diff
changeset
|
383 |
If we rename a volume group (VG), we have to change all of them. The lines look like this one: |
4e45b9a177c2
examples: relpipe-tr-sed
František Kučera <franta-hg@frantovo.cz>
parents:
198
diff
changeset
|
384 |
</p> |
4e45b9a177c2
examples: relpipe-tr-sed
František Kučera <franta-hg@frantovo.cz>
parents:
198
diff
changeset
|
385 |
|
4e45b9a177c2
examples: relpipe-tr-sed
František Kučera <franta-hg@frantovo.cz>
parents:
198
diff
changeset
|
386 |
<pre>/dev/alpha/photos /mnt/photos/ btrfs noauto,noatime,nodiratime 0 0</pre> |
4e45b9a177c2
examples: relpipe-tr-sed
František Kučera <franta-hg@frantovo.cz>
parents:
198
diff
changeset
|
387 |
|
4e45b9a177c2
examples: relpipe-tr-sed
František Kučera <franta-hg@frantovo.cz>
parents:
198
diff
changeset
|
388 |
<p> |
4e45b9a177c2
examples: relpipe-tr-sed
František Kučera <franta-hg@frantovo.cz>
parents:
198
diff
changeset
|
389 |
We want to change all lines from <code>alpha</code> to <code>beta</code> (the new VG name). |
208
91a210d3b66a
examples: relpipe-tr-grep
František Kučera <franta-hg@frantovo.cz>
parents:
207
diff
changeset
|
390 |
This can be done by the power of regular expressions<m:podČarou>see <a href="https://en.wikibooks.org/wiki/Regular_Expressions/Simple_Regular_Expressions">Regular Expressions</a> at Wikibooks</m:podČarou> and this pipeline: |
207
4e45b9a177c2
examples: relpipe-tr-sed
František Kučera <franta-hg@frantovo.cz>
parents:
198
diff
changeset
|
391 |
</p> |
4e45b9a177c2
examples: relpipe-tr-sed
František Kučera <franta-hg@frantovo.cz>
parents:
198
diff
changeset
|
392 |
|
4e45b9a177c2
examples: relpipe-tr-sed
František Kučera <franta-hg@frantovo.cz>
parents:
198
diff
changeset
|
393 |
<m:pre jazyk="bash"><![CDATA[relpipe-in-fstab \ |
4e45b9a177c2
examples: relpipe-tr-sed
František Kučera <franta-hg@frantovo.cz>
parents:
198
diff
changeset
|
394 |
| relpipe-tr-sed 'fstab' 'device' '^/dev/alpha/' '/dev/beta/' \ |
4e45b9a177c2
examples: relpipe-tr-sed
František Kučera <franta-hg@frantovo.cz>
parents:
198
diff
changeset
|
395 |
| relpipe-out-fstab]]></m:pre> |
4e45b9a177c2
examples: relpipe-tr-sed
František Kučera <franta-hg@frantovo.cz>
parents:
198
diff
changeset
|
396 |
|
4e45b9a177c2
examples: relpipe-tr-sed
František Kučera <franta-hg@frantovo.cz>
parents:
198
diff
changeset
|
397 |
<p> |
209 | 398 |
The <code>relpipe-tr-sed</code> tool works only with given relation (<code>fstab</code>) and given attribute (<code>device</code>) |
207
4e45b9a177c2
examples: relpipe-tr-sed
František Kučera <franta-hg@frantovo.cz>
parents:
198
diff
changeset
|
399 |
and it would leave untouched other relations and attributes in the stream. |
4e45b9a177c2
examples: relpipe-tr-sed
František Kučera <franta-hg@frantovo.cz>
parents:
198
diff
changeset
|
400 |
So it would not replace the strings on unwanted places (if there are any random matches). |
4e45b9a177c2
examples: relpipe-tr-sed
František Kučera <franta-hg@frantovo.cz>
parents:
198
diff
changeset
|
401 |
</p> |
4e45b9a177c2
examples: relpipe-tr-sed
František Kučera <franta-hg@frantovo.cz>
parents:
198
diff
changeset
|
402 |
|
4e45b9a177c2
examples: relpipe-tr-sed
František Kučera <franta-hg@frantovo.cz>
parents:
198
diff
changeset
|
403 |
<p> |
4e45b9a177c2
examples: relpipe-tr-sed
František Kučera <franta-hg@frantovo.cz>
parents:
198
diff
changeset
|
404 |
Even the relation names and attribute names are specified as a regular expression, so we can (purposefully) modify multiple relations or attributes. |
4e45b9a177c2
examples: relpipe-tr-sed
František Kučera <franta-hg@frantovo.cz>
parents:
198
diff
changeset
|
405 |
For example we can put zeroes in both <code>dump</code> and <code>pass</code> attributes: |
4e45b9a177c2
examples: relpipe-tr-sed
František Kučera <franta-hg@frantovo.cz>
parents:
198
diff
changeset
|
406 |
</p> |
4e45b9a177c2
examples: relpipe-tr-sed
František Kučera <franta-hg@frantovo.cz>
parents:
198
diff
changeset
|
407 |
|
223
6402cd6921c5
examples: small sed fix: .* → .+
František Kučera <franta-hg@frantovo.cz>
parents:
216
diff
changeset
|
408 |
<m:pre jazyk="bash"><![CDATA[relpipe-in-fstab | relpipe-tr-sed 'fstab' 'dump|pass' '.+' '0' | relpipe-out-fstab]]></m:pre> |
207
4e45b9a177c2
examples: relpipe-tr-sed
František Kučera <franta-hg@frantovo.cz>
parents:
198
diff
changeset
|
409 |
|
4e45b9a177c2
examples: relpipe-tr-sed
František Kučera <franta-hg@frantovo.cz>
parents:
198
diff
changeset
|
410 |
<p> |
4e45b9a177c2
examples: relpipe-tr-sed
František Kučera <franta-hg@frantovo.cz>
parents:
198
diff
changeset
|
411 |
n.b. the data types must be respected, we can not e.g. put <code>abc</code> in the <code>pass</code> attribute because it is declared as <code>integer</code>. |
4e45b9a177c2
examples: relpipe-tr-sed
František Kučera <franta-hg@frantovo.cz>
parents:
198
diff
changeset
|
412 |
</p> |
4e45b9a177c2
examples: relpipe-tr-sed
František Kučera <franta-hg@frantovo.cz>
parents:
198
diff
changeset
|
413 |
|
4e45b9a177c2
examples: relpipe-tr-sed
František Kučera <franta-hg@frantovo.cz>
parents:
198
diff
changeset
|
414 |
<h2>Using relpipe-tr-sed with groups and backreferences</h2> |
4e45b9a177c2
examples: relpipe-tr-sed
František Kučera <franta-hg@frantovo.cz>
parents:
198
diff
changeset
|
415 |
|
4e45b9a177c2
examples: relpipe-tr-sed
František Kučera <franta-hg@frantovo.cz>
parents:
198
diff
changeset
|
416 |
<p> |
4e45b9a177c2
examples: relpipe-tr-sed
František Kučera <franta-hg@frantovo.cz>
parents:
198
diff
changeset
|
417 |
This tool also support regex groups and backreferences. Thus we can use parts of the matched string in our replacement string: |
4e45b9a177c2
examples: relpipe-tr-sed
František Kučera <franta-hg@frantovo.cz>
parents:
198
diff
changeset
|
418 |
</p> |
4e45b9a177c2
examples: relpipe-tr-sed
František Kučera <franta-hg@frantovo.cz>
parents:
198
diff
changeset
|
419 |
|
4e45b9a177c2
examples: relpipe-tr-sed
František Kučera <franta-hg@frantovo.cz>
parents:
198
diff
changeset
|
420 |
<m:pre jazyk="bash"><![CDATA[relpipe-in-cli generate r 1 a string "some string xxx_123 some zzz_456 other" \ |
4e45b9a177c2
examples: relpipe-tr-sed
František Kučera <franta-hg@frantovo.cz>
parents:
198
diff
changeset
|
421 |
| relpipe-tr-sed 'r' 'a' '([a-z]{3})_([0-9]+)' '$2:$1' \ |
4e45b9a177c2
examples: relpipe-tr-sed
František Kučera <franta-hg@frantovo.cz>
parents:
198
diff
changeset
|
422 |
| relpipe-out-tabular]]></m:pre> |
4e45b9a177c2
examples: relpipe-tr-sed
František Kučera <franta-hg@frantovo.cz>
parents:
198
diff
changeset
|
423 |
|
4e45b9a177c2
examples: relpipe-tr-sed
František Kučera <franta-hg@frantovo.cz>
parents:
198
diff
changeset
|
424 |
<p>Which would convert this:</p> |
4e45b9a177c2
examples: relpipe-tr-sed
František Kučera <franta-hg@frantovo.cz>
parents:
198
diff
changeset
|
425 |
<pre><![CDATA[r: |
4e45b9a177c2
examples: relpipe-tr-sed
František Kučera <franta-hg@frantovo.cz>
parents:
198
diff
changeset
|
426 |
╭────────────────────────────────────────╮ |
4e45b9a177c2
examples: relpipe-tr-sed
František Kučera <franta-hg@frantovo.cz>
parents:
198
diff
changeset
|
427 |
│ a (string) │ |
4e45b9a177c2
examples: relpipe-tr-sed
František Kučera <franta-hg@frantovo.cz>
parents:
198
diff
changeset
|
428 |
├────────────────────────────────────────┤ |
4e45b9a177c2
examples: relpipe-tr-sed
František Kučera <franta-hg@frantovo.cz>
parents:
198
diff
changeset
|
429 |
│ some string xxx_123 some zzz_456 other │ |
4e45b9a177c2
examples: relpipe-tr-sed
František Kučera <franta-hg@frantovo.cz>
parents:
198
diff
changeset
|
430 |
╰────────────────────────────────────────╯ |
4e45b9a177c2
examples: relpipe-tr-sed
František Kučera <franta-hg@frantovo.cz>
parents:
198
diff
changeset
|
431 |
Record count: 1]]></pre> |
4e45b9a177c2
examples: relpipe-tr-sed
František Kučera <franta-hg@frantovo.cz>
parents:
198
diff
changeset
|
432 |
|
4e45b9a177c2
examples: relpipe-tr-sed
František Kučera <franta-hg@frantovo.cz>
parents:
198
diff
changeset
|
433 |
<p>into this:</p> |
4e45b9a177c2
examples: relpipe-tr-sed
František Kučera <franta-hg@frantovo.cz>
parents:
198
diff
changeset
|
434 |
<pre><![CDATA[r: |
4e45b9a177c2
examples: relpipe-tr-sed
František Kučera <franta-hg@frantovo.cz>
parents:
198
diff
changeset
|
435 |
╭────────────────────────────────────────╮ |
4e45b9a177c2
examples: relpipe-tr-sed
František Kučera <franta-hg@frantovo.cz>
parents:
198
diff
changeset
|
436 |
│ a (string) │ |
4e45b9a177c2
examples: relpipe-tr-sed
František Kučera <franta-hg@frantovo.cz>
parents:
198
diff
changeset
|
437 |
├────────────────────────────────────────┤ |
4e45b9a177c2
examples: relpipe-tr-sed
František Kučera <franta-hg@frantovo.cz>
parents:
198
diff
changeset
|
438 |
│ some string 123:xxx some 456:zzz other │ |
4e45b9a177c2
examples: relpipe-tr-sed
František Kučera <franta-hg@frantovo.cz>
parents:
198
diff
changeset
|
439 |
╰────────────────────────────────────────╯ |
4e45b9a177c2
examples: relpipe-tr-sed
František Kučera <franta-hg@frantovo.cz>
parents:
198
diff
changeset
|
440 |
Record count: 1]]></pre> |
4e45b9a177c2
examples: relpipe-tr-sed
František Kučera <franta-hg@frantovo.cz>
parents:
198
diff
changeset
|
441 |
|
4e45b9a177c2
examples: relpipe-tr-sed
František Kučera <franta-hg@frantovo.cz>
parents:
198
diff
changeset
|
442 |
<p> |
4e45b9a177c2
examples: relpipe-tr-sed
František Kučera <franta-hg@frantovo.cz>
parents:
198
diff
changeset
|
443 |
If there were any other relations or attributes in the stream, they would be unaffected by this transformation, |
4e45b9a177c2
examples: relpipe-tr-sed
František Kučera <franta-hg@frantovo.cz>
parents:
198
diff
changeset
|
444 |
becase we specified <code>'r' 'a'</code> instead of some wider regular expression that would match more relations or attributes. |
4e45b9a177c2
examples: relpipe-tr-sed
František Kučera <franta-hg@frantovo.cz>
parents:
198
diff
changeset
|
445 |
</p> |
4e45b9a177c2
examples: relpipe-tr-sed
František Kučera <franta-hg@frantovo.cz>
parents:
198
diff
changeset
|
446 |
|
208
91a210d3b66a
examples: relpipe-tr-grep
František Kučera <franta-hg@frantovo.cz>
parents:
207
diff
changeset
|
447 |
<h2>Filter /etc/fstab using relpipe-tr-grep</h2> |
91a210d3b66a
examples: relpipe-tr-grep
František Kučera <franta-hg@frantovo.cz>
parents:
207
diff
changeset
|
448 |
|
91a210d3b66a
examples: relpipe-tr-grep
František Kučera <franta-hg@frantovo.cz>
parents:
207
diff
changeset
|
449 |
<p> |
91a210d3b66a
examples: relpipe-tr-grep
František Kučera <franta-hg@frantovo.cz>
parents:
207
diff
changeset
|
450 |
If we are interested only in certain records in some relation, we can filter it using <code>relpipe-tr-grep</code>. |
91a210d3b66a
examples: relpipe-tr-grep
František Kučera <franta-hg@frantovo.cz>
parents:
207
diff
changeset
|
451 |
If we want to list e.g. only Btrfs and XFS file systems from our <code>fstab</code> (see above), we will run: |
91a210d3b66a
examples: relpipe-tr-grep
František Kučera <franta-hg@frantovo.cz>
parents:
207
diff
changeset
|
452 |
</p> |
91a210d3b66a
examples: relpipe-tr-grep
František Kučera <franta-hg@frantovo.cz>
parents:
207
diff
changeset
|
453 |
|
91a210d3b66a
examples: relpipe-tr-grep
František Kučera <franta-hg@frantovo.cz>
parents:
207
diff
changeset
|
454 |
|
91a210d3b66a
examples: relpipe-tr-grep
František Kučera <franta-hg@frantovo.cz>
parents:
207
diff
changeset
|
455 |
<m:pre jazyk="bash"><![CDATA[relpipe-in-fstab | relpipe-tr-grep 'fstab' 'type' 'btrfs|xfs' | relpipe-out-tabular]]></m:pre> |
91a210d3b66a
examples: relpipe-tr-grep
František Kučera <franta-hg@frantovo.cz>
parents:
207
diff
changeset
|
456 |
|
91a210d3b66a
examples: relpipe-tr-grep
František Kučera <franta-hg@frantovo.cz>
parents:
207
diff
changeset
|
457 |
<p>and we will get following filtered result:</p> |
91a210d3b66a
examples: relpipe-tr-grep
František Kučera <franta-hg@frantovo.cz>
parents:
207
diff
changeset
|
458 |
<pre><![CDATA[fstab: |
91a210d3b66a
examples: relpipe-tr-grep
František Kučera <franta-hg@frantovo.cz>
parents:
207
diff
changeset
|
459 |
╭─────────────────┬──────────────────────────────────────┬──────────────────────┬───────────────┬──────────────────┬────────────────┬────────────────╮ |
91a210d3b66a
examples: relpipe-tr-grep
František Kučera <franta-hg@frantovo.cz>
parents:
207
diff
changeset
|
460 |
│ scheme (string) │ device (string) │ mount_point (string) │ type (string) │ options (string) │ dump (integer) │ pass (integer) │ |
91a210d3b66a
examples: relpipe-tr-grep
František Kučera <franta-hg@frantovo.cz>
parents:
207
diff
changeset
|
461 |
├─────────────────┼──────────────────────────────────────┼──────────────────────┼───────────────┼──────────────────┼────────────────┼────────────────┤ |
91a210d3b66a
examples: relpipe-tr-grep
František Kučera <franta-hg@frantovo.cz>
parents:
207
diff
changeset
|
462 |
│ UUID │ a2b5f230-a795-4f6f-a39b-9b57686c86d5 │ /home │ btrfs │ relatime │ 0 │ 2 │ |
91a210d3b66a
examples: relpipe-tr-grep
František Kučera <franta-hg@frantovo.cz>
parents:
207
diff
changeset
|
463 |
│ │ /dev/mapper/sdf_crypt │ /mnt/private │ xfs │ relatime │ 0 │ 2 │ |
91a210d3b66a
examples: relpipe-tr-grep
František Kučera <franta-hg@frantovo.cz>
parents:
207
diff
changeset
|
464 |
╰─────────────────┴──────────────────────────────────────┴──────────────────────┴───────────────┴──────────────────┴────────────────┴────────────────╯ |
91a210d3b66a
examples: relpipe-tr-grep
František Kučera <franta-hg@frantovo.cz>
parents:
207
diff
changeset
|
465 |
Record count: 2]]></pre> |
91a210d3b66a
examples: relpipe-tr-grep
František Kučera <franta-hg@frantovo.cz>
parents:
207
diff
changeset
|
466 |
|
91a210d3b66a
examples: relpipe-tr-grep
František Kučera <franta-hg@frantovo.cz>
parents:
207
diff
changeset
|
467 |
<p> |
91a210d3b66a
examples: relpipe-tr-grep
František Kučera <franta-hg@frantovo.cz>
parents:
207
diff
changeset
|
468 |
Command arguments are similar to <code>relpipe-tr-sed</code>. |
91a210d3b66a
examples: relpipe-tr-grep
František Kučera <franta-hg@frantovo.cz>
parents:
207
diff
changeset
|
469 |
Everything is a regular expression. |
91a210d3b66a
examples: relpipe-tr-grep
František Kučera <franta-hg@frantovo.cz>
parents:
207
diff
changeset
|
470 |
Only relations matching the regex will be filtered, others will flow through the pipeline unmodified. |
91a210d3b66a
examples: relpipe-tr-grep
František Kučera <franta-hg@frantovo.cz>
parents:
207
diff
changeset
|
471 |
If the attribute regex matches more attribute names, filtering will be done with logical OR |
91a210d3b66a
examples: relpipe-tr-grep
František Kučera <franta-hg@frantovo.cz>
parents:
207
diff
changeset
|
472 |
i.e. the record is included if at least one of that attributes matches the search regex. |
91a210d3b66a
examples: relpipe-tr-grep
František Kučera <franta-hg@frantovo.cz>
parents:
207
diff
changeset
|
473 |
</p> |
91a210d3b66a
examples: relpipe-tr-grep
František Kučera <franta-hg@frantovo.cz>
parents:
207
diff
changeset
|
474 |
|
91a210d3b66a
examples: relpipe-tr-grep
František Kučera <franta-hg@frantovo.cz>
parents:
207
diff
changeset
|
475 |
<p> |
91a210d3b66a
examples: relpipe-tr-grep
František Kučera <franta-hg@frantovo.cz>
parents:
207
diff
changeset
|
476 |
If we need exact match of the whole attribute, we have to use something like <code>'^btrfs|xfs$'</code>, |
91a210d3b66a
examples: relpipe-tr-grep
František Kučera <franta-hg@frantovo.cz>
parents:
207
diff
changeset
|
477 |
otherwise mere substring-match is enough to include the record. |
91a210d3b66a
examples: relpipe-tr-grep
František Kučera <franta-hg@frantovo.cz>
parents:
207
diff
changeset
|
478 |
</p> |
91a210d3b66a
examples: relpipe-tr-grep
František Kučera <franta-hg@frantovo.cz>
parents:
207
diff
changeset
|
479 |
|
212
bf9a704dc916
examples: relpipe-tr-cut
František Kučera <franta-hg@frantovo.cz>
parents:
210
diff
changeset
|
480 |
<h2>SELECT mount_point FROM fstab WHERE type IN ('btrfs', 'xfs')</h2> |
bf9a704dc916
examples: relpipe-tr-cut
František Kučera <franta-hg@frantovo.cz>
parents:
210
diff
changeset
|
481 |
|
bf9a704dc916
examples: relpipe-tr-cut
František Kučera <franta-hg@frantovo.cz>
parents:
210
diff
changeset
|
482 |
<p> |
bf9a704dc916
examples: relpipe-tr-cut
František Kučera <franta-hg@frantovo.cz>
parents:
210
diff
changeset
|
483 |
While reading classic pipelines involving <code>grep</code> and <code>cut</code> commands |
bf9a704dc916
examples: relpipe-tr-cut
František Kučera <franta-hg@frantovo.cz>
parents:
210
diff
changeset
|
484 |
we must notice that there is some similarity with simple SQL queries looking like: |
bf9a704dc916
examples: relpipe-tr-cut
František Kučera <franta-hg@frantovo.cz>
parents:
210
diff
changeset
|
485 |
</p> |
bf9a704dc916
examples: relpipe-tr-cut
František Kučera <franta-hg@frantovo.cz>
parents:
210
diff
changeset
|
486 |
|
bf9a704dc916
examples: relpipe-tr-cut
František Kučera <franta-hg@frantovo.cz>
parents:
210
diff
changeset
|
487 |
<m:pre jazyk="SQL">SELECT "some", "cut", "fields" FROM stdin WHERE grep_matches(whole_line);</m:pre> |
bf9a704dc916
examples: relpipe-tr-cut
František Kučera <franta-hg@frantovo.cz>
parents:
210
diff
changeset
|
488 |
|
bf9a704dc916
examples: relpipe-tr-cut
František Kučera <franta-hg@frantovo.cz>
parents:
210
diff
changeset
|
489 |
<p> |
bf9a704dc916
examples: relpipe-tr-cut
František Kučera <franta-hg@frantovo.cz>
parents:
210
diff
changeset
|
490 |
And that is true: <code>grep</code> does restriction<m:podČarou> |
bf9a704dc916
examples: relpipe-tr-cut
František Kučera <franta-hg@frantovo.cz>
parents:
210
diff
changeset
|
491 |
<a href="https://en.wikipedia.org/wiki/Selection_(relational_algebra)">selecting</a> only certain records from the original relation according to their match with given conditions</m:podČarou> |
bf9a704dc916
examples: relpipe-tr-cut
František Kučera <franta-hg@frantovo.cz>
parents:
210
diff
changeset
|
492 |
and <code>cut</code> does projection<m:podČarou>limited subset of what <a href="https://en.wikipedia.org/wiki/Projection_(relational_algebra)">projection</a> means</m:podČarou>. |
bf9a704dc916
examples: relpipe-tr-cut
František Kučera <franta-hg@frantovo.cz>
parents:
210
diff
changeset
|
493 |
Now we can do these relational operations using our relational tools called <code>relpipe-tr-grep</code> and <code>relpipe-tr-cut</code>. |
bf9a704dc916
examples: relpipe-tr-cut
František Kučera <franta-hg@frantovo.cz>
parents:
210
diff
changeset
|
494 |
</p> |
bf9a704dc916
examples: relpipe-tr-cut
František Kučera <franta-hg@frantovo.cz>
parents:
210
diff
changeset
|
495 |
|
bf9a704dc916
examples: relpipe-tr-cut
František Kučera <franta-hg@frantovo.cz>
parents:
210
diff
changeset
|
496 |
<p> |
bf9a704dc916
examples: relpipe-tr-cut
František Kučera <franta-hg@frantovo.cz>
parents:
210
diff
changeset
|
497 |
Assume that we need only <code>mount_point</code> fields from our <code>fstab</code> where <code>type</code> is <code>btrfs</code> or <code>xfs</code> |
bf9a704dc916
examples: relpipe-tr-cut
František Kučera <franta-hg@frantovo.cz>
parents:
210
diff
changeset
|
498 |
and we want to do something (a shell script block) with these directory paths. |
bf9a704dc916
examples: relpipe-tr-cut
František Kučera <franta-hg@frantovo.cz>
parents:
210
diff
changeset
|
499 |
</p> |
bf9a704dc916
examples: relpipe-tr-cut
František Kučera <franta-hg@frantovo.cz>
parents:
210
diff
changeset
|
500 |
|
bf9a704dc916
examples: relpipe-tr-cut
František Kučera <franta-hg@frantovo.cz>
parents:
210
diff
changeset
|
501 |
<m:pre jazyk="bash"><![CDATA[relpipe-in-fstab \ |
bf9a704dc916
examples: relpipe-tr-cut
František Kučera <franta-hg@frantovo.cz>
parents:
210
diff
changeset
|
502 |
| relpipe-tr-grep 'fstab' 'type' '^btrfs|xfs$' \ |
bf9a704dc916
examples: relpipe-tr-cut
František Kučera <franta-hg@frantovo.cz>
parents:
210
diff
changeset
|
503 |
| relpipe-tr-cut 'fstab' 'mount_point' \ |
bf9a704dc916
examples: relpipe-tr-cut
František Kučera <franta-hg@frantovo.cz>
parents:
210
diff
changeset
|
504 |
| relpipe-out-nullbyte \ |
bf9a704dc916
examples: relpipe-tr-cut
František Kučera <franta-hg@frantovo.cz>
parents:
210
diff
changeset
|
505 |
| while read -r -d '' m; do |
bf9a704dc916
examples: relpipe-tr-cut
František Kučera <franta-hg@frantovo.cz>
parents:
210
diff
changeset
|
506 |
echo "$m"; |
bf9a704dc916
examples: relpipe-tr-cut
František Kučera <franta-hg@frantovo.cz>
parents:
210
diff
changeset
|
507 |
done]]></m:pre> |
bf9a704dc916
examples: relpipe-tr-cut
František Kučera <franta-hg@frantovo.cz>
parents:
210
diff
changeset
|
508 |
|
bf9a704dc916
examples: relpipe-tr-cut
František Kučera <franta-hg@frantovo.cz>
parents:
210
diff
changeset
|
509 |
<p> |
bf9a704dc916
examples: relpipe-tr-cut
František Kučera <franta-hg@frantovo.cz>
parents:
210
diff
changeset
|
510 |
The <code>relpipe-tr-cut</code> tool has similar syntax to its <em>grep</em> and <em>sed</em> siblings and also uses the power of regular expressions. |
bf9a704dc916
examples: relpipe-tr-cut
František Kučera <franta-hg@frantovo.cz>
parents:
210
diff
changeset
|
511 |
In this case it modifies on-the-fly the <code>fstab</code> relation and drops all its attributes except the <code>mount_point</code> one. |
bf9a704dc916
examples: relpipe-tr-cut
František Kučera <franta-hg@frantovo.cz>
parents:
210
diff
changeset
|
512 |
</p> |
bf9a704dc916
examples: relpipe-tr-cut
František Kučera <franta-hg@frantovo.cz>
parents:
210
diff
changeset
|
513 |
|
bf9a704dc916
examples: relpipe-tr-cut
František Kučera <franta-hg@frantovo.cz>
parents:
210
diff
changeset
|
514 |
<p> |
214
63f31bf2697f
examples: Writing an output filter in Bash
František Kučera <franta-hg@frantovo.cz>
parents:
213
diff
changeset
|
515 |
Then we pass the data to the Bash <code>while</code> cycle. |
63f31bf2697f
examples: Writing an output filter in Bash
František Kučera <franta-hg@frantovo.cz>
parents:
213
diff
changeset
|
516 |
In such simple scenario (just <code>echo</code>), we could use <code>xargs</code> as in examples above, |
212
bf9a704dc916
examples: relpipe-tr-cut
František Kučera <franta-hg@frantovo.cz>
parents:
210
diff
changeset
|
517 |
but in this syntax, we can write whole block of shell commands for each record/value and do more complex actions with them. |
bf9a704dc916
examples: relpipe-tr-cut
František Kučera <franta-hg@frantovo.cz>
parents:
210
diff
changeset
|
518 |
</p> |
bf9a704dc916
examples: relpipe-tr-cut
František Kučera <franta-hg@frantovo.cz>
parents:
210
diff
changeset
|
519 |
|
bf9a704dc916
examples: relpipe-tr-cut
František Kučera <franta-hg@frantovo.cz>
parents:
210
diff
changeset
|
520 |
<h2>More projections with relpipe-tr-cut</h2> |
bf9a704dc916
examples: relpipe-tr-cut
František Kučera <franta-hg@frantovo.cz>
parents:
210
diff
changeset
|
521 |
|
bf9a704dc916
examples: relpipe-tr-cut
František Kučera <franta-hg@frantovo.cz>
parents:
210
diff
changeset
|
522 |
<p> |
bf9a704dc916
examples: relpipe-tr-cut
František Kučera <franta-hg@frantovo.cz>
parents:
210
diff
changeset
|
523 |
Assume that we have a simple relation containing numbers: |
bf9a704dc916
examples: relpipe-tr-cut
František Kučera <franta-hg@frantovo.cz>
parents:
210
diff
changeset
|
524 |
</p> |
bf9a704dc916
examples: relpipe-tr-cut
František Kučera <franta-hg@frantovo.cz>
parents:
210
diff
changeset
|
525 |
|
bf9a704dc916
examples: relpipe-tr-cut
František Kučera <franta-hg@frantovo.cz>
parents:
210
diff
changeset
|
526 |
<m:pre jazyk="bash"><![CDATA[seq 0 8 \ |
bf9a704dc916
examples: relpipe-tr-cut
František Kučera <franta-hg@frantovo.cz>
parents:
210
diff
changeset
|
527 |
| tr \\n \\0 \ |
bf9a704dc916
examples: relpipe-tr-cut
František Kučera <franta-hg@frantovo.cz>
parents:
210
diff
changeset
|
528 |
| relpipe-in-cli generate-from-stdin numbers 3 a integer b integer c integer \ |
bf9a704dc916
examples: relpipe-tr-cut
František Kučera <franta-hg@frantovo.cz>
parents:
210
diff
changeset
|
529 |
> numbers.rp]]></m:pre> |
bf9a704dc916
examples: relpipe-tr-cut
František Kučera <franta-hg@frantovo.cz>
parents:
210
diff
changeset
|
530 |
|
bf9a704dc916
examples: relpipe-tr-cut
František Kučera <franta-hg@frantovo.cz>
parents:
210
diff
changeset
|
531 |
<p>and second one containing letters:</p> |
bf9a704dc916
examples: relpipe-tr-cut
František Kučera <franta-hg@frantovo.cz>
parents:
210
diff
changeset
|
532 |
|
bf9a704dc916
examples: relpipe-tr-cut
František Kučera <franta-hg@frantovo.cz>
parents:
210
diff
changeset
|
533 |
<m:pre jazyk="bash"><![CDATA[relpipe-in-cli generate letters 2 a string b string A B C D > letters.rp]]></m:pre> |
bf9a704dc916
examples: relpipe-tr-cut
František Kučera <franta-hg@frantovo.cz>
parents:
210
diff
changeset
|
534 |
|
bf9a704dc916
examples: relpipe-tr-cut
František Kučera <franta-hg@frantovo.cz>
parents:
210
diff
changeset
|
535 |
<p>We saved them into two files and then combined them into a single file. We will work with them as they are a single stream of relations:</p> |
bf9a704dc916
examples: relpipe-tr-cut
František Kučera <franta-hg@frantovo.cz>
parents:
210
diff
changeset
|
536 |
|
bf9a704dc916
examples: relpipe-tr-cut
František Kučera <franta-hg@frantovo.cz>
parents:
210
diff
changeset
|
537 |
<m:pre jazyk="bash"><![CDATA[cat numbers.rp letters.rp > both.rp; |
bf9a704dc916
examples: relpipe-tr-cut
František Kučera <franta-hg@frantovo.cz>
parents:
210
diff
changeset
|
538 |
cat both.rp | relpipe-out-tabular]]></m:pre> |
bf9a704dc916
examples: relpipe-tr-cut
František Kučera <franta-hg@frantovo.cz>
parents:
210
diff
changeset
|
539 |
|
bf9a704dc916
examples: relpipe-tr-cut
František Kučera <franta-hg@frantovo.cz>
parents:
210
diff
changeset
|
540 |
<p>Will print:</p> |
bf9a704dc916
examples: relpipe-tr-cut
František Kučera <franta-hg@frantovo.cz>
parents:
210
diff
changeset
|
541 |
|
bf9a704dc916
examples: relpipe-tr-cut
František Kučera <franta-hg@frantovo.cz>
parents:
210
diff
changeset
|
542 |
<pre><![CDATA[numbers: |
bf9a704dc916
examples: relpipe-tr-cut
František Kučera <franta-hg@frantovo.cz>
parents:
210
diff
changeset
|
543 |
╭─────────────┬─────────────┬─────────────╮ |
bf9a704dc916
examples: relpipe-tr-cut
František Kučera <franta-hg@frantovo.cz>
parents:
210
diff
changeset
|
544 |
│ a (integer) │ b (integer) │ c (integer) │ |
bf9a704dc916
examples: relpipe-tr-cut
František Kučera <franta-hg@frantovo.cz>
parents:
210
diff
changeset
|
545 |
├─────────────┼─────────────┼─────────────┤ |
bf9a704dc916
examples: relpipe-tr-cut
František Kučera <franta-hg@frantovo.cz>
parents:
210
diff
changeset
|
546 |
│ 0 │ 1 │ 2 │ |
bf9a704dc916
examples: relpipe-tr-cut
František Kučera <franta-hg@frantovo.cz>
parents:
210
diff
changeset
|
547 |
│ 3 │ 4 │ 5 │ |
bf9a704dc916
examples: relpipe-tr-cut
František Kučera <franta-hg@frantovo.cz>
parents:
210
diff
changeset
|
548 |
│ 6 │ 7 │ 8 │ |
bf9a704dc916
examples: relpipe-tr-cut
František Kučera <franta-hg@frantovo.cz>
parents:
210
diff
changeset
|
549 |
╰─────────────┴─────────────┴─────────────╯ |
bf9a704dc916
examples: relpipe-tr-cut
František Kučera <franta-hg@frantovo.cz>
parents:
210
diff
changeset
|
550 |
Record count: 3 |
bf9a704dc916
examples: relpipe-tr-cut
František Kučera <franta-hg@frantovo.cz>
parents:
210
diff
changeset
|
551 |
letters: |
bf9a704dc916
examples: relpipe-tr-cut
František Kučera <franta-hg@frantovo.cz>
parents:
210
diff
changeset
|
552 |
╭─────────────┬─────────────╮ |
bf9a704dc916
examples: relpipe-tr-cut
František Kučera <franta-hg@frantovo.cz>
parents:
210
diff
changeset
|
553 |
│ a (string) │ b (string) │ |
bf9a704dc916
examples: relpipe-tr-cut
František Kučera <franta-hg@frantovo.cz>
parents:
210
diff
changeset
|
554 |
├─────────────┼─────────────┤ |
bf9a704dc916
examples: relpipe-tr-cut
František Kučera <franta-hg@frantovo.cz>
parents:
210
diff
changeset
|
555 |
│ A │ B │ |
bf9a704dc916
examples: relpipe-tr-cut
František Kučera <franta-hg@frantovo.cz>
parents:
210
diff
changeset
|
556 |
│ C │ D │ |
bf9a704dc916
examples: relpipe-tr-cut
František Kučera <franta-hg@frantovo.cz>
parents:
210
diff
changeset
|
557 |
╰─────────────┴─────────────╯ |
bf9a704dc916
examples: relpipe-tr-cut
František Kučera <franta-hg@frantovo.cz>
parents:
210
diff
changeset
|
558 |
Record count: 2]]></pre> |
bf9a704dc916
examples: relpipe-tr-cut
František Kučera <franta-hg@frantovo.cz>
parents:
210
diff
changeset
|
559 |
|
bf9a704dc916
examples: relpipe-tr-cut
František Kučera <franta-hg@frantovo.cz>
parents:
210
diff
changeset
|
560 |
<p>We can put away the <code>a</code> attribute from the <code>numbers</code> relation:</p> |
bf9a704dc916
examples: relpipe-tr-cut
František Kučera <franta-hg@frantovo.cz>
parents:
210
diff
changeset
|
561 |
|
bf9a704dc916
examples: relpipe-tr-cut
František Kučera <franta-hg@frantovo.cz>
parents:
210
diff
changeset
|
562 |
<m:pre jazyk="bash">cat both.rp | relpipe-tr-cut 'numbers' 'b|c' | relpipe-out-tabular</m:pre> |
bf9a704dc916
examples: relpipe-tr-cut
František Kučera <franta-hg@frantovo.cz>
parents:
210
diff
changeset
|
563 |
|
bf9a704dc916
examples: relpipe-tr-cut
František Kučera <franta-hg@frantovo.cz>
parents:
210
diff
changeset
|
564 |
<p>and leave the <code>letters</code> relation unaffected:</p> |
bf9a704dc916
examples: relpipe-tr-cut
František Kučera <franta-hg@frantovo.cz>
parents:
210
diff
changeset
|
565 |
|
bf9a704dc916
examples: relpipe-tr-cut
František Kučera <franta-hg@frantovo.cz>
parents:
210
diff
changeset
|
566 |
<pre><![CDATA[numbers: |
bf9a704dc916
examples: relpipe-tr-cut
František Kučera <franta-hg@frantovo.cz>
parents:
210
diff
changeset
|
567 |
╭─────────────┬─────────────╮ |
bf9a704dc916
examples: relpipe-tr-cut
František Kučera <franta-hg@frantovo.cz>
parents:
210
diff
changeset
|
568 |
│ b (integer) │ c (integer) │ |
bf9a704dc916
examples: relpipe-tr-cut
František Kučera <franta-hg@frantovo.cz>
parents:
210
diff
changeset
|
569 |
├─────────────┼─────────────┤ |
bf9a704dc916
examples: relpipe-tr-cut
František Kučera <franta-hg@frantovo.cz>
parents:
210
diff
changeset
|
570 |
│ 1 │ 2 │ |
bf9a704dc916
examples: relpipe-tr-cut
František Kučera <franta-hg@frantovo.cz>
parents:
210
diff
changeset
|
571 |
│ 4 │ 5 │ |
bf9a704dc916
examples: relpipe-tr-cut
František Kučera <franta-hg@frantovo.cz>
parents:
210
diff
changeset
|
572 |
│ 7 │ 8 │ |
bf9a704dc916
examples: relpipe-tr-cut
František Kučera <franta-hg@frantovo.cz>
parents:
210
diff
changeset
|
573 |
╰─────────────┴─────────────╯ |
bf9a704dc916
examples: relpipe-tr-cut
František Kučera <franta-hg@frantovo.cz>
parents:
210
diff
changeset
|
574 |
Record count: 3 |
bf9a704dc916
examples: relpipe-tr-cut
František Kučera <franta-hg@frantovo.cz>
parents:
210
diff
changeset
|
575 |
letters: |
bf9a704dc916
examples: relpipe-tr-cut
František Kučera <franta-hg@frantovo.cz>
parents:
210
diff
changeset
|
576 |
╭─────────────┬─────────────╮ |
bf9a704dc916
examples: relpipe-tr-cut
František Kučera <franta-hg@frantovo.cz>
parents:
210
diff
changeset
|
577 |
│ a (string) │ b (string) │ |
bf9a704dc916
examples: relpipe-tr-cut
František Kučera <franta-hg@frantovo.cz>
parents:
210
diff
changeset
|
578 |
├─────────────┼─────────────┤ |
bf9a704dc916
examples: relpipe-tr-cut
František Kučera <franta-hg@frantovo.cz>
parents:
210
diff
changeset
|
579 |
│ A │ B │ |
bf9a704dc916
examples: relpipe-tr-cut
František Kučera <franta-hg@frantovo.cz>
parents:
210
diff
changeset
|
580 |
│ C │ D │ |
bf9a704dc916
examples: relpipe-tr-cut
František Kučera <franta-hg@frantovo.cz>
parents:
210
diff
changeset
|
581 |
╰─────────────┴─────────────╯ |
bf9a704dc916
examples: relpipe-tr-cut
František Kučera <franta-hg@frantovo.cz>
parents:
210
diff
changeset
|
582 |
Record count: 2]]></pre> |
bf9a704dc916
examples: relpipe-tr-cut
František Kučera <franta-hg@frantovo.cz>
parents:
210
diff
changeset
|
583 |
|
bf9a704dc916
examples: relpipe-tr-cut
František Kučera <franta-hg@frantovo.cz>
parents:
210
diff
changeset
|
584 |
<p>Or we can remove <code>a</code> from both relations resp. keep there only attributes whose names match <code>'b|c'</code> regex:</p> |
bf9a704dc916
examples: relpipe-tr-cut
František Kučera <franta-hg@frantovo.cz>
parents:
210
diff
changeset
|
585 |
|
bf9a704dc916
examples: relpipe-tr-cut
František Kučera <franta-hg@frantovo.cz>
parents:
210
diff
changeset
|
586 |
<m:pre jazyk="bash">cat both.rp | relpipe-tr-cut '.*' 'b|c' | relpipe-out-tabular</m:pre> |
bf9a704dc916
examples: relpipe-tr-cut
František Kučera <franta-hg@frantovo.cz>
parents:
210
diff
changeset
|
587 |
|
bf9a704dc916
examples: relpipe-tr-cut
František Kučera <franta-hg@frantovo.cz>
parents:
210
diff
changeset
|
588 |
<p>Instead of <code>'.*'</code> we could use <code>'numbers|letters'</code> and in this case it will give the same result:</p> |
bf9a704dc916
examples: relpipe-tr-cut
František Kučera <franta-hg@frantovo.cz>
parents:
210
diff
changeset
|
589 |
|
bf9a704dc916
examples: relpipe-tr-cut
František Kučera <franta-hg@frantovo.cz>
parents:
210
diff
changeset
|
590 |
<pre><![CDATA[numbers: |
bf9a704dc916
examples: relpipe-tr-cut
František Kučera <franta-hg@frantovo.cz>
parents:
210
diff
changeset
|
591 |
╭─────────────┬─────────────╮ |
bf9a704dc916
examples: relpipe-tr-cut
František Kučera <franta-hg@frantovo.cz>
parents:
210
diff
changeset
|
592 |
│ b (integer) │ c (integer) │ |
bf9a704dc916
examples: relpipe-tr-cut
František Kučera <franta-hg@frantovo.cz>
parents:
210
diff
changeset
|
593 |
├─────────────┼─────────────┤ |
bf9a704dc916
examples: relpipe-tr-cut
František Kučera <franta-hg@frantovo.cz>
parents:
210
diff
changeset
|
594 |
│ 1 │ 2 │ |
bf9a704dc916
examples: relpipe-tr-cut
František Kučera <franta-hg@frantovo.cz>
parents:
210
diff
changeset
|
595 |
│ 4 │ 5 │ |
bf9a704dc916
examples: relpipe-tr-cut
František Kučera <franta-hg@frantovo.cz>
parents:
210
diff
changeset
|
596 |
│ 7 │ 8 │ |
bf9a704dc916
examples: relpipe-tr-cut
František Kučera <franta-hg@frantovo.cz>
parents:
210
diff
changeset
|
597 |
╰─────────────┴─────────────╯ |
bf9a704dc916
examples: relpipe-tr-cut
František Kučera <franta-hg@frantovo.cz>
parents:
210
diff
changeset
|
598 |
Record count: 3 |
bf9a704dc916
examples: relpipe-tr-cut
František Kučera <franta-hg@frantovo.cz>
parents:
210
diff
changeset
|
599 |
letters: |
bf9a704dc916
examples: relpipe-tr-cut
František Kučera <franta-hg@frantovo.cz>
parents:
210
diff
changeset
|
600 |
╭─────────────╮ |
bf9a704dc916
examples: relpipe-tr-cut
František Kučera <franta-hg@frantovo.cz>
parents:
210
diff
changeset
|
601 |
│ b (string) │ |
bf9a704dc916
examples: relpipe-tr-cut
František Kučera <franta-hg@frantovo.cz>
parents:
210
diff
changeset
|
602 |
├─────────────┤ |
bf9a704dc916
examples: relpipe-tr-cut
František Kučera <franta-hg@frantovo.cz>
parents:
210
diff
changeset
|
603 |
│ B │ |
bf9a704dc916
examples: relpipe-tr-cut
František Kučera <franta-hg@frantovo.cz>
parents:
210
diff
changeset
|
604 |
│ D │ |
bf9a704dc916
examples: relpipe-tr-cut
František Kučera <franta-hg@frantovo.cz>
parents:
210
diff
changeset
|
605 |
╰─────────────╯ |
bf9a704dc916
examples: relpipe-tr-cut
František Kučera <franta-hg@frantovo.cz>
parents:
210
diff
changeset
|
606 |
Record count: 2]]></pre> |
bf9a704dc916
examples: relpipe-tr-cut
František Kučera <franta-hg@frantovo.cz>
parents:
210
diff
changeset
|
607 |
|
bf9a704dc916
examples: relpipe-tr-cut
František Kučera <franta-hg@frantovo.cz>
parents:
210
diff
changeset
|
608 |
<p>All the time, we are reducing the attributes. But we can also multiply them or change their order:</p> |
bf9a704dc916
examples: relpipe-tr-cut
František Kučera <franta-hg@frantovo.cz>
parents:
210
diff
changeset
|
609 |
|
bf9a704dc916
examples: relpipe-tr-cut
František Kučera <franta-hg@frantovo.cz>
parents:
210
diff
changeset
|
610 |
<m:pre jazyk="bash">cat both.rp | relpipe-tr-cut 'numbers' 'b|a|c' 'b' 'a' 'a' | relpipe-out-tabular</m:pre> |
bf9a704dc916
examples: relpipe-tr-cut
František Kučera <franta-hg@frantovo.cz>
parents:
210
diff
changeset
|
611 |
|
bf9a704dc916
examples: relpipe-tr-cut
František Kučera <franta-hg@frantovo.cz>
parents:
210
diff
changeset
|
612 |
<p> |
bf9a704dc916
examples: relpipe-tr-cut
František Kučera <franta-hg@frantovo.cz>
parents:
210
diff
changeset
|
613 |
n.b. the order in <code>'b|a|c'</code> does not matter and if such regex matches, it preserves the original order of the attributes; |
213
cbf25a63a43f
specification: relpipe-tr-cut
František Kučera <franta-hg@frantovo.cz>
parents:
212
diff
changeset
|
614 |
but if we use multiple regexes to specify attributes, their order and count matters: |
212
bf9a704dc916
examples: relpipe-tr-cut
František Kučera <franta-hg@frantovo.cz>
parents:
210
diff
changeset
|
615 |
</p> |
bf9a704dc916
examples: relpipe-tr-cut
František Kučera <franta-hg@frantovo.cz>
parents:
210
diff
changeset
|
616 |
|
bf9a704dc916
examples: relpipe-tr-cut
František Kučera <franta-hg@frantovo.cz>
parents:
210
diff
changeset
|
617 |
<pre><![CDATA[numbers: |
bf9a704dc916
examples: relpipe-tr-cut
František Kučera <franta-hg@frantovo.cz>
parents:
210
diff
changeset
|
618 |
╭─────────────┬─────────────┬─────────────┬─────────────┬─────────────┬─────────────╮ |
bf9a704dc916
examples: relpipe-tr-cut
František Kučera <franta-hg@frantovo.cz>
parents:
210
diff
changeset
|
619 |
│ a (integer) │ b (integer) │ c (integer) │ b (integer) │ a (integer) │ a (integer) │ |
bf9a704dc916
examples: relpipe-tr-cut
František Kučera <franta-hg@frantovo.cz>
parents:
210
diff
changeset
|
620 |
├─────────────┼─────────────┼─────────────┼─────────────┼─────────────┼─────────────┤ |
bf9a704dc916
examples: relpipe-tr-cut
František Kučera <franta-hg@frantovo.cz>
parents:
210
diff
changeset
|
621 |
│ 0 │ 1 │ 2 │ 1 │ 0 │ 0 │ |
bf9a704dc916
examples: relpipe-tr-cut
František Kučera <franta-hg@frantovo.cz>
parents:
210
diff
changeset
|
622 |
│ 3 │ 4 │ 5 │ 4 │ 3 │ 3 │ |
bf9a704dc916
examples: relpipe-tr-cut
František Kučera <franta-hg@frantovo.cz>
parents:
210
diff
changeset
|
623 |
│ 6 │ 7 │ 8 │ 7 │ 6 │ 6 │ |
bf9a704dc916
examples: relpipe-tr-cut
František Kučera <franta-hg@frantovo.cz>
parents:
210
diff
changeset
|
624 |
╰─────────────┴─────────────┴─────────────┴─────────────┴─────────────┴─────────────╯ |
bf9a704dc916
examples: relpipe-tr-cut
František Kučera <franta-hg@frantovo.cz>
parents:
210
diff
changeset
|
625 |
Record count: 3 |
bf9a704dc916
examples: relpipe-tr-cut
František Kučera <franta-hg@frantovo.cz>
parents:
210
diff
changeset
|
626 |
letters: |
bf9a704dc916
examples: relpipe-tr-cut
František Kučera <franta-hg@frantovo.cz>
parents:
210
diff
changeset
|
627 |
╭─────────────┬─────────────╮ |
bf9a704dc916
examples: relpipe-tr-cut
František Kučera <franta-hg@frantovo.cz>
parents:
210
diff
changeset
|
628 |
│ a (string) │ b (string) │ |
bf9a704dc916
examples: relpipe-tr-cut
František Kučera <franta-hg@frantovo.cz>
parents:
210
diff
changeset
|
629 |
├─────────────┼─────────────┤ |
bf9a704dc916
examples: relpipe-tr-cut
František Kučera <franta-hg@frantovo.cz>
parents:
210
diff
changeset
|
630 |
│ A │ B │ |
bf9a704dc916
examples: relpipe-tr-cut
František Kučera <franta-hg@frantovo.cz>
parents:
210
diff
changeset
|
631 |
│ C │ D │ |
bf9a704dc916
examples: relpipe-tr-cut
František Kučera <franta-hg@frantovo.cz>
parents:
210
diff
changeset
|
632 |
╰─────────────┴─────────────╯ |
bf9a704dc916
examples: relpipe-tr-cut
František Kučera <franta-hg@frantovo.cz>
parents:
210
diff
changeset
|
633 |
Record count: 2]]></pre> |
bf9a704dc916
examples: relpipe-tr-cut
František Kučera <franta-hg@frantovo.cz>
parents:
210
diff
changeset
|
634 |
|
bf9a704dc916
examples: relpipe-tr-cut
František Kučera <franta-hg@frantovo.cz>
parents:
210
diff
changeset
|
635 |
<p> |
bf9a704dc916
examples: relpipe-tr-cut
František Kučera <franta-hg@frantovo.cz>
parents:
210
diff
changeset
|
636 |
The <code>letters</code> relation stays rock steady and <code>relpipe-tr-cut 'numbers'</code> does not affect it in any way. |
bf9a704dc916
examples: relpipe-tr-cut
František Kučera <franta-hg@frantovo.cz>
parents:
210
diff
changeset
|
637 |
</p> |
bf9a704dc916
examples: relpipe-tr-cut
František Kučera <franta-hg@frantovo.cz>
parents:
210
diff
changeset
|
638 |
|
bf9a704dc916
examples: relpipe-tr-cut
František Kučera <franta-hg@frantovo.cz>
parents:
210
diff
changeset
|
639 |
|
232
c7d05cf04b76
examples: Read an Atom feed using XQuery and relpipe-in-xml
František Kučera <franta-hg@frantovo.cz>
parents:
223
diff
changeset
|
640 |
<h2>Read an Atom feed using XQuery and relpipe-in-xml</h2> |
c7d05cf04b76
examples: Read an Atom feed using XQuery and relpipe-in-xml
František Kučera <franta-hg@frantovo.cz>
parents:
223
diff
changeset
|
641 |
|
c7d05cf04b76
examples: Read an Atom feed using XQuery and relpipe-in-xml
František Kučera <franta-hg@frantovo.cz>
parents:
223
diff
changeset
|
642 |
<p> |
c7d05cf04b76
examples: Read an Atom feed using XQuery and relpipe-in-xml
František Kučera <franta-hg@frantovo.cz>
parents:
223
diff
changeset
|
643 |
Atom Syndication Format is a standard for publishing web feeds a.k.a web syndication. |
c7d05cf04b76
examples: Read an Atom feed using XQuery and relpipe-in-xml
František Kučera <franta-hg@frantovo.cz>
parents:
223
diff
changeset
|
644 |
These feeds are usually consumed by a <em>feed reeder</em> that aggregates news from many websites and displays them in a uniform format. |
c7d05cf04b76
examples: Read an Atom feed using XQuery and relpipe-in-xml
František Kučera <franta-hg@frantovo.cz>
parents:
223
diff
changeset
|
645 |
The Atom feed is an XML with a list of recent news containing their titles, URLs and short annotations. |
c7d05cf04b76
examples: Read an Atom feed using XQuery and relpipe-in-xml
František Kučera <franta-hg@frantovo.cz>
parents:
223
diff
changeset
|
646 |
It also contains some metadata (website author, title etc.). |
c7d05cf04b76
examples: Read an Atom feed using XQuery and relpipe-in-xml
František Kučera <franta-hg@frantovo.cz>
parents:
223
diff
changeset
|
647 |
</p> |
c7d05cf04b76
examples: Read an Atom feed using XQuery and relpipe-in-xml
František Kučera <franta-hg@frantovo.cz>
parents:
223
diff
changeset
|
648 |
<p> |
c7d05cf04b76
examples: Read an Atom feed using XQuery and relpipe-in-xml
František Kučera <franta-hg@frantovo.cz>
parents:
223
diff
changeset
|
649 |
Using this simple XQuery<m:podČarou>see <a href="https://en.wikibooks.org/wiki/XQuery">XQuery</a> at Wikibooks</m:podČarou> |
c7d05cf04b76
examples: Read an Atom feed using XQuery and relpipe-in-xml
František Kučera <franta-hg@frantovo.cz>
parents:
223
diff
changeset
|
650 |
<em>FLWOR Expression</em> |
c7d05cf04b76
examples: Read an Atom feed using XQuery and relpipe-in-xml
František Kučera <franta-hg@frantovo.cz>
parents:
223
diff
changeset
|
651 |
we convert the Atom feed into the XML serialization of relational data: |
c7d05cf04b76
examples: Read an Atom feed using XQuery and relpipe-in-xml
František Kučera <franta-hg@frantovo.cz>
parents:
223
diff
changeset
|
652 |
</p> |
c7d05cf04b76
examples: Read an Atom feed using XQuery and relpipe-in-xml
František Kučera <franta-hg@frantovo.cz>
parents:
223
diff
changeset
|
653 |
|
c7d05cf04b76
examples: Read an Atom feed using XQuery and relpipe-in-xml
František Kučera <franta-hg@frantovo.cz>
parents:
223
diff
changeset
|
654 |
<m:pre jazyk="xq" src="examples/atom.xq" odkaz="ano"/> |
c7d05cf04b76
examples: Read an Atom feed using XQuery and relpipe-in-xml
František Kučera <franta-hg@frantovo.cz>
parents:
223
diff
changeset
|
655 |
|
c7d05cf04b76
examples: Read an Atom feed using XQuery and relpipe-in-xml
František Kučera <franta-hg@frantovo.cz>
parents:
223
diff
changeset
|
656 |
<p> |
c7d05cf04b76
examples: Read an Atom feed using XQuery and relpipe-in-xml
František Kučera <franta-hg@frantovo.cz>
parents:
223
diff
changeset
|
657 |
This is similar operation to <a href="https://www.postgresql.org/docs/current/functions-xml.html">xmltable</a> used in SQL databases. |
c7d05cf04b76
examples: Read an Atom feed using XQuery and relpipe-in-xml
František Kučera <franta-hg@frantovo.cz>
parents:
223
diff
changeset
|
658 |
It converts an XML tree structure to the relational form. |
c7d05cf04b76
examples: Read an Atom feed using XQuery and relpipe-in-xml
František Kučera <franta-hg@frantovo.cz>
parents:
223
diff
changeset
|
659 |
In our case, the output is still XML, but in a format that can be read by <code>relpipe-in-xml</code>. |
c7d05cf04b76
examples: Read an Atom feed using XQuery and relpipe-in-xml
František Kučera <franta-hg@frantovo.cz>
parents:
223
diff
changeset
|
660 |
All put together in a single shell script: |
c7d05cf04b76
examples: Read an Atom feed using XQuery and relpipe-in-xml
František Kučera <franta-hg@frantovo.cz>
parents:
223
diff
changeset
|
661 |
</p> |
c7d05cf04b76
examples: Read an Atom feed using XQuery and relpipe-in-xml
František Kučera <franta-hg@frantovo.cz>
parents:
223
diff
changeset
|
662 |
|
c7d05cf04b76
examples: Read an Atom feed using XQuery and relpipe-in-xml
František Kučera <franta-hg@frantovo.cz>
parents:
223
diff
changeset
|
663 |
<m:pre jazyk="bash" src="examples/atom.sh"/> |
c7d05cf04b76
examples: Read an Atom feed using XQuery and relpipe-in-xml
František Kučera <franta-hg@frantovo.cz>
parents:
223
diff
changeset
|
664 |
|
c7d05cf04b76
examples: Read an Atom feed using XQuery and relpipe-in-xml
František Kučera <franta-hg@frantovo.cz>
parents:
223
diff
changeset
|
665 |
<p>Will generate a table with web news:</p> |
c7d05cf04b76
examples: Read an Atom feed using XQuery and relpipe-in-xml
František Kučera <franta-hg@frantovo.cz>
parents:
223
diff
changeset
|
666 |
|
c7d05cf04b76
examples: Read an Atom feed using XQuery and relpipe-in-xml
František Kučera <franta-hg@frantovo.cz>
parents:
223
diff
changeset
|
667 |
<m:pre jazyk="text" src="examples/atom.txt"/> |
c7d05cf04b76
examples: Read an Atom feed using XQuery and relpipe-in-xml
František Kučera <franta-hg@frantovo.cz>
parents:
223
diff
changeset
|
668 |
|
c7d05cf04b76
examples: Read an Atom feed using XQuery and relpipe-in-xml
František Kučera <franta-hg@frantovo.cz>
parents:
223
diff
changeset
|
669 |
<p> |
c7d05cf04b76
examples: Read an Atom feed using XQuery and relpipe-in-xml
František Kučera <franta-hg@frantovo.cz>
parents:
223
diff
changeset
|
670 |
For frequent usage we can create a script or funcrion called <code>relpipe-in-atom</code> |
c7d05cf04b76
examples: Read an Atom feed using XQuery and relpipe-in-xml
František Kučera <franta-hg@frantovo.cz>
parents:
223
diff
changeset
|
671 |
that reads Atom XML on STDIN and generates relational data on STDOUT. |
c7d05cf04b76
examples: Read an Atom feed using XQuery and relpipe-in-xml
František Kučera <franta-hg@frantovo.cz>
parents:
223
diff
changeset
|
672 |
And then do any of these: |
c7d05cf04b76
examples: Read an Atom feed using XQuery and relpipe-in-xml
František Kučera <franta-hg@frantovo.cz>
parents:
223
diff
changeset
|
673 |
</p> |
c7d05cf04b76
examples: Read an Atom feed using XQuery and relpipe-in-xml
František Kučera <franta-hg@frantovo.cz>
parents:
223
diff
changeset
|
674 |
|
c7d05cf04b76
examples: Read an Atom feed using XQuery and relpipe-in-xml
František Kučera <franta-hg@frantovo.cz>
parents:
223
diff
changeset
|
675 |
<m:pre jazyk="bash"><![CDATA[wget … | relpipe-in-atom | relpipe-out-tabular |
c7d05cf04b76
examples: Read an Atom feed using XQuery and relpipe-in-xml
František Kučera <franta-hg@frantovo.cz>
parents:
223
diff
changeset
|
676 |
wget … | relpipe-in-atom | relpipe-out-csv |
c7d05cf04b76
examples: Read an Atom feed using XQuery and relpipe-in-xml
František Kučera <franta-hg@frantovo.cz>
parents:
223
diff
changeset
|
677 |
wget … | relpipe-in-atom | relpipe-out-gui |
c7d05cf04b76
examples: Read an Atom feed using XQuery and relpipe-in-xml
František Kučera <franta-hg@frantovo.cz>
parents:
223
diff
changeset
|
678 |
wget … | relpipe-in-atom | relpipe-out-nullbyte | while read_nullbyte published title url; do echo "$title"; done |
c7d05cf04b76
examples: Read an Atom feed using XQuery and relpipe-in-xml
František Kučera <franta-hg@frantovo.cz>
parents:
223
diff
changeset
|
679 |
wget … | relpipe-in-atom | relpipe-out-csv | csv2rec | … |
c7d05cf04b76
examples: Read an Atom feed using XQuery and relpipe-in-xml
František Kučera <franta-hg@frantovo.cz>
parents:
223
diff
changeset
|
680 |
]]></m:pre> |
c7d05cf04b76
examples: Read an Atom feed using XQuery and relpipe-in-xml
František Kučera <franta-hg@frantovo.cz>
parents:
223
diff
changeset
|
681 |
|
c7d05cf04b76
examples: Read an Atom feed using XQuery and relpipe-in-xml
František Kučera <franta-hg@frantovo.cz>
parents:
223
diff
changeset
|
682 |
<p> |
c7d05cf04b76
examples: Read an Atom feed using XQuery and relpipe-in-xml
František Kučera <franta-hg@frantovo.cz>
parents:
223
diff
changeset
|
683 |
There are several implementations of XQuery. |
c7d05cf04b76
examples: Read an Atom feed using XQuery and relpipe-in-xml
František Kučera <franta-hg@frantovo.cz>
parents:
223
diff
changeset
|
684 |
<a href="http://galax.sourceforge.net/">Galax</a> is one of them. |
c7d05cf04b76
examples: Read an Atom feed using XQuery and relpipe-in-xml
František Kučera <franta-hg@frantovo.cz>
parents:
223
diff
changeset
|
685 |
<a href="http://xqilla.sourceforge.net/">XQilla</a> or |
c7d05cf04b76
examples: Read an Atom feed using XQuery and relpipe-in-xml
František Kučera <franta-hg@frantovo.cz>
parents:
223
diff
changeset
|
686 |
<a href="http://basex.org/basex/xquery/">BaseX</a> are another ones (and support newer versions of the standard). |
c7d05cf04b76
examples: Read an Atom feed using XQuery and relpipe-in-xml
František Kučera <franta-hg@frantovo.cz>
parents:
223
diff
changeset
|
687 |
There are also XSLT processors like <a href="http://xmlsoft.org/XSLT/xsltproc2.html">xsltproc</a>. |
c7d05cf04b76
examples: Read an Atom feed using XQuery and relpipe-in-xml
František Kučera <franta-hg@frantovo.cz>
parents:
223
diff
changeset
|
688 |
BaseX can be used instead of Galax – we just replace |
c7d05cf04b76
examples: Read an Atom feed using XQuery and relpipe-in-xml
František Kučera <franta-hg@frantovo.cz>
parents:
223
diff
changeset
|
689 |
<code>galax-run -context-item /dev/stdin</code> with <code>basex -i /dev/stdin</code>. |
c7d05cf04b76
examples: Read an Atom feed using XQuery and relpipe-in-xml
František Kučera <franta-hg@frantovo.cz>
parents:
223
diff
changeset
|
690 |
</p> |
c7d05cf04b76
examples: Read an Atom feed using XQuery and relpipe-in-xml
František Kučera <franta-hg@frantovo.cz>
parents:
223
diff
changeset
|
691 |
|
c7d05cf04b76
examples: Read an Atom feed using XQuery and relpipe-in-xml
František Kučera <franta-hg@frantovo.cz>
parents:
223
diff
changeset
|
692 |
<p> |
c7d05cf04b76
examples: Read an Atom feed using XQuery and relpipe-in-xml
František Kučera <franta-hg@frantovo.cz>
parents:
223
diff
changeset
|
693 |
Reading Atom feeds in a terminal might not be the best way to get news from a website, |
c7d05cf04b76
examples: Read an Atom feed using XQuery and relpipe-in-xml
František Kučera <franta-hg@frantovo.cz>
parents:
223
diff
changeset
|
694 |
but this simple example learns us how to convert arbitrary XML to relational data. |
c7d05cf04b76
examples: Read an Atom feed using XQuery and relpipe-in-xml
František Kučera <franta-hg@frantovo.cz>
parents:
223
diff
changeset
|
695 |
And of course, we can generate multiple relations from a single XML using a single XQuery script. |
c7d05cf04b76
examples: Read an Atom feed using XQuery and relpipe-in-xml
František Kučera <franta-hg@frantovo.cz>
parents:
223
diff
changeset
|
696 |
XQuery can be also used for operations like JOIN or UNION and for filtering and other transformations |
c7d05cf04b76
examples: Read an Atom feed using XQuery and relpipe-in-xml
František Kučera <franta-hg@frantovo.cz>
parents:
223
diff
changeset
|
697 |
as will be shown in further examples. |
c7d05cf04b76
examples: Read an Atom feed using XQuery and relpipe-in-xml
František Kučera <franta-hg@frantovo.cz>
parents:
223
diff
changeset
|
698 |
</p> |
c7d05cf04b76
examples: Read an Atom feed using XQuery and relpipe-in-xml
František Kučera <franta-hg@frantovo.cz>
parents:
223
diff
changeset
|
699 |
|
c7d05cf04b76
examples: Read an Atom feed using XQuery and relpipe-in-xml
František Kučera <franta-hg@frantovo.cz>
parents:
223
diff
changeset
|
700 |
|
87
25dec6931f18
Lepší odsazení, tabulátory.
František Kučera <franta-hg@frantovo.cz>
parents:
23
diff
changeset
|
701 |
</text> |
4
1bb39595a51c
genrování hlavní nabídky #1
František Kučera <franta-hg@frantovo.cz>
parents:
2
diff
changeset
|
702 |
|
1 | 703 |
</stránka> |