|
1 <stránka |
|
2 xmlns="https://trac.frantovo.cz/xml-web-generator/wiki/xmlns/strana" |
|
3 xmlns:m="https://trac.frantovo.cz/xml-web-generator/wiki/xmlns/makro"> |
|
4 |
|
5 <nadpis>Aggregating data with AWK</nadpis> |
|
6 <perex>counting records and computing sum or appending new records</perex> |
|
7 <m:pořadí-příkladu>02600</m:pořadí-příkladu> |
|
8 |
|
9 <text xmlns="http://www.w3.org/1999/xhtml"> |
|
10 |
|
11 <p> |
|
12 We have filtered records, modified attribute values, added and removed attributes, dropped a relation… |
|
13 and there is one more operation that we can do with AWK: <code>INSERT</code> resp. appending or preppending additional records to the relation |
|
14 – and we can also completely replace the record set by skipping the original records. |
|
15 </p> |
|
16 |
|
17 <h2>Adding records</h2> |
|
18 |
|
19 <p> |
|
20 Using options <code>--before-records</code> and <code>--after-records</code> we can pass additional AWK code that will be executed – once for given relation. |
|
21 The <code>record()</code> function will then generate an additional record (can be called multiple times and generate more records): |
|
22 </p> |
|
23 |
|
24 <m:pre jazyk="bash"><![CDATA[awkCode=' |
|
25 scheme = ""; |
|
26 device = "/dev/vg1/volume123"; |
|
27 mount_point = "/mnt/v123"; |
|
28 type = "btrfs"; |
|
29 options = "relatime"; |
|
30 dump = 0; |
|
31 pass = 2; |
|
32 |
|
33 record(); |
|
34 '; |
|
35 |
|
36 relpipe-in-fstab \ |
|
37 | relpipe-tr-awk \ |
|
38 --relation '.*' \ |
|
39 --for-each '1' \ |
|
40 --after-records "$awkCode" \ |
|
41 | relpipe-out-tabular]]></m:pre> |
|
42 |
|
43 <p>Which will <code>INSERT</code> one new record:</p> |
|
44 |
|
45 <pre><![CDATA[fstab: |
|
46 ╭─────────────────┬──────────────────────────────────────┬──────────────────────┬───────────────┬───────────────────────────────────────┬────────────────┬────────────────╮ |
|
47 │ scheme (string) │ device (string) │ mount_point (string) │ type (string) │ options (string) │ dump (integer) │ pass (integer) │ |
|
48 ├─────────────────┼──────────────────────────────────────┼──────────────────────┼───────────────┼───────────────────────────────────────┼────────────────┼────────────────┤ |
|
49 │ UUID │ 29758270-fd25-4a6c-a7bb-9a18302816af │ / │ ext4 │ relatime,user_xattr,errors=remount-ro │ 0 │ 1 │ |
|
50 │ │ /dev/sr0 │ /media/cdrom0 │ udf,iso9660 │ user,noauto │ 0 │ 0 │ |
|
51 │ │ /dev/sde │ /mnt/data │ ext4 │ relatime,user_xattr,errors=remount-ro │ 0 │ 2 │ |
|
52 │ UUID │ a2b5f230-a795-4f6f-a39b-9b57686c86d5 │ /home │ btrfs │ relatime │ 0 │ 2 │ |
|
53 │ │ /dev/mapper/sdf_crypt │ /mnt/private │ xfs │ relatime │ 0 │ 2 │ |
|
54 │ │ /dev/vg1/volume123 │ /mnt/v123 │ btrfs │ relatime │ 0 │ 2 │ |
|
55 ╰─────────────────┴──────────────────────────────────────┴──────────────────────┴───────────────┴───────────────────────────────────────┴────────────────┴────────────────╯ |
|
56 Record count: 6]]></pre> |
|
57 |
|
58 <h2>Counting and summarizing values</h2> |
|
59 |
|
60 <p>We can also compute some statistics like <code>COUNT()</code> and <code>SUM()</code>:</p> |
|
61 |
|
62 <m:pre jazyk="bash"><![CDATA[find -print0 | relpipe-in-filesystem \ |
|
63 | relpipe-tr-awk \ |
|
64 --relation '.*' \ |
|
65 --before-records 'count = 0; total_size = 0;' \ |
|
66 --for-each '{ count++; total_size += size; }' \ |
|
67 --after-records 'record();' \ |
|
68 --output-attribute count integer \ |
|
69 --output-attribute total_size integer \ |
|
70 | relpipe-out-tabular]]></m:pre> |
|
71 |
|
72 <p>and get result:</p> |
|
73 |
|
74 <pre><![CDATA[filesystem: |
|
75 ╭─────────────────┬──────────────────────╮ |
|
76 │ count (integer) │ total_size (integer) │ |
|
77 ├─────────────────┼──────────────────────┤ |
|
78 │ 9 │ 818747 │ |
|
79 ╰─────────────────┴──────────────────────╯ |
|
80 Record count: 1]]></pre> |
|
81 |
|
82 <p>Where the <code>total_size</code> is the same as will <code>du</code> compute:</p> |
|
83 |
|
84 <pre>find . -type f -print0 | du -b -c --files0-from=-</pre> |
|
85 |
|
86 <p>Analogously we can compute minimum, maximum etc. using AWK transformation.</p> |
|
87 |
|
88 </text> |
|
89 |
|
90 </stránka> |