|
1 <stránka |
|
2 xmlns="https://trac.frantovo.cz/xml-web-generator/wiki/xmlns/strana" |
|
3 xmlns:m="https://trac.frantovo.cz/xml-web-generator/wiki/xmlns/makro"> |
|
4 |
|
5 <nadpis>Complex filtering with Guile</nadpis> |
|
6 <perex>filtering records with AND, OR and functions</perex> |
|
7 <m:pořadí-příkladu>01400</m:pořadí-příkladu> |
|
8 |
|
9 <text xmlns="http://www.w3.org/1999/xhtml"> |
|
10 |
|
11 <p> |
|
12 For simple filtering, we can use <code>relpipe-tr-grep</code>. |
|
13 But what if we need to write some complex query that contains AND and OR operators? |
|
14 What if we need e.g. compare numbers – not only match texts against regular expressions? |
|
15 There is a tool capable to do this and much more: <code>relpipe-tr-guile</code>! |
|
16 </p> |
|
17 |
|
18 <p> |
|
19 <a href="https://www.gnu.org/software/guile/">Guile</a> is the GNU implementation of Scheme language (something like Lisp and also full of parenthesis). |
|
20 The <code>relpipe-tr-guile</code> uses GNU Guile as a library, puts data in the Guile context and evaluates Guile expressions and then reads data from the Guile context back and generates relational output from them. |
|
21 Good news are that it is not necessary to know Lisp/Scheme to use this tool. For the first steps, it can be used just as a query language – like SQL, just a bit Polish. |
|
22 </p> |
|
23 |
|
24 <h2>Filtering numbers</h2> |
|
25 |
|
26 <p> |
|
27 We are looking for „satanistic“ icons in our filesystem – those that have size = 666 bytes. |
|
28 </p> |
|
29 |
|
30 <m:pre jazyk="bash"><![CDATA[$ find /usr/share/icons/ -type f -print0 \ |
|
31 | relpipe-in-filesystem \ |
|
32 | relpipe-tr-guile --relation 'files.*' --where '(= $size 666)' \ |
|
33 | relpipe-out-tabular]]></m:pre> |
|
34 |
|
35 <p>Well, well… here we are:</p> |
|
36 |
|
37 <m:pre jazyk="text"><![CDATA[filesystem: |
|
38 ╭───────────────────────────────────────────────────────────────────────┬───────────────┬────────────────┬────────────────┬────────────────╮ |
|
39 │ path (string) │ type (string) │ size (integer) │ owner (string) │ group (string) │ |
|
40 ├───────────────────────────────────────────────────────────────────────┼───────────────┼────────────────┼────────────────┼────────────────┤ |
|
41 │ /usr/share/icons/elementary-xfce/actions/24/tab-new.png │ f │ 666 │ root │ root │ |
|
42 │ /usr/share/icons/elementary-xfce/apps/16/clock.png │ f │ 666 │ root │ root │ |
|
43 │ /usr/share/icons/elementary-xfce/mimes/22/x-office-spreadsheet.png │ f │ 666 │ root │ root │ |
|
44 │ /usr/share/icons/Tango/22x22/apps/office-calendar.png │ f │ 666 │ root │ root │ |
|
45 │ /usr/share/icons/Tango/16x16/actions/process-stop.png │ f │ 666 │ root │ root │ |
|
46 │ /usr/share/icons/breeze/actions/24/align-vertical-center.svg │ f │ 666 │ root │ root │ |
|
47 │ /usr/share/icons/breeze/devices/22/camera-photo.svg │ f │ 666 │ root │ root │ |
|
48 │ /usr/share/icons/oxygen/base/48x48/actions/tab-detach.png │ f │ 666 │ root │ root │ |
|
49 │ /usr/share/icons/oxygen/base/32x32/actions/insert-horizontal-rule.png │ f │ 666 │ root │ root │ |
|
50 │ /usr/share/icons/breeze-dark/actions/24/align-vertical-center.svg │ f │ 666 │ root │ root │ |
|
51 │ /usr/share/icons/breeze-dark/devices/22/camera-photo.svg │ f │ 666 │ root │ root │ |
|
52 │ /usr/share/icons/gnome/22x22/status/weather-overcast.png │ f │ 666 │ root │ root │ |
|
53 │ /usr/share/icons/gnome/16x16/actions/go-home.png │ f │ 666 │ root │ root │ |
|
54 ╰───────────────────────────────────────────────────────────────────────┴───────────────┴────────────────┴────────────────┴────────────────╯ |
|
55 Record count: 13]]></m:pre> |
|
56 |
|
57 <p>The <code>--relation 'files.*'</code> is a regular expression that says which relations should be processed in Guile – others are passed through unchanged.</p> |
|
58 |
|
59 <p> |
|
60 The <code>--where '(= $size 666)'</code> is our condition. |
|
61 The Polish<m:podČarou>see <a href="https://en.wikipedia.org/wiki/Polish_notation">Polish notation</a></m:podČarou> thing means that we write <code>= $size 666</code> instead of <code>$size = 666</code>. |
|
62 It seems a bit weird but it makes sense – the <code>=</code> is a function that compares two numbers and returns a boolean value – |
|
63 so we just call this function and pass <code>$size</code> and <code>666</code> arguments to it. |
|
64 And because it is a function, there are <code>(</code>parentheses<code>)</code>. |
|
65 </p> |
|
66 |
|
67 <p> |
|
68 Relational attributes are mapped to Guile variables with same name, just prefixed with <code>$</code>. |
|
69 (we considered <code> |
|
70 <abbr title="Bitcoin">₿</abbr> |
|
71 </code> symbol, but <code>$</code> seems to be still more common on keyboards in 2019) |
|
72 While relational attribute name is an arbitrary string, Guile variable names have some limitations, thus not all attributes can be mapped – those with spaces and some special characters are currently unsupported (this will be fixed in later versions by some kind of encoding/escaping). |
|
73 </p> |
|
74 |
|
75 <p> |
|
76 We can also look for |
|
77 <code>--where '(> $size 100)'</code> which means „size is greater than 100“ |
|
78 or |
|
79 <code>--where '(< $size 100)'</code> which means „size is smaller than 100“. |
|
80 The <code>>=</code> and <code><=</code> also work as expected. |
|
81 </p> |
|
82 |
|
83 <h2>Filtering strings</h2> |
|
84 |
|
85 <p> |
|
86 Scheme is strongly typed language and we have to use proper functions/operators for each type. |
|
87 For strings, it is <code>string=</code> instead of <code>=</code> function: |
|
88 </p> |
|
89 |
|
90 <m:pre jazyk="bash"><![CDATA[relpipe-in-fstab \ |
|
91 | relpipe-tr-guile --relation 'fstab' --where '(string= $type "btrfs")' \ |
|
92 | relpipe-out-tabular]]></m:pre> |
|
93 |
|
94 <p>The Btrfs filesystems in our <code>fstab</code>:</p> |
|
95 |
|
96 <m:pre jazyk="text"><![CDATA[fstab: |
|
97 ╭─────────────────┬──────────────────────────────────────┬──────────────────────┬───────────────┬──────────────────┬────────────────┬────────────────╮ |
|
98 │ scheme (string) │ device (string) │ mount_point (string) │ type (string) │ options (string) │ dump (integer) │ pass (integer) │ |
|
99 ├─────────────────┼──────────────────────────────────────┼──────────────────────┼───────────────┼──────────────────┼────────────────┼────────────────┤ |
|
100 │ UUID │ a2b5f230-a795-4f6f-a39b-9b57686c86d5 │ /home │ btrfs │ relatime │ 0 │ 2 │ |
|
101 ╰─────────────────┴──────────────────────────────────────┴──────────────────────┴───────────────┴──────────────────┴────────────────┴────────────────╯ |
|
102 Record count: 1]]></m:pre> |
|
103 |
|
104 <p> |
|
105 There is also <code>string-prefix?</code> which evaluates whether the first string is a prefix of the second string: |
|
106 </p> |
|
107 |
|
108 <m:pre jazyk="bash"><![CDATA[relpipe-in-fstab \ |
|
109 | relpipe-tr-guile --relation 'fstab' --where '(string-prefix? "/mnt" $mount_point)' \ |
|
110 | relpipe-out-tabular]]></m:pre> |
|
111 |
|
112 <p>So we can find filesystems mounted somewhere under <code>/mnt</code>:</p> |
|
113 |
|
114 <m:pre jazyk="bash"><![CDATA[fstab: |
|
115 ╭─────────────────┬───────────────────────┬──────────────────────┬───────────────┬───────────────────────────────────────┬────────────────┬────────────────╮ |
|
116 │ scheme (string) │ device (string) │ mount_point (string) │ type (string) │ options (string) │ dump (integer) │ pass (integer) │ |
|
117 ├─────────────────┼───────────────────────┼──────────────────────┼───────────────┼───────────────────────────────────────┼────────────────┼────────────────┤ |
|
118 │ │ /dev/sde │ /mnt/data │ ext4 │ relatime,user_xattr,errors=remount-ro │ 0 │ 2 │ |
|
119 │ │ /dev/mapper/sdf_crypt │ /mnt/private │ xfs │ relatime │ 0 │ 2 │ |
|
120 ╰─────────────────┴───────────────────────┴──────────────────────┴───────────────┴───────────────────────────────────────┴────────────────┴────────────────╯ |
|
121 Record count: 2]]></m:pre> |
|
122 |
|
123 <p> |
|
124 There are much more functions – can be found in the <a href="https://www.gnu.org/software/guile/manual/guile.html">Guile documentation</a> |
|
125 – like case-insensitive variants (e.g. <code>string-ci=</code>) or regular expression search (<code>string-match</code>). |
|
126 </p> |
|
127 |
|
128 |
|
129 <h2>AND and OR</h2> |
|
130 |
|
131 <p> |
|
132 Like in SQL, we can join multiple conditions together with logical operators AND and OR. |
|
133 In Guile/Scheme these operators are also functions – they are written in the same <code>(</code>fashion<code>)</code>. |
|
134 </p> |
|
135 |
|
136 <p> |
|
137 So we can e.g. look for icons that are „satanistic“ or „Orwellian“: |
|
138 </p> |
|
139 |
|
140 <m:pre jazyk="bash"><![CDATA[find /usr/share/icons/ -type f -print0 \ |
|
141 | relpipe-in-filesystem --file path --file size \ |
|
142 | relpipe-tr-guile --relation 'files.*' --where '(or (= $size 666) (= $size 1984) )' \ |
|
143 | relpipe-out-tabular]]></m:pre> |
|
144 |
|
145 <p>Files with sizes 666 bytes or 1984 bytes:</p> |
|
146 |
|
147 <m:pre jazyk="text"><![CDATA[filesystem: |
|
148 ╭───────────────────────────────────────────────────────────────────────┬────────────────╮ |
|
149 │ path (string) │ size (integer) │ |
|
150 ├───────────────────────────────────────────────────────────────────────┼────────────────┤ |
|
151 │ /usr/share/icons/elementary-xfce/actions/48/mail-mark-important.png │ 1984 │ |
|
152 │ /usr/share/icons/elementary-xfce/actions/24/tab-new.png │ 666 │ |
|
153 │ /usr/share/icons/elementary-xfce/apps/16/clock.png │ 666 │ |
|
154 │ /usr/share/icons/elementary-xfce/mimes/22/x-office-spreadsheet.png │ 666 │ |
|
155 │ /usr/share/icons/Humanity-Dark/status/22/krb-no-valid-ticket.svg │ 1984 │ |
|
156 │ /usr/share/icons/Tango/22x22/apps/office-calendar.png │ 666 │ |
|
157 │ /usr/share/icons/Tango/16x16/actions/process-stop.png │ 666 │ |
|
158 │ /usr/share/icons/breeze/actions/24/align-vertical-center.svg │ 666 │ |
|
159 │ /usr/share/icons/breeze/devices/22/camera-photo.svg │ 666 │ |
|
160 │ /usr/share/icons/oxygen/base/48x48/actions/tab-detach.png │ 666 │ |
|
161 │ /usr/share/icons/oxygen/base/32x32/actions/insert-horizontal-rule.png │ 666 │ |
|
162 │ /usr/share/icons/Humanity/status/22/krb-no-valid-ticket.svg │ 1984 │ |
|
163 │ /usr/share/icons/breeze-dark/actions/24/align-vertical-center.svg │ 666 │ |
|
164 │ /usr/share/icons/breeze-dark/devices/22/camera-photo.svg │ 666 │ |
|
165 │ /usr/share/icons/gnome/48x48/status/user-busy.png │ 1984 │ |
|
166 │ /usr/share/icons/gnome/22x22/status/weather-overcast.png │ 666 │ |
|
167 │ /usr/share/icons/gnome/16x16/actions/go-home.png │ 666 │ |
|
168 ╰───────────────────────────────────────────────────────────────────────┴────────────────╯ |
|
169 Record count: 17]]></m:pre> |
|
170 |
|
171 <p>Or we can look for icons that are in SVG format and (at the same time) Orwellian:</p> |
|
172 |
|
173 <m:pre jazyk="bash"><![CDATA[find /usr/share/icons/ -type f -print0 \ |
|
174 | relpipe-in-filesystem --file path --file size \ |
|
175 | relpipe-tr-guile \ |
|
176 --relation 'files.*' \ |
|
177 --where '(and (string-suffix? ".svg" $path) (= $size 1984) )' \ |
|
178 | relpipe-out-tabular]]></m:pre> |
|
179 |
|
180 <p>Which is quite rare and we have only two such icons:</p> |
|
181 |
|
182 <m:pre jazyk="text"><![CDATA[filesystem: |
|
183 ╭──────────────────────────────────────────────────────────────────┬────────────────╮ |
|
184 │ path (string) │ size (integer) │ |
|
185 ├──────────────────────────────────────────────────────────────────┼────────────────┤ |
|
186 │ /usr/share/icons/Humanity-Dark/status/22/krb-no-valid-ticket.svg │ 1984 │ |
|
187 │ /usr/share/icons/Humanity/status/22/krb-no-valid-ticket.svg │ 1984 │ |
|
188 ╰──────────────────────────────────────────────────────────────────┴────────────────╯ |
|
189 Record count: 2]]></m:pre> |
|
190 |
|
191 <p> |
|
192 We can nest ANDs and ORs and other functions as deep as we need and build even very complex queries. |
|
193 Prentheses nesting is fun, isn't it? |
|
194 </p> |
|
195 |
|
196 |
|
197 |
|
198 |
|
199 |
|
200 </text> |
|
201 |
|
202 </stránka> |