|
1 <stránka |
|
2 xmlns="https://trac.frantovo.cz/xml-web-generator/wiki/xmlns/strana" |
|
3 xmlns:m="https://trac.frantovo.cz/xml-web-generator/wiki/xmlns/makro"> |
|
4 |
|
5 <nadpis>Release v0.17.1</nadpis> |
|
6 <perex>new public release of Relational pipes</perex> |
|
7 <m:release>v0.17.1</m:release> |
|
8 |
|
9 <text xmlns="http://www.w3.org/1999/xhtml"> |
|
10 <p> |
|
11 We are pleased to introduce you the new development version of <m:name/>. |
|
12 This release improves and standardizes the CLI interface of several tools, brings a new output module + several smaller improvements. |
|
13 </p> |
|
14 |
|
15 <p> |
|
16 Note: the only difference between <m:a href="release-v0.17">v0.17</m:a> and v0.17.1 are two fixes in <code>relpipe-out-gui</code>. |
|
17 </p> |
|
18 |
|
19 <ul> |
|
20 <li> |
|
21 <strong>The <code>relpipe-in-cli</code> module has changed CLI interface</strong>: see details below |
|
22 </li> |
|
23 <li> |
|
24 <strong>The <code>relpipe-in-csv</code> module has changed CLI interface</strong>: see details below |
|
25 </li> |
|
26 <li> |
|
27 <strong>The <code>relpipe-out-csv</code> module has new CLI interface</strong>: see details below |
|
28 </li> |
|
29 <li> |
|
30 <strong>The <code>relpipe-out-tabular</code> module has new CLI interface</strong>: see details below |
|
31 </li> |
|
32 <li> |
|
33 <strong>The <code>relpipe-tr-guile</code> was renamed to <code>relpipe-tr-scheme</code></strong>: see details below |
|
34 </li> |
|
35 <li> |
|
36 <strong>New tool <code>relpipe-out-jack</code> for generating MIDI through JACK</strong>: see details below |
|
37 </li> |
|
38 </ul> |
|
39 |
|
40 <p> |
|
41 See the <m:a href="examples">examples</m:a> and <m:a href="screenshots">screenshots</m:a> pages for details. |
|
42 </p> |
|
43 |
|
44 <p> |
|
45 Please note that this is still a development release and thus the API (libraries, CLI arguments, formats) might and will change. |
|
46 Any suggestions, ideas and bug reports are welcome in our <m:a href="contact">mailing list</m:a>. |
|
47 </p> |
|
48 |
|
49 <h2>New CLI interfaces</h2> |
|
50 |
|
51 <p> |
|
52 Several earlier tools have non-standard CLI interface and does not conform with <m:a href="principles">principle</m:a> „Use --long-options“. |
|
53 Posistional parameters might be shorter but they are also cryptic and less flexible. |
|
54 In this release the <code>relpipe-in-cli</code>, <code>relpipe-in-csv</code> tools dropped their old CLI interface |
|
55 and got a new one based on named parameters and long options. |
|
56 The <code>relpipe-out-csv</code> and <code>relpipe-out-tabular</code> had no parameters in previous versions and now have standard CLI interface. |
|
57 </p> |
|
58 |
|
59 <p> |
|
60 In <code>relpipe-out-tabular</code> we can now optionally omit several parts: |
|
61 <code>--write-record-count false</code>, |
|
62 <code>--write-types false</code>, |
|
63 <code>--write-relation-name false</code> |
|
64 and thus simplify the output if we do not need given information. |
|
65 </p> |
|
66 |
|
67 <pre><![CDATA[ ╭───────┬────────╮ |
|
68 │ first │ second │ |
|
69 ├───────┼────────┤ |
|
70 │ 1 │ abc │ |
|
71 │ 2 │ def │ |
|
72 ╰───────┴────────╯]]></pre> |
|
73 |
|
74 <p> |
|
75 These options may be set globally or for particular relation using the <code>--relation</code> option with pattern parameter. |
|
76 Thus we may e.g. print some relations with data types and some without them. |
|
77 </p> |
|
78 |
|
79 |
|
80 <p> |
|
81 Remaining tools that will be transformed in the same way: <code>relpipe-tr-grep</code>, <code>relpipe-tr-cut</code>, <code>relpipe-tr-sed</code> |
|
82 and will get standard CLI interface. |
|
83 </p> |
|
84 |
|
85 <h2>JACK (MIDI) output module</h2> |
|
86 |
|
87 <p> |
|
88 A powerful audio system called <a href="https://jackaudio.org/">JACK</a> allows us to |
|
89 build pipelines consisting of audio interfaces, players, recorders, filters and effects… |
|
90 and route sound streams (both PCM and MIDI) through them. |
|
91 MIDI messages can come from keyboards or other hardware MIDI controllers or from MIDI players and other software. |
|
92 </p> |
|
93 <p> |
|
94 In the <m:a href="release-v0.16">v0.16</m:a> release, the <code>relpipe-in-jack</code> was introduced. |
|
95 This tools allowed capturing the MIDI events from JACK and sending them to the relational pipeline. |
|
96 Now, in v0.17, we introduce the counterpart tool <code>relpipe-out-jack</code> that reads relational data |
|
97 and sends SysEx messages, MIDI notes and other events to the JACK pipeline. |
|
98 </p> |
|
99 |
|
100 <m:img src="img/jack-connections-2.png"/> |
|
101 |
|
102 <p> |
|
103 We can join the JACK graph with <code>relpipe-out-jack</code> command. |
|
104 It does not produce anything on STDOUT, it sends MIDI events to JACK instead, so no other output data are needed. |
|
105 </p> |
|
106 |
|
107 <p> |
|
108 Besides capturing and playing the MIDI, we can use the <code>relpipe-in-jack</code> and <code>relpipe-out-jack</code> |
|
109 to tak a snapshot of the JACK graph (connections between ports) and recreate it later. |
|
110 Or establish any new connections or break existing ones. |
|
111 Either through CLI commands of <code>relpipe-in-jack</code> or through relational data (list of connection and disconnection commands) |
|
112 sent to <code>relpipe-out-jack</code>. |
|
113 </p> |
|
114 |
|
115 <p> |
|
116 Both commands have also the <code>--required-connections</code> and <code>--connect-to</code> parameters, |
|
117 so they can react on reaching particular connection count or automatically connect their port to another one specified on the CLI. |
|
118 The bash-completion suggests available port names. |
|
119 </p> |
|
120 |
|
121 <p> |
|
122 More details in the examples: |
|
123 <m:a href="examples-jack-ports">Working with JACK ports</m:a> |
|
124 and <m:a href="examples-jack-midi-generating-1">Generating and sending MIDI messages</m:a>. |
|
125 </p> |
|
126 |
|
127 <h2>Feature overview</h2> |
|
128 |
|
129 <h3>Data types</h3> |
|
130 <ul> |
|
131 <li m:since="v0.8">boolean</li> |
|
132 <li m:since="v0.15">variable-length signed integer (SLEB128)</li> |
|
133 <li m:since="v0.8">string in UTF-8</li> |
|
134 </ul> |
|
135 <h3>Inputs</h3> |
|
136 <ul> |
|
137 <li m:since="v0.11">Recfile</li> |
|
138 <li m:since="v0.9">XML</li> |
|
139 <li m:since="v0.13">XMLTable</li> |
|
140 <li m:since="v0.9">CSV</li> |
|
141 <li m:since="v0.9">file system</li> |
|
142 <li m:since="v0.8">CLI</li> |
|
143 <li m:since="v0.8">fstab</li> |
|
144 <li m:since="v0.14">SQL script</li> |
|
145 <li m:since="v0.16">JACK</li> |
|
146 </ul> |
|
147 <h3>Transformations</h3> |
|
148 <ul> |
|
149 <li m:since="v0.13">sql: filtering and transformations using the SQL language</li> |
|
150 <li m:since="v0.12">awk: filtering and transformations using the classic AWK tool and language</li> |
|
151 <li m:since="v0.10">scheme: filtering and transformations defined in the Scheme language using GNU Guile</li> |
|
152 <li m:since="v0.8">grep: regular expression filter, removes unwanted records from the relation</li> |
|
153 <li m:since="v0.8">cut: regular expression attribute cutter (removes or duplicates attributes and can also DROP whole relation)</li> |
|
154 <li m:since="v0.8">sed: regular expression replacer</li> |
|
155 <li m:since="v0.8">validator: just a pass-through filter that crashes on invalid data</li> |
|
156 <li m:since="v0.8">python: highly experimental</li> |
|
157 </ul> |
|
158 <h3>Streamlets</h3> |
|
159 <ul> |
|
160 <li m:since="v0.15">xpath (example, unstable)</li> |
|
161 <li m:since="v0.15">hash (example, unstable)</li> |
|
162 <li m:since="v0.15">jar_info (example, unstable)</li> |
|
163 <li m:since="v0.15">mime_type (example, unstable)</li> |
|
164 <li m:since="v0.15">exiftool (example, unstable)</li> |
|
165 <li m:since="v0.15">pid (example, unstable)</li> |
|
166 <li m:since="v0.15">cloc (example, unstable)</li> |
|
167 <li m:since="v0.15">exiv2 (example, unstable)</li> |
|
168 <li m:since="v0.15">inode (example, unstable)</li> |
|
169 <li m:since="v0.15">lines_count (example, unstable)</li> |
|
170 <li m:since="v0.15">pdftotext (example, unstable)</li> |
|
171 <li m:since="v0.15">pdfinfo (example, unstable)</li> |
|
172 <li m:since="v0.15">tesseract (example, unstable)</li> |
|
173 </ul> |
|
174 <h3>Outputs</h3> |
|
175 <ul> |
|
176 <li m:since="v0.11">ASN.1 BER</li> |
|
177 <li m:since="v0.11">Recfile</li> |
|
178 <li m:since="v0.9">CSV</li> |
|
179 <li m:since="v0.8">tabular</li> |
|
180 <li m:since="v0.8">XML</li> |
|
181 <li m:since="v0.8">nullbyte</li> |
|
182 <li m:since="v0.8">GUI in Qt</li> |
|
183 <li m:since="v0.8">ODS (LibreOffice)</li> |
|
184 <li m:since="v0.17">JACK</li> |
|
185 </ul> |
|
186 |
|
187 <h2>New examples</h2> |
|
188 <ul> |
|
189 <li><m:a href="examples-jack-ports">Working with JACK ports</m:a></li> |
|
190 <li><m:a href="examples-jack-midi-generating-1">Generating and sending MIDI messages</m:a></li> |
|
191 </ul> |
|
192 |
|
193 <h2>Backward incompatible changes</h2> |
|
194 |
|
195 <p> |
|
196 The <code>relpipe-tr-guile</code> has been renamed to <code>relpipe-tr-scheme</code>, so it is necessary to modify existing scripts. |
|
197 However the CLI interface has not been changed, thus it can be simply updated e.g. by: |
|
198 </p> |
|
199 <m:pre jazyk="bash">sed s/relpipe-tr-guile/relpipe-tr-scheme/g -i script.sh</m:pre> |
|
200 <p> |
|
201 The reason behind this rename is that Scheme is a programming language and Guile is one of implementations of this language. |
|
202 And we prefer <a href="https://sane-software.globalcode.info/v_0/#a0f42ec9-5032-4f6d-a50a-4b7bddde77f0">depending on interfaces</a> rather than on particular implementations. |
|
203 Similar situation is <code>relpipe-tr-sql</code>: this module is also named after the language (SQL), not after the default implementation (SQLite) |
|
204 and this module can use <m:a href="examples-tr-sql-odbc">any SQL engine</m:a>. |
|
205 </p> |
|
206 |
|
207 <p> |
|
208 The CLI interface of the <code>relpipe-in-cli</code> tool has been standardized and made consistent with other tools. |
|
209 Now, it uses named parameters instead of cryptic positional ones. |
|
210 The format of the STDIN stream remains unchanged (values separated by <code>\0</code> byte). |
|
211 </p> |
|
212 |
|
213 <m:pre jazyk="bash"><![CDATA[# Old syntax before v0.17: |
|
214 relpipe-in-cli generate "relation_from_cli" 3 \ |
|
215 "a" "integer" \ |
|
216 "b" "string" \ |
|
217 "c" "boolean" \ |
|
218 "1" "Hello" "true" \ |
|
219 "2" "World!" "false" |
|
220 |
|
221 # New syntax: |
|
222 relpipe-in-cli \ |
|
223 --relation "relation_from_cli" \ |
|
224 --attribute "a" integer \ |
|
225 --attribute "b" string \ |
|
226 --attribute "c" boolean \ |
|
227 --record "1" "Hello" "true" \ |
|
228 --record "2" "World!" "false" |
|
229 ]]></m:pre> |
|
230 |
|
231 <p> |
|
232 We can repeat these parameters and generate several relations (that was not possibel in the older version). |
|
233 One relation may be filled with values from STDIN: <code>--records-on-stdin true</code> (replaces the old <code>relpipe-in-cli generate-from-stdin</code> syntax). |
|
234 And the last relation may be filled with values from remaining CLI arguments: <code>--records "1" "Hello" "true" "2" "World!" "false"</code>. |
|
235 The demo (<code>relpipe-in-cli demo</code>) has been removed (sample data can be easily generated by setting the CLI arguments). |
|
236 </p> |
|
237 |
|
238 <p> |
|
239 The CLI interface of the <code>relpipe-in-csv</code> tool has also been standardized and made consistent with other tools. |
|
240 If we have attribute names on the first line of the CSV file (which is quite common), the usage is same as in previous versions: |
|
241 </p> |
|
242 |
|
243 <m:pre jazyk="bash"><![CDATA[cat file.csv | relpipe-in-csv]]></m:pre> |
|
244 |
|
245 <p>If we have only data in the CSV file (no headers) and thus want to specify them on CLI, the syntax is different:</p> |
|
246 |
|
247 <m:pre jazyk="bash"><![CDATA[# Old syntax before v0.17: |
|
248 cat file.csv | relpipe-in-csv "my_relation" "a" integer "b" boolean |
|
249 |
|
250 # New syntax: |
|
251 cat file.csv | relpipe-in-csv \ |
|
252 --relation "my_relation" \ |
|
253 --attribute "a" integer \ |
|
254 --attribute "b" boolean |
|
255 ]]></m:pre> |
|
256 |
|
257 <p>Bash-completion scripts for all tools are provided.</p> |
|
258 |
|
259 <p> |
|
260 Future versions should allow optionally storing the attribute data types in the CSV header and parsing them back (together with the attribute names). |
|
261 </p> |
|
262 |
|
263 <h2>Installation</h2> |
|
264 |
|
265 <p> |
|
266 Instalation was tested on Debian GNU/Linux 10.2. |
|
267 The process should be similar on other distributions. |
|
268 </p> |
|
269 |
|
270 <m:pre src="examples/release-v0.17.1.sh" jazyk="bash" odkaz="ano"/> |
|
271 |
|
272 <p> |
|
273 <m:name/> are modular thus you can download and install only parts you need (the libraries are needed always). |
|
274 Tools <code>out-gui.qt</code> and <code>tr-python</code> require additional libraries and are not built by default. |
|
275 </p> |
|
276 |
|
277 </text> |
|
278 |
|
279 </stránka> |