|
1 <stránka |
|
2 xmlns="https://trac.frantovo.cz/xml-web-generator/wiki/xmlns/strana" |
|
3 xmlns:m="https://trac.frantovo.cz/xml-web-generator/wiki/xmlns/makro"> |
|
4 |
|
5 <nadpis>Release v0.17</nadpis> |
|
6 <perex>new public release of Relational pipes</perex> |
|
7 <m:release>v0.17</m:release> |
|
8 |
|
9 <text xmlns="http://www.w3.org/1999/xhtml"> |
|
10 <p> |
|
11 We are pleased to introduce you the new development version of <m:name/>. |
|
12 This release improves and standardizes the CLI interface of several tools, brings a new output module + several smaller improvements. |
|
13 </p> |
|
14 |
|
15 <ul> |
|
16 <li> |
|
17 <strong>The <code>relpipe-in-cli</code> module has changed CLI interface</strong>: see details below |
|
18 </li> |
|
19 <li> |
|
20 <strong>The <code>relpipe-in-csv</code> module has changed CLI interface</strong>: see details below |
|
21 </li> |
|
22 <li> |
|
23 <strong>The <code>relpipe-out-csv</code> module has new CLI interface</strong>: see details below |
|
24 </li> |
|
25 <li> |
|
26 <strong>The <code>relpipe-out-tabular</code> module has new CLI interface</strong>: see details below |
|
27 </li> |
|
28 <li> |
|
29 <strong>The <code>relpipe-tr-guile</code> was renamed to <code>relpipe-tr-scheme</code></strong>: see details below |
|
30 </li> |
|
31 <li> |
|
32 <strong>New tool <code>relpipe-out-jack</code> for generating MIDI through JACK</strong>: see details below |
|
33 </li> |
|
34 </ul> |
|
35 |
|
36 <p> |
|
37 See the <m:a href="examples">examples</m:a> and <m:a href="screenshots">screenshots</m:a> pages for details. |
|
38 </p> |
|
39 |
|
40 <p> |
|
41 Please note that this is still a development release and thus the API (libraries, CLI arguments, formats) might and will change. |
|
42 Any suggestions, ideas and bug reports are welcome in our <m:a href="contact">mailing list</m:a>. |
|
43 </p> |
|
44 |
|
45 <h2>New CLI interfaces</h2> |
|
46 |
|
47 <p> |
|
48 Several earlier tools have non-standard CLI interface and does not conform with <m:a href="principles">principle</m:a> „Use --long-options“. |
|
49 Posistional parameters might be shorter but they are also cryptic and less flexible. |
|
50 In this release the <code>relpipe-in-cli</code>, <code>relpipe-in-csv</code> tools dropped their old CLI interface |
|
51 and got a new one based on named parameters and long options. |
|
52 The <code>relpipe-out-csv</code> and <code>relpipe-out-tabular</code> had no parameters in previous versions and now have standard CLI interface. |
|
53 </p> |
|
54 |
|
55 <p> |
|
56 In <code>relpipe-out-tabular</code> we can now optionally omit several parts: |
|
57 <code>--write-record-count false</code>, |
|
58 <code>--write-types false</code>, |
|
59 <code>--write-relation-name false</code> |
|
60 and thus simplify the output if we do not need given information. |
|
61 </p> |
|
62 |
|
63 <pre><![CDATA[ ╭───────┬────────╮ |
|
64 │ first │ second │ |
|
65 ├───────┼────────┤ |
|
66 │ 1 │ abc │ |
|
67 │ 2 │ def │ |
|
68 ╰───────┴────────╯]]></pre> |
|
69 |
|
70 <p> |
|
71 These options may be set globally or for particular relation using the <code>--relation</code> option with pattern parameter. |
|
72 Thus we may e.g. print some relations with data types and some without them. |
|
73 </p> |
|
74 |
|
75 |
|
76 <p> |
|
77 Remaining tools that will be transformed in the same way: <code>relpipe-tr-grep</code>, <code>relpipe-tr-cut</code>, <code>relpipe-tr-sed</code> |
|
78 and will get standard CLI interface. |
|
79 </p> |
|
80 |
|
81 <h2>JACK (MIDI) output module</h2> |
|
82 |
|
83 <p> |
|
84 A powerful audio system called <a href="https://jackaudio.org/">JACK</a> allows us to |
|
85 build pipelines consisting of audio interfaces, players, recorders, filters and effects… |
|
86 and route sound streams (both PCM and MIDI) through them. |
|
87 MIDI messages can come from keyboards or other hardware MIDI controllers or from MIDI players and other software. |
|
88 </p> |
|
89 <p> |
|
90 In the <m:a href="release-v0.16">v0.16</m:a> release, the <code>relpipe-in-jack</code> was introduced. |
|
91 This tools allowed capturing the MIDI events from JACK and sending them to the relational pipeline. |
|
92 Now, in v0.17, we introduce the counterpart tool <code>relpipe-out-jack</code> that reads relational data |
|
93 and sends SysEx messages, MIDI notes and other events to the JACK pipeline. |
|
94 </p> |
|
95 |
|
96 <m:img src="img/jack-connections-2.png"/> |
|
97 |
|
98 <p> |
|
99 We can join the JACK graph with <code>relpipe-out-jack</code> command. |
|
100 It does not produce anything on STDOUT, it sends MIDI events to JACK instead, so no other output data are needed. |
|
101 </p> |
|
102 |
|
103 <p> |
|
104 Besides capturing and playing the MIDI, we can use the <code>relpipe-in-jack</code> and <code>relpipe-out-jack</code> |
|
105 to tak a snapshot of the JACK graph (connections between ports) and recreate it later. |
|
106 Or establish any new connections or break existing ones. |
|
107 Either through CLI commands of <code>relpipe-in-jack</code> or through relational data (list of connection and disconnection commands) |
|
108 sent to <code>relpipe-out-jack</code>. |
|
109 </p> |
|
110 |
|
111 <p> |
|
112 Both commands have also the <code>--required-connections</code> and <code>--connect-to</code> parameters, |
|
113 so they can react on reaching particular connection count or automatically connect their port to another one specified on the CLI. |
|
114 The bash-completion suggests available port names. |
|
115 </p> |
|
116 |
|
117 <p> |
|
118 More details in the examples: |
|
119 <m:a href="examples-jack-ports">Working with JACK ports</m:a> |
|
120 and <m:a href="examples-jack-midi-generating-1">Generating and sending MIDI messages</m:a>. |
|
121 </p> |
|
122 |
|
123 <h2>Feature overview</h2> |
|
124 |
|
125 <h3>Data types</h3> |
|
126 <ul> |
|
127 <li m:since="v0.8">boolean</li> |
|
128 <li m:since="v0.15">variable-length signed integer (SLEB128)</li> |
|
129 <li m:since="v0.8">string in UTF-8</li> |
|
130 </ul> |
|
131 <h3>Inputs</h3> |
|
132 <ul> |
|
133 <li m:since="v0.11">Recfile</li> |
|
134 <li m:since="v0.9">XML</li> |
|
135 <li m:since="v0.13">XMLTable</li> |
|
136 <li m:since="v0.9">CSV</li> |
|
137 <li m:since="v0.9">file system</li> |
|
138 <li m:since="v0.8">CLI</li> |
|
139 <li m:since="v0.8">fstab</li> |
|
140 <li m:since="v0.14">SQL script</li> |
|
141 <li m:since="v0.16">JACK</li> |
|
142 </ul> |
|
143 <h3>Transformations</h3> |
|
144 <ul> |
|
145 <li m:since="v0.13">sql: filtering and transformations using the SQL language</li> |
|
146 <li m:since="v0.12">awk: filtering and transformations using the classic AWK tool and language</li> |
|
147 <li m:since="v0.10">scheme: filtering and transformations defined in the Scheme language using GNU Guile</li> |
|
148 <li m:since="v0.8">grep: regular expression filter, removes unwanted records from the relation</li> |
|
149 <li m:since="v0.8">cut: regular expression attribute cutter (removes or duplicates attributes and can also DROP whole relation)</li> |
|
150 <li m:since="v0.8">sed: regular expression replacer</li> |
|
151 <li m:since="v0.8">validator: just a pass-through filter that crashes on invalid data</li> |
|
152 <li m:since="v0.8">python: highly experimental</li> |
|
153 </ul> |
|
154 <h3>Streamlets</h3> |
|
155 <ul> |
|
156 <li m:since="v0.15">xpath (example, unstable)</li> |
|
157 <li m:since="v0.15">hash (example, unstable)</li> |
|
158 <li m:since="v0.15">jar_info (example, unstable)</li> |
|
159 <li m:since="v0.15">mime_type (example, unstable)</li> |
|
160 <li m:since="v0.15">exiftool (example, unstable)</li> |
|
161 <li m:since="v0.15">pid (example, unstable)</li> |
|
162 <li m:since="v0.15">cloc (example, unstable)</li> |
|
163 <li m:since="v0.15">exiv2 (example, unstable)</li> |
|
164 <li m:since="v0.15">inode (example, unstable)</li> |
|
165 <li m:since="v0.15">lines_count (example, unstable)</li> |
|
166 <li m:since="v0.15">pdftotext (example, unstable)</li> |
|
167 <li m:since="v0.15">pdfinfo (example, unstable)</li> |
|
168 <li m:since="v0.15">tesseract (example, unstable)</li> |
|
169 </ul> |
|
170 <h3>Outputs</h3> |
|
171 <ul> |
|
172 <li m:since="v0.11">ASN.1 BER</li> |
|
173 <li m:since="v0.11">Recfile</li> |
|
174 <li m:since="v0.9">CSV</li> |
|
175 <li m:since="v0.8">tabular</li> |
|
176 <li m:since="v0.8">XML</li> |
|
177 <li m:since="v0.8">nullbyte</li> |
|
178 <li m:since="v0.8">GUI in Qt</li> |
|
179 <li m:since="v0.8">ODS (LibreOffice)</li> |
|
180 <li m:since="v0.17">JACK</li> |
|
181 </ul> |
|
182 |
|
183 <h2>New examples</h2> |
|
184 <ul> |
|
185 <li><m:a href="examples-jack-ports">Working with JACK ports</m:a></li> |
|
186 <li><m:a href="examples-jack-midi-generating-1">Generating and sending MIDI messages</m:a></li> |
|
187 </ul> |
|
188 |
|
189 <h2>Backward incompatible changes</h2> |
|
190 |
|
191 <p> |
|
192 The <code>relpipe-tr-guile</code> has been renamed to <code>relpipe-tr-scheme</code>, so it is necessary to modify existing scripts. |
|
193 However the CLI interface has not been changed, thus it can be simply updated e.g. by: |
|
194 </p> |
|
195 <m:pre jazyk="bash">sed s/relpipe-tr-guile/relpipe-tr-scheme/g -i script.sh</m:pre> |
|
196 <p> |
|
197 The reason behind this rename is that Scheme is a programming language and Guile is one of implementations of this language. |
|
198 And we prefer <a href="https://sane-software.globalcode.info/v_0/#a0f42ec9-5032-4f6d-a50a-4b7bddde77f0">depending on interfaces</a> rather than on particular implementations. |
|
199 Similar situation is <code>relpipe-tr-sql</code>: this module is also named after the language (SQL), not after the default implementation (SQLite) |
|
200 and this module can use <m:a href="examples-tr-sql-odbc">any SQL engine</m:a>. |
|
201 </p> |
|
202 |
|
203 <p> |
|
204 The CLI interface of the <code>relpipe-in-cli</code> tool has been standardized and made consistent with other tools. |
|
205 Now, it uses named parameters instead of cryptic positional ones. |
|
206 The format of the STDIN stream remains unchanged (values separated by <code>\0</code> byte). |
|
207 </p> |
|
208 |
|
209 <m:pre jazyk="bash"><![CDATA[# Old syntax before v0.17: |
|
210 relpipe-in-cli generate "relation_from_cli" 3 \ |
|
211 "a" "integer" \ |
|
212 "b" "string" \ |
|
213 "c" "boolean" \ |
|
214 "1" "Hello" "true" \ |
|
215 "2" "World!" "false" |
|
216 |
|
217 # New syntax: |
|
218 relpipe-in-cli \ |
|
219 --relation "relation_from_cli" \ |
|
220 --attribute "a" integer \ |
|
221 --attribute "b" string \ |
|
222 --attribute "c" boolean \ |
|
223 --record "1" "Hello" "true" \ |
|
224 --record "2" "World!" "false" |
|
225 ]]></m:pre> |
|
226 |
|
227 <p> |
|
228 We can repeat these parameters and generate several relations (that was not possibel in the older version). |
|
229 One relation may be filled with values from STDIN: <code>--records-on-stdin true</code> (replaces the old <code>relpipe-in-cli generate-from-stdin</code> syntax). |
|
230 And the last relation may be filled with values from remaining CLI arguments: <code>--records "1" "Hello" "true" "2" "World!" "false"</code>. |
|
231 The demo (<code>relpipe-in-cli demo</code>) has been removed (sample data can be easily generated by setting the CLI arguments). |
|
232 </p> |
|
233 |
|
234 <p> |
|
235 The CLI interface of the <code>relpipe-in-csv</code> tool has also been standardized and made consistent with other tools. |
|
236 If we have attribute names on the first line of the CSV file (which is quite common), the usage is same as in previous versions: |
|
237 </p> |
|
238 |
|
239 <m:pre jazyk="bash"><![CDATA[cat file.csv | relpipe-in-csv]]></m:pre> |
|
240 |
|
241 <p>If we have only data in the CSV file (no headers) and thus want to specify them on CLI, the syntax is different:</p> |
|
242 |
|
243 <m:pre jazyk="bash"><![CDATA[# Old syntax before v0.17: |
|
244 cat file.csv | relpipe-in-csv "my_relation" "a" integer "b" boolean |
|
245 |
|
246 # New syntax: |
|
247 cat file.csv | relpipe-in-csv \ |
|
248 --relation "my_relation" \ |
|
249 --attribute "a" integer \ |
|
250 --attribute "b" boolean |
|
251 ]]></m:pre> |
|
252 |
|
253 <p>Bash-completion scripts for all tools are provided.</p> |
|
254 |
|
255 <p> |
|
256 Future versions should allow optionally storing the attribute data types in the CSV header and parsing them back (together with the attribute names). |
|
257 </p> |
|
258 |
|
259 <h2>Installation</h2> |
|
260 |
|
261 <p> |
|
262 Instalation was tested on Debian GNU/Linux 10.2. |
|
263 The process should be similar on other distributions. |
|
264 </p> |
|
265 |
|
266 <m:pre src="examples/release-v0.17.sh" jazyk="bash" odkaz="ano"/> |
|
267 |
|
268 <p> |
|
269 <m:name/> are modular thus you can download and install only parts you need (the libraries are needed always). |
|
270 Tools <code>out-gui.qt</code> and <code>tr-python</code> require additional libraries and are not built by default. |
|
271 </p> |
|
272 |
|
273 </text> |
|
274 |
|
275 </stránka> |