--- /dev/null Thu Jan 01 00:00:00 1970 +0000
+++ b/relpipe-data/release-v0.17.xml Thu Oct 22 01:51:32 2020 +0200
@@ -0,0 +1,275 @@
+<stránka
+ xmlns="https://trac.frantovo.cz/xml-web-generator/wiki/xmlns/strana"
+ xmlns:m="https://trac.frantovo.cz/xml-web-generator/wiki/xmlns/makro">
+
+ <nadpis>Release v0.17</nadpis>
+ <perex>new public release of Relational pipes</perex>
+ <m:release>v0.17</m:release>
+
+ <text xmlns="http://www.w3.org/1999/xhtml">
+ <p>
+ We are pleased to introduce you the new development version of <m:name/>.
+ This release improves and standardizes the CLI interface of several tools, brings a new output module + several smaller improvements.
+ </p>
+
+ <ul>
+ <li>
+ <strong>The <code>relpipe-in-cli</code> module has changed CLI interface</strong>: see details below
+ </li>
+ <li>
+ <strong>The <code>relpipe-in-csv</code> module has changed CLI interface</strong>: see details below
+ </li>
+ <li>
+ <strong>The <code>relpipe-out-csv</code> module has new CLI interface</strong>: see details below
+ </li>
+ <li>
+ <strong>The <code>relpipe-out-tabular</code> module has new CLI interface</strong>: see details below
+ </li>
+ <li>
+ <strong>The <code>relpipe-tr-guile</code> was renamed to <code>relpipe-tr-scheme</code></strong>: see details below
+ </li>
+ <li>
+ <strong>New tool <code>relpipe-out-jack</code> for generating MIDI through JACK</strong>: see details below
+ </li>
+ </ul>
+
+ <p>
+ See the <m:a href="examples">examples</m:a> and <m:a href="screenshots">screenshots</m:a> pages for details.
+ </p>
+
+ <p>
+ Please note that this is still a development release and thus the API (libraries, CLI arguments, formats) might and will change.
+ Any suggestions, ideas and bug reports are welcome in our <m:a href="contact">mailing list</m:a>.
+ </p>
+
+ <h2>New CLI interfaces</h2>
+
+ <p>
+ Several earlier tools have non-standard CLI interface and does not conform with <m:a href="principles">principle</m:a> „Use --long-options“.
+ Posistional parameters might be shorter but they are also cryptic and less flexible.
+ In this release the <code>relpipe-in-cli</code>, <code>relpipe-in-csv</code> tools dropped their old CLI interface
+ and got a new one based on named parameters and long options.
+ The <code>relpipe-out-csv</code> and <code>relpipe-out-tabular</code> had no parameters in previous versions and now have standard CLI interface.
+ </p>
+
+ <p>
+ In <code>relpipe-out-tabular</code> we can now optionally omit several parts:
+ <code>--write-record-count false</code>,
+ <code>--write-types false</code>,
+ <code>--write-relation-name false</code>
+ and thus simplify the output if we do not need given information.
+ </p>
+
+ <pre><![CDATA[ ╭───────┬────────╮
+ │ first │ second │
+ ├───────┼────────┤
+ │ 1 │ abc │
+ │ 2 │ def │
+ ╰───────┴────────╯]]></pre>
+
+ <p>
+ These options may be set globally or for particular relation using the <code>--relation</code> option with pattern parameter.
+ Thus we may e.g. print some relations with data types and some without them.
+ </p>
+
+
+ <p>
+ Remaining tools that will be transformed in the same way: <code>relpipe-tr-grep</code>, <code>relpipe-tr-cut</code>, <code>relpipe-tr-sed</code>
+ and will get standard CLI interface.
+ </p>
+
+ <h2>JACK (MIDI) output module</h2>
+
+ <p>
+ A powerful audio system called <a href="https://jackaudio.org/">JACK</a> allows us to
+ build pipelines consisting of audio interfaces, players, recorders, filters and effects…
+ and route sound streams (both PCM and MIDI) through them.
+ MIDI messages can come from keyboards or other hardware MIDI controllers or from MIDI players and other software.
+ </p>
+ <p>
+ In the <m:a href="release-v0.16">v0.16</m:a> release, the <code>relpipe-in-jack</code> was introduced.
+ This tools allowed capturing the MIDI events from JACK and sending them to the relational pipeline.
+ Now, in v0.17, we introduce the counterpart tool <code>relpipe-out-jack</code> that reads relational data
+ and sends SysEx messages, MIDI notes and other events to the JACK pipeline.
+ </p>
+
+ <m:img src="img/jack-connections-2.png"/>
+
+ <p>
+ We can join the JACK graph with <code>relpipe-out-jack</code> command.
+ It does not produce anything on STDOUT, it sends MIDI events to JACK instead, so no other output data are needed.
+ </p>
+
+ <p>
+ Besides capturing and playing the MIDI, we can use the <code>relpipe-in-jack</code> and <code>relpipe-out-jack</code>
+ to tak a snapshot of the JACK graph (connections between ports) and recreate it later.
+ Or establish any new connections or break existing ones.
+ Either through CLI commands of <code>relpipe-in-jack</code> or through relational data (list of connection and disconnection commands)
+ sent to <code>relpipe-out-jack</code>.
+ </p>
+
+ <p>
+ Both commands have also the <code>--required-connections</code> and <code>--connect-to</code> parameters,
+ so they can react on reaching particular connection count or automatically connect their port to another one specified on the CLI.
+ The bash-completion suggests available port names.
+ </p>
+
+ <p>
+ More details in the examples:
+ <m:a href="examples-jack-ports">Working with JACK ports</m:a>
+ and <m:a href="examples-jack-midi-generating-1">Generating and sending MIDI messages</m:a>.
+ </p>
+
+ <h2>Feature overview</h2>
+
+ <h3>Data types</h3>
+ <ul>
+ <li m:since="v0.8">boolean</li>
+ <li m:since="v0.15">variable-length signed integer (SLEB128)</li>
+ <li m:since="v0.8">string in UTF-8</li>
+ </ul>
+ <h3>Inputs</h3>
+ <ul>
+ <li m:since="v0.11">Recfile</li>
+ <li m:since="v0.9">XML</li>
+ <li m:since="v0.13">XMLTable</li>
+ <li m:since="v0.9">CSV</li>
+ <li m:since="v0.9">file system</li>
+ <li m:since="v0.8">CLI</li>
+ <li m:since="v0.8">fstab</li>
+ <li m:since="v0.14">SQL script</li>
+ <li m:since="v0.16">JACK</li>
+ </ul>
+ <h3>Transformations</h3>
+ <ul>
+ <li m:since="v0.13">sql: filtering and transformations using the SQL language</li>
+ <li m:since="v0.12">awk: filtering and transformations using the classic AWK tool and language</li>
+ <li m:since="v0.10">scheme: filtering and transformations defined in the Scheme language using GNU Guile</li>
+ <li m:since="v0.8">grep: regular expression filter, removes unwanted records from the relation</li>
+ <li m:since="v0.8">cut: regular expression attribute cutter (removes or duplicates attributes and can also DROP whole relation)</li>
+ <li m:since="v0.8">sed: regular expression replacer</li>
+ <li m:since="v0.8">validator: just a pass-through filter that crashes on invalid data</li>
+ <li m:since="v0.8">python: highly experimental</li>
+ </ul>
+ <h3>Streamlets</h3>
+ <ul>
+ <li m:since="v0.15">xpath (example, unstable)</li>
+ <li m:since="v0.15">hash (example, unstable)</li>
+ <li m:since="v0.15">jar_info (example, unstable)</li>
+ <li m:since="v0.15">mime_type (example, unstable)</li>
+ <li m:since="v0.15">exiftool (example, unstable)</li>
+ <li m:since="v0.15">pid (example, unstable)</li>
+ <li m:since="v0.15">cloc (example, unstable)</li>
+ <li m:since="v0.15">exiv2 (example, unstable)</li>
+ <li m:since="v0.15">inode (example, unstable)</li>
+ <li m:since="v0.15">lines_count (example, unstable)</li>
+ <li m:since="v0.15">pdftotext (example, unstable)</li>
+ <li m:since="v0.15">pdfinfo (example, unstable)</li>
+ <li m:since="v0.15">tesseract (example, unstable)</li>
+ </ul>
+ <h3>Outputs</h3>
+ <ul>
+ <li m:since="v0.11">ASN.1 BER</li>
+ <li m:since="v0.11">Recfile</li>
+ <li m:since="v0.9">CSV</li>
+ <li m:since="v0.8">tabular</li>
+ <li m:since="v0.8">XML</li>
+ <li m:since="v0.8">nullbyte</li>
+ <li m:since="v0.8">GUI in Qt</li>
+ <li m:since="v0.8">ODS (LibreOffice)</li>
+ <li m:since="v0.17">JACK</li>
+ </ul>
+
+ <h2>New examples</h2>
+ <ul>
+ <li><m:a href="examples-jack-ports">Working with JACK ports</m:a></li>
+ <li><m:a href="examples-jack-midi-generating-1">Generating and sending MIDI messages</m:a></li>
+ </ul>
+
+ <h2>Backward incompatible changes</h2>
+
+ <p>
+ The <code>relpipe-tr-guile</code> has been renamed to <code>relpipe-tr-scheme</code>, so it is necessary to modify existing scripts.
+ However the CLI interface has not been changed, thus it can be simply updated e.g. by:
+ </p>
+ <m:pre jazyk="bash">sed s/relpipe-tr-guile/relpipe-tr-scheme/g -i script.sh</m:pre>
+ <p>
+ The reason behind this rename is that Scheme is a programming language and Guile is one of implementations of this language.
+ And we prefer <a href="https://sane-software.globalcode.info/v_0/#a0f42ec9-5032-4f6d-a50a-4b7bddde77f0">depending on interfaces</a> rather than on particular implementations.
+ Similar situation is <code>relpipe-tr-sql</code>: this module is also named after the language (SQL), not after the default implementation (SQLite)
+ and this module can use <m:a href="examples-tr-sql-odbc">any SQL engine</m:a>.
+ </p>
+
+ <p>
+ The CLI interface of the <code>relpipe-in-cli</code> tool has been standardized and made consistent with other tools.
+ Now, it uses named parameters instead of cryptic positional ones.
+ The format of the STDIN stream remains unchanged (values separated by <code>\0</code> byte).
+ </p>
+
+ <m:pre jazyk="bash"><![CDATA[# Old syntax before v0.17:
+relpipe-in-cli generate "relation_from_cli" 3 \
+ "a" "integer" \
+ "b" "string" \
+ "c" "boolean" \
+ "1" "Hello" "true" \
+ "2" "World!" "false"
+
+# New syntax:
+relpipe-in-cli \
+ --relation "relation_from_cli" \
+ --attribute "a" integer \
+ --attribute "b" string \
+ --attribute "c" boolean \
+ --record "1" "Hello" "true" \
+ --record "2" "World!" "false"
+]]></m:pre>
+
+ <p>
+ We can repeat these parameters and generate several relations (that was not possibel in the older version).
+ One relation may be filled with values from STDIN: <code>--records-on-stdin true</code> (replaces the old <code>relpipe-in-cli generate-from-stdin</code> syntax).
+ And the last relation may be filled with values from remaining CLI arguments: <code>--records "1" "Hello" "true" "2" "World!" "false"</code>.
+ The demo (<code>relpipe-in-cli demo</code>) has been removed (sample data can be easily generated by setting the CLI arguments).
+ </p>
+
+ <p>
+ The CLI interface of the <code>relpipe-in-csv</code> tool has also been standardized and made consistent with other tools.
+ If we have attribute names on the first line of the CSV file (which is quite common), the usage is same as in previous versions:
+ </p>
+
+ <m:pre jazyk="bash"><![CDATA[cat file.csv | relpipe-in-csv]]></m:pre>
+
+ <p>If we have only data in the CSV file (no headers) and thus want to specify them on CLI, the syntax is different:</p>
+
+ <m:pre jazyk="bash"><![CDATA[# Old syntax before v0.17:
+cat file.csv | relpipe-in-csv "my_relation" "a" integer "b" boolean
+
+# New syntax:
+cat file.csv | relpipe-in-csv \
+ --relation "my_relation" \
+ --attribute "a" integer \
+ --attribute "b" boolean
+]]></m:pre>
+
+ <p>Bash-completion scripts for all tools are provided.</p>
+
+ <p>
+ Future versions should allow optionally storing the attribute data types in the CSV header and parsing them back (together with the attribute names).
+ </p>
+
+ <h2>Installation</h2>
+
+ <p>
+ Instalation was tested on Debian GNU/Linux 10.2.
+ The process should be similar on other distributions.
+ </p>
+
+ <m:pre src="examples/release-v0.17.sh" jazyk="bash" odkaz="ano"/>
+
+ <p>
+ <m:name/> are modular thus you can download and install only parts you need (the libraries are needed always).
+ Tools <code>out-gui.qt</code> and <code>tr-python</code> require additional libraries and are not built by default.
+ </p>
+
+ </text>
+
+</stránka>
\ No newline at end of file