diff -r d7ae02390fac -r fce3d6290c40 relpipe-data/release-v0.17.xml --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/relpipe-data/release-v0.17.xml Thu Oct 22 01:51:32 2020 +0200 @@ -0,0 +1,275 @@ + + + Release v0.17 + new public release of Relational pipes + v0.17 + + +

+ We are pleased to introduce you the new development version of . + This release improves and standardizes the CLI interface of several tools, brings a new output module + several smaller improvements. +

+ +

+ The relpipe-in-cli module has changed CLI interface: see details below +
+ The relpipe-in-csv module has changed CLI interface: see details below +
+ The relpipe-out-csv module has new CLI interface: see details below +
+ The relpipe-out-tabular module has new CLI interface: see details below +
+ The relpipe-tr-guile was renamed to relpipe-tr-scheme: see details below +
+ New tool relpipe-out-jack for generating MIDI through JACK: see details below +

+ +

+ See the examples and screenshots pages for details. +

+ +

+ Please note that this is still a development release and thus the API (libraries, CLI arguments, formats) might and will change. + Any suggestions, ideas and bug reports are welcome in our mailing list. +

+ +

New CLI interfaces

+ +

+ Several earlier tools have non-standard CLI interface and does not conform with principle „Use --long-options“. + Posistional parameters might be shorter but they are also cryptic and less flexible. + In this release the relpipe-in-cli, relpipe-in-csv tools dropped their old CLI interface + and got a new one based on named parameters and long options. + The relpipe-out-csv and relpipe-out-tabular had no parameters in previous versions and now have standard CLI interface. +

+ +

+ In relpipe-out-tabular we can now optionally omit several parts: + --write-record-count false, + --write-types false, + --write-relation-name false + and thus simplify the output if we do not need given information. +

+ +

+ These options may be set globally or for particular relation using the --relation option with pattern parameter. + Thus we may e.g. print some relations with data types and some without them. +

+ + +

+ Remaining tools that will be transformed in the same way: relpipe-tr-grep, relpipe-tr-cut, relpipe-tr-sed + and will get standard CLI interface. +

+ +

JACK (MIDI) output module

+ +

+ A powerful audio system called JACK allows us to + build pipelines consisting of audio interfaces, players, recorders, filters and effects… + and route sound streams (both PCM and MIDI) through them. + MIDI messages can come from keyboards or other hardware MIDI controllers or from MIDI players and other software. +

+ In the v0.16 release, the relpipe-in-jack was introduced. + This tools allowed capturing the MIDI events from JACK and sending them to the relational pipeline. + Now, in v0.17, we introduce the counterpart tool relpipe-out-jack that reads relational data + and sends SysEx messages, MIDI notes and other events to the JACK pipeline. +

+ + + +

+ We can join the JACK graph with relpipe-out-jack command. + It does not produce anything on STDOUT, it sends MIDI events to JACK instead, so no other output data are needed. +

+ +

+ Besides capturing and playing the MIDI, we can use the relpipe-in-jack and relpipe-out-jack + to tak a snapshot of the JACK graph (connections between ports) and recreate it later. + Or establish any new connections or break existing ones. + Either through CLI commands of relpipe-in-jack or through relational data (list of connection and disconnection commands) + sent to relpipe-out-jack. +

+ +

+ Both commands have also the --required-connections and --connect-to parameters, + so they can react on reaching particular connection count or automatically connect their port to another one specified on the CLI. + The bash-completion suggests available port names. +

+ +

+ More details in the examples: + Working with JACK ports + and Generating and sending MIDI messages. +

+ +

Feature overview

+ +

Data types

boolean
variable-length signed integer (SLEB128)
string in UTF-8

Inputs

Recfile
XML
XMLTable
CSV
file system
CLI
fstab
SQL script
JACK

Transformations

sql: filtering and transformations using the SQL language
awk: filtering and transformations using the classic AWK tool and language
scheme: filtering and transformations defined in the Scheme language using GNU Guile
grep: regular expression filter, removes unwanted records from the relation
cut: regular expression attribute cutter (removes or duplicates attributes and can also DROP whole relation)
sed: regular expression replacer
validator: just a pass-through filter that crashes on invalid data
python: highly experimental

Streamlets

xpath (example, unstable)
hash (example, unstable)
jar_info (example, unstable)
mime_type (example, unstable)
exiftool (example, unstable)
pid (example, unstable)
cloc (example, unstable)
exiv2 (example, unstable)
inode (example, unstable)
lines_count (example, unstable)
pdftotext (example, unstable)
pdfinfo (example, unstable)
tesseract (example, unstable)

Outputs

ASN.1 BER
Recfile
CSV
tabular
XML
nullbyte
GUI in Qt
ODS (LibreOffice)
JACK

+ +

New examples

Working with JACK ports
Generating and sending MIDI messages

+ +

Backward incompatible changes

+ +

+ The relpipe-tr-guile has been renamed to relpipe-tr-scheme, so it is necessary to modify existing scripts. + However the CLI interface has not been changed, thus it can be simply updated e.g. by: +

+ sed s/relpipe-tr-guile/relpipe-tr-scheme/g -i script.sh +

+ The reason behind this rename is that Scheme is a programming language and Guile is one of implementations of this language. + And we prefer depending on interfaces rather than on particular implementations. + Similar situation is relpipe-tr-sql: this module is also named after the language (SQL), not after the default implementation (SQLite) + and this module can use any SQL engine. +

+ +

+ The CLI interface of the relpipe-in-cli tool has been standardized and made consistent with other tools. + Now, it uses named parameters instead of cryptic positional ones. + The format of the STDIN stream remains unchanged (values separated by \0 byte). +

+ + + +

+ We can repeat these parameters and generate several relations (that was not possibel in the older version). + One relation may be filled with values from STDIN: --records-on-stdin true (replaces the old relpipe-in-cli generate-from-stdin syntax). + And the last relation may be filled with values from remaining CLI arguments: --records "1" "Hello" "true" "2" "World!" "false". + The demo (relpipe-in-cli demo) has been removed (sample data can be easily generated by setting the CLI arguments). +

+ +

+ The CLI interface of the relpipe-in-csv tool has also been standardized and made consistent with other tools. + If we have attribute names on the first line of the CSV file (which is quite common), the usage is same as in previous versions: +

+ + + +

If we have only data in the CSV file (no headers) and thus want to specify them on CLI, the syntax is different:

+ + + +

Bash-completion scripts for all tools are provided.

+ +

+ Future versions should allow optionally storing the attribute data types in the CSV header and parsing them back (together with the attribute names). +

+ +

Installation

+ +

+ Instalation was tested on Debian GNU/Linux 10.2. + The process should be similar on other distributions. +

+ + + +

+ are modular thus you can download and install only parts you need (the libraries are needed always). + Tools out-gui.qt and tr-python require additional libraries and are not built by default. +

+ + + + \ No newline at end of file