# HG changeset patch
# User František Kučera
+ Build instructions are in release notes below.
+ Released versions
+
This command generates relational data.
In order to see them, we need to convert them to some other format.
- For now, we will use the "tabular" format and pipe relational data to the relpipe-out-tabular
.
+ For now, we will use the „tabular“ format and pipe relational data to the relpipe-out-tabular
.
Output:
@@ -46,8 +48,8 @@ ]]>- The syntax is simple as we see above. We specify the name of the relation, number of attributes, - and then their definitions (names and types), + The syntax is simple as we see above. We specify the name of the relation + and the names and types of attributes followed by the data.
@@ -55,8 +57,8 @@ A single stream may contain multiple relations: -@@ -77,6 +79,38 @@ │ world │ ╰────────────╯ Record count: 1]]> + +
+ In the example above, we call relpipe-in-cli
twice and let the shell combine their outputs.
+ This approach is useful when we want to combine relational data from various sources: different relpipe-in-*
tools, files etc.
+ But when we work with relpipe-in-cli
only, we can ask it to create several relations during one run:
+
+ The result will be the same.
+ We can also use the --records
option instead of the --record
option – then the rest of the CLI arguments is treated as data of given relation.
+ Obviously this option can be used only once for the last relation.
+
+ It will generate the same relation_from_cli
table as above.
+ If we have more data (especially from external sources), we can use the --records-on-stdin
option and pass them through the
+ A powerful audio system called JACK allows us to + build pipelines consisting of audio interfaces, players, recorders, filters and effects… + and route sound streams (both PCM and MIDI) through them. + MIDI messages can come from keyboards or other hardware MIDI controllers or from MIDI players and other software. +
+ +
+ While relpipe-in-jack
can be used for relpipe-out-jack
(since
+ The relpipe-out-jack
reads relational data in the same format as relpipe-in-jack
.
+ So we can capture some MIDI data and play it later.
+ But this is bit boring and any music sequencer would do a better job.
+ We can have more fun with programmatically generating the „music“ (or rather just sounds in beginnings).
+ Thanks to the nature of
+ Before doing anything audible, we will start with saying hello to our sound module.
+ We need a relation with two attributes. The first one is event
and says what should be done.
+ The second one raw
contains the raw MIDI data. It is a SysEx message specific to Roland MT-32,
+ but it can be any MIDI event:
+
+ If we omit the --connect-to j2a_bridge:playback
part, the relpipe-out-jack
command
+ will wait util we connect it somewhere e.g. using QjackCtl.
+ We can control this behavior by explicitly specifying --required-connections N
(default N
is 1)
+ and say e.g. „Wait until you have 3 connections and then start sending“.
+ But because we specified the destination port, the relpipe-out-jack
command immediately
+ connects to it and sends data there.
+
+ The j2a_bridge:playback
port was created by the j2amidi_bridge
+ that bridges JACK-MIDI and ALSA-MIDI.
+ In QjackCtl or other tool, we can connect the ALSA end of this bridge to a physical interface (sound module, synthesizer).
+ If we have no audio hardware, we omit this part and stay in the JACK world – we can connect our MIDI ports to various
+ applications like Yoshimi, QMidiArp, Qtractor etc.
+
+ See more details on MT-32 and MIDI SysEx in:
+ Instruments (like particular pianos, guitars, drums, special effects etc.) can be set on our synthesizer for particular channels.
+ However for reproducible results and more comfort, it is better to configure the instruments through MIDI commands (Patch change).
+ Currently we misuse the sysex
event for this and send this configuration as raw MIDI commands
+ (the c0 0e
and c1 58
below).
+ In later versions of relpipe-out-jack
, there might be better support for instrument configuration.
+
+ The list of standard instruments is available at MIDI.org: General MIDI: GM 1 Sound Set. +
+ +
+ Specifying the notes as raw MIDI messages is possible but uncomfortable.
+ So we will put note
in the event
attribute
+ and specify also time
, channel
, note_on
, note_pitch
and note_velocity
attributes.
+ The relpipe-out-jack
will compose the raw MIDI event from these values.
+
+ We are not going to create any tables or insert any records. + Everything in this example is done in-memory by a single SELECT statement executed on an empty database. + But of course, we can persist our creations in database tables and later play the music by SELECTing from them. +
+ +
+ The SQL engine (SQLite by default) called from relpipe-in-sql
converts our script into a relation.
+ Then the relpipe-out-jack
translates this relation to actual MIDI events and sends them to JACK system.
+
+ We connect it to two ports, so we can hear the sounds and at the same time, we can monitor the MIDI events
+ (through relpipe-in-jack
and usually relpipe-out-csv
).
+ We can also use relpipe-out-tabular
instead of relpipe-out-jack
and check generated data without sending them to JACK.
+ For visual check, we can connect to the JACK port of a MIDI program like Qtractor and record there our creation (like recording from a MIDI keyboard).
+ We can also do some real-time post-processing e.g. using QMidiArp (arpeggiator).
+
+ If we have some hardware, it should look and sound like this: +
+ + + ++ In this video, the Roland MT-32 and SC-88 Pro are daisy-chained and most sounds come from the SC-88. +
+ ++ n.b. the MIDI channel 1 (index 0) is usually silent on MT-32 (default configuration). +
+ + + +
+ The relpipe-out-jack
supports following event types:
+
note
: press or release (depending on note_on
) of particular key on keyboard
+ control
: change value of a controller (knob, slider, switch etc.)
+ sysex
: SystemExclusive command specific for particular device or manufacturer (e.g. show some text on the display)
+ connect
: link two JACK ports using a virtual cable
+ disconnect
: put that cable away
+ + Because the events of various types are usually interleaved, we pass them as records of one relation. + Each type uses a different set of attributes: +
+ +event |
+ note |
+ control |
+ sysex |
+ connect |
+ disconnect |
+
time |
+ ✔ | +✔ | +✔ | ++ | + |
channel |
+ ✔ | +✔ | +✔ | ++ | + |
note_on |
+ ✔ | ++ | + | + | + |
note_pitch |
+ ✔ | ++ | + | + | + |
note_velocity |
+ ✔ | ++ | + | + | + |
controller_id |
+ + | ✔ | ++ | + | + |
controller_value |
+ + | ✔ | ++ | + | + |
raw |
+ + | + | ✔ | ++ | + |
source_port |
+ + | + | + | ✔ | +✔ | +
destination_port |
+ + | + | + | ✔ | +✔ | +
+ Note: this design pattern is one of possible ways how to implement the inheritance (an object-oriented concept) in the relational world. + The relation contains one attribute that determines the type (or class) + and then union of all attributes of all that types. + Each type uses just its relevant attributes and the rest is empty. +
+ ++ The meaning of the attributes is following: +
+ +attribute | +type | +description | +
+ event
+ |
+
+ string
+ |
+ type of the record | +
+ time
+ |
+
+ integer
+ |
+ time in microseconds since start of the playback | +
+ channel
+ |
+
+ integer
+ |
+ number of the MIDI channel; starts from 0 | +
+ note_on
+ |
+
+ boolean
+ |
+ whether the key was pressed (true ) or released (false ) |
+
+ note_pitch
+ |
+
+ integer
+ |
+ pitch or tone; starts from 0; e.g. C4 is 60 | +
+ note_velocity
+ |
+
+ integer
+ |
+ force with which a note is played; 0-127 | +
+ controller_id
+ |
+
+ integer
+ |
+ number of the controller; starts from 0 | +
+ controller_value
+ |
+
+ integer
+ |
+ value of the controller; 0-127 | +
+ raw
+ |
+
+ string
+ |
+ raw MIDI bytes written in hexadecimal format; e.g. 91 3c 41 |
+
+ source_port
+ |
+
+ string
+ |
+ name of the JACK output port; e.g. system:capture_1 |
+
+ destination_port
+ |
+
+ string
+ |
+ name of the JACK input port; e.g. system:playback_1 |
+
+ Besides the event
we need to specify only attributes we use.
+ So if we e.g. only send SysEx messages, we need only event
and raw
attributes.
+ If the time
is missing, the event is processed as soon as possible (in the next real-time cycle).
+ The time
attribute is used for precise timing
+ – so the process is not driven by the time when the event arrives on STDIN of the relpipe-out-jack
.
+
+ A powerful audio system called JACK allows us to + build pipelines consisting of audio interfaces, players, recorders, filters and effects… + and route sound streams (both PCM and MIDI) through them. + MIDI messages can come from keyboards or other hardware MIDI controllers or from MIDI players and other software. +
+ ++ In JACK, we have clients (applications) and they have ports (input and output channels). + The JACK graph consists of connections between input and output ports. + There might be multiple connections from or to a single port – singals are then replicated or combined. + Obviously, we can connect MIDI ports only to other MIDI ports and PCM/audio ports only to other ports of the same kind. + However there can be applications with both kinds of ports that e.g. translate MIDI signal to audio signal (synthesizers). +
+ ++ The JACK daemon is responsible for maintaining the graph and transmitting the messages. + Clients connect to the daemon. +
+ +
+ Since relpipe-in-jack
command:
+
and see it as a relation:
+ + + +
+ Port names are prefixed with the names of clients.
+ We see the port direction in the input
and output
attributes
+ and can distinguish MIDI and audio ports by the value of the midi
attribute.
+
+ We can also list the connections using the --list-connections true
option:
+
+ Instead of formatting as a table, we can direct this relation into a file and later recreate the saved connections: +
+ +
+ If we want to store the connections in a human readable format in order to be able to edit them,
+ we should add e.g. relpipe-out-csv
(or XML, or Recfile) stage before redirecting the stream to the file
+ and add relpipe-in-csv
etc. stage to the second pipeline.
+
+ Or we can write the file by hand and just send it to the JACK output module: +
+ +
+ The primary purpose of the relpipe-out-jack
is sending MIDI messages.
+ And because it does not make much sense to send them nowhere, it waits for at least one connection.
+ But if we are not sending any messages and are using this tool just to connect or disconnect the ports,
+ we should set --required-connections 0
.
+
+ We can also connect or disconnect particular ports specified on the command line: +
+ +
+ The primary purpose of the relpipe-in-jack
command is monitoring the MIDI commands.
+ If we want just to connect or disconnect some ports, we can disable monitoring by setting --list-midi-messages false
+ and the command will return immediately.
+
+ The port names are suggested by the Bash-completion, so we do not have to memorize them. + Because these names contain the colon, we should use quotes (otherwise the Bash-completion may be confused). +
+ +
+ Disconnecting works the same way – just use the --disconnect-ports
option
+ or specify the disconnect
command in the event
attribute.
+
+ Note: there are sophisticated GUI tools focused on working with the JACK graph + like QjackCtl or KXStudio/Claudia. + But sometimes it is useful to have a simple CLI tool for this job. +
+ +
+ We are pleased to introduce you the new development version of
relpipe-in-cli
module has changed CLI interface: see details below
+ relpipe-in-csv
module has changed CLI interface: see details below
+ relpipe-out-csv
module has new CLI interface: see details below
+ relpipe-out-tabular
module has new CLI interface: see details below
+ relpipe-tr-guile
was renamed to relpipe-tr-scheme
: see details below
+ relpipe-out-jack
for generating MIDI through JACK: see details below
+
+ See the
+ Please note that this is still a development release and thus the API (libraries, CLI arguments, formats) might and will change.
+ Any suggestions, ideas and bug reports are welcome in our
+ Several earlier tools have non-standard CLI interface and does not conform with relpipe-in-cli
, relpipe-in-csv
tools dropped their old CLI interface
+ and got a new one based on named parameters and long options.
+ The relpipe-out-csv
and relpipe-out-tabular
had no parameters in previous versions and now have standard CLI interface.
+
+ In relpipe-out-tabular
we can now optionally omit several parts:
+ --write-record-count false
,
+ --write-types false
,
+ --write-relation-name false
+ and thus simplify the output if we do not need given information.
+
+ These options may be set globally or for particular relation using the --relation
option with pattern parameter.
+ Thus we may e.g. print some relations with data types and some without them.
+
+ Remaining tools that will be transformed in the same way: relpipe-tr-grep
, relpipe-tr-cut
, relpipe-tr-sed
+ and will get standard CLI interface.
+
+ A powerful audio system called JACK allows us to + build pipelines consisting of audio interfaces, players, recorders, filters and effects… + and route sound streams (both PCM and MIDI) through them. + MIDI messages can come from keyboards or other hardware MIDI controllers or from MIDI players and other software. +
+
+ In the relpipe-in-jack
was introduced.
+ This tools allowed capturing the MIDI events from JACK and sending them to the relational pipeline.
+ Now, in v0.17, we introduce the counterpart tool relpipe-out-jack
that reads relational data
+ and sends SysEx messages, MIDI notes and other events to the JACK pipeline.
+
+ We can join the JACK graph with relpipe-out-jack
command.
+ It does not produce anything on STDOUT, it sends MIDI events to JACK instead, so no other output data are needed.
+
+ Besides capturing and playing the MIDI, we can use the relpipe-in-jack
and relpipe-out-jack
+ to tak a snapshot of the JACK graph (connections between ports) and recreate it later.
+ Or establish any new connections or break existing ones.
+ Either through CLI commands of relpipe-in-jack
or through relational data (list of connection and disconnection commands)
+ sent to relpipe-out-jack
.
+
+ Both commands have also the --required-connections
and --connect-to
parameters,
+ so they can react on reaching particular connection count or automatically connect their port to another one specified on the CLI.
+ The bash-completion suggests available port names.
+
+ More details in the examples:
+
+ The relpipe-tr-guile
has been renamed to relpipe-tr-scheme
, so it is necessary to modify existing scripts.
+ However the CLI interface has not been changed, thus it can be simply updated e.g. by:
+
+ The reason behind this rename is that Scheme is a programming language and Guile is one of implementations of this language.
+ And we prefer depending on interfaces rather than on particular implementations.
+ Similar situation is relpipe-tr-sql
: this module is also named after the language (SQL), not after the default implementation (SQLite)
+ and this module can use
+ The CLI interface of the relpipe-in-cli
tool has been standardized and made consistent with other tools.
+ Now, it uses named parameters instead of cryptic positional ones.
+ The format of the STDIN stream remains unchanged (values separated by \0
byte).
+
+ We can repeat these parameters and generate several relations (that was not possibel in the older version).
+ One relation may be filled with values from STDIN: --records-on-stdin true
(replaces the old relpipe-in-cli generate-from-stdin
syntax).
+ And the last relation may be filled with values from remaining CLI arguments: --records "1" "Hello" "true" "2" "World!" "false"
.
+ The demo (relpipe-in-cli demo
) has been removed (sample data can be easily generated by setting the CLI arguments).
+
+ The CLI interface of the relpipe-in-csv
tool has also been standardized and made consistent with other tools.
+ If we have attribute names on the first line of the CSV file (which is quite common), the usage is same as in previous versions:
+
If we have only data in the CSV file (no headers) and thus want to specify them on CLI, the syntax is different:
+ +Bash-completion scripts for all tools are provided.
+ ++ Future versions should allow optionally storing the attribute data types in the CSV header and parsing them back (together with the attribute names). +
+ ++ Instalation was tested on Debian GNU/Linux 10.2. + The process should be similar on other distributions. +
+ +
+ out-gui.qt
and tr-python
require additional libraries and are not built by default.
+