relpipe-data/index.xml
author František Kučera <franta-hg@frantovo.cz>
Sun, 25 Nov 2018 01:03:26 +0100
branchv_0
changeset 143 297da74fcab2
parent 138 5d892b169e32
child 144 ee7e96151673
permissions -rw-r--r--
introduction
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
23
0d2729ed16ed zkouška interního odkazu
František Kučera <franta-hg@frantovo.cz>
parents: 18
diff changeset
     1
<stránka
0d2729ed16ed zkouška interního odkazu
František Kučera <franta-hg@frantovo.cz>
parents: 18
diff changeset
     2
	xmlns="https://trac.frantovo.cz/xml-web-generator/wiki/xmlns/strana"
0d2729ed16ed zkouška interního odkazu
František Kučera <franta-hg@frantovo.cz>
parents: 18
diff changeset
     3
	xmlns:m="https://trac.frantovo.cz/xml-web-generator/wiki/xmlns/makro">
0d2729ed16ed zkouška interního odkazu
František Kučera <franta-hg@frantovo.cz>
parents: 18
diff changeset
     4
	
137
ecfe2980ae1f logo and css
František Kučera <franta-hg@frantovo.cz>
parents: 136
diff changeset
     5
	<nadpis>Relational pipes</nadpis>
ecfe2980ae1f logo and css
František Kučera <franta-hg@frantovo.cz>
parents: 136
diff changeset
     6
	<perex>Official homepage of Relational pipes.</perex>
4
1bb39595a51c genrování hlavní nabídky #1
František Kučera <franta-hg@frantovo.cz>
parents: 2
diff changeset
     7
	<pořadí>10</pořadí>
1bb39595a51c genrování hlavní nabídky #1
František Kučera <franta-hg@frantovo.cz>
parents: 2
diff changeset
     8
2
ab9099ff88fa vkládání zápatí, jmenné prostory, saxon
František Kučera <franta-hg@frantovo.cz>
parents: 1
diff changeset
     9
	<text xmlns="http://www.w3.org/1999/xhtml">
ab9099ff88fa vkládání zápatí, jmenné prostory, saxon
František Kučera <franta-hg@frantovo.cz>
parents: 1
diff changeset
    10
		<p>
143
297da74fcab2 introduction
František Kučera <franta-hg@frantovo.cz>
parents: 138
diff changeset
    11
			One of the great parts of the <m:unix/>
297da74fcab2 introduction
František Kučera <franta-hg@frantovo.cz>
parents: 138
diff changeset
    12
			<m:podČarou> 
297da74fcab2 introduction
František Kučera <franta-hg@frantovo.cz>
parents: 138
diff changeset
    13
				<m:unix tvar="vysvětlivka"/> 
297da74fcab2 introduction
František Kučera <franta-hg@frantovo.cz>
parents: 138
diff changeset
    14
			</m:podČarou> culture is the invention<m:podČarou>which is attributed to Doug McIlroy, see <a href="http://www.catb.org/~esr/writings/taoup/html/ch07s02.html#plumbing">The Art of Unix Programming: Pipes, Redirection, and Filters</a></m:podČarou>
297da74fcab2 introduction
František Kučera <franta-hg@frantovo.cz>
parents: 138
diff changeset
    15
			of <em>pipes</em> and the idea<m:podČarou>see <a href="http://www.catb.org/~esr/writings/taoup/html/ch01s06.html">The Art of Unix Programming: Basics of the Unix Philosophy</a></m:podČarou> 
297da74fcab2 introduction
František Kučera <franta-hg@frantovo.cz>
parents: 138
diff changeset
    16
			that <em>one program should do one thing and do it well</em>.
87
25dec6931f18 Lepší odsazení, tabulátory.
František Kučera <franta-hg@frantovo.cz>
parents: 23
diff changeset
    17
		</p>
138
5d892b169e32 metadata, footer
František Kučera <franta-hg@frantovo.cz>
parents: 137
diff changeset
    18
		
143
297da74fcab2 introduction
František Kučera <franta-hg@frantovo.cz>
parents: 138
diff changeset
    19
		<p>
297da74fcab2 introduction
František Kučera <franta-hg@frantovo.cz>
parents: 138
diff changeset
    20
			Each running program (process) has one input stream (called standard input or STDIN) and one output stream (called standard output or STDOUT) and also one additional output stream for logging/errors/warnings (STDERR).
297da74fcab2 introduction
František Kučera <franta-hg@frantovo.cz>
parents: 138
diff changeset
    21
			We can connect programs and pass the STDOUT of first one to the STDIN of the second one (etc.) using pipes.
297da74fcab2 introduction
František Kučera <franta-hg@frantovo.cz>
parents: 138
diff changeset
    22
		</p>
297da74fcab2 introduction
František Kučera <franta-hg@frantovo.cz>
parents: 138
diff changeset
    23
297da74fcab2 introduction
František Kučera <franta-hg@frantovo.cz>
parents: 138
diff changeset
    24
		<!--		
297da74fcab2 introduction
František Kučera <franta-hg@frantovo.cz>
parents: 138
diff changeset
    25
		<pre>cat /etc/fstab | dd 2>/tmp/dd.log | grep tmpfs</pre>
297da74fcab2 introduction
František Kučera <franta-hg@frantovo.cz>
parents: 138
diff changeset
    26
		<p></p>
297da74fcab2 introduction
František Kučera <franta-hg@frantovo.cz>
parents: 138
diff changeset
    27
		<m:diagram orientace="vodorovně">
297da74fcab2 introduction
František Kučera <franta-hg@frantovo.cz>
parents: 138
diff changeset
    28
			node[shape=box];
297da74fcab2 introduction
František Kučera <franta-hg@frantovo.cz>
parents: 138
diff changeset
    29
			
297da74fcab2 introduction
František Kučera <franta-hg@frantovo.cz>
parents: 138
diff changeset
    30
			cat  [label="cat /etc/fstab"];
297da74fcab2 introduction
František Kučera <franta-hg@frantovo.cz>
parents: 138
diff changeset
    31
			dd   [];
297da74fcab2 introduction
František Kučera <franta-hg@frantovo.cz>
parents: 138
diff changeset
    32
			grep [label="grep tmpfs"];
297da74fcab2 introduction
František Kučera <franta-hg@frantovo.cz>
parents: 138
diff changeset
    33
			log  [label="/tmp/dd.log"];
297da74fcab2 introduction
František Kučera <franta-hg@frantovo.cz>
parents: 138
diff changeset
    34
			
297da74fcab2 introduction
František Kučera <franta-hg@frantovo.cz>
parents: 138
diff changeset
    35
			cat -> dd  [label="STDOUT → STDIN"];
297da74fcab2 introduction
František Kučera <franta-hg@frantovo.cz>
parents: 138
diff changeset
    36
			dd -> grep [label="STDOUT → STDIN"];
297da74fcab2 introduction
František Kučera <franta-hg@frantovo.cz>
parents: 138
diff changeset
    37
			dd -> log  [label="STDERR → file"];
297da74fcab2 introduction
František Kučera <franta-hg@frantovo.cz>
parents: 138
diff changeset
    38
		</m:diagram>
297da74fcab2 introduction
František Kučera <franta-hg@frantovo.cz>
parents: 138
diff changeset
    39
		-->
297da74fcab2 introduction
František Kučera <franta-hg@frantovo.cz>
parents: 138
diff changeset
    40
		
297da74fcab2 introduction
František Kučera <franta-hg@frantovo.cz>
parents: 138
diff changeset
    41
		<p>
297da74fcab2 introduction
František Kučera <franta-hg@frantovo.cz>
parents: 138
diff changeset
    42
			According to this principle we can build complex and powerful programs (pipelines) by composing several simple, single-purpose and reusable programs.
297da74fcab2 introduction
František Kučera <franta-hg@frantovo.cz>
parents: 138
diff changeset
    43
			Such single-purpose programs (often called <em>filters</em>) are much easier to create, test and optimize and their authors don't have to bother about the complexity of the final pipeline.
297da74fcab2 introduction
František Kučera <franta-hg@frantovo.cz>
parents: 138
diff changeset
    44
			They even don't have to know, how their programs will be used in the future by others.
297da74fcab2 introduction
František Kučera <franta-hg@frantovo.cz>
parents: 138
diff changeset
    45
			This is a great design principle that brings us advanced flexibility, reusability, efficiency and reliability. Simply: awesome.
297da74fcab2 introduction
František Kučera <franta-hg@frantovo.cz>
parents: 138
diff changeset
    46
			Being in any role (author of a filter, builder of a pipeline etc.), we can always focus on our task only and do it well.
297da74fcab2 introduction
František Kučera <franta-hg@frantovo.cz>
parents: 138
diff changeset
    47
			And we can collaborate with others even if we don't know about them and we don't know that we are collaborating.
297da74fcab2 introduction
František Kučera <franta-hg@frantovo.cz>
parents: 138
diff changeset
    48
			Now think about putting this together with the free software ideas...  How very!
297da74fcab2 introduction
František Kučera <franta-hg@frantovo.cz>
parents: 138
diff changeset
    49
		</p>
297da74fcab2 introduction
František Kučera <franta-hg@frantovo.cz>
parents: 138
diff changeset
    50
		
297da74fcab2 introduction
František Kučera <franta-hg@frantovo.cz>
parents: 138
diff changeset
    51
		<!--
138
5d892b169e32 metadata, footer
František Kučera <franta-hg@frantovo.cz>
parents: 137
diff changeset
    52
		<m:diagram orientace="vodorovně">
5d892b169e32 metadata, footer
František Kučera <franta-hg@frantovo.cz>
parents: 137
diff changeset
    53
			compound=true;
5d892b169e32 metadata, footer
František Kučera <franta-hg@frantovo.cz>
parents: 137
diff changeset
    54
			node[shape=box];
5d892b169e32 metadata, footer
František Kučera <franta-hg@frantovo.cz>
parents: 137
diff changeset
    55
			
5d892b169e32 metadata, footer
František Kučera <franta-hg@frantovo.cz>
parents: 137
diff changeset
    56
			subgraph cluster_in {
5d892b169e32 metadata, footer
František Kučera <franta-hg@frantovo.cz>
parents: 137
diff changeset
    57
			label = "Inputs:";
5d892b169e32 metadata, footer
František Kučera <franta-hg@frantovo.cz>
parents: 137
diff changeset
    58
			cli;
5d892b169e32 metadata, footer
František Kučera <franta-hg@frantovo.cz>
parents: 137
diff changeset
    59
			fstab;
5d892b169e32 metadata, footer
František Kučera <franta-hg@frantovo.cz>
parents: 137
diff changeset
    60
			}
5d892b169e32 metadata, footer
František Kučera <franta-hg@frantovo.cz>
parents: 137
diff changeset
    61
			
5d892b169e32 metadata, footer
František Kučera <franta-hg@frantovo.cz>
parents: 137
diff changeset
    62
			subgraph cluster_tr {
5d892b169e32 metadata, footer
František Kučera <franta-hg@frantovo.cz>
parents: 137
diff changeset
    63
			label = "Transformations:";
5d892b169e32 metadata, footer
František Kučera <franta-hg@frantovo.cz>
parents: 137
diff changeset
    64
			grep;
5d892b169e32 metadata, footer
František Kučera <franta-hg@frantovo.cz>
parents: 137
diff changeset
    65
			sed;
5d892b169e32 metadata, footer
František Kučera <franta-hg@frantovo.cz>
parents: 137
diff changeset
    66
			}
5d892b169e32 metadata, footer
František Kučera <franta-hg@frantovo.cz>
parents: 137
diff changeset
    67
			
5d892b169e32 metadata, footer
František Kučera <franta-hg@frantovo.cz>
parents: 137
diff changeset
    68
			subgraph cluster_out {
5d892b169e32 metadata, footer
František Kučera <franta-hg@frantovo.cz>
parents: 137
diff changeset
    69
			label = "Outputs:";
5d892b169e32 metadata, footer
František Kučera <franta-hg@frantovo.cz>
parents: 137
diff changeset
    70
			xml;
5d892b169e32 metadata, footer
František Kučera <franta-hg@frantovo.cz>
parents: 137
diff changeset
    71
			tabular;
5d892b169e32 metadata, footer
František Kučera <franta-hg@frantovo.cz>
parents: 137
diff changeset
    72
			gui;
5d892b169e32 metadata, footer
František Kučera <franta-hg@frantovo.cz>
parents: 137
diff changeset
    73
			}
5d892b169e32 metadata, footer
František Kučera <franta-hg@frantovo.cz>
parents: 137
diff changeset
    74
			
5d892b169e32 metadata, footer
František Kučera <franta-hg@frantovo.cz>
parents: 137
diff changeset
    75
			cli -> grep  [ltail=cluster_in, lhead=cluster_tr];
5d892b169e32 metadata, footer
František Kučera <franta-hg@frantovo.cz>
parents: 137
diff changeset
    76
			grep -> xml [ltail=cluster_tr, lhead=cluster_out];
5d892b169e32 metadata, footer
František Kučera <franta-hg@frantovo.cz>
parents: 137
diff changeset
    77
			// cli -> xml [ltail=cluster_in, lhead=cluster_out];
5d892b169e32 metadata, footer
František Kučera <franta-hg@frantovo.cz>
parents: 137
diff changeset
    78
			
5d892b169e32 metadata, footer
František Kučera <franta-hg@frantovo.cz>
parents: 137
diff changeset
    79
		</m:diagram>
143
297da74fcab2 introduction
František Kučera <franta-hg@frantovo.cz>
parents: 138
diff changeset
    80
		-->
297da74fcab2 introduction
František Kučera <franta-hg@frantovo.cz>
parents: 138
diff changeset
    81
		
297da74fcab2 introduction
František Kučera <franta-hg@frantovo.cz>
parents: 138
diff changeset
    82
		<h2>What <m:name/> are?</h2>
297da74fcab2 introduction
František Kučera <franta-hg@frantovo.cz>
parents: 138
diff changeset
    83
		
297da74fcab2 introduction
František Kučera <franta-hg@frantovo.cz>
parents: 138
diff changeset
    84
		<p>
297da74fcab2 introduction
František Kučera <franta-hg@frantovo.cz>
parents: 138
diff changeset
    85
			<m:name/> are an open <em>data format</em> designed for streaming structured data between two processes. 
297da74fcab2 introduction
František Kučera <franta-hg@frantovo.cz>
parents: 138
diff changeset
    86
			Simultaneously with the format specification, we are also developing a <em>reference implementation</em> (libraries and tools) as a free software.
297da74fcab2 introduction
František Kučera <franta-hg@frantovo.cz>
parents: 138
diff changeset
    87
			Although we believe in the specification-first (or contract-first) approach, we always look and check, whether the theoretic concepts are feasible and whether they can be reasonably and reliably implemented.
297da74fcab2 introduction
František Kučera <franta-hg@frantovo.cz>
parents: 138
diff changeset
    88
			So befeore publishing any new specification or its version, we will verify it by creating a reference implementation at least in one programming language.
297da74fcab2 introduction
František Kučera <franta-hg@frantovo.cz>
parents: 138
diff changeset
    89
		</p>
297da74fcab2 introduction
František Kučera <franta-hg@frantovo.cz>
parents: 138
diff changeset
    90
		<p>
297da74fcab2 introduction
František Kučera <franta-hg@frantovo.cz>
parents: 138
diff changeset
    91
			More generally, <m:name/> are a philosophical continuation of the classic <m:unix/> pipelines and the relational model.
297da74fcab2 introduction
František Kučera <franta-hg@frantovo.cz>
parents: 138
diff changeset
    92
		</p>
297da74fcab2 introduction
František Kučera <franta-hg@frantovo.cz>
parents: 138
diff changeset
    93
		
297da74fcab2 introduction
František Kučera <franta-hg@frantovo.cz>
parents: 138
diff changeset
    94
		
297da74fcab2 introduction
František Kučera <franta-hg@frantovo.cz>
parents: 138
diff changeset
    95
		<h2>What <m:name/> are not?</h2>
297da74fcab2 introduction
František Kučera <franta-hg@frantovo.cz>
parents: 138
diff changeset
    96
			
297da74fcab2 introduction
František Kučera <franta-hg@frantovo.cz>
parents: 138
diff changeset
    97
		<p>
297da74fcab2 introduction
František Kučera <franta-hg@frantovo.cz>
parents: 138
diff changeset
    98
			<m:name/> respect the existing ecosystem and are rather an improvement or supplement than a replacement.
297da74fcab2 introduction
František Kučera <franta-hg@frantovo.cz>
parents: 138
diff changeset
    99
			So <m:name/> are not a:
297da74fcab2 introduction
František Kučera <franta-hg@frantovo.cz>
parents: 138
diff changeset
   100
		</p>
297da74fcab2 introduction
František Kučera <franta-hg@frantovo.cz>
parents: 138
diff changeset
   101
		
297da74fcab2 introduction
František Kučera <franta-hg@frantovo.cz>
parents: 138
diff changeset
   102
		<ul>
297da74fcab2 introduction
František Kučera <franta-hg@frantovo.cz>
parents: 138
diff changeset
   103
			<li>Shell – we use existing shells (e.g. GNU Bash), work with any shell and even without a shell (e.g. as a stream format passed through a network or stored in a file).</li>
297da74fcab2 introduction
František Kučera <franta-hg@frantovo.cz>
parents: 138
diff changeset
   104
			<li>Terminal emulator – same as with shells, we use existing terminals and we can use <m:name/> also outside any terminal; if we interact with any terminal, we use standard means as Unicode, ANSI escape sequences etc.</li>
297da74fcab2 introduction
František Kučera <franta-hg@frantovo.cz>
parents: 138
diff changeset
   105
			<li>IDE – we use standard <m:unix/> tools as an IDE (GNU Screen, Make etc.) or any other IDE.</li>
297da74fcab2 introduction
František Kučera <franta-hg@frantovo.cz>
parents: 138
diff changeset
   106
			<li>Programming language – <m:name/> are language-independent data format and can be produced or consumed in any programming language.</li>
297da74fcab2 introduction
František Kučera <franta-hg@frantovo.cz>
parents: 138
diff changeset
   107
			<li>Query language – although some of our tools are doing queries, filtering or transformations, we are not inventing a new query language – instead, we use existing languages like SQL, XPath or regular expressions.</li>
297da74fcab2 introduction
František Kučera <franta-hg@frantovo.cz>
parents: 138
diff changeset
   108
			<!--<li>Text editor – </li>-->
297da74fcab2 introduction
František Kučera <franta-hg@frantovo.cz>
parents: 138
diff changeset
   109
			<li>Database system, DBMS – we focus on the stream processing rather than data storage. Although sometimes it makes sense to pipe data to a file and continue with the processing later.</li>
297da74fcab2 introduction
František Kučera <franta-hg@frantovo.cz>
parents: 138
diff changeset
   110
		</ul>
297da74fcab2 introduction
František Kučera <franta-hg@frantovo.cz>
parents: 138
diff changeset
   111
		
297da74fcab2 introduction
František Kučera <franta-hg@frantovo.cz>
parents: 138
diff changeset
   112
		
297da74fcab2 introduction
František Kučera <franta-hg@frantovo.cz>
parents: 138
diff changeset
   113
		<h2>Project status</h2>
297da74fcab2 introduction
František Kučera <franta-hg@frantovo.cz>
parents: 138
diff changeset
   114
		
297da74fcab2 introduction
František Kučera <franta-hg@frantovo.cz>
parents: 138
diff changeset
   115
		<p>
297da74fcab2 introduction
František Kučera <franta-hg@frantovo.cz>
parents: 138
diff changeset
   116
			The main ideas and the roadmap are quite clear, but many things will change (including the format internals and interfaces of the libraries and tools).
297da74fcab2 introduction
František Kučera <franta-hg@frantovo.cz>
parents: 138
diff changeset
   117
			Because we know how important the API and ABI stability is, we are not ready to publish the version 1.0 yet.
297da74fcab2 introduction
František Kučera <franta-hg@frantovo.cz>
parents: 138
diff changeset
   118
		</p>
297da74fcab2 introduction
František Kučera <franta-hg@frantovo.cz>
parents: 138
diff changeset
   119
		<p>
297da74fcab2 introduction
František Kučera <franta-hg@frantovo.cz>
parents: 138
diff changeset
   120
			On the other hand, the already published tools (tagged as v0.x in v_0 branch) should work quite well (should compile, should run, should not segfault often, should not wipe your hard drive or kill your cat),
297da74fcab2 introduction
František Kučera <franta-hg@frantovo.cz>
parents: 138
diff changeset
   121
			so they might be useful for someone who likes our ideas and who is prepared to update own programs and scripts when the new version is ready.
297da74fcab2 introduction
František Kučera <franta-hg@frantovo.cz>
parents: 138
diff changeset
   122
		</p>
138
5d892b169e32 metadata, footer
František Kučera <franta-hg@frantovo.cz>
parents: 137
diff changeset
   123
5d892b169e32 metadata, footer
František Kučera <franta-hg@frantovo.cz>
parents: 137
diff changeset
   124
		
87
25dec6931f18 Lepší odsazení, tabulátory.
František Kučera <franta-hg@frantovo.cz>
parents: 23
diff changeset
   125
	</text>
4
1bb39595a51c genrování hlavní nabídky #1
František Kučera <franta-hg@frantovo.cz>
parents: 2
diff changeset
   126
1
a05c6f3cbc3e základ, první verze
František Kučera <franta-hg@frantovo.cz>
parents:
diff changeset
   127
</stránka>
87
25dec6931f18 Lepší odsazení, tabulátory.
František Kučera <franta-hg@frantovo.cz>
parents: 23
diff changeset
   128