author | František Kučera <franta-hg@frantovo.cz> |
Sun, 25 Nov 2018 01:03:26 +0100 | |
branch | v_0 |
changeset 143 | 297da74fcab2 |
parent 138 | 5d892b169e32 |
child 144 | ee7e96151673 |
permissions | -rw-r--r-- |
23
0d2729ed16ed
zkouška interního odkazu
František Kučera <franta-hg@frantovo.cz>
parents:
18
diff
changeset
|
1 |
<stránka |
0d2729ed16ed
zkouška interního odkazu
František Kučera <franta-hg@frantovo.cz>
parents:
18
diff
changeset
|
2 |
xmlns="https://trac.frantovo.cz/xml-web-generator/wiki/xmlns/strana" |
0d2729ed16ed
zkouška interního odkazu
František Kučera <franta-hg@frantovo.cz>
parents:
18
diff
changeset
|
3 |
xmlns:m="https://trac.frantovo.cz/xml-web-generator/wiki/xmlns/makro"> |
0d2729ed16ed
zkouška interního odkazu
František Kučera <franta-hg@frantovo.cz>
parents:
18
diff
changeset
|
4 |
|
137 | 5 |
<nadpis>Relational pipes</nadpis> |
6 |
<perex>Official homepage of Relational pipes.</perex> |
|
4
1bb39595a51c
genrování hlavní nabídky #1
František Kučera <franta-hg@frantovo.cz>
parents:
2
diff
changeset
|
7 |
<pořadí>10</pořadí> |
1bb39595a51c
genrování hlavní nabídky #1
František Kučera <franta-hg@frantovo.cz>
parents:
2
diff
changeset
|
8 |
|
2
ab9099ff88fa
vkládání zápatí, jmenné prostory, saxon
František Kučera <franta-hg@frantovo.cz>
parents:
1
diff
changeset
|
9 |
<text xmlns="http://www.w3.org/1999/xhtml"> |
ab9099ff88fa
vkládání zápatí, jmenné prostory, saxon
František Kučera <franta-hg@frantovo.cz>
parents:
1
diff
changeset
|
10 |
<p> |
143 | 11 |
One of the great parts of the <m:unix/> |
12 |
<m:podČarou> |
|
13 |
<m:unix tvar="vysvětlivka"/> |
|
14 |
</m:podČarou> culture is the invention<m:podČarou>which is attributed to Doug McIlroy, see <a href="http://www.catb.org/~esr/writings/taoup/html/ch07s02.html#plumbing">The Art of Unix Programming: Pipes, Redirection, and Filters</a></m:podČarou> |
|
15 |
of <em>pipes</em> and the idea<m:podČarou>see <a href="http://www.catb.org/~esr/writings/taoup/html/ch01s06.html">The Art of Unix Programming: Basics of the Unix Philosophy</a></m:podČarou> |
|
16 |
that <em>one program should do one thing and do it well</em>. |
|
87
25dec6931f18
Lepší odsazení, tabulátory.
František Kučera <franta-hg@frantovo.cz>
parents:
23
diff
changeset
|
17 |
</p> |
138 | 18 |
|
143 | 19 |
<p> |
20 |
Each running program (process) has one input stream (called standard input or STDIN) and one output stream (called standard output or STDOUT) and also one additional output stream for logging/errors/warnings (STDERR). |
|
21 |
We can connect programs and pass the STDOUT of first one to the STDIN of the second one (etc.) using pipes. |
|
22 |
</p> |
|
23 |
||
24 |
<!-- |
|
25 |
<pre>cat /etc/fstab | dd 2>/tmp/dd.log | grep tmpfs</pre> |
|
26 |
<p></p> |
|
27 |
<m:diagram orientace="vodorovně"> |
|
28 |
node[shape=box]; |
|
29 |
||
30 |
cat [label="cat /etc/fstab"]; |
|
31 |
dd []; |
|
32 |
grep [label="grep tmpfs"]; |
|
33 |
log [label="/tmp/dd.log"]; |
|
34 |
||
35 |
cat -> dd [label="STDOUT → STDIN"]; |
|
36 |
dd -> grep [label="STDOUT → STDIN"]; |
|
37 |
dd -> log [label="STDERR → file"]; |
|
38 |
</m:diagram> |
|
39 |
--> |
|
40 |
||
41 |
<p> |
|
42 |
According to this principle we can build complex and powerful programs (pipelines) by composing several simple, single-purpose and reusable programs. |
|
43 |
Such single-purpose programs (often called <em>filters</em>) are much easier to create, test and optimize and their authors don't have to bother about the complexity of the final pipeline. |
|
44 |
They even don't have to know, how their programs will be used in the future by others. |
|
45 |
This is a great design principle that brings us advanced flexibility, reusability, efficiency and reliability. Simply: awesome. |
|
46 |
Being in any role (author of a filter, builder of a pipeline etc.), we can always focus on our task only and do it well. |
|
47 |
And we can collaborate with others even if we don't know about them and we don't know that we are collaborating. |
|
48 |
Now think about putting this together with the free software ideas... How very! |
|
49 |
</p> |
|
50 |
||
51 |
<!-- |
|
138 | 52 |
<m:diagram orientace="vodorovně"> |
53 |
compound=true; |
|
54 |
node[shape=box]; |
|
55 |
||
56 |
subgraph cluster_in { |
|
57 |
label = "Inputs:"; |
|
58 |
cli; |
|
59 |
fstab; |
|
60 |
} |
|
61 |
||
62 |
subgraph cluster_tr { |
|
63 |
label = "Transformations:"; |
|
64 |
grep; |
|
65 |
sed; |
|
66 |
} |
|
67 |
||
68 |
subgraph cluster_out { |
|
69 |
label = "Outputs:"; |
|
70 |
xml; |
|
71 |
tabular; |
|
72 |
gui; |
|
73 |
} |
|
74 |
||
75 |
cli -> grep [ltail=cluster_in, lhead=cluster_tr]; |
|
76 |
grep -> xml [ltail=cluster_tr, lhead=cluster_out]; |
|
77 |
// cli -> xml [ltail=cluster_in, lhead=cluster_out]; |
|
78 |
||
79 |
</m:diagram> |
|
143 | 80 |
--> |
81 |
||
82 |
<h2>What <m:name/> are?</h2> |
|
83 |
||
84 |
<p> |
|
85 |
<m:name/> are an open <em>data format</em> designed for streaming structured data between two processes. |
|
86 |
Simultaneously with the format specification, we are also developing a <em>reference implementation</em> (libraries and tools) as a free software. |
|
87 |
Although we believe in the specification-first (or contract-first) approach, we always look and check, whether the theoretic concepts are feasible and whether they can be reasonably and reliably implemented. |
|
88 |
So befeore publishing any new specification or its version, we will verify it by creating a reference implementation at least in one programming language. |
|
89 |
</p> |
|
90 |
<p> |
|
91 |
More generally, <m:name/> are a philosophical continuation of the classic <m:unix/> pipelines and the relational model. |
|
92 |
</p> |
|
93 |
||
94 |
||
95 |
<h2>What <m:name/> are not?</h2> |
|
96 |
||
97 |
<p> |
|
98 |
<m:name/> respect the existing ecosystem and are rather an improvement or supplement than a replacement. |
|
99 |
So <m:name/> are not a: |
|
100 |
</p> |
|
101 |
||
102 |
<ul> |
|
103 |
<li>Shell – we use existing shells (e.g. GNU Bash), work with any shell and even without a shell (e.g. as a stream format passed through a network or stored in a file).</li> |
|
104 |
<li>Terminal emulator – same as with shells, we use existing terminals and we can use <m:name/> also outside any terminal; if we interact with any terminal, we use standard means as Unicode, ANSI escape sequences etc.</li> |
|
105 |
<li>IDE – we use standard <m:unix/> tools as an IDE (GNU Screen, Make etc.) or any other IDE.</li> |
|
106 |
<li>Programming language – <m:name/> are language-independent data format and can be produced or consumed in any programming language.</li> |
|
107 |
<li>Query language – although some of our tools are doing queries, filtering or transformations, we are not inventing a new query language – instead, we use existing languages like SQL, XPath or regular expressions.</li> |
|
108 |
<!--<li>Text editor – </li>--> |
|
109 |
<li>Database system, DBMS – we focus on the stream processing rather than data storage. Although sometimes it makes sense to pipe data to a file and continue with the processing later.</li> |
|
110 |
</ul> |
|
111 |
||
112 |
||
113 |
<h2>Project status</h2> |
|
114 |
||
115 |
<p> |
|
116 |
The main ideas and the roadmap are quite clear, but many things will change (including the format internals and interfaces of the libraries and tools). |
|
117 |
Because we know how important the API and ABI stability is, we are not ready to publish the version 1.0 yet. |
|
118 |
</p> |
|
119 |
<p> |
|
120 |
On the other hand, the already published tools (tagged as v0.x in v_0 branch) should work quite well (should compile, should run, should not segfault often, should not wipe your hard drive or kill your cat), |
|
121 |
so they might be useful for someone who likes our ideas and who is prepared to update own programs and scripts when the new version is ready. |
|
122 |
</p> |
|
138 | 123 |
|
124 |
||
87
25dec6931f18
Lepší odsazení, tabulátory.
František Kučera <franta-hg@frantovo.cz>
parents:
23
diff
changeset
|
125 |
</text> |
4
1bb39595a51c
genrování hlavní nabídky #1
František Kučera <franta-hg@frantovo.cz>
parents:
2
diff
changeset
|
126 |
|
1 | 127 |
</stránka> |
87
25dec6931f18
Lepší odsazení, tabulátory.
František Kučera <franta-hg@frantovo.cz>
parents:
23
diff
changeset
|
128 |