43 |
43 |
44 <p> |
44 <p> |
45 According to this principle we can build complex and powerful programs (pipelines) by composing several simple, single-purpose and reusable programs. |
45 According to this principle we can build complex and powerful programs (pipelines) by composing several simple, single-purpose and reusable programs. |
46 Such single-purpose programs (often called <em>filters</em>) are much easier to create, test and optimize and their authors don't have to bother about the complexity of the final pipeline. |
46 Such single-purpose programs (often called <em>filters</em>) are much easier to create, test and optimize and their authors don't have to bother about the complexity of the final pipeline. |
47 They even don't have to know, how their programs will be used in the future by others. |
47 They even don't have to know, how their programs will be used in the future by others. |
48 This is a great design principle that brings us advanced flexibility, reusability, efficiency and reliability. |
48 This is a great design principle that brings us advanced flexibility, reusability, efficiency and reliability. |
49 Being in any role (author of a filter, builder of a pipeline etc.), we can always focus on our task only and do it well.<m:podČarou>see <a href="http://wiki.apidesign.org/wiki/Cluelessness">cluelessness</a> by Jaroslav Tulach in his <em>Practical API Design. Confessions of a Java Framework Architect</em></m:podČarou> |
49 Being in any role (author of a filter, builder of a pipeline etc.), we can always focus on our task only and do it well.<m:podČarou>see <a href="http://wiki.apidesign.org/wiki/Cluelessness">cluelessness</a> by Jaroslav Tulach in his <em>Practical API Design. Confessions of a Java Framework Architect</em></m:podČarou> |
50 And we can collaborate with others even if we don't know about them and we don't know that we are collaborating. |
50 And we can collaborate with others even if we don't know about them and we don't know that we are collaborating. |
51 Now think about putting this together with the free software ideas... How very! |
51 Now think about putting this together with the free software ideas... How very! |
52 </p> |
52 </p> |
53 |
53 |
54 <!-- |
54 <!-- |
97 <blockquote>Everything should be made as simple as possible, but not simpler.</blockquote> |
97 <blockquote>Everything should be made as simple as possible, but not simpler.</blockquote> |
98 --> |
98 --> |
99 |
99 |
100 <p> |
100 <p> |
101 It is not about the shape of the brackets, apostrophes, quotes or text vs. binary. |
101 It is not about the shape of the brackets, apostrophes, quotes or text vs. binary. |
102 It is not a technical question – it is in the semantic layer and human brain. |
102 It is not a technical question – it is in the semantic layer and human brain. |
103 Generic formats and their <em>arbitrary object trees/graphs</em> are (for humans, not for computers) difficult to understand and work with |
103 Generic formats and their <em>arbitrary object trees/graphs</em> are (for humans, not for computers) difficult to understand and work with |
104 – compared to simpler structures like arrays, maps or matrixes. |
104 – compared to simpler structures like arrays, maps or matrixes. |
105 </p> |
105 </p> |
106 |
106 |
107 <p> |
107 <p> |
115 This logical model is powerful enough to describe almost any data and – at the same time – it is still simple and easy to be understood by humans. |
115 This logical model is powerful enough to describe almost any data and – at the same time – it is still simple and easy to be understood by humans. |
116 </p> |
116 </p> |
117 |
117 |
118 <p> |
118 <p> |
119 Thus the <m:name/> are streams containing zero or more relations. |
119 Thus the <m:name/> are streams containing zero or more relations. |
120 Each relation has a name, one or more attributes and zero or more records (tuples). |
120 Each relation has a name, one or more attributes and zero or more records (tuples). |
121 Each attribute has a name and a data-type. |
121 Each attribute has a name and a data-type. |
122 Records contain attribute values. |
122 Records contain attribute values. |
123 We can imagine this stream as a sequence of tables (but the table is only one of many possible visual representations of such relational data). |
123 We can imagine this stream as a sequence of tables (but the table is only one of many possible visual representations of such relational data). |
124 </p> |
124 </p> |
125 |
125 |
126 <h2>What <m:name/> are?</h2> |
126 <h2>What <m:name/> are?</h2> |
127 |
127 |
128 <p> |
128 <p> |
129 <m:name/> are an open <em>data format</em> designed for streaming structured data between two processes. |
129 <m:name/> are an open <em>data format</em> designed for streaming structured data between two processes. |
130 Simultaneously with the format specification, we are also developing a <em>reference implementation</em> (libraries and tools) as a free software. |
130 Simultaneously with the format specification, we are also developing a <em>reference implementation</em> (libraries and tools) as a free software. |
131 Although we believe in the specification-first (or contract-first) approach, we always look and check, whether the theoretic concepts are feasible and whether they can be reasonably and reliably implemented. |
131 Although we believe in the specification-first (or contract-first) approach, we always look and check, whether the theoretic concepts are feasible and whether they can be reasonably and reliably implemented. |
132 So befeore publishing any new specification or its version, we will verify it by creating a reference implementation at least in one programming language. |
132 So befeore publishing any new specification or its version, we will verify it by creating a reference implementation at least in one programming language. |
133 </p> |
133 </p> |
134 <p> |
134 <p> |
135 More generally, <m:name/> are a philosophical continuation of the classic <m:unix/> pipelines and the relational model. |
135 More generally, <m:name/> are a philosophical continuation of the classic <m:unix/> pipelines and the relational model. |
136 </p> |
136 </p> |
137 |
137 |
138 |
138 |
139 <h2>What <m:name/> are not?</h2> |
139 <h2>What <m:name/> are not?</h2> |
140 |
140 |
141 <p> |
141 <p> |
142 <m:name/> respect the existing ecosystem and are rather an improvement or supplement than a replacement. |
142 <m:name/> respect the existing ecosystem and are rather an improvement or supplement than a replacement. |
143 So the <m:name/> are not a: |
143 So the <m:name/> are not a: |
144 </p> |
144 </p> |
145 |
145 |
146 <ul> |
146 <ul> |
147 <li>Shell – we use existing shells (e.g. GNU Bash), work with any shell and even without a shell (e.g. as a stream format passed through a network or stored in a file).</li> |
147 <li>Shell – we use existing shells (e.g. GNU Bash), work with any shell and even without a shell (e.g. as a stream format passed through a network or stored in a file).</li> |
148 <li>Terminal emulator – same as with shells, we use existing terminals and we can use <m:name/> also outside any terminal; if we interact with the terminal, we use standard means like Unicode, ANSI escape sequences etc.</li> |
148 <li>Terminal emulator – same as with shells, we use existing terminals and we can use <m:name/> also outside any terminal; if we interact with the terminal, we use standard means like Unicode, ANSI escape sequences etc.</li> |
149 <li>IDE – we can use standard <m:unix/> tools as an IDE (GNU Screen, Emacs, Make etc.) or any other IDE.</li> |
149 <li>IDE – we can use standard <m:unix/> tools as an IDE (GNU Screen, Emacs, Make etc.) or any other IDE.</li> |
150 <li>Programming language – <m:name/> are language-independent data format and can be produced or consumed in any programming language.</li> |
150 <li>Programming language – <m:name/> are language-independent data format and can be produced or consumed in any programming language.</li> |
151 <li>Query language – although some of our tools are doing queries, filtering or transformations, we are not inventing a new query language – instead, we use existing languages like SQL, XPath, Guile/Scheme, AWK or regular expressions.</li> |
151 <li>Query language – although some of our tools are doing queries, filtering or transformations, we are not inventing a new query language – instead, we use existing languages like SQL, XPath, Guile/Scheme, AWK or regular expressions.</li> |
152 <!--<li>Text editor – </li>--> |
152 <!--<li>Text editor – </li>--> |
153 <li>Database system, DBMS – we focus on the stream processing rather than data storage. Although sometimes it makes sense to redirect data to a file and continue with the processing later.</li> |
153 <li>Database system, DBMS – we focus on the stream processing rather than data storage. Although sometimes it makes sense to redirect data to a file and continue with the processing later.</li> |
154 </ul> |
154 </ul> |
155 |
155 |
156 |
156 |
157 <h2>Project status</h2> |
157 <h2>Project status</h2> |
158 |
158 |