--- a/relpipe-data/principles.xml Thu Dec 06 17:11:29 2018 +0100
+++ b/relpipe-data/principles.xml Thu Dec 06 18:38:34 2018 +0100
@@ -124,6 +124,26 @@
The data that are not written don't need to be compressed and thus have the best compression ratio.
</p>
+ <h2>Streaming</h2>
+
+ <p>
+ Relational tools should process streams of data and should hold only necessary data in the memory
+ i.e. the tool should produce the output (the first record) as soon as possible while still reading the input (following records).
+ Thus the memory usage does not depend on the volume of processed data.
+ </p>
+
+ <p>
+ However, there are cases where such streaming is not feasible e.g. if we need to compute some statistics or a column widths while printing a table in the terminal.
+ In such situation, we must read the whole relation and only then generate the output.
+ But we should still be able to do streaming on the relations level e.i. if there are more relation, we always hold only one of them in the memory.
+ </p>
+
+ <p>
+ This rule is important not only from the performance point of view but also for user experience.
+ The user should see the output as soon as possible i.e. the longer running processes will produce result continuously instead of flushing everything at the end.
+ This is also good for debugging and <em>looking inside the things</em>.
+ </p>
+
<h2>Unambiguity</h2>
<p>