principles: streaming v_0
authorFrantišek Kučera <franta-hg@frantovo.cz>
Thu, 06 Dec 2018 18:38:34 +0100
branchv_0
changeset 188 5b0fab48d59e
parent 187 c952261978e8
child 189 47907749817f
principles: streaming
relpipe-data/principles.xml
--- a/relpipe-data/principles.xml	Thu Dec 06 17:11:29 2018 +0100
+++ b/relpipe-data/principles.xml	Thu Dec 06 18:38:34 2018 +0100
@@ -124,6 +124,26 @@
 			The data that are not written don't need to be compressed and thus have the best compression ratio.
 		</p>
 		
+		<h2>Streaming</h2>
+		
+		<p>
+			Relational tools should process streams of data and should hold only necessary data in the memory
+			i.e. the tool should produce the output (the first record) as soon as possible while still reading the input (following records).
+			Thus the memory usage does not depend on the volume of processed data.
+		</p>
+		
+		<p>
+			However, there are cases where such streaming is not feasible e.g. if we need to compute some statistics or a column widths while printing a table in the terminal.
+			In such situation, we must read the whole relation and only then generate the output.
+			But we should still be able to do streaming on the relations level e.i. if there are more relation, we always hold only one of them in the memory.
+		</p>
+		
+		<p>
+			This rule is important not only from the performance point of view but also for user experience.
+			The user should see the output as soon as possible i.e. the longer running processes will produce result continuously instead of flushing everything at the end.
+			This is also good for debugging and <em>looking inside the things</em>. 
+		</p>
+		
 		<h2>Unambiguity</h2>
 		
 		<p>