relpipe-data/faq.xml
author František Kučera <franta-hg@frantovo.cz>
Tue, 24 Sep 2019 17:07:22 +0200
branchv_0
changeset 270 e6bac9ff0d87
parent 226 fc68cd31db78
child 275 1cdb74e845d0
permissions -rw-r--r--
Sane software manifesto
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
23
0d2729ed16ed zkouška interního odkazu
František Kučera <franta-hg@frantovo.cz>
parents: 18
diff changeset
     1
<stránka
0d2729ed16ed zkouška interního odkazu
František Kučera <franta-hg@frantovo.cz>
parents: 18
diff changeset
     2
	xmlns="https://trac.frantovo.cz/xml-web-generator/wiki/xmlns/strana"
0d2729ed16ed zkouška interního odkazu
František Kučera <franta-hg@frantovo.cz>
parents: 18
diff changeset
     3
	xmlns:m="https://trac.frantovo.cz/xml-web-generator/wiki/xmlns/makro">
0d2729ed16ed zkouška interního odkazu
František Kučera <franta-hg@frantovo.cz>
parents: 18
diff changeset
     4
	
147
c004a45502b3 new pages: principles, roadmap, faq
František Kučera <franta-hg@frantovo.cz>
parents: 139
diff changeset
     5
	<nadpis>FAQ</nadpis>
c004a45502b3 new pages: principles, roadmap, faq
František Kučera <franta-hg@frantovo.cz>
parents: 139
diff changeset
     6
	<perex>Frequently asked questions</perex>
c004a45502b3 new pages: principles, roadmap, faq
František Kučera <franta-hg@frantovo.cz>
parents: 139
diff changeset
     7
	<pořadí>16</pořadí>
4
1bb39595a51c genrování hlavní nabídky #1
František Kučera <franta-hg@frantovo.cz>
parents: 2
diff changeset
     8
2
ab9099ff88fa vkládání zápatí, jmenné prostory, saxon
František Kučera <franta-hg@frantovo.cz>
parents: 1
diff changeset
     9
	<text xmlns="http://www.w3.org/1999/xhtml">
149
František Kučera <franta-hg@frantovo.cz>
parents: 147
diff changeset
    10
		
2
ab9099ff88fa vkládání zápatí, jmenné prostory, saxon
František Kučera <franta-hg@frantovo.cz>
parents: 1
diff changeset
    11
		<p>
149
František Kučera <franta-hg@frantovo.cz>
parents: 147
diff changeset
    12
			<strong>When the stable version will be released?</strong>		
František Kučera <franta-hg@frantovo.cz>
parents: 147
diff changeset
    13
			<br/>
František Kučera <franta-hg@frantovo.cz>
parents: 147
diff changeset
    14
			We don't know – there is no exact date.
František Kučera <franta-hg@frantovo.cz>
parents: 147
diff changeset
    15
			<m:name/> are something that should be released about twenty years ago. But real work started in 2018.
František Kučera <franta-hg@frantovo.cz>
parents: 147
diff changeset
    16
			So it is not a big difference whether it will be released this month or the next one.
František Kučera <franta-hg@frantovo.cz>
parents: 147
diff changeset
    17
			We understand the <em>release early, release often</em> rule.
František Kučera <franta-hg@frantovo.cz>
parents: 147
diff changeset
    18
			But it fits better to application software than to standards and APIs.
151
5697a01db388 roadmap
František Kučera <franta-hg@frantovo.cz>
parents: 149
diff changeset
    19
			Of course, we expect some evolution after the v1.0.0 release, but we need to stabilize and verify many things before the release in order to be able to maintain backward compatibility in future.
149
František Kučera <franta-hg@frantovo.cz>
parents: 147
diff changeset
    20
		</p>
František Kučera <franta-hg@frantovo.cz>
parents: 147
diff changeset
    21
		
František Kučera <franta-hg@frantovo.cz>
parents: 147
diff changeset
    22
		<p>
František Kučera <franta-hg@frantovo.cz>
parents: 147
diff changeset
    23
			<strong>How can I help you?</strong>		
František Kučera <franta-hg@frantovo.cz>
parents: 147
diff changeset
    24
			<br/>
177
a64671d5f997 faq: help
František Kučera <franta-hg@frantovo.cz>
parents: 164
diff changeset
    25
			<ul>
a64671d5f997 faq: help
František Kučera <franta-hg@frantovo.cz>
parents: 164
diff changeset
    26
				<li>Suggest more examples how <m:name/> can be used; especially how YOU would like to use it.</li>
a64671d5f997 faq: help
František Kučera <franta-hg@frantovo.cz>
parents: 164
diff changeset
    27
				<li>We are looking for illustrations that would supplement our documentation and website.</li>
a64671d5f997 faq: help
František Kučera <franta-hg@frantovo.cz>
parents: 164
diff changeset
    28
				<li>
a64671d5f997 faq: help
František Kučera <franta-hg@frantovo.cz>
parents: 164
diff changeset
    29
					As an author of a program that generates or consumes some data, you could add relational input and output to your program. 
a64671d5f997 faq: help
František Kučera <franta-hg@frantovo.cz>
parents: 164
diff changeset
    30
					But please mention that we do not have v1.0 yet, so these features should be marked as experimental. 
a64671d5f997 faq: help
František Kučera <franta-hg@frantovo.cz>
parents: 164
diff changeset
    31
					The API might/will change. 
a64671d5f997 faq: help
František Kučera <franta-hg@frantovo.cz>
parents: 164
diff changeset
    32
					Other (and maybe better for now) option is to add input/output of values separated by null byte (<code>\0</code>).
a64671d5f997 faq: help
František Kučera <franta-hg@frantovo.cz>
parents: 164
diff changeset
    33
					This "API" will be supported for sure and data are simply the attribute values. There are no record separators (we know the number of attributes, so they are not needed).
a64671d5f997 faq: help
František Kučera <franta-hg@frantovo.cz>
parents: 164
diff changeset
    34
					Disadvantage of this approach is that the stream can contain only a single relation; and that the metadata are not embedded in the stream and must be passed separately.
a64671d5f997 faq: help
František Kučera <franta-hg@frantovo.cz>
parents: 164
diff changeset
    35
				</li>
a64671d5f997 faq: help
František Kučera <franta-hg@frantovo.cz>
parents: 164
diff changeset
    36
				<li>Review our source code and suggest improvements and fixes. Constructive criticism is always welcome. This is one of reasons why we publish our programs as free software.</li>
a64671d5f997 faq: help
František Kučera <franta-hg@frantovo.cz>
parents: 164
diff changeset
    37
				<li>Native speakers could suggest improvements and corrections of our English texts.</li>
a64671d5f997 faq: help
František Kučera <franta-hg@frantovo.cz>
parents: 164
diff changeset
    38
			</ul>
149
František Kučera <franta-hg@frantovo.cz>
parents: 147
diff changeset
    39
		</p>
František Kučera <franta-hg@frantovo.cz>
parents: 147
diff changeset
    40
		
František Kučera <franta-hg@frantovo.cz>
parents: 147
diff changeset
    41
		<p>
164
56eb59640688 faq: relation vs. table
František Kučera <franta-hg@frantovo.cz>
parents: 163
diff changeset
    42
			<strong>Why do you speak about <em>relations</em> instead of <em>tables</em>?</strong>
149
František Kučera <franta-hg@frantovo.cz>
parents: 147
diff changeset
    43
			<br/>
152
f876683324c2 roadmap: more formats
František Kučera <franta-hg@frantovo.cz>
parents: 151
diff changeset
    44
			It might be uncommon terminology for someone, but <em>relations</em> and <em>attributes</em> symbolizes
f876683324c2 roadmap: more formats
František Kučera <franta-hg@frantovo.cz>
parents: 151
diff changeset
    45
			that we focus on substance of the data. Pure data are conveyed through the pipelines 
f876683324c2 roadmap: more formats
František Kučera <franta-hg@frantovo.cz>
parents: 151
diff changeset
    46
			and the presentation of such data is only the last step.
f876683324c2 roadmap: more formats
František Kučera <franta-hg@frantovo.cz>
parents: 151
diff changeset
    47
			The data might be presented/visualized in many various forms.
f876683324c2 roadmap: more formats
František Kučera <franta-hg@frantovo.cz>
parents: 151
diff changeset
    48
			And tables (consisting of rows and columns) are only one of many possible options.
149
František Kučera <franta-hg@frantovo.cz>
parents: 147
diff changeset
    49
		</p>
František Kučera <franta-hg@frantovo.cz>
parents: 147
diff changeset
    50
		
164
56eb59640688 faq: relation vs. table
František Kučera <franta-hg@frantovo.cz>
parents: 163
diff changeset
    51
		<m:tabulka>
56eb59640688 faq: relation vs. table
František Kučera <franta-hg@frantovo.cz>
parents: 163
diff changeset
    52
			Relational	SQL	alternative terms
56eb59640688 faq: relation vs. table
František Kučera <franta-hg@frantovo.cz>
parents: 163
diff changeset
    53
			relation	table
56eb59640688 faq: relation vs. table
František Kučera <franta-hg@frantovo.cz>
parents: 163
diff changeset
    54
			attribute	column	field
56eb59640688 faq: relation vs. table
František Kučera <franta-hg@frantovo.cz>
parents: 163
diff changeset
    55
			record	row	tuple
56eb59640688 faq: relation vs. table
František Kučera <franta-hg@frantovo.cz>
parents: 163
diff changeset
    56
		</m:tabulka>
56eb59640688 faq: relation vs. table
František Kučera <franta-hg@frantovo.cz>
parents: 163
diff changeset
    57
		
224
9ea7e5c65107 faq: duplicate records and the relational model
František Kučera <franta-hg@frantovo.cz>
parents: 200
diff changeset
    58
		<p>
9ea7e5c65107 faq: duplicate records and the relational model
František Kučera <franta-hg@frantovo.cz>
parents: 200
diff changeset
    59
			<strong>What about duplicate records?</strong>
9ea7e5c65107 faq: duplicate records and the relational model
František Kučera <franta-hg@frantovo.cz>
parents: 200
diff changeset
    60
			<br/>
9ea7e5c65107 faq: duplicate records and the relational model
František Kučera <franta-hg@frantovo.cz>
parents: 200
diff changeset
    61
			In the relational model, the records must be unique.
9ea7e5c65107 faq: duplicate records and the relational model
František Kučera <franta-hg@frantovo.cz>
parents: 200
diff changeset
    62
			In <m:name/> there is no central authority that would prevent you from appending duplicate records to the relational stream.
9ea7e5c65107 faq: duplicate records and the relational model
František Kučera <franta-hg@frantovo.cz>
parents: 200
diff changeset
    63
			It means that in some points in the relational pipeline there might occur data that do not fit the rules of the relational model.
9ea7e5c65107 faq: duplicate records and the relational model
František Kučera <franta-hg@frantovo.cz>
parents: 200
diff changeset
    64
			The deduplication is generally not done on the output side of particular steps, but is postponed and done on the input side of steps, where uniqueness is important (e.g. JOIN or UNION).
9ea7e5c65107 faq: duplicate records and the relational model
František Kučera <franta-hg@frantovo.cz>
parents: 200
diff changeset
    65
			You should not put duplicate records in the relational stream, but you can.
9ea7e5c65107 faq: duplicate records and the relational model
František Kučera <franta-hg@frantovo.cz>
parents: 200
diff changeset
    66
			Duplicates can also occur after some transformations like <code>relpipe-tr-cut</code> (e.g. if you choose only <code>dump</code> or <code>type</code> attributes from your <code>fstab</code> and omit the primary/unique key field).
9ea7e5c65107 faq: duplicate records and the relational model
František Kučera <franta-hg@frantovo.cz>
parents: 200
diff changeset
    67
			Such data are not considered invalid, but should be processed like there are no duplicates (if uniqueness is important for particular step)
9ea7e5c65107 faq: duplicate records and the relational model
František Kučera <franta-hg@frantovo.cz>
parents: 200
diff changeset
    68
			or should be passed through if it is not in conflict with the goal of given step (e.g. calling <code>uppercase()</code> function on some field or doing UNION ALL).
9ea7e5c65107 faq: duplicate records and the relational model
František Kučera <franta-hg@frantovo.cz>
parents: 200
diff changeset
    69
			Each tool must document how it handles duplicate records.
9ea7e5c65107 faq: duplicate records and the relational model
František Kučera <franta-hg@frantovo.cz>
parents: 200
diff changeset
    70
		</p>
9ea7e5c65107 faq: duplicate records and the relational model
František Kučera <franta-hg@frantovo.cz>
parents: 200
diff changeset
    71
		
9ea7e5c65107 faq: duplicate records and the relational model
František Kučera <franta-hg@frantovo.cz>
parents: 200
diff changeset
    72
		<p>
9ea7e5c65107 faq: duplicate records and the relational model
František Kučera <franta-hg@frantovo.cz>
parents: 200
diff changeset
    73
			The reasons for this <em>transient tolerance of duplicate records</em> are two.
9ea7e5c65107 faq: duplicate records and the relational model
František Kučera <franta-hg@frantovo.cz>
parents: 200
diff changeset
    74
			1) Performance: guaranteeing the uniqueness in every moment would negate streaming and would require holding whole relation in memory and always sorting the records.
9ea7e5c65107 faq: duplicate records and the relational model
František Kučera <franta-hg@frantovo.cz>
parents: 200
diff changeset
    75
			2) Modularity: many tasks would have to be done by a single bulky tool that does everything e.g. if you want to cut only the <code>type</code> field from your <code>fstab</code> and then count statistics how many times particular filesystems are used.
9ea7e5c65107 faq: duplicate records and the relational model
František Kučera <franta-hg@frantovo.cz>
parents: 200
diff changeset
    76
		</p>
9ea7e5c65107 faq: duplicate records and the relational model
František Kučera <franta-hg@frantovo.cz>
parents: 200
diff changeset
    77
		
152
f876683324c2 roadmap: more formats
František Kučera <franta-hg@frantovo.cz>
parents: 151
diff changeset
    78
		<!--
149
František Kučera <franta-hg@frantovo.cz>
parents: 147
diff changeset
    79
		<p>
František Kučera <franta-hg@frantovo.cz>
parents: 147
diff changeset
    80
			<strong>?</strong>		
František Kučera <franta-hg@frantovo.cz>
parents: 147
diff changeset
    81
			<br/>
František Kučera <franta-hg@frantovo.cz>
parents: 147
diff changeset
    82
			...
87
25dec6931f18 Lepší odsazení, tabulátory.
František Kučera <franta-hg@frantovo.cz>
parents: 23
diff changeset
    83
		</p>
149
František Kučera <franta-hg@frantovo.cz>
parents: 147
diff changeset
    84
		
František Kučera <franta-hg@frantovo.cz>
parents: 147
diff changeset
    85
		<p>
František Kučera <franta-hg@frantovo.cz>
parents: 147
diff changeset
    86
			<strong>Why don't build on XML? It is a standard since 1998 and there are many tools and libraries for it.</strong>		
František Kučera <franta-hg@frantovo.cz>
parents: 147
diff changeset
    87
			<br/>
František Kučera <franta-hg@frantovo.cz>
parents: 147
diff changeset
    88
			XML is a great and mature (meta)format and its ecosystem is respectable and inspiring.
František Kučera <franta-hg@frantovo.cz>
parents: 147
diff changeset
    89
			But the XML does not conform to our <m:a href="principles">principles</m:a>, especially the ability to concatenate multiple files/streams and to append new records to an already existing relation.
František Kučera <franta-hg@frantovo.cz>
parents: 147
diff changeset
    90
			XML is also not concise. 
František Kučera <franta-hg@frantovo.cz>
parents: 147
diff changeset
    91
			And the implementation of the XML parser in various environments would be <em>a bit more complex</em>.
František Kučera <franta-hg@frantovo.cz>
parents: 147
diff changeset
    92
		</p>
František Kučera <franta-hg@frantovo.cz>
parents: 147
diff changeset
    93
		<p>
František Kučera <franta-hg@frantovo.cz>
parents: 147
diff changeset
    94
			We prefer XML as an input and output format and look forward to cooperation with XML ecosystem (XSD, XPath, XSLT, XQuery etc.).
František Kučera <franta-hg@frantovo.cz>
parents: 147
diff changeset
    95
			Such steps might be at the beginning, at the end, or even in the middle of the relational pipeline.
František Kučera <franta-hg@frantovo.cz>
parents: 147
diff changeset
    96
		</p>
František Kučera <franta-hg@frantovo.cz>
parents: 147
diff changeset
    97
		
František Kučera <franta-hg@frantovo.cz>
parents: 147
diff changeset
    98
		<p>
František Kučera <franta-hg@frantovo.cz>
parents: 147
diff changeset
    99
			<strong>?</strong>		
František Kučera <franta-hg@frantovo.cz>
parents: 147
diff changeset
   100
			<br/>
František Kučera <franta-hg@frantovo.cz>
parents: 147
diff changeset
   101
			...
František Kučera <franta-hg@frantovo.cz>
parents: 147
diff changeset
   102
		</p>
František Kučera <franta-hg@frantovo.cz>
parents: 147
diff changeset
   103
		-->
František Kučera <franta-hg@frantovo.cz>
parents: 147
diff changeset
   104
		
163
580c0e195817 faq: why c++
František Kučera <franta-hg@frantovo.cz>
parents: 152
diff changeset
   105
		<p>
580c0e195817 faq: why c++
František Kučera <franta-hg@frantovo.cz>
parents: 152
diff changeset
   106
			<strong>Why C++?</strong>		
580c0e195817 faq: why c++
František Kučera <franta-hg@frantovo.cz>
parents: 152
diff changeset
   107
			<br/>
580c0e195817 faq: why c++
František Kučera <franta-hg@frantovo.cz>
parents: 152
diff changeset
   108
			Firstly, <m:name/> are a specification of a data format and as such are not bound to any programming language.
164
56eb59640688 faq: relation vs. table
František Kučera <franta-hg@frantovo.cz>
parents: 163
diff changeset
   109
			This specification is totally language- and platform- independent.
163
580c0e195817 faq: why c++
František Kučera <franta-hg@frantovo.cz>
parents: 152
diff changeset
   110
		</p>
580c0e195817 faq: why c++
František Kučera <franta-hg@frantovo.cz>
parents: 152
diff changeset
   111
		<p>
580c0e195817 faq: why c++
František Kučera <franta-hg@frantovo.cz>
parents: 152
diff changeset
   112
			The ideal/perfect language does not exist and our implementations will be written in various languages.
200
44661c8005f6 faq: fix english
František Kučera <franta-hg@frantovo.cz>
parents: 182
diff changeset
   113
			We started our prototype and first real implementations in C++ for several reasons:
163
580c0e195817 faq: why c++
František Kučera <franta-hg@frantovo.cz>
parents: 152
diff changeset
   114
		</p>
580c0e195817 faq: why c++
František Kučera <franta-hg@frantovo.cz>
parents: 152
diff changeset
   115
		<ul>
580c0e195817 faq: why c++
František Kučera <franta-hg@frantovo.cz>
parents: 152
diff changeset
   116
			<li>It is mature and widespread: GCC runs almost everywhere and other compilers/toolchains are also available.</li>
580c0e195817 faq: why c++
František Kučera <franta-hg@frantovo.cz>
parents: 152
diff changeset
   117
			<li>Programs written in C++ starts immediately: very important for CLI tools.</li>
580c0e195817 faq: why c++
František Kučera <franta-hg@frantovo.cz>
parents: 152
diff changeset
   118
			<li>Can be seamlessly mixed with C and its libraries. Is good for interaction with the operating system.</li>
580c0e195817 faq: why c++
František Kučera <franta-hg@frantovo.cz>
parents: 152
diff changeset
   119
			<li>Modern C++ is a quite good language.</li>
580c0e195817 faq: why c++
František Kučera <franta-hg@frantovo.cz>
parents: 152
diff changeset
   120
			<li>We are not C++ gurus and C++ is not our first-choice language i.e. the fact that we are able to do implementation in C++ proves that the specification is simple enough to be reasonably implemented by an average software engineer in any other language :-)</li>
580c0e195817 faq: why c++
František Kučera <franta-hg@frantovo.cz>
parents: 152
diff changeset
   121
		</ul>
580c0e195817 faq: why c++
František Kučera <franta-hg@frantovo.cz>
parents: 152
diff changeset
   122
		
580c0e195817 faq: why c++
František Kučera <franta-hg@frantovo.cz>
parents: 152
diff changeset
   123
		<p>Implementation in other languages will follow. Java is the next one. Then probably Perl, Python, Rust, Go, PHP etc. (depends on community involvement).</p>
580c0e195817 faq: why c++
František Kučera <franta-hg@frantovo.cz>
parents: 152
diff changeset
   124
		
149
František Kučera <franta-hg@frantovo.cz>
parents: 147
diff changeset
   125
		<p>			
František Kučera <franta-hg@frantovo.cz>
parents: 147
diff changeset
   126
			<strong>Have you seen <a href="https://xkcd.com/927/">XKCD 927</a>?</strong>		
František Kučera <franta-hg@frantovo.cz>
parents: 147
diff changeset
   127
			<br/>
František Kučera <franta-hg@frantovo.cz>
parents: 147
diff changeset
   128
			Yes. And we liked it so much that we followed their instructions and created <m:name/>.
František Kučera <franta-hg@frantovo.cz>
parents: 147
diff changeset
   129
		</p>
František Kučera <franta-hg@frantovo.cz>
parents: 147
diff changeset
   130
			
František Kučera <franta-hg@frantovo.cz>
parents: 147
diff changeset
   131
		<p>
František Kučera <franta-hg@frantovo.cz>
parents: 147
diff changeset
   132
			<strong>Are <m:name/> compatible with cloud, IoT, SPA/PWA, AI, blockchain and mobile-first? Should our DevOps use it in our serverless hipster fintech app with strong focus on SEO, UX and machine learning?</strong>
František Kučera <franta-hg@frantovo.cz>
parents: 147
diff changeset
   133
			<br/>
František Kučera <franta-hg@frantovo.cz>
parents: 147
diff changeset
   134
			Go @#$%&amp; yourself. We are pretty old school hackers and we enjoy our green screen terminals!<br/>
František Kučera <franta-hg@frantovo.cz>
parents: 147
diff changeset
   135
			Of course, you can use <m:name/> anywhere if it makes sense for you.
František Kučera <franta-hg@frantovo.cz>
parents: 147
diff changeset
   136
			<m:name/> are designed to be generic enough – i.e. not specific to any industry (banking, telecommunications, embedded etc.) nor platform.
163
580c0e195817 faq: why c++
František Kučera <franta-hg@frantovo.cz>
parents: 152
diff changeset
   137
			Data in this format are very concise, so can be used even in very small devices.
149
František Kučera <franta-hg@frantovo.cz>
parents: 147
diff changeset
   138
			Its native data structure is a relation (table) but it can also handle tree-structured data (i.e. any data).
František Kučera <franta-hg@frantovo.cz>
parents: 147
diff changeset
   139
			It is designed rather for streaming than for storage (but under some circumstances it is also meaningful to use it for storage).
František Kučera <franta-hg@frantovo.cz>
parents: 147
diff changeset
   140
		</p>
František Kučera <franta-hg@frantovo.cz>
parents: 147
diff changeset
   141
		
182
17caf5443ff7 faq: hobby
František Kučera <franta-hg@frantovo.cz>
parents: 177
diff changeset
   142
		<p>
17caf5443ff7 faq: hobby
František Kučera <franta-hg@frantovo.cz>
parents: 177
diff changeset
   143
			<strong>What about your hobbies?</strong>
17caf5443ff7 faq: hobby
František Kučera <franta-hg@frantovo.cz>
parents: 177
diff changeset
   144
			<br/>
226
fc68cd31db78 move common XML code to relpipe-lib-xmlwriter (a header-only library)
František Kučera <franta-hg@frantovo.cz>
parents: 224
diff changeset
   145
			It is a bit personal question, but I can unveil that I collect signed photos of Ally Sheedy, Winona Ryder and Richard Stallman.
182
17caf5443ff7 faq: hobby
František Kučera <franta-hg@frantovo.cz>
parents: 177
diff changeset
   146
		</p>
17caf5443ff7 faq: hobby
František Kučera <franta-hg@frantovo.cz>
parents: 177
diff changeset
   147
		
87
25dec6931f18 Lepší odsazení, tabulátory.
František Kučera <franta-hg@frantovo.cz>
parents: 23
diff changeset
   148
	</text>
4
1bb39595a51c genrování hlavní nabídky #1
František Kučera <franta-hg@frantovo.cz>
parents: 2
diff changeset
   149
1
a05c6f3cbc3e základ, první verze
František Kučera <franta-hg@frantovo.cz>
parents:
diff changeset
   150
</stránka>