relpipe-data/examples-in-sql-selecting-existing-database.xml
author František Kučera <franta-hg@frantovo.cz>
Mon, 21 Feb 2022 00:43:11 +0100
branchv_0
changeset 329 5bc2bb8b7946
parent 297 192b0059a6c4
permissions -rw-r--r--
Release v0.18
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
279
de1b49ba06f1 examples: SELECTing from SQLite databases
František Kučera <franta-hg@frantovo.cz>
parents:
diff changeset
     1
<stránka
de1b49ba06f1 examples: SELECTing from SQLite databases
František Kučera <franta-hg@frantovo.cz>
parents:
diff changeset
     2
	xmlns="https://trac.frantovo.cz/xml-web-generator/wiki/xmlns/strana"
de1b49ba06f1 examples: SELECTing from SQLite databases
František Kučera <franta-hg@frantovo.cz>
parents:
diff changeset
     3
	xmlns:m="https://trac.frantovo.cz/xml-web-generator/wiki/xmlns/makro">
de1b49ba06f1 examples: SELECTing from SQLite databases
František Kučera <franta-hg@frantovo.cz>
parents:
diff changeset
     4
	
de1b49ba06f1 examples: SELECTing from SQLite databases
František Kučera <franta-hg@frantovo.cz>
parents:
diff changeset
     5
	<nadpis>SELECTing from SQLite databases</nadpis>
de1b49ba06f1 examples: SELECTing from SQLite databases
František Kučera <franta-hg@frantovo.cz>
parents:
diff changeset
     6
	<perex>read from an existing SQLite file or update it</perex>
de1b49ba06f1 examples: SELECTing from SQLite databases
František Kučera <franta-hg@frantovo.cz>
parents:
diff changeset
     7
	<m:pořadí-příkladu>03300</m:pořadí-příkladu>
de1b49ba06f1 examples: SELECTing from SQLite databases
František Kučera <franta-hg@frantovo.cz>
parents:
diff changeset
     8
de1b49ba06f1 examples: SELECTing from SQLite databases
František Kučera <franta-hg@frantovo.cz>
parents:
diff changeset
     9
	<text xmlns="http://www.w3.org/1999/xhtml">
de1b49ba06f1 examples: SELECTing from SQLite databases
František Kučera <franta-hg@frantovo.cz>
parents:
diff changeset
    10
		
de1b49ba06f1 examples: SELECTing from SQLite databases
František Kučera <franta-hg@frantovo.cz>
parents:
diff changeset
    11
		<p>
de1b49ba06f1 examples: SELECTing from SQLite databases
František Kučera <franta-hg@frantovo.cz>
parents:
diff changeset
    12
			Both the <code>relpipe-tr-sql</code> and <code>relpipe-in-sql</code>
297
192b0059a6c4 Release v0.16 – ODBC in SQL + JACK MIDI
František Kučera <franta-hg@frantovo.cz>
parents: 279
diff changeset
    13
			support options <code>--data-source-name</code> and <code>--data-source-string</code>
192b0059a6c4 Release v0.16 – ODBC in SQL + JACK MIDI
František Kučera <franta-hg@frantovo.cz>
parents: 279
diff changeset
    14
			that can be used for connecting to a different database than the in-memory SQLite one.
192b0059a6c4 Release v0.16 – ODBC in SQL + JACK MIDI
František Kučera <franta-hg@frantovo.cz>
parents: 279
diff changeset
    15
			This is useful if we want to offload to HDD and be able to process more data whithout consuming too much RAM.
192b0059a6c4 Release v0.16 – ODBC in SQL + JACK MIDI
František Kučera <franta-hg@frantovo.cz>
parents: 279
diff changeset
    16
			Or if we want to access an existing database instead of just transforming records from the STDIN to STDOUT.
192b0059a6c4 Release v0.16 – ODBC in SQL + JACK MIDI
František Kučera <franta-hg@frantovo.cz>
parents: 279
diff changeset
    17
			These options allow connecting to any DBMS.
279
de1b49ba06f1 examples: SELECTing from SQLite databases
František Kučera <franta-hg@frantovo.cz>
parents:
diff changeset
    18
		</p>
de1b49ba06f1 examples: SELECTing from SQLite databases
František Kučera <franta-hg@frantovo.cz>
parents:
diff changeset
    19
		
de1b49ba06f1 examples: SELECTing from SQLite databases
František Kučera <franta-hg@frantovo.cz>
parents:
diff changeset
    20
		<p>
de1b49ba06f1 examples: SELECTing from SQLite databases
František Kučera <franta-hg@frantovo.cz>
parents:
diff changeset
    21
			Thus we can use the <code>relpipe-in-sql</code> as a database client to access existing SQLite files and SELECT from them (and then convert the results to any supported format).
297
192b0059a6c4 Release v0.16 – ODBC in SQL + JACK MIDI
František Kučera <franta-hg@frantovo.cz>
parents: 279
diff changeset
    22
			And we can use the <code>relpipe-tr-sql</code> as an output filter which converts relational data to a SQLite file that can be queried later.
279
de1b49ba06f1 examples: SELECTing from SQLite databases
František Kučera <franta-hg@frantovo.cz>
parents:
diff changeset
    23
		</p>
de1b49ba06f1 examples: SELECTing from SQLite databases
František Kučera <franta-hg@frantovo.cz>
parents:
diff changeset
    24
		
de1b49ba06f1 examples: SELECTing from SQLite databases
František Kučera <franta-hg@frantovo.cz>
parents:
diff changeset
    25
		<p>
de1b49ba06f1 examples: SELECTing from SQLite databases
František Kučera <franta-hg@frantovo.cz>
parents:
diff changeset
    26
			SQLite is a popular format used by many applications to store structured data.
de1b49ba06f1 examples: SELECTing from SQLite databases
František Kučera <franta-hg@frantovo.cz>
parents:
diff changeset
    27
			Version control systems (VCS) like <a href="https://www.monotone.ca/">Monotone</a> or <a href="https://fossil-scm.org/">Fossil</a>
de1b49ba06f1 examples: SELECTing from SQLite databases
František Kučera <franta-hg@frantovo.cz>
parents:
diff changeset
    28
			use it as a repository format and put whole history, data and metadata into it.
de1b49ba06f1 examples: SELECTing from SQLite databases
František Kučera <franta-hg@frantovo.cz>
parents:
diff changeset
    29
		</p>
de1b49ba06f1 examples: SELECTing from SQLite databases
František Kučera <franta-hg@frantovo.cz>
parents:
diff changeset
    30
		
de1b49ba06f1 examples: SELECTing from SQLite databases
František Kučera <franta-hg@frantovo.cz>
parents:
diff changeset
    31
		<p>
de1b49ba06f1 examples: SELECTing from SQLite databases
František Kučera <franta-hg@frantovo.cz>
parents:
diff changeset
    32
			So if we clone a Fossil repository (the SQLite one in this case):
de1b49ba06f1 examples: SELECTing from SQLite databases
František Kučera <franta-hg@frantovo.cz>
parents:
diff changeset
    33
		</p>
de1b49ba06f1 examples: SELECTing from SQLite databases
František Kučera <franta-hg@frantovo.cz>
parents:
diff changeset
    34
		
de1b49ba06f1 examples: SELECTing from SQLite databases
František Kučera <franta-hg@frantovo.cz>
parents:
diff changeset
    35
		<m:pre jazyk="bash"><![CDATA[fossil clone http://www.sqlite.org/cgi/src sqlite.fossil]]></m:pre>
de1b49ba06f1 examples: SELECTing from SQLite databases
František Kučera <franta-hg@frantovo.cz>
parents:
diff changeset
    36
		
de1b49ba06f1 examples: SELECTing from SQLite databases
František Kučera <franta-hg@frantovo.cz>
parents:
diff changeset
    37
		<p>
297
192b0059a6c4 Release v0.16 – ODBC in SQL + JACK MIDI
František Kučera <franta-hg@frantovo.cz>
parents: 279
diff changeset
    38
			We can then query the database file <code>sqlite.fossil</code> using <m:name/> tools:
279
de1b49ba06f1 examples: SELECTing from SQLite databases
František Kučera <franta-hg@frantovo.cz>
parents:
diff changeset
    39
		</p>
de1b49ba06f1 examples: SELECTing from SQLite databases
František Kučera <franta-hg@frantovo.cz>
parents:
diff changeset
    40
		
de1b49ba06f1 examples: SELECTing from SQLite databases
František Kučera <franta-hg@frantovo.cz>
parents:
diff changeset
    41
		<m:pre jazyk="bash"><![CDATA[relpipe-in-sql \
297
192b0059a6c4 Release v0.16 – ODBC in SQL + JACK MIDI
František Kučera <franta-hg@frantovo.cz>
parents: 279
diff changeset
    42
	--data-source-string 'Driver=SQLite3;Database=file:sqlite.fossil' \
279
de1b49ba06f1 examples: SELECTing from SQLite databases
František Kučera <franta-hg@frantovo.cz>
parents:
diff changeset
    43
	--relation tickets "
de1b49ba06f1 examples: SELECTing from SQLite databases
František Kučera <franta-hg@frantovo.cz>
parents:
diff changeset
    44
		SELECT 
de1b49ba06f1 examples: SELECTing from SQLite databases
František Kučera <franta-hg@frantovo.cz>
parents:
diff changeset
    45
			type,
de1b49ba06f1 examples: SELECTing from SQLite databases
František Kučera <franta-hg@frantovo.cz>
parents:
diff changeset
    46
			status,
de1b49ba06f1 examples: SELECTing from SQLite databases
František Kučera <franta-hg@frantovo.cz>
parents:
diff changeset
    47
			count(*) AS count
de1b49ba06f1 examples: SELECTing from SQLite databases
František Kučera <franta-hg@frantovo.cz>
parents:
diff changeset
    48
		FROM ticket 
de1b49ba06f1 examples: SELECTing from SQLite databases
František Kučera <franta-hg@frantovo.cz>
parents:
diff changeset
    49
		GROUP BY type, status
de1b49ba06f1 examples: SELECTing from SQLite databases
František Kučera <franta-hg@frantovo.cz>
parents:
diff changeset
    50
		ORDER BY count DESC;
de1b49ba06f1 examples: SELECTing from SQLite databases
František Kučera <franta-hg@frantovo.cz>
parents:
diff changeset
    51
	" | relpipe-out-tabular]]></m:pre>
de1b49ba06f1 examples: SELECTing from SQLite databases
František Kučera <franta-hg@frantovo.cz>
parents:
diff changeset
    52
	
de1b49ba06f1 examples: SELECTing from SQLite databases
František Kučera <franta-hg@frantovo.cz>
parents:
diff changeset
    53
		<p>and get some aggregated statistics:</p>
de1b49ba06f1 examples: SELECTing from SQLite databases
František Kučera <franta-hg@frantovo.cz>
parents:
diff changeset
    54
		
de1b49ba06f1 examples: SELECTing from SQLite databases
František Kučera <franta-hg@frantovo.cz>
parents:
diff changeset
    55
		<m:pre jazyk="text"><![CDATA[tickets:
de1b49ba06f1 examples: SELECTing from SQLite databases
František Kučera <franta-hg@frantovo.cz>
parents:
diff changeset
    56
 ╭───────────────────┬─────────────────┬────────────────╮
de1b49ba06f1 examples: SELECTing from SQLite databases
František Kučera <franta-hg@frantovo.cz>
parents:
diff changeset
    57
 │ type     (string) │ status (string) │ count (string) │
de1b49ba06f1 examples: SELECTing from SQLite databases
František Kučera <franta-hg@frantovo.cz>
parents:
diff changeset
    58
 ├───────────────────┼─────────────────┼────────────────┤
de1b49ba06f1 examples: SELECTing from SQLite databases
František Kučera <franta-hg@frantovo.cz>
parents:
diff changeset
    59
 │ Code_Defect       │ Fixed           │ 394            │
de1b49ba06f1 examples: SELECTing from SQLite databases
František Kučera <franta-hg@frantovo.cz>
parents:
diff changeset
    60
 │ Code_Defect       │ Closed          │ 115            │
de1b49ba06f1 examples: SELECTing from SQLite databases
František Kučera <franta-hg@frantovo.cz>
parents:
diff changeset
    61
 │ Feature_Request   │ Closed          │ 36             │
de1b49ba06f1 examples: SELECTing from SQLite databases
František Kučera <franta-hg@frantovo.cz>
parents:
diff changeset
    62
 │ Feature_Request   │ Fixed           │ 16             │
de1b49ba06f1 examples: SELECTing from SQLite databases
František Kučera <franta-hg@frantovo.cz>
parents:
diff changeset
    63
 │ Build_Problem     │ Closed          │ 10             │
de1b49ba06f1 examples: SELECTing from SQLite databases
František Kučera <franta-hg@frantovo.cz>
parents:
diff changeset
    64
 │ Documentation     │ Fixed           │ 10             │
de1b49ba06f1 examples: SELECTing from SQLite databases
František Kučera <franta-hg@frantovo.cz>
parents:
diff changeset
    65
 │ Documentation     │ Closed          │ 8              │
de1b49ba06f1 examples: SELECTing from SQLite databases
František Kučera <franta-hg@frantovo.cz>
parents:
diff changeset
    66
 │ Incident          │ Closed          │ 4              │
de1b49ba06f1 examples: SELECTing from SQLite databases
František Kučera <franta-hg@frantovo.cz>
parents:
diff changeset
    67
 │ Build_Problem     │ Fixed           │ 3              │
de1b49ba06f1 examples: SELECTing from SQLite databases
František Kučera <franta-hg@frantovo.cz>
parents:
diff changeset
    68
 │ Performance_Issue │ Closed          │ 3              │
de1b49ba06f1 examples: SELECTing from SQLite databases
František Kučera <franta-hg@frantovo.cz>
parents:
diff changeset
    69
 │ Support_Request   │ Closed          │ 3              │
de1b49ba06f1 examples: SELECTing from SQLite databases
František Kučera <franta-hg@frantovo.cz>
parents:
diff changeset
    70
 │ Code_Defect       │ Open            │ 2              │
de1b49ba06f1 examples: SELECTing from SQLite databases
František Kučera <franta-hg@frantovo.cz>
parents:
diff changeset
    71
 │ Feature_Request   │ Open            │ 2              │
de1b49ba06f1 examples: SELECTing from SQLite databases
František Kučera <franta-hg@frantovo.cz>
parents:
diff changeset
    72
 │ Performance_Issue │ Fixed           │ 2              │
de1b49ba06f1 examples: SELECTing from SQLite databases
František Kučera <franta-hg@frantovo.cz>
parents:
diff changeset
    73
 │ Code_Defect       │ Deferred        │ 1              │
de1b49ba06f1 examples: SELECTing from SQLite databases
František Kučera <franta-hg@frantovo.cz>
parents:
diff changeset
    74
 │ Compiler_Warning  │ Closed          │ 1              │
de1b49ba06f1 examples: SELECTing from SQLite databases
František Kučera <franta-hg@frantovo.cz>
parents:
diff changeset
    75
 │ Compiler_Warning  │ Fixed           │ 1              │
de1b49ba06f1 examples: SELECTing from SQLite databases
František Kučera <franta-hg@frantovo.cz>
parents:
diff changeset
    76
 │ Not_A_Bug         │ Closed          │ 1              │
de1b49ba06f1 examples: SELECTing from SQLite databases
František Kučera <franta-hg@frantovo.cz>
parents:
diff changeset
    77
 │ Performance_Issue │ Open            │ 1              │
de1b49ba06f1 examples: SELECTing from SQLite databases
František Kučera <franta-hg@frantovo.cz>
parents:
diff changeset
    78
 │ Portability       │ Fixed           │ 1              │
de1b49ba06f1 examples: SELECTing from SQLite databases
František Kučera <franta-hg@frantovo.cz>
parents:
diff changeset
    79
 ╰───────────────────┴─────────────────┴────────────────╯
de1b49ba06f1 examples: SELECTing from SQLite databases
František Kučera <franta-hg@frantovo.cz>
parents:
diff changeset
    80
Record count: 20]]></m:pre>
de1b49ba06f1 examples: SELECTing from SQLite databases
František Kučera <franta-hg@frantovo.cz>
parents:
diff changeset
    81
		
de1b49ba06f1 examples: SELECTing from SQLite databases
František Kučera <franta-hg@frantovo.cz>
parents:
diff changeset
    82
		<p>
de1b49ba06f1 examples: SELECTing from SQLite databases
František Kučera <franta-hg@frantovo.cz>
parents:
diff changeset
    83
			Accessing internal data structures of another application might be bit tricky (if the application does not guarantee it as a stable API),
de1b49ba06f1 examples: SELECTing from SQLite databases
František Kučera <franta-hg@frantovo.cz>
parents:
diff changeset
    84
			but it can be used for ad-hoc digging and even in regular use, there are chances that such „API“ will not break for a reasonable period of time.
de1b49ba06f1 examples: SELECTing from SQLite databases
František Kučera <franta-hg@frantovo.cz>
parents:
diff changeset
    85
		</p>
de1b49ba06f1 examples: SELECTing from SQLite databases
František Kučera <franta-hg@frantovo.cz>
parents:
diff changeset
    86
		
de1b49ba06f1 examples: SELECTing from SQLite databases
František Kučera <franta-hg@frantovo.cz>
parents:
diff changeset
    87
		<p>
de1b49ba06f1 examples: SELECTing from SQLite databases
František Kučera <franta-hg@frantovo.cz>
parents:
diff changeset
    88
			However, best results can be achieved, if the database scheme is under our control or defined as a stable API by someone else.
de1b49ba06f1 examples: SELECTing from SQLite databases
František Kučera <franta-hg@frantovo.cz>
parents:
diff changeset
    89
		</p>
de1b49ba06f1 examples: SELECTing from SQLite databases
František Kučera <franta-hg@frantovo.cz>
parents:
diff changeset
    90
		
de1b49ba06f1 examples: SELECTing from SQLite databases
František Kučera <franta-hg@frantovo.cz>
parents:
diff changeset
    91
		<p>
de1b49ba06f1 examples: SELECTing from SQLite databases
František Kučera <franta-hg@frantovo.cz>
parents:
diff changeset
    92
			All relations or their subset can be dumped even without writing any SELECTs: just use <code>--copy '.*'</code>
de1b49ba06f1 examples: SELECTing from SQLite databases
František Kučera <franta-hg@frantovo.cz>
parents:
diff changeset
    93
			or e.g. <code>--copy 'tickets|tag|evet'</code> or <code>--copy-renamed 'tickets|tag|evet' 'copy_of_$0'</code> and get exported some relations under modified names (using regular expressions).
de1b49ba06f1 examples: SELECTing from SQLite databases
František Kučera <franta-hg@frantovo.cz>
parents:
diff changeset
    94
			Such export can be converted e.g. to Recfile or XML format using corresponding output filter.
de1b49ba06f1 examples: SELECTing from SQLite databases
František Kučera <franta-hg@frantovo.cz>
parents:
diff changeset
    95
		</p>
de1b49ba06f1 examples: SELECTing from SQLite databases
František Kučera <franta-hg@frantovo.cz>
parents:
diff changeset
    96
		
de1b49ba06f1 examples: SELECTing from SQLite databases
František Kučera <franta-hg@frantovo.cz>
parents:
diff changeset
    97
	</text>
de1b49ba06f1 examples: SELECTing from SQLite databases
František Kučera <franta-hg@frantovo.cz>
parents:
diff changeset
    98
de1b49ba06f1 examples: SELECTing from SQLite databases
František Kučera <franta-hg@frantovo.cz>
parents:
diff changeset
    99
</stránka>