<stránka

	xmlns="https://trac.frantovo.cz/xml-web-generator/wiki/xmlns/strana"

	xmlns:m="https://trac.frantovo.cz/xml-web-generator/wiki/xmlns/makro">

	<nadpis>Reading SQL scripts</nadpis>

	<perex>read an SQL file and display it in a tabular way</perex>

	<m:pořadí-příkladu>03200</m:pořadí-příkladu>

	<text xmlns="http://www.w3.org/1999/xhtml">

		<p>

			SQL scripts containing

			DDL (Data Definition Language) and DML (Data Manipulation Language)

			contain both data structures (relations) and data (records).

			Simple example:

		</p>

		<m:pre jazyk="sql" src="examples/relpipe-in-sql-1.sql"/>

		<p>

			We can read such data using the <code>relpipe-in-sql</code> command 

			in a similar way we read CSV, XML or Recfile streams – just pipe the stream into the particular input filter

			and let it convert data to the relational format:

		</p>

		<m:pre jazyk="bash"><![CDATA[cat relpipe-in-sql-1.sql | relpipe-in-sql | relpipe-out-tabular]]></m:pre>

		<p>

			And in the next step we use an output filter and covert relational data to some other format e.g. the tabular output displayed in our terminal:

		</p>

		<m:pre jazyk="text" src="examples/relpipe-in-sql-1.txt"/>

		<p>

			Of course, we can add further steps in our pipeline and use any transformation tool for filtering or modifying data:

		</p>

		<m:pre jazyk="bash"><![CDATA[# AWK transformation:

cat relpipe-in-sql-1.sql \

	| relpipe-in-sql \

	| relpipe-tr-awk \

		--relation 'a' --where 'message == "Hello,"' \

		--relation '.*' --drop \

	| relpipe-out-tabular

# Scheme transformation:

cat relpipe-in-sql-1.sql \

	| relpipe-in-sql \

	| relpipe-tr-scheme \

		--relation 'a' --where '(string= $message "Hello,")' \

		--relation '.*' --drop \

	| relpipe-out-tabular]]></m:pre>

		<p>and get filtered output:</p>

		<m:pre jazyk="text" src="examples/relpipe-in-sql-1.filtered.txt"/>

		<p>

			However, it is usually not necessary, because once we have data in an in-memory database (which happens on-the-fly in the <code>relpipe-in-sql</code> step),

			we can use the SQL language for filtering and transformations and get the same output as above:

		</p>

		<m:pre jazyk="bash"><![CDATA[cat relpipe-in-sql-1.sql \

	| relpipe-in-sql \

		--relation 'a' "SELECT * FROM a WHERE message = 'Hello,'" \

	| relpipe-out-tabular]]></m:pre>

	<p>

		Actually, the <code>relpipe-in-sql</code> is just a symbolic link to the <code>relpipe-tr-sql</code> 

		and have the same capabilities (with just bit different default behavior to match general logic of the input filters).

		So if we do not need special feautres of Scheme, AWK or other transformation tool, it is better to stay with SQL (when we already use the SQL module).

	</p>

	<p>

		The difference between <code>relpipe-tr-sql</code> and <code>relpipe-in-sql</code> is that

		the SQL input filter reads SQL scripts from the STDIN while the SQL transformation reads relational data from the STDIN.

		And the input filter has implicit <code>--copy '.*'</code> option, if executed without any arguments (so it passes through all relations from the input without need of writing any SELECTs).

		We can override this default behavior by using any argument – we can e.g. copy only certain relations using <code>--copy 'a|c'</code> (it is a regular expression for matching relation names)

		or rename them: <code>--copy-renamed 'a|c' 'copy_of_$0'</code> or run arbitrary SELECT: <code>--relation 'a' "SELECT * FROM …"</code> as we have seen above.

	</p>

	<p>

		SQL scripts can be used for storing relational data as an alternative to other human-readable and human-editable text formats like XML, CSV or Recfiles.

		And compared to the other formats, SQL scripts may contain even some logic (e.g. call SQL functions) or views.

	</p>

	<p>

		n.b. The SQL script does not contain only data – it is an executable script and running it might be dangerous.

		Depending on SQL engine used (the default one is SQLite, but others like PostgreSQL or MySQL/MariaDB can be used),

		such script may call various functions and some of them might read or write local files or do some other unsafe operations.

		Thus the SQL scripts comming from untrusted sources must be carefully reviewed or executed in an isolated environment (sandbox).

		We can run <code>relpipe-in-sql</code> using <code>sudo</code> under an unprivileged account or using <code>ssh</code> doing the same even on a remote machine (virtual od physical one or a container dedicated for such dirty work).

		Or we can use the <code>--data-source-name</code> or <code>--data-source-string</code> options and run such script on a remote DBMS under an unprivileged database account or on a sandbox database that will be destroyed or refreshed after use.

	</p>

	</text>

</stránka>