<stránka
	xmlns="https://trac.frantovo.cz/xml-web-generator/wiki/xmlns/strana"
	xmlns:m="https://trac.frantovo.cz/xml-web-generator/wiki/xmlns/makro">
	
	<nadpis>Integrating Relational pipes with GNU Recutils</nadpis>
	<perex>using recfile format as input and output + filtering</perex>
	<m:pořadí-příkladu>01900</m:pořadí-příkladu>

	<text xmlns="http://www.w3.org/1999/xhtml">
		
		<p>
			Recfile is the native format of <a href="https://www.gnu.org/software/recutils/">GNU Recutils</a>.
			Recfiles are text files that contain records of various types.
			They are human-editable and serve as simple databases.
			<m:name/> support input and output in this format since v0.11.
		</p>
		
		
		<p>
			We can convert any relational data to the recfile format by using <code>relpipe-out-recfile</code> – e.g. our <code>fstab</code> will look like this:
		</p>

		<m:pre jazyk="text" src="examples/relpipe-out-fstab.rec.txt"/>
		
		<p>
			Then we can edit this data (e.g. in GNU Emacs which has mode for this format) or store it in a version control system like Mercurial or Git.
			Because it is a text format (like XML, which is also supported and good for this purpose),
			we can efficiently track changes in data across versions, do <code>diff</code> or (with some care) even <code>patch</code>.
			And we can use whole GNU Recutils toolchain while working with such data.
		</p>
		
		<p>
			Obligatory example of filtering our <code>fstab</code>:
		</p>
		
		<m:pre jazyk="bash"><![CDATA[relpipe-in-fstab | relpipe-out-recfile | recsel -e "type = 'btrfs' || type = 'xfs'"]]></m:pre>
		
		<p>Will give us a recfile:</p>
		
		<m:pre jazyk="text"><![CDATA[scheme: UUID
device: a2b5f230-a795-4f6f-a39b-9b57686c86d5
mount_point: /home
type: btrfs
options: relatime
dump: 0
pass: 2

scheme:
device: /dev/mapper/sdf_crypt
mount_point: /mnt/private
type: xfs
options: relatime
dump: 0
pass: 2]]></m:pre>
		
		<p>And we can convert it back to the relational format using <code>relpipe-in-recfile</code>:</p>		
		<m:pre jazyk="bash"><![CDATA[relpipe-in-fstab \
	| relpipe-out-recfile \
	| recsel -e "type = 'btrfs' || type = 'xfs'" \
	| relpipe-in-recfile \
	| relpipe-out-tabular]]></m:pre>
		
		<p>and print as a table in our terminal:</p>		
		<m:pre jazyk="text"><![CDATA[recfile:
 ╭─────────────────┬──────────────────────────────────────┬──────────────────────┬───────────────┬──────────────────┬───────────────┬───────────────╮
 │ scheme (string) │ device                      (string) │ mount_point (string) │ type (string) │ options (string) │ dump (string) │ pass (string) │
 ├─────────────────┼──────────────────────────────────────┼──────────────────────┼───────────────┼──────────────────┼───────────────┼───────────────┤
 │ UUID            │ a2b5f230-a795-4f6f-a39b-9b57686c86d5 │ /home                │ btrfs         │ relatime         │ 0             │ 2             │
 │                 │ /dev/mapper/sdf_crypt                │ /mnt/private         │ xfs           │ relatime         │ 0             │ 2             │
 ╰─────────────────┴──────────────────────────────────────┴──────────────────────┴───────────────┴──────────────────┴───────────────┴───────────────╯
Record count: 2]]></m:pre>
		
		<p>
			n.b. in the v0.11 the conversion to recfiles and back is not 100% lossless (unlike XML)
			because <m:name/> support only three data types (string, unsigned integer and boolean) in this version;
			this will be improved in later releases (more data types are planned before v1.0)
		</p>
		
		<p>
			Because some web browsers or tools can store the original URL in extended attributes while downloading a file,
			we can use <code>recsel</code> to find files downloaded from some particular domain:
		</p>
		<m:pre jazyk="bash"><![CDATA[find -print0 | relpipe-in-filesystem \
	--file path \
	--file size \
	--file type \
	--xattr xdg.origin.url --as url \
	| relpipe-out-recfile \
	| recsel -e 'url ~ "^https?://([^/]*\.)?archive\.org/"']]></m:pre>
		
		<p>
			<m:name/> can be also used together with <a href="https://sql-dk.globalcode.info/">SQL-DK</a> (in 2019-03-05 development version)
			to pipe data from big relational databases like PostgreSQL or MariaDB to other formats like recfiles.
			Having a script:
		</p>
				
		<m:pre jazyk="bash" src="examples/sql-dk_pg_1.sh" odkaz="ano"/>
		
		<p>
			We can convert result sets from any SQL queries to relational format and then work with such data without connection to the original database.
			Thus we can cache (<em>materialize</em>) the results locally in a file and use them even offline.
			Or we can run the SQL query each time and have fresh data:
		</p>
		
		<m:pre jazyk="text"><![CDATA[sql-dk_pg_1.sh | relpipe-out-recfile]]></m:pre>
		
		<p>Will result in:</p>
		<m:pre jazyk="text" src="examples/sql-dk_pg_1.rec.txt"/>
		
		<p>Or we can view the data in classic tabular way using <code>relpipe-out-tabular</code>:</p>
		<m:pre jazyk="text" src="examples/sql-dk_pg_1.tabular.txt"/>
		
		<p>
			Materialized (or fresh) data from the database can be further transformed 
			using <code>relpipe-tr-*</code> commands like grep, sed, cut, guile, 
			or (through the recfile conversion) by the <code>recsel</code> command from GNU Recutils.
		</p>
		
		<p>
			The <code>relpipe-in-recfile</code> will help with conversion of recfiles to various formats like XHTML,
			pretty-printing or with xargs-like processing
			(using <code>relpipe-out-nullbyte</code> and regular <code>xargs</code> or <code>read_nullbyte</code> function
			as described in the <m:a href="examples-out-bash">Writing an output filter in Bash</m:a> example).
			Thus we can have data-driven Bash scripts based on our recfiles.
		</p>

		
	</text>

</stránka>