|
1 <stránka |
|
2 xmlns="https://trac.frantovo.cz/xml-web-generator/wiki/xmlns/strana" |
|
3 xmlns:m="https://trac.frantovo.cz/xml-web-generator/wiki/xmlns/makro"> |
|
4 |
|
5 <nadpis>Release v0.13</nadpis> |
|
6 <perex>sixth public release of Relational pipes</perex> |
|
7 <m:release>v0.13</m:release> |
|
8 |
|
9 <text xmlns="http://www.w3.org/1999/xhtml"> |
|
10 <p> |
|
11 We are pleased to introduce you the new development version of <m:name/>. |
|
12 This release brings the XMLTable input filter, the SQL transformation and a small improvement in the filesystem input filter: |
|
13 </p> |
|
14 |
|
15 <ul> |
|
16 <li> |
|
17 <strong>XMLTable input</strong>: |
|
18 This tool does similar job like the <a href="https://www.postgresql.org/docs/current/functions-xml.html">xmltable</a> function known from SQL. |
|
19 It uses the <a href="https://www.w3.org/TR/xpath/all/">XPath</a> language for selecting parts of the input XML – one XPath expression points to record nodes |
|
20 and one or more XPath expressions point to attribute nodes/values relatively to particular record node. |
|
21 Thus it is able to produce one or more relations from an arbitrary XML input. |
|
22 The input is parsed at once and converted to DOM in memory i.e. no streaming – thus processing of huge XML files requires appropriate amounts of RAM, on the other hand: |
|
23 our expression can access whole XML document and pick values not only from currently processed record node. |
|
24 This tool uses the <a href="http://xmlsoft.org/">Libxml2</a> library (XML parser and XPath processor). |
|
25 </li> |
|
26 |
|
27 <li> |
|
28 <strong>SQL transformation</strong>: |
|
29 SQL is one of most powerful languages for processing relational data and the most widespread one. |
|
30 Now it can be used even on-the-fly in shell pipelines – without having any database server running. |
|
31 It is useful for record filtering, JOINing several relations together, doing aggregations or computations. |
|
32 By default everything is done in memory, but with the <code>--file</code> parameter we can use a temporary file |
|
33 and with <code>--keep-file</code> we can make it not so temporary. |
|
34 This tool uses the <a href="https://www.sqlite.org/">SQLite</a> library. |
|
35 </li> |
|
36 |
|
37 <li> |
|
38 <strong>file system input</strong>: |
|
39 new optional attribute has been added: <code>--file content</code> |
|
40 which allows getting the text content (currently only in the UTF-8 encoding) of the file which allows us using the file system as a simple database. |
|
41 </li> |
|
42 |
|
43 </ul> |
|
44 |
|
45 <p> |
|
46 See the <m:a href="examples">examples</m:a> and <m:a href="screenshots">screenshots</m:a> pages for details. |
|
47 </p> |
|
48 |
|
49 <p> |
|
50 Please note that this is still a development relasease and thus the API (libraries, CLI arguments, formats) might and will change. |
|
51 Any suggestions, ideas and bug reports are welcome in our <m:a href="contact">mailing list</m:a>. |
|
52 </p> |
|
53 |
|
54 <h3>Data types</h3> |
|
55 <ul> |
|
56 <li m:since="v0.8">boolean</li> |
|
57 <li m:since="v0.8">variable unsigned integer (prototype)</li> |
|
58 <li m:since="v0.8">string in UTF-8</li> |
|
59 </ul> |
|
60 <h3>Inputs</h3> |
|
61 <ul> |
|
62 <li m:since="v0.11">Recfile</li> |
|
63 <li m:since="v0.9">XML</li> |
|
64 <li m:since="v0.13">XMLTable</li> |
|
65 <li m:since="v0.9">CSV</li> |
|
66 <li m:since="v0.9">file system</li> |
|
67 <li m:since="v0.8">CLI</li> |
|
68 <li m:since="v0.8">fstab</li> |
|
69 </ul> |
|
70 <h3>Transformations</h3> |
|
71 <ul> |
|
72 <li m:since="v0.13">sql: filtering and transformations using the SQL language</li> |
|
73 <li m:since="v0.12">awk: filtering and transformations using the classic AWK tool and language</li> |
|
74 <li m:since="v0.10">guile: filtering and transformations defined in the Scheme language using GNU Guile</li> |
|
75 <li m:since="v0.8">grep: regular expression filter, removes unwanted records from the relation</li> |
|
76 <li m:since="v0.8">cut: regular expression attribute cutter (removes or duplicates attributes and can also DROP whole relation)</li> |
|
77 <li m:since="v0.8">sed: regular expression replacer</li> |
|
78 <li m:since="v0.8">validator: just a pass-through filter that crashes on invalid data</li> |
|
79 <li m:since="v0.8">python: highly experimental</li> |
|
80 </ul> |
|
81 <h3>Outputs</h3> |
|
82 <ul> |
|
83 <li m:since="v0.11">ASN.1 BER</li> |
|
84 <li m:since="v0.11">Recfile</li> |
|
85 <li m:since="v0.9">CSV</li> |
|
86 <li m:since="v0.8">tabular</li> |
|
87 <li m:since="v0.8">XML</li> |
|
88 <li m:since="v0.8">nullbyte</li> |
|
89 <li m:since="v0.8">GUI in Qt</li> |
|
90 <li m:since="v0.8">ODS (LibreOffice)</li> |
|
91 </ul> |
|
92 |
|
93 <p> |
|
94 Instalation was tested on Debian GNU/Linux 9.6. |
|
95 The process should be similar on other distributions. |
|
96 </p> |
|
97 |
|
98 <m:pre src="examples/release-v0.13.sh" jazyk="bash" odkaz="ano"/> |
|
99 |
|
100 <p> |
|
101 <m:name/> are modular thus you can download and install only parts you need (the libraries are needed always). |
|
102 Tools <code>out-gui.qt</code> and <code>tr-python</code> require additional libraries and are not built by default. |
|
103 </p> |
|
104 |
|
105 <p> |
|
106 The module <code>relpipe-in-filesystem</code> uses C++ filesystem API which is supported since GCC 8. |
|
107 This module can be compiled and seems usable even with GCC 6, but requires some patching (switch to the experimental API): |
|
108 </p> |
|
109 |
|
110 <m:pre jazyk="bash"><![CDATA[sed 's@#include <filesystem>@#include <experimental/filesystem>@g' -i "$RELPIPE_SRC"/relpipe-in-filesystem.cpp/src/FileAttributeFinder.h "$RELPIPE_SRC"/relpipe-in-filesystem.cpp/src/XattrAttributeFinder.h "$RELPIPE_SRC"/relpipe-in-filesystem.cpp/src/FilesystemCommand.h "$RELPIPE_SRC"/relpipe-in-filesystem.cpp/src/AttributeFinder.h |
|
111 sed 's@std::filesystem@std::experimental::filesystem@g' -i "$RELPIPE_SRC"/relpipe-in-filesystem.cpp/src/FileAttributeFinder.h "$RELPIPE_SRC"/relpipe-in-filesystem.cpp/src/XattrAttributeFinder.h "$RELPIPE_SRC"/relpipe-in-filesystem.cpp/src/FilesystemCommand.h "$RELPIPE_SRC"/relpipe-in-filesystem.cpp/src/AttributeFinder.h |
|
112 sed 's/.*PROPERTY CXX_STANDARD.*/#\0/g' -i "$RELPIPE_SRC"/relpipe-in-filesystem.cpp/src/CMakeLists.txt]]></m:pre> |
|
113 |
|
114 <p> |
|
115 The module <code>relpipe-tr-guile</code> uses GNU Guile 2.2 but can also work with 2.0. |
|
116 In such case, it requires this patch: |
|
117 </p> |
|
118 |
|
119 <m:pre jazyk="bash"><![CDATA[sed 's/guile-2\.2/guile-2.0/g' -i "$RELPIPE_SRC"/relpipe-tr-guile.cpp/src/CMakeLists.txt]]></m:pre> |
|
120 |
|
121 </text> |
|
122 |
|
123 </stránka> |