# HG changeset patch
# User František Kučera
- In
- One day, there might also be a translator that parses SQL code and generates Guile code,
+ One day, there might also be a translator that parses SQL code and generates Scheme code,
so we could have advantages of both worlds
a) concise and declarative syntax of SQL and
b) streaming – which means no need for putting all the data in the RAM or on the disk.
diff -r d4c2968a391f -r d7ae02390fac relpipe-data/examples-guile-filtering.xml
--- a/relpipe-data/examples-guile-filtering.xml Sat Sep 12 13:20:21 2020 +0200
+++ b/relpipe-data/examples-guile-filtering.xml Fri Sep 25 14:38:24 2020 +0200
@@ -2,7 +2,7 @@
xmlns="https://trac.frantovo.cz/xml-web-generator/wiki/xmlns/strana"
xmlns:m="https://trac.frantovo.cz/xml-web-generator/wiki/xmlns/makro">
-
Guile is the GNU implementation of Scheme language (something like Lisp and also full of parenthesis).
- The Well, well… here we are: The The
The
- Relational attributes are mapped to Guile variables with same name, just prefixed with
@@ -88,7 +88,7 @@
The Btrfs filesystems in our relpipe-tr-guile
we can generate new records – not only modify records from the input.
- There is --has-more-records
option which – if evaluated as true – says: „read one more record from the Guile context and call me again“.
+ In relpipe-tr-scheme
we can generate new records – not only modify records from the input.
+ There is --has-more-records
option which – if evaluated as true – says: „read one more record from the Scheme context and call me again“.
We can also suppress all original records by --where '#f'
.
And we can also change the structure of the relation (see previous examples).
Thus we can iterate through a relation but completely replace its structure and content.
@@ -49,7 +49,7 @@
relpipe-tr-grep
.
But what if we need to write some complex query that contains AND and OR operators?
What if we need e.g. compare numbers – not only match texts against regular expressions?
- There is a tool capable to do this and much more: relpipe-tr-guile
!
+ There is a tool capable to do this and much more: relpipe-tr-scheme
!
relpipe-tr-guile
uses GNU Guile as a library, puts data in the Guile context and evaluates Guile expressions and then reads data from the Guile context back and generates relational output from them.
+ The relpipe-tr-scheme
reference implementation uses GNU Guile as a library, puts data in the Scheme context and evaluates Scheme expressions and then reads data from the Scheme context back and generates relational output from them.
Good news are that it is not necessary to know Lisp/Scheme to use this tool. For the first steps, it can be used just as a query language – like SQL, just a bit Polish.
--relation 'files.*'
is a regular expression that says which relations should be processed in Guile – others are passed through unchanged.--relation 'files.*'
is a regular expression that says which relations should be processed in Scheme – others are passed through unchanged.--where '(= $size 666)'
is our condition.
@@ -65,11 +65,11 @@
$
.
+ Relational attributes are mapped to Scheme variables with same name, just prefixed with $
.
(we considered
₿
symbol, but $
seems to be still more common on keyboards in 2019)
- While relational attribute name is an arbitrary string, Guile variable names have some limitations, thus not all attributes can be mapped – those with spaces and some special characters are currently unsupported (this will be fixed in later versions by some kind of encoding/escaping).
+ While relational attribute name is an arbitrary string, Scheme variable names have some limitations, thus not all attributes can be mapped – those with spaces and some special characters are currently unsupported (this will be fixed in later versions by some kind of encoding/escaping).
fstab
:
So we can find filesystems mounted somewhere under /mnt
:
Like in SQL, we can join multiple conditions together with logical operators AND and OR.
- In Guile/Scheme these operators are also functions – they are written in the same (
fashion)
.
+ In Scheme these operators are also functions – they are written in the same (
fashion)
.
@@ -139,7 +139,7 @@
Files with sizes 666 bytes or 1984 bytes:
@@ -172,7 +172,7 @@
- The relpipe-tr-guile
is capable to process multiple relations in a single pass.
+ The relpipe-tr-scheme
is capable to process multiple relations in a single pass.
So we can filter some relations and let others flow unaffected through this pipeline step.
- In Guile, we have filtered the fstab
and filesystem
relations
+ In Scheme, we have filtered the fstab
and filesystem
relations
while the seq
relation was kept intact.
- If we define some variables in the Guile context, they will stay there – so we can pass data across relations. + If we define some variables in the Scheme context, they will stay there – so we can pass data across relations. Thus we can do even JOIN, if we really want.
diff -r d4c2968a391f -r d7ae02390fac relpipe-data/examples-guile-parametrized-queries.xml --- a/relpipe-data/examples-guile-parametrized-queries.xml Sat Sep 12 13:20:21 2020 +0200 +++ b/relpipe-data/examples-guile-parametrized-queries.xml Fri Sep 25 14:38:24 2020 +0200 @@ -2,7 +2,7 @@ xmlns="https://trac.frantovo.cz/xml-web-generator/wiki/xmlns/strana" xmlns:m="https://trac.frantovo.cz/xml-web-generator/wiki/xmlns/makro"> -
- But it is fundamentally wrong. The input parameter is blindly pasted in middle of the Guile code.
+ But it is fundamentally wrong. The input parameter is blindly pasted in middle of the Scheme code.
So if we call e.g. fstab-where-type 'ext4"'
, it crashes terribly.
Do you remember SQL injections in your first PHP scripts when you were 14?
Do not do it again!
- The relpipe-tr-guile
tool has a safe way for passing parameters from the outside. And such parameters are even strongly typed.
+ The relpipe-tr-scheme
tool has a safe way for passing parameters from the outside. And such parameters are even strongly typed.
So this is, how our program should be written:
- The relpipe-tr-guile
can not only filter records,
+ The relpipe-tr-scheme
can not only filter records,
but can also modify them and even modify the structure of the relation – add or remove attributes.
- Then, we can modify such relation using Guile – e.g. we can make the name
uppercase and increase id
by 1000:
+ Then, we can modify such relation using Scheme – e.g. we can make the name
uppercase and increase id
by 1000:
Actually, the relpipe-in-sql
is just a symbolic link to the relpipe-tr-sql
and have the same capabilities (with just bit different default behavior to match general logic of the input filters).
- So if we do not need special feautres of Guile, AWK or other transformation tool, it is better to stay with SQL (when we already use the SQL module).
+ So if we do not need special feautres of Scheme, AWK or other transformation tool, it is better to stay with SQL (when we already use the SQL module).
diff -r d4c2968a391f -r d7ae02390fac relpipe-data/examples-parallel-hashes.xml --- a/relpipe-data/examples-parallel-hashes.xml Sat Sep 12 13:20:21 2020 +0200 +++ b/relpipe-data/examples-parallel-hashes.xml Fri Sep 25 14:38:24 2020 +0200 @@ -56,7 +56,7 @@
aggregate
– uses SQL to order the records and SQL window function to show, how many files have the same content;
- in this step we could use also relpipe-tr-awk
or relpipe-tr-guile
if we prefer AWK or Guile/Scheme to SQL
+ in this step we could use also relpipe-tr-awk
or relpipe-tr-scheme
if we prefer AWK or Scheme to SQL
relpipe-out-tabular
diff -r d4c2968a391f -r d7ae02390fac relpipe-data/examples-runnable-jars.xml
--- a/relpipe-data/examples-runnable-jars.xml Sat Sep 12 13:20:21 2020 +0200
+++ b/relpipe-data/examples-runnable-jars.xml Fri Sep 25 14:38:24 2020 +0200
@@ -43,7 +43,7 @@
filterRunable
– uses AWK to skip the records (files) that does not have a main class;
- in this step we could use also relpipe-tr-sql
or relpipe-tr-guile
if we prefer SQL or Guile/Scheme to AWK
+ in this step we could use also relpipe-tr-sql
or relpipe-tr-scheme
if we prefer SQL or Scheme to AWK
shortenPath
diff -r d4c2968a391f -r d7ae02390fac relpipe-data/examples/awk-through-xml.sh
--- a/relpipe-data/examples/awk-through-xml.sh Sat Sep 12 13:20:21 2020 +0200
+++ b/relpipe-data/examples/awk-through-xml.sh Fri Sep 25 14:38:24 2020 +0200
@@ -41,8 +41,8 @@
# "SELECT * FROM policy WHERE policy = 'allow' AND user <> 'root'"
# --copy 'limit'
- # Or using Guile (Scheme):
- # relpipe-tr-guile \
+ # Or using Scheme:
+ # relpipe-tr-scheme \
# --relation policy \
# --where '(and (string= $policy "allow") (not (string= $user "root")) )'
}
diff -r d4c2968a391f -r d7ae02390fac relpipe-data/examples/guile-file-count-size-sum.sh
--- a/relpipe-data/examples/guile-file-count-size-sum.sh Sat Sep 12 13:20:21 2020 +0200
+++ b/relpipe-data/examples/guile-file-count-size-sum.sh Fri Sep 25 14:38:24 2020 +0200
@@ -7,7 +7,7 @@
| relpipe-in-filesystem \
--file path \
--file size \
- | relpipe-tr-guile \
+ | relpipe-tr-scheme \
--relation 'f.*' \
--output-attribute 'count' integer \
--output-attribute 'sum' integer \
diff -r d4c2968a391f -r d7ae02390fac relpipe-data/implementation.xml
--- a/relpipe-data/implementation.xml Sat Sep 12 13:20:21 2020 +0200
+++ b/relpipe-data/implementation.xml Fri Sep 25 14:38:24 2020 +0200
@@ -38,7 +38,7 @@
relpipe-tr-awk.cpp executable transformation c++ GNU GPLv3
relpipe-tr-cut.cpp executable transformation c++ GNU GPLv3
relpipe-tr-grep.cpp executable transformation c++ GNU GPLv3
- relpipe-tr-guile.cpp executable transformation c++ GNU GPLv3
+ relpipe-tr-scheme.cpp executable transformation c++ GNU GPLv3
relpipe-tr-python.cpp executable transformation c++ GNU GPLv3
relpipe-tr-sed.cpp executable transformation c++ GNU GPLv3
relpipe-tr-sql.cpp executable transformation c++ GNU GPLv3
diff -r d4c2968a391f -r d7ae02390fac relpipe-data/index.xml
--- a/relpipe-data/index.xml Sat Sep 12 13:20:21 2020 +0200
+++ b/relpipe-data/index.xml Fri Sep 25 14:38:24 2020 +0200
@@ -148,7 +148,7 @@