# HG changeset patch # User František Kučera # Date 1600991956 -7200 # Node ID e87c231afb77f2732e367672557af12619eb9c6f # Parent 2354c9058fb6833cbcb0bea2910c33a2f68ef17d relpipe-tr-guile → relpipe-tr-scheme diff -r 2354c9058fb6 -r e87c231afb77 CMakeLists.txt --- a/CMakeLists.txt Sat Jun 06 01:50:45 2020 +0200 +++ b/CMakeLists.txt Fri Sep 25 01:59:16 2020 +0200 @@ -13,6 +13,6 @@ # You should have received a copy of the GNU General Public License # along with this program. If not, see . -project (relpipe-tr-guile.cpp) +project (relpipe-tr-scheme.cpp) cmake_minimum_required(VERSION 3.7.2) add_subdirectory (src) diff -r 2354c9058fb6 -r e87c231afb77 bash-completion.sh --- a/bash-completion.sh Sat Jun 06 01:50:45 2020 +0200 +++ b/bash-completion.sh Fri Sep 25 01:59:16 2020 +0200 @@ -13,7 +13,7 @@ # You should have received a copy of the GNU General Public License # along with this program. If not, see . -_relpipe_tr_guile_completion() { +_relpipe_tr_scheme_completion() { local w0 w1 w2 COMPREPLY=() @@ -57,5 +57,5 @@ fi } -complete -F _relpipe_tr_guile_completion relpipe-tr-guile -complete -F _relpipe_tr_guile_completion relpipe-in-guile +complete -F _relpipe_tr_scheme_completion relpipe-tr-scheme +complete -F _relpipe_tr_scheme_completion relpipe-in-scheme diff -r 2354c9058fb6 -r e87c231afb77 nbproject/configurations.xml --- a/nbproject/configurations.xml Sat Jun 06 01:50:45 2020 +0200 +++ b/nbproject/configurations.xml Fri Sep 25 01:59:16 2020 +0200 @@ -42,7 +42,7 @@ - relpipe-tr-guile.cpp + relpipe-tr-scheme.cpp build/Debug ${MAKE} -f Makefile ${MAKE} -f Makefile clean - build/Debug/src/relpipe-tr-guile + build/Debug/src/relpipe-tr-scheme ../relpipe-lib-reader.cpp/include ../relpipe-lib-writer.cpp/include + ../relpipe-lib-common.cpp/include ../relpipe-lib-cli.cpp/include /usr/include/guile/2.2 build/Debug/src @@ -93,7 +94,7 @@ true - + @@ -133,7 +134,7 @@ true - + diff -r 2354c9058fb6 -r e87c231afb77 nbproject/project.xml --- a/nbproject/project.xml Sat Jun 06 01:50:45 2020 +0200 +++ b/nbproject/project.xml Fri Sep 25 01:59:16 2020 +0200 @@ -42,7 +42,7 @@ org.netbeans.modules.cnd.makeproject - relpipe-tr-guile.cpp + relpipe-tr-scheme.cpp

cpp

h diff -r 2354c9058fb6 -r e87c231afb77 src/CLIParser.h --- a/src/CLIParser.h Sat Jun 06 01:50:45 2020 +0200 +++ b/src/CLIParser.h Fri Sep 25 01:59:16 2020 +0200 @@ -26,7 +26,7 @@ namespace relpipe { namespace tr { -namespace guile { +namespace scheme { class CLIParser { private: @@ -75,11 +75,11 @@ for (int i = 0; i < arguments.size();) { string_t option = readNext(arguments, i); - if (option == OPTION_BEFORE_RECORDS) currentRelation.guileBeforeRecords = readNext(arguments, i); - else if (option == OPTION_AFTER_RECORDS) currentRelation.guileAfterRecords = readNext(arguments, i); - else if (option == OPTION_FOR_EACH) currentRelation.guileForEach = readNext(arguments, i); - else if (option == OPTION_WHERE) currentRelation.guileWhere = readNext(arguments, i); - else if (option == OPTION_HAS_MORE_RECORDS) currentRelation.guileHasMoreRecords = readNext(arguments, i); + if (option == OPTION_BEFORE_RECORDS) currentRelation.schemeBeforeRecords = readNext(arguments, i); + else if (option == OPTION_AFTER_RECORDS) currentRelation.schemeAfterRecords = readNext(arguments, i); + else if (option == OPTION_FOR_EACH) currentRelation.schemeForEach = readNext(arguments, i); + else if (option == OPTION_WHERE) currentRelation.schemeWhere = readNext(arguments, i); + else if (option == OPTION_HAS_MORE_RECORDS) currentRelation.schemeHasMoreRecords = readNext(arguments, i); else if (option == OPTION_DROP) currentRelation.drop = true; else if (option == OPTION_INPUT_ATTRIBUTES_APPEND) currentRelation.inputAttributesAppend = true; else if (option == OPTION_INPUT_ATTRIBUTES_PREPEND) currentRelation.inputAttributesPrepend = true; diff -r 2354c9058fb6 -r e87c231afb77 src/CMakeLists.txt --- a/src/CMakeLists.txt Sat Jun 06 01:50:45 2020 +0200 +++ b/src/CMakeLists.txt Fri Sep 25 01:59:16 2020 +0200 @@ -13,7 +13,7 @@ # You should have received a copy of the GNU General Public License # along with this program. If not, see . -set(EXECUTABLE_FILE "relpipe-tr-guile") +set(EXECUTABLE_FILE "relpipe-tr-scheme") # Relpipe libraries: INCLUDE(FindPkgConfig) @@ -29,7 +29,7 @@ # Executable output: add_executable( ${EXECUTABLE_FILE} - relpipe-tr-guile.cpp + relpipe-tr-scheme.cpp ) # Link libraries: diff -r 2354c9058fb6 -r e87c231afb77 src/Configuration.h --- a/src/Configuration.h Sat Jun 06 01:50:45 2020 +0200 +++ b/src/Configuration.h Fri Sep 25 01:59:16 2020 +0200 @@ -24,7 +24,7 @@ namespace relpipe { namespace tr { -namespace guile { +namespace scheme { class DefinitionRecipe { public: @@ -44,19 +44,19 @@ } relpipe::writer::string_t relation; - relpipe::writer::string_t guileBeforeRecords; - relpipe::writer::string_t guileAfterRecords; - relpipe::writer::string_t guileForEach; - relpipe::writer::string_t guileWhere; - relpipe::writer::string_t guileHasMoreRecords; + relpipe::writer::string_t schemeBeforeRecords; + relpipe::writer::string_t schemeAfterRecords; + relpipe::writer::string_t schemeForEach; + relpipe::writer::string_t schemeWhere; + relpipe::writer::string_t schemeHasMoreRecords; /** - * If true, additional relation will be generated: mapping between relpipe attribute names and Guile variable names + * If true, additional relation will be generated: mapping between relpipe attribute names and Scheme variable names */ bool debugVariableMapping = false; /** - * If true, Guile code will be executed, but no output will be generated. + * If true, Scheme code will be executed, but no output will be generated. */ bool drop = false; @@ -94,9 +94,9 @@ */ std::vector definitions; - // TODO: --create t2 3 a integer b boolean '…guile…' – allow creating new relations? Or allow calling startRelation() and attribute() from Guile? - // TODO: --has-more-relations '…guile…' – called after all relations … ? - // TODO: --dynamic-output-attributes '…guile…' – used instead or together with --output-attribute + // TODO: --create t2 3 a integer b boolean '…scheme…' – allow creating new relations? Or allow calling startRelation() and attribute() from Scheme? + // TODO: --has-more-relations '…scheme…' – called after all relations … ? + // TODO: --dynamic-output-attributes '…scheme…' – used instead or together with --output-attribute virtual ~Configuration() { } diff -r 2354c9058fb6 -r e87c231afb77 src/GuileException.h --- a/src/GuileException.h Sat Jun 06 01:50:45 2020 +0200 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,43 +0,0 @@ -/** - * Relational pipes - * Copyright © 2019 František Kučera (Frantovo.cz, GlobalCode.info) - * - * This program is free software: you can redistribute it and/or modify - * it under the terms of the GNU General Public License as published by - * the Free Software Foundation, version 3 of the License. - * - * This program is distributed in the hope that it will be useful, - * but WITHOUT ANY WARRANTY; without even the implied warranty of - * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the - * GNU General Public License for more details. - * - * You should have received a copy of the GNU General Public License - * along with this program. If not, see . - */ -#pragma once - -#include - -using namespace std; - -namespace relpipe { -namespace tr { -namespace guile { - -class GuileException { -private: - wstring message; -public: - - GuileException(wstring message) : message(message) { - } - - wstring getMessge() { - return message; - } - -}; - -} -} -} \ No newline at end of file diff -r 2354c9058fb6 -r e87c231afb77 src/GuileHandler.h --- a/src/GuileHandler.h Sat Jun 06 01:50:45 2020 +0200 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,321 +0,0 @@ -/** - * Relational pipes - * Copyright © 2019 František Kučera (Frantovo.cz, GlobalCode.info) - * - * This program is free software: you can redistribute it and/or modify - * it under the terms of the GNU General Public License as published by - * the Free Software Foundation, version 3 of the License. - * - * This program is distributed in the hope that it will be useful, - * but WITHOUT ANY WARRANTY; without even the implied warranty of - * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the - * GNU General Public License for more details. - * - * You should have received a copy of the GNU General Public License - * along with this program. If not, see . - */ -#pragma once - -#include -#include -#include -#include -#include -#include -#include -#include -#include -#include - -#include - -#include -#include -#include -#include - -#include - -#include - -#include "Configuration.h" -#include "GuileException.h" - -namespace relpipe { -namespace tr { -namespace guile { - -using namespace std; -using namespace relpipe; -using namespace relpipe::reader; -using namespace relpipe::reader::handlers; - -class GuileHandler : public RelationalReaderValueHandler { -private: - std::wstring_convert> convertor; // TODO: support also other encodings or use always UTF-8 between C++ and Guile - - Configuration configuration; - writer::RelationalWriter* relationalWriter; - - RelationConfiguration* currentRelationConfiguration = nullptr; - vector currentReaderMetadata; - vector currentWriterMetadata; - std::map currenVariablesMapping; - integer_t currentAttributeIndex = 0; - boolean_t includeCurrentRecord = false; - - void add(vector& readerAttributes, vector& writerAttributes) { - for (AttributeMetadata readerAttributes : readerAttributes) - writerAttributes.push_back({ - readerAttributes.getAttributeName(), - relationalWriter->toTypeId(readerAttributes.getTypeName()) - }); - } - - void generateVariableMappings() { - currenVariablesMapping.clear(); - for (AttributeMetadata m : currentReaderMetadata) currenVariablesMapping[m.getAttributeName()] = L""; - for (writer::AttributeMetadata m : currentWriterMetadata) currenVariablesMapping[m.attributeName] = L""; - - for (std::pair m : currenVariablesMapping) { - currenVariablesMapping[m.first] = escapeAwkVariableName(m.first); - } - } - - /** - * @param attributeName name from relational pipe - * @return variable name in Guile - */ - string_t a2v(const string_t& attributeName) { - if (currenVariablesMapping.find(attributeName) != currenVariablesMapping.end()) return currenVariablesMapping[attributeName]; - else throw GuileException(L"Unable to find value in currenVariablesMapping"); - } - - template bool containsValue(std::map map, V value) { // TODO: common function (Guile, AWK) - for (std::pair p : map) if (p.second == value) return true; - return false; - } - - string_t escapeAwkVariableName(const string_t& attributeName, bool addPrefix = true) { - std::wregex badCharacters(L"\\s"); - string_t name = std::regex_replace(attributeName, badCharacters, L"-"); - - if (addPrefix) name = L"$" + name; // $ = standard attribute-variable prefix - - if (containsValue(currenVariablesMapping, name)) return escapeAwkVariableName(L"$" + name, false); // $ = different prefix added to distinguish two attributes with ambiguous names - else return name; - - } - - void debugVariableMapping(const string_t& relationName) { - relationalWriter->startRelation(relationName + L".variableMapping",{ - {L"attribute", writer::TypeId::STRING}, - {L"variable", writer::TypeId::STRING}, - }, true); - - for (std::pair m : currenVariablesMapping) { - relationalWriter->writeAttribute(m.first); - relationalWriter->writeAttribute(m.second); - } - } - - SCM toGuileSymbol(const string_t& name) { - return scm_string_to_symbol(scm_from_locale_string(convertor.to_bytes(name).c_str())); - } - - /** - * @param code guile source code e.g. (+ 1 2 3) or #t - * @param defaultReturnValue is returned if code is empty - * @return result of code execution or defaultReturnValue - */ - SCM evalGuileCode(const string_t& code, SCM defaultReturnValue = SCM_BOOL_F) { - if (code.size()) return scm_eval_string(toGuileValue(&code, typeid (string_t), TypeId::STRING)); - else return defaultReturnValue; - } - - SCM toGuileValue(const void* value, const std::type_info& typeInfo, TypeId type) { - switch (type) { - case TypeId::BOOLEAN: - { - assert(typeInfo == typeid (boolean_t)); - auto* typedValue = static_cast (value); - return *typedValue ? SCM_BOOL_T : SCM_BOOL_F; - } - case TypeId::INTEGER: - { - assert(typeInfo == typeid (integer_t)); - auto* typedValue = static_cast (value); - return scm_from_int64(*typedValue); - } - case TypeId::STRING: - { - assert(typeInfo == typeid (string_t)); - auto* typedValue = static_cast (value); - return scm_from_locale_string(convertor.to_bytes(*typedValue).c_str()); - } - default: - throw cli::RelpipeCLIException(L"Unsupported type in toGuileValue()", cli::CLI::EXIT_CODE_UNEXPECTED_ERROR); - } - } - - void defineGuileVariable(const string_t& name, const void* value, const std::type_info& typeInfo, TypeId type) { - scm_define(toGuileSymbol(name), toGuileValue(value, typeInfo, type)); - } - - /** - * TODO: use a common method - */ - bool parseBoolean(const string_t& value) { - if (value == L"true") return true; - else if (value == L"false") return false; - else throw relpipe::cli::RelpipeCLIException(L"Unable to parse boolean value: " + value + L" (expecting true or false)", relpipe::cli::CLI::EXIT_CODE_BAD_CLI_ARGUMENTS); - } - - void defineGuileVariable(const DefinitionRecipe& definition) { - switch (relationalWriter->toTypeId(definition.type)) { - case writer::TypeId::BOOLEAN: - { - boolean_t value = parseBoolean(definition.value); - defineGuileVariable(definition.name, &value, typeid (value), TypeId::BOOLEAN); - break; - } - case writer::TypeId::INTEGER: - { - integer_t value = stol(definition.value); - defineGuileVariable(definition.name, &value, typeid (value), TypeId::INTEGER); - break; - } - case writer::TypeId::STRING: - { - defineGuileVariable(definition.name, &definition.value, typeid (definition.value), TypeId::STRING); - break; - } - default: - throw cli::RelpipeCLIException(L"Unsupported type in defineGuileVariable(): " + definition.type, cli::CLI::EXIT_CODE_UNEXPECTED_ERROR); - } - } - - void undefineGuileVariable(const string_t& name) { - scm_define(toGuileSymbol(name), scm_make_undefined_variable()); // undefined != (define n) - // TODO: or use: scm_variable_unset_x() ? - } - - void writeGuileValueToAttribute(const writer::AttributeMetadata& attribute) { - string_t variableName = a2v(attribute.attributeName); - SCM guileValue = scm_eval_string(toGuileValue(&variableName, typeid (variableName), TypeId::STRING)); - - switch (attribute.typeId) { - case writer::TypeId::BOOLEAN: - { - boolean_t value = scm_to_bool(guileValue); - return relationalWriter->writeAttribute(&value, typeid (value)); - } - case writer::TypeId::INTEGER: - { - integer_t value = scm_to_int64(guileValue); - return relationalWriter->writeAttribute(&value, typeid (value)); - } - case writer::TypeId::STRING: - { - char* ch = scm_to_locale_string(guileValue); - string_t value = convertor.from_bytes(ch); - free(ch); - return relationalWriter->writeAttribute(&value, typeid (value)); - } - default: - throw cli::RelpipeCLIException(L"Unsupported type in writeGuileValueToAttribute()", cli::CLI::EXIT_CODE_UNEXPECTED_ERROR); - } - } - - /** - * Read from the Guile variables and write to relational output stream. - */ - void writeCurrentRecord() { - for (auto attribute : currentWriterMetadata) writeGuileValueToAttribute(attribute); - } - - void writeMoreRecords() { - while (scm_to_bool(evalGuileCode(currentRelationConfiguration->guileHasMoreRecords, SCM_BOOL_F))) writeCurrentRecord(); - } - -public: - - GuileHandler(writer::RelationalWriter* relationalWriter, Configuration& configuration) : relationalWriter(relationalWriter), configuration(configuration) { - } - - void startRelation(string_t name, vector attributes) override { - if (currentRelationConfiguration) { - evalGuileCode(currentRelationConfiguration->guileAfterRecords); - writeMoreRecords(); - for (DefinitionRecipe definition : currentRelationConfiguration->definitions) undefineGuileVariable(definition.name); - } - for (auto attribute : currentReaderMetadata) undefineGuileVariable(attribute.getAttributeName()); - - for (DefinitionRecipe definition : configuration.definitions) defineGuileVariable(definition); - - currentRelationConfiguration = nullptr; - for (int i = 0; i < configuration.relationConfigurations.size(); i++) { - if (regex_match(name, wregex(configuration.relationConfigurations[i].relation))) { - currentRelationConfiguration = &configuration.relationConfigurations[i]; - for (DefinitionRecipe definition : currentRelationConfiguration->definitions) defineGuileVariable(definition); - break; // it there are multiple matches, only the first configuration is used - } - } - - currentReaderMetadata = attributes; - // TODO: move to a reusable method (or use same metadata on both reader and writer side?) - currentWriterMetadata.clear(); - if (currentRelationConfiguration && currentRelationConfiguration->writerMetadata.size()) { - if (currentRelationConfiguration->inputAttributesPrepend) add(currentReaderMetadata, currentWriterMetadata); - currentWriterMetadata.insert(currentWriterMetadata.end(), currentRelationConfiguration->writerMetadata.begin(), currentRelationConfiguration->writerMetadata.end()); - if (currentRelationConfiguration->inputAttributesAppend) add(currentReaderMetadata, currentWriterMetadata); - } else { - add(currentReaderMetadata, currentWriterMetadata); - } - - generateVariableMappings(); - - if (currentRelationConfiguration && currentRelationConfiguration->debugVariableMapping) debugVariableMapping(name); - - if (!currentRelationConfiguration || !currentRelationConfiguration->drop) relationalWriter->startRelation(name, currentWriterMetadata, true); - - if (currentRelationConfiguration) { - // TODO: better variable name, object, function? - defineGuileVariable(L"relpipe-relation-name", &name, typeid (name), TypeId::STRING); - evalGuileCode(currentRelationConfiguration->guileBeforeRecords); - } - } - - void attribute(const void* value, const std::type_info& type) override { - if (currentRelationConfiguration) { - defineGuileVariable(a2v(currentReaderMetadata[currentAttributeIndex].getAttributeName()), value, type, currentReaderMetadata[currentAttributeIndex].getTypeId()); - - currentAttributeIndex++; - - // TODO: > 0 ?: - if (currentAttributeIndex > 0 && currentAttributeIndex % currentReaderMetadata.size() == 0) { - evalGuileCode(currentRelationConfiguration->guileForEach); - includeCurrentRecord = scm_to_bool(evalGuileCode(currentRelationConfiguration->guileWhere, SCM_BOOL_T)); - if (includeCurrentRecord && !currentRelationConfiguration->drop) writeCurrentRecord(); - includeCurrentRecord = false; - writeMoreRecords(); - } - - currentAttributeIndex = currentAttributeIndex % currentReaderMetadata.size(); - } else { - relationalWriter->writeAttribute(value, type); - } - } - - void endOfPipe() { - if (currentRelationConfiguration) { - evalGuileCode(currentRelationConfiguration->guileAfterRecords); - writeMoreRecords(); - } - } - -}; - -} -} -} diff -r 2354c9058fb6 -r e87c231afb77 src/SchemeException.h --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/src/SchemeException.h Fri Sep 25 01:59:16 2020 +0200 @@ -0,0 +1,43 @@ +/** + * Relational pipes + * Copyright © 2019 František Kučera (Frantovo.cz, GlobalCode.info) + * + * This program is free software: you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation, version 3 of the License. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program. If not, see . + */ +#pragma once + +#include + +using namespace std; + +namespace relpipe { +namespace tr { +namespace scheme { + +class SchemeException { +private: + wstring message; +public: + + SchemeException(wstring message) : message(message) { + } + + wstring getMessge() { + return message; + } + +}; + +} +} +} \ No newline at end of file diff -r 2354c9058fb6 -r e87c231afb77 src/SchemeHandler.h --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/src/SchemeHandler.h Fri Sep 25 01:59:16 2020 +0200 @@ -0,0 +1,321 @@ +/** + * Relational pipes + * Copyright © 2019 František Kučera (Frantovo.cz, GlobalCode.info) + * + * This program is free software: you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation, version 3 of the License. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program. If not, see . + */ +#pragma once + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include + +#include +#include +#include +#include + +#include + +#include + +#include "Configuration.h" +#include "SchemeException.h" + +namespace relpipe { +namespace tr { +namespace scheme { + +using namespace std; +using namespace relpipe; +using namespace relpipe::reader; +using namespace relpipe::reader::handlers; + +class SchemeHandler : public RelationalReaderValueHandler { +private: + std::wstring_convert> convertor; // TODO: support also other encodings or use always UTF-8 between C++ and Scheme + + Configuration configuration; + writer::RelationalWriter* relationalWriter; + + RelationConfiguration* currentRelationConfiguration = nullptr; + vector currentReaderMetadata; + vector currentWriterMetadata; + std::map currenVariablesMapping; + integer_t currentAttributeIndex = 0; + boolean_t includeCurrentRecord = false; + + void add(vector& readerAttributes, vector& writerAttributes) { + for (AttributeMetadata readerAttributes : readerAttributes) + writerAttributes.push_back({ + readerAttributes.getAttributeName(), + relationalWriter->toTypeId(readerAttributes.getTypeName()) + }); + } + + void generateVariableMappings() { + currenVariablesMapping.clear(); + for (AttributeMetadata m : currentReaderMetadata) currenVariablesMapping[m.getAttributeName()] = L""; + for (writer::AttributeMetadata m : currentWriterMetadata) currenVariablesMapping[m.attributeName] = L""; + + for (std::pair m : currenVariablesMapping) { + currenVariablesMapping[m.first] = escapeAwkVariableName(m.first); + } + } + + /** + * @param attributeName name from relational pipe + * @return variable name in Scheme + */ + string_t a2v(const string_t& attributeName) { + if (currenVariablesMapping.find(attributeName) != currenVariablesMapping.end()) return currenVariablesMapping[attributeName]; + else throw SchemeException(L"Unable to find value in currenVariablesMapping"); + } + + template bool containsValue(std::map map, V value) { // TODO: common function (Scheme, AWK) + for (std::pair p : map) if (p.second == value) return true; + return false; + } + + string_t escapeAwkVariableName(const string_t& attributeName, bool addPrefix = true) { + std::wregex badCharacters(L"\\s"); + string_t name = std::regex_replace(attributeName, badCharacters, L"-"); + + if (addPrefix) name = L"$" + name; // $ = standard attribute-variable prefix + + if (containsValue(currenVariablesMapping, name)) return escapeAwkVariableName(L"$" + name, false); // $ = different prefix added to distinguish two attributes with ambiguous names + else return name; + + } + + void debugVariableMapping(const string_t& relationName) { + relationalWriter->startRelation(relationName + L".variableMapping",{ + {L"attribute", writer::TypeId::STRING}, + {L"variable", writer::TypeId::STRING}, + }, true); + + for (std::pair m : currenVariablesMapping) { + relationalWriter->writeAttribute(m.first); + relationalWriter->writeAttribute(m.second); + } + } + + SCM toSchemeSymbol(const string_t& name) { + return scm_string_to_symbol(scm_from_locale_string(convertor.to_bytes(name).c_str())); + } + + /** + * @param code scheme source code e.g. (+ 1 2 3) or #t + * @param defaultReturnValue is returned if code is empty + * @return result of code execution or defaultReturnValue + */ + SCM evalSchemeCode(const string_t& code, SCM defaultReturnValue = SCM_BOOL_F) { + if (code.size()) return scm_eval_string(toSchemeValue(&code, typeid (string_t), TypeId::STRING)); + else return defaultReturnValue; + } + + SCM toSchemeValue(const void* value, const std::type_info& typeInfo, TypeId type) { + switch (type) { + case TypeId::BOOLEAN: + { + assert(typeInfo == typeid (boolean_t)); + auto* typedValue = static_cast (value); + return *typedValue ? SCM_BOOL_T : SCM_BOOL_F; + } + case TypeId::INTEGER: + { + assert(typeInfo == typeid (integer_t)); + auto* typedValue = static_cast (value); + return scm_from_int64(*typedValue); + } + case TypeId::STRING: + { + assert(typeInfo == typeid (string_t)); + auto* typedValue = static_cast (value); + return scm_from_locale_string(convertor.to_bytes(*typedValue).c_str()); + } + default: + throw cli::RelpipeCLIException(L"Unsupported type in toSchemeValue()", cli::CLI::EXIT_CODE_UNEXPECTED_ERROR); + } + } + + void defineSchemeVariable(const string_t& name, const void* value, const std::type_info& typeInfo, TypeId type) { + scm_define(toSchemeSymbol(name), toSchemeValue(value, typeInfo, type)); + } + + /** + * TODO: use a common method + */ + bool parseBoolean(const string_t& value) { + if (value == L"true") return true; + else if (value == L"false") return false; + else throw relpipe::cli::RelpipeCLIException(L"Unable to parse boolean value: " + value + L" (expecting true or false)", relpipe::cli::CLI::EXIT_CODE_BAD_CLI_ARGUMENTS); + } + + void defineSchemeVariable(const DefinitionRecipe& definition) { + switch (relationalWriter->toTypeId(definition.type)) { + case writer::TypeId::BOOLEAN: + { + boolean_t value = parseBoolean(definition.value); + defineSchemeVariable(definition.name, &value, typeid (value), TypeId::BOOLEAN); + break; + } + case writer::TypeId::INTEGER: + { + integer_t value = stol(definition.value); + defineSchemeVariable(definition.name, &value, typeid (value), TypeId::INTEGER); + break; + } + case writer::TypeId::STRING: + { + defineSchemeVariable(definition.name, &definition.value, typeid (definition.value), TypeId::STRING); + break; + } + default: + throw cli::RelpipeCLIException(L"Unsupported type in defineSchemeVariable(): " + definition.type, cli::CLI::EXIT_CODE_UNEXPECTED_ERROR); + } + } + + void undefineSchemeVariable(const string_t& name) { + scm_define(toSchemeSymbol(name), scm_make_undefined_variable()); // undefined != (define n) + // TODO: or use: scm_variable_unset_x() ? + } + + void writeSchemeValueToAttribute(const writer::AttributeMetadata& attribute) { + string_t variableName = a2v(attribute.attributeName); + SCM schemeValue = scm_eval_string(toSchemeValue(&variableName, typeid (variableName), TypeId::STRING)); + + switch (attribute.typeId) { + case writer::TypeId::BOOLEAN: + { + boolean_t value = scm_to_bool(schemeValue); + return relationalWriter->writeAttribute(&value, typeid (value)); + } + case writer::TypeId::INTEGER: + { + integer_t value = scm_to_int64(schemeValue); + return relationalWriter->writeAttribute(&value, typeid (value)); + } + case writer::TypeId::STRING: + { + char* ch = scm_to_locale_string(schemeValue); + string_t value = convertor.from_bytes(ch); + free(ch); + return relationalWriter->writeAttribute(&value, typeid (value)); + } + default: + throw cli::RelpipeCLIException(L"Unsupported type in writeSchemeValueToAttribute()", cli::CLI::EXIT_CODE_UNEXPECTED_ERROR); + } + } + + /** + * Read from the Scheme variables and write to relational output stream. + */ + void writeCurrentRecord() { + for (auto attribute : currentWriterMetadata) writeSchemeValueToAttribute(attribute); + } + + void writeMoreRecords() { + while (scm_to_bool(evalSchemeCode(currentRelationConfiguration->schemeHasMoreRecords, SCM_BOOL_F))) writeCurrentRecord(); + } + +public: + + SchemeHandler(writer::RelationalWriter* relationalWriter, Configuration& configuration) : relationalWriter(relationalWriter), configuration(configuration) { + } + + void startRelation(string_t name, vector attributes) override { + if (currentRelationConfiguration) { + evalSchemeCode(currentRelationConfiguration->schemeAfterRecords); + writeMoreRecords(); + for (DefinitionRecipe definition : currentRelationConfiguration->definitions) undefineSchemeVariable(definition.name); + } + for (auto attribute : currentReaderMetadata) undefineSchemeVariable(attribute.getAttributeName()); + + for (DefinitionRecipe definition : configuration.definitions) defineSchemeVariable(definition); + + currentRelationConfiguration = nullptr; + for (int i = 0; i < configuration.relationConfigurations.size(); i++) { + if (regex_match(name, wregex(configuration.relationConfigurations[i].relation))) { + currentRelationConfiguration = &configuration.relationConfigurations[i]; + for (DefinitionRecipe definition : currentRelationConfiguration->definitions) defineSchemeVariable(definition); + break; // it there are multiple matches, only the first configuration is used + } + } + + currentReaderMetadata = attributes; + // TODO: move to a reusable method (or use same metadata on both reader and writer side?) + currentWriterMetadata.clear(); + if (currentRelationConfiguration && currentRelationConfiguration->writerMetadata.size()) { + if (currentRelationConfiguration->inputAttributesPrepend) add(currentReaderMetadata, currentWriterMetadata); + currentWriterMetadata.insert(currentWriterMetadata.end(), currentRelationConfiguration->writerMetadata.begin(), currentRelationConfiguration->writerMetadata.end()); + if (currentRelationConfiguration->inputAttributesAppend) add(currentReaderMetadata, currentWriterMetadata); + } else { + add(currentReaderMetadata, currentWriterMetadata); + } + + generateVariableMappings(); + + if (currentRelationConfiguration && currentRelationConfiguration->debugVariableMapping) debugVariableMapping(name); + + if (!currentRelationConfiguration || !currentRelationConfiguration->drop) relationalWriter->startRelation(name, currentWriterMetadata, true); + + if (currentRelationConfiguration) { + // TODO: better variable name, object, function? + defineSchemeVariable(L"relpipe-relation-name", &name, typeid (name), TypeId::STRING); + evalSchemeCode(currentRelationConfiguration->schemeBeforeRecords); + } + } + + void attribute(const void* value, const std::type_info& type) override { + if (currentRelationConfiguration) { + defineSchemeVariable(a2v(currentReaderMetadata[currentAttributeIndex].getAttributeName()), value, type, currentReaderMetadata[currentAttributeIndex].getTypeId()); + + currentAttributeIndex++; + + // TODO: > 0 ?: + if (currentAttributeIndex > 0 && currentAttributeIndex % currentReaderMetadata.size() == 0) { + evalSchemeCode(currentRelationConfiguration->schemeForEach); + includeCurrentRecord = scm_to_bool(evalSchemeCode(currentRelationConfiguration->schemeWhere, SCM_BOOL_T)); + if (includeCurrentRecord && !currentRelationConfiguration->drop) writeCurrentRecord(); + includeCurrentRecord = false; + writeMoreRecords(); + } + + currentAttributeIndex = currentAttributeIndex % currentReaderMetadata.size(); + } else { + relationalWriter->writeAttribute(value, type); + } + } + + void endOfPipe() { + if (currentRelationConfiguration) { + evalSchemeCode(currentRelationConfiguration->schemeAfterRecords); + writeMoreRecords(); + } + } + +}; + +} +} +} diff -r 2354c9058fb6 -r e87c231afb77 src/relpipe-tr-guile.cpp --- a/src/relpipe-tr-guile.cpp Sat Jun 06 01:50:45 2020 +0200 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,78 +0,0 @@ -/** - * Relational pipes - * Copyright © 2019 František Kučera (Frantovo.cz, GlobalCode.info) - * - * This program is free software: you can redistribute it and/or modify - * it under the terms of the GNU General Public License as published by - * the Free Software Foundation, version 3 of the License. - * - * This program is distributed in the hope that it will be useful, - * but WITHOUT ANY WARRANTY; without even the implied warranty of - * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the - * GNU General Public License for more details. - * - * You should have received a copy of the GNU General Public License - * along with this program. If not, see . - */ - -#include -#include -#include - -#include -#include - -#include -#include -#include - -#include -#include -#include -#include - -#include "GuileHandler.h" -#include "CLIParser.h" - -using namespace relpipe::cli; -using namespace relpipe::tr::guile; - -static void relpipeMain(void *closure, int argc, char **argv) { - setlocale(LC_ALL, ""); - CLI::untieStdIO(); - CLI cli(argc, argv); - - int resultCode = CLI::EXIT_CODE_UNEXPECTED_ERROR; - - try { - CLIParser cliParser; - Configuration configuration = cliParser.parse(cli.arguments()); - std::shared_ptr reader(reader::Factory::create(std::cin)); - std::shared_ptr writer(writer::Factory::create(std::cout)); - GuileHandler handler(writer.get(), configuration); - reader->addHandler(&handler); - reader->process(); - - resultCode = CLI::EXIT_CODE_SUCCESS; - - } catch (RelpipeCLIException& e) { - fwprintf(stderr, L"Caught CLI exception: %ls\n", e.getMessge().c_str()); - fwprintf(stderr, L"Debug: Input stream: eof=%ls, lastRead=%d\n", (cin.eof() ? L"true" : L"false"), cin.gcount()); - resultCode = e.getExitCode(); - } catch (RelpipeReaderException& e) { - fwprintf(stderr, L"Caught Reader exception: %ls\n", e.getMessge().c_str()); - fwprintf(stderr, L"Debug: Input stream: eof=%ls, lastRead=%d\n", (cin.eof() ? L"true" : L"false"), cin.gcount()); - resultCode = CLI::EXIT_CODE_DATA_ERROR; - } catch (GuileException& e) { - fwprintf(stderr, L"Caught Guile exception: %ls\n", e.getMessge().c_str()); - fwprintf(stderr, L"Debug: Input stream: eof=%ls, lastRead=%d\n", (cin.eof() ? L"true" : L"false"), cin.gcount()); - resultCode = CLI::EXIT_CODE_UNEXPECTED_ERROR; - } - - exit(resultCode); -} - -int main(int argc, char**argv) { - scm_boot_guile(argc, argv, relpipeMain, nullptr); - return 222; // never reached – see exit(resultCode) above -} diff -r 2354c9058fb6 -r e87c231afb77 src/relpipe-tr-scheme.cpp --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/src/relpipe-tr-scheme.cpp Fri Sep 25 01:59:16 2020 +0200 @@ -0,0 +1,78 @@ +/** + * Relational pipes + * Copyright © 2019 František Kučera (Frantovo.cz, GlobalCode.info) + * + * This program is free software: you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation, version 3 of the License. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program. If not, see . + */ + +#include +#include +#include + +#include +#include + +#include +#include +#include + +#include +#include +#include +#include + +#include "SchemeHandler.h" +#include "CLIParser.h" + +using namespace relpipe::cli; +using namespace relpipe::tr::scheme; + +static void relpipeMain(void *closure, int argc, char **argv) { + setlocale(LC_ALL, ""); + CLI::untieStdIO(); + CLI cli(argc, argv); + + int resultCode = CLI::EXIT_CODE_UNEXPECTED_ERROR; + + try { + CLIParser cliParser; + Configuration configuration = cliParser.parse(cli.arguments()); + std::shared_ptr reader(reader::Factory::create(std::cin)); + std::shared_ptr writer(writer::Factory::create(std::cout)); + SchemeHandler handler(writer.get(), configuration); + reader->addHandler(&handler); + reader->process(); + + resultCode = CLI::EXIT_CODE_SUCCESS; + + } catch (RelpipeCLIException& e) { + fwprintf(stderr, L"Caught CLI exception: %ls\n", e.getMessge().c_str()); + fwprintf(stderr, L"Debug: Input stream: eof=%ls, lastRead=%d\n", (cin.eof() ? L"true" : L"false"), cin.gcount()); + resultCode = e.getExitCode(); + } catch (RelpipeReaderException& e) { + fwprintf(stderr, L"Caught Reader exception: %ls\n", e.getMessge().c_str()); + fwprintf(stderr, L"Debug: Input stream: eof=%ls, lastRead=%d\n", (cin.eof() ? L"true" : L"false"), cin.gcount()); + resultCode = CLI::EXIT_CODE_DATA_ERROR; + } catch (SchemeException& e) { + fwprintf(stderr, L"Caught Scheme exception: %ls\n", e.getMessge().c_str()); + fwprintf(stderr, L"Debug: Input stream: eof=%ls, lastRead=%d\n", (cin.eof() ? L"true" : L"false"), cin.gcount()); + resultCode = CLI::EXIT_CODE_UNEXPECTED_ERROR; + } + + exit(resultCode); +} + +int main(int argc, char**argv) { + scm_boot_guile(argc, argv, relpipeMain, nullptr); + return 222; // never reached – see exit(resultCode) above +}