relpipe-data/examples-in-filesystem-tr-sql-indexing.xml
branchv_0
changeset 297 192b0059a6c4
parent 281 0b6b1781a0a5
equal deleted inserted replaced
296:418e11eb6fea 297:192b0059a6c4
    24 			the <code>/bin/</code> and <code>/usr/bin/</code> directories:
    24 			the <code>/bin/</code> and <code>/usr/bin/</code> directories:
    25 		</p>
    25 		</p>
    26 		
    26 		
    27 		<m:pre jazyk="bash"><![CDATA[find /bin/ /usr/bin/ -print0 \
    27 		<m:pre jazyk="bash"><![CDATA[find /bin/ /usr/bin/ -print0 \
    28 	| relpipe-in-filesystem --relation "program" \
    28 	| relpipe-in-filesystem --relation "program" \
    29 	| relpipe-tr-sql --file bin.sqlite --file-keep true]]></m:pre>
    29 	| relpipe-tr-sql --data-source-string 'Driver=SQLite3;Database=file:bin.sqlite']]></m:pre>
    30 	
    30 	
    31 		<p>
    31 		<p>
    32 			This index allows us to do fast searches and various analysis.
    32 			This index allows us to do fast searches and various analysis.
    33 			We can e.g. find 20 largest binaries:
    33 			We can e.g. find 20 largest binaries:
    34 		</p>
    34 		</p>
    35 		
    35 		
    36 		<m:pre jazyk="bash"><![CDATA[relpipe-in-sql \
    36 		<m:pre jazyk="bash"><![CDATA[relpipe-in-sql \
    37 	--file bin.sqlite \
    37 	--data-source-string 'Driver=SQLite3;Database=file:bin.sqlite' \
    38 	--relation "largest" \
    38 	--relation "largest" \
    39 		"SELECT path, size FROM program WHERE type = 'f' ORDER BY size DESC LIMIT 20" \
    39 		"SELECT path, size FROM program WHERE type = 'f' ORDER BY size DESC LIMIT 20" \
    40 	| relpipe-out-tabular]]></m:pre>
    40 	| relpipe-out-tabular]]></m:pre>
    41 	
    41 	
    42 		<p>How very:</p>
    42 		<p>How very:</p>
    73 			In this example, we get lists of dynamically linked libraries using the <code>ldd</code> tool
    73 			In this example, we get lists of dynamically linked libraries using the <code>ldd</code> tool
    74 			for each binary and store the lists in our index:
    74 			for each binary and store the lists in our index:
    75 		</p>
    75 		</p>
    76 
    76 
    77 		<m:pre jazyk="bash"><![CDATA[relpipe-in-sql \
    77 		<m:pre jazyk="bash"><![CDATA[relpipe-in-sql \
    78 		--file bin.sqlite \
    78 		--data-source-string 'Driver=SQLite3;Database=file:bin.sqlite' \
    79 		--relation bin "SELECT path FROM program WHERE type = 'f'" \
    79 		--relation bin "SELECT path FROM program WHERE type = 'f'" \
    80 	| relpipe-out-nullbyte \
    80 	| relpipe-out-nullbyte \
    81 	| while read_nullbyte f; do 
    81 	| while read_nullbyte f; do 
    82 		ldd "$f" | perl -ne 'if (/ => (.*) \(/) { print "$ENV{f},$1\n"; }';
    82 		ldd "$f" | perl -ne 'if (/ => (.*) \(/) { print "$ENV{f},$1\n"; }';
    83 	  done \
    83 	  done \
    84 	| relpipe-in-csv \
    84 	| relpipe-in-csv \
    85 		"dependency" \
    85 		"dependency" \
    86 			"program" string \
    86 			"program" string \
    87 			"library" string \
    87 			"library" string \
    88 	| relpipe-tr-sql --file bin.sqlite]]></m:pre>
    88 	| relpipe-tr-sql --data-source-string 'Driver=SQLite3;Database=file:bin.sqlite']]></m:pre>
    89 	
    89 	
    90 		<p>And then we can make a „popularity contest“ and find 20 most often used libraries:</p>
    90 		<p>And then we can make a „popularity contest“ and find 20 most often used libraries:</p>
    91 		
    91 		
    92 		<m:pre jazyk="bash"><![CDATA[relpipe-in-sql \
    92 		<m:pre jazyk="bash"><![CDATA[relpipe-in-sql \
    93 	--file bin.sqlite \
    93 	--data-source-string 'Driver=SQLite3;Database=file:bin.sqlite' \
    94 	--relation "popular_libraries" "
    94 	--relation "popular_libraries" "
    95 		SELECT 
    95 		SELECT 
    96 			d.library, 
    96 			d.library, 
    97 			count(*) AS count 
    97 			count(*) AS count 
    98 		FROM dependency AS d 
    98 		FROM dependency AS d 
   136 			In future versions there might be an option to gather more file metadata like hashes, Exif etc.
   136 			In future versions there might be an option to gather more file metadata like hashes, Exif etc.
   137 			But even in the current version, it is possible to gather any literally metadata using a custom script (as we have shown with <code>ldd</code> above).
   137 			But even in the current version, it is possible to gather any literally metadata using a custom script (as we have shown with <code>ldd</code> above).
   138 			Extended attributes are already supported (the <code>--xattr</code> option).
   138 			Extended attributes are already supported (the <code>--xattr</code> option).
   139 		</p>
   139 		</p>
   140 		
   140 		
       
   141 		<p>
       
   142 			n.b. if we use a database frequently it is convenient to configure it as a data source in the <code>~/.odbc.ini</code> file
       
   143 			– and then connect to it using the <code>--data-source-name</code> option and its name.
       
   144 		</p>
       
   145 		
   141 	</text>
   146 	</text>
   142 
   147 
   143 </stránka>
   148 </stránka>