From 09e330aa24e2015be72ad0f6412adcd64cf3499e Mon Sep 17 00:00:00 2001 From: Olli-Antti Kivilahti Date: Tue, 3 Feb 2015 13:52:26 +0200 Subject: [PATCH] Bug 13660: Exclude export phase and use existing exported MARCXML - rebuild_zebra_sliced.sh When looking for a bad MARC Record using the rebuild_zebra_sliced.sh, it is useful to skip the complete MARCXML exporting from Koha and reuse the exported files for Zebra indexing. This patch adds a new parameter: -x | --exclude-export Do not export Biblios from Koha, but use the existing export-dir Which depends on the: -d | --export-dir Where rebuild_zebra.pl will export data Default: $EXPORTDIR !---------! ! TEST PLAN ! !---------! 1. Run "./rebuild_zebra_sliced.sh --length 1000" to export 1000 MARC Records and slice them to one big 1000-Record chunk. 2. Realize that you get an imaginary "stack smashing detected"-error crashing your indexing at some Record you dont know of and can't make out from the indexing logging. 3. Start looking for the bad Record by running: "./rebuild_zebra_sliced.sh --exlude-export --chunk-size 10" To skip Biblios export from Koha which takes ~2h and get straight into splitting your exported biblios to chunks of 10, and indexing them. You know which chunk fails so it is much easier to find the issue there. Signed-off-by: Katrin Fischer Signed-off-by: Marcel de Rooy Signed-off-by: Jonathan Druart --- misc/migration_tools/rebuild_zebra_sliced.sh | 15 ++++++++++++--- 1 file changed, 12 insertions(+), 3 deletions(-) diff --git a/misc/migration_tools/rebuild_zebra_sliced.sh b/misc/migration_tools/rebuild_zebra_sliced.sh index 7ddf32e9c5..1eeb551b31 100755 --- a/misc/migration_tools/rebuild_zebra_sliced.sh +++ b/misc/migration_tools/rebuild_zebra_sliced.sh @@ -24,6 +24,8 @@ $scriptname -h Default: $CHUNKSSIZE -d | --export-dir Where rebuild_zebra.pl will export data Default: $EXPORTDIR + -x | --exclude-export Do not export Biblios from Koha, but use the existing + export-dir -L | --log-dir Log directory Default: $LOGDIR -r | --remove-logs Clean log directory before start @@ -110,6 +112,7 @@ OFFSET=0 LENGTH= CHUNKSSIZE=10000 EXPORTDIR=/tmp/rebuild/export +EXCLUDEEXPORT=no LOGDIR=/tmp/rebuild/logs RMLOGS=no NOCONFIRM=no @@ -140,6 +143,9 @@ while [ $1 ]; do shift LOGDIR=$1 ;; + -x | --exclude-export ) + EXCLUDEEXPORT=yes + ;; -r | --remove-logs ) RMLOGS=yes ;; @@ -235,6 +241,7 @@ echo "Start at offset: $OFFSET" echo "Total number of records to index: $LENGTH" echo "Initial chunk size: $CHUNKSSIZE" echo "Export directory: $EXPORTDIR" +echo "Exclude re-exporting: $EXCLUDEEXPORT" echo "Log directory: $LOGDIR" echo "Remove logs before start? $RMLOGS" echo "Type of record: $TYPE" @@ -274,9 +281,11 @@ if [ $RMLOGS = "yes" ]; then rm -f $LOGDIR/*.log fi -REBUILDZEBRA_CMD="$REBUILDZEBRA $TYPESWITCH -v -x -k -d $EXPORTDIR --offset $OFFSET --length $LENGTH --skip-index" -echo "\n$REBUILDZEBRA_CMD" -$REBUILDZEBRA_CMD +if [ $EXCLUDEEXPORT = "no" ]; then + REBUILDZEBRA_CMD="$REBUILDZEBRA $TYPESWITCH -v -x -k -d $EXPORTDIR --offset $OFFSET --length $LENGTH --skip-index" + echo "\n$REBUILDZEBRA_CMD" + $REBUILDZEBRA_CMD +fi EXPORTFILE= case $TYPE in -- 2.39.5