Commit graph

68 commits

Author SHA1 Message Date
Galen Charlton
ad4e02f91d warn on attempts to add duplicate item barcodes during batch import
Signed-off-by: Chris Cormack <crc@liblime.com>
Signed-off-by: Joshua Ferraro <jmf@liblime.com>
2007-12-02 15:06:24 -06:00
Paul POULAIN
262a6e2a9a Updating rebuild_zebra.pl : now uses etc config files
There are only 2 UNIMARC specific files (.abs and .chr), they have been moved to etc/zebradb

The rebuild_zebra.pl takes all config file from this location now.
the misc/zebra/ can be removed (and will be soon)

Signed-off-by: Joshua Ferraro <jmf@liblime.com>
2007-11-25 17:07:46 -06:00
Joshua Ferraro
cee40a741d adding $DEBUG warnings to nozebra
Signed-off-by: Chris Cormack <crc@liblime.com>
Signed-off-by: Joshua Ferraro <jmf@liblime.com>
2007-11-24 09:07:44 -06:00
Paul POULAIN
f38b7598fc still handling better dirty MARC records
this time it's when a biblio don't have biblionumber, has a 100$a field, and it's invalid.

1 biblio in my 300 000 DB (and it was biblio 294 359, of course !)

Signed-off-by: Chris Cormack <crc@liblime.com>
Signed-off-by: Joshua Ferraro <jmf@liblime.com>
2007-11-20 16:20:50 -06:00
Mason James
78abbe94d3 little SQL typo fix, now builds 'NoZebraIndexes' index mapping correctly.
Signed-off-by: Chris Cormack <crc@liblime.com>
Signed-off-by: Joshua Ferraro <jmf@liblime.com>
2007-11-20 16:19:05 -06:00
Paul POULAIN
f1bca9ba50 missing biblionumber AND missing unimarc 100 was not properly handled
now, adding both on the fly when needed.
(had 2 biblios like that in a 290 000 DB, but was enought to have M::F::X complaining & diing !)

Signed-off-by: Chris Cormack <crc@liblime.com>
Signed-off-by: Joshua Ferraro <jmf@liblime.com>
2007-11-17 11:25:07 -06:00
Paul POULAIN
ef1ac56857 handling wrong MARC record better
Signed-off-by: Chris Cormack <crc@liblime.com>
Signed-off-by: Joshua Ferraro <jmf@liblime.com>
2007-11-12 17:13:00 -06:00
Mason James
c846ed00db utf8 handling fixes 'Wide character in print at' encoding errors.
Signed-off-by: Chris Cormack <crc@liblime.com>
Signed-off-by: Joshua Ferraro <jmf@liblime.com>
2007-11-12 17:10:17 -06:00
Mason James
a51118833c wrapping AddBiblio(), and AddItem() in evals{} to protect import from failure due to bad records.
Signed-off-by: Chris Cormack <crc@liblime.com>
Signed-off-by: Joshua Ferraro <jmf@liblime.com>
2007-11-11 18:44:13 -06:00
Mason James
f6b17c1de9 wrapping write to *.iso file in eval{}, to handle failure, caused by bad record.
Signed-off-by: Chris Cormack <crc@liblime.com>
Signed-off-by: Joshua Ferraro <jmf@liblime.com>
2007-11-11 18:44:12 -06:00
Paul POULAIN
9149a711fb bugfixes to config files for zebra 2.0.18
those 2 lines are invalid

Signed-off-by: Chris Cormack <crc@liblime.com>
Signed-off-by: Joshua Ferraro <jmf@liblime.com>
2007-11-08 17:50:00 -06:00
Paul POULAIN
b7eb9e1b5c rebuild_zebra now handle correctly improper authorities records
(missing 100 field are automatically added)

Signed-off-by: Chris Cormack <crc@liblime.com>
Signed-off-by: Joshua Ferraro <jmf@liblime.com>
2007-11-07 08:18:24 -06:00
Paul POULAIN
bb5cea8e56 deal with wrong authorities when exporting for zebra
(authorities that don't have a 001 field containing authid)

also comment some code when exporting biblios (NOT tested, hdl,pls confirm this commit)

Signed-off-by: Chris Cormack <crc@liblime.com>
Signed-off-by: Joshua Ferraro <jmf@liblime.com>
2007-11-07 08:18:19 -06:00
Paul POULAIN
89b9e8f8c1 skip empty records (new GetMarcRecord behaviour that returns empty string and not empty MARC::Record)
Signed-off-by: Chris Cormack <crc@liblime.com>
Signed-off-by: Joshua Ferraro <jmf@liblime.com>
2007-10-31 19:41:49 -05:00
Paul POULAIN
1cd11f4d54 fixes in NoZebra search & indexing
- the quotemeta was wrong (and introduced some bugs in diacritics)
- fixing some bugs that appear only sometimes : the union was done including weight, which is wrong & resulted in missing some results (when various weighting)

Signed-off-by: Chris Cormack <crc@liblime.com>
Signed-off-by: Joshua Ferraro <jmf@liblime.com>
2007-10-31 05:53:36 -05:00
Paul POULAIN
fa26bcc037 rebuild_unimarc_100 : better handling of unusual cases
If 100$a repeated, the scripts failed to handle that correctly

Signed-off-by: Chris Cormack <crc@liblime.com>
Signed-off-by: Joshua Ferraro <jmf@liblime.com>
2007-10-24 17:08:56 -05:00
Paul POULAIN
cd8a565a6a temp
Signed-off-by: Chris Cormack <crc@liblime.com>
Signed-off-by: Joshua Ferraro <jmf@liblime.com>
2007-10-24 17:08:40 -05:00
Paul POULAIN
837e5c5e94 less verbose
Signed-off-by: Chris Cormack <crc@liblime.com>
Signed-off-by: Joshua Ferraro <jmf@liblime.com>
2007-10-24 17:06:36 -05:00
Joshua Ferraro
9d29ce5d58 improvements to zebra configuration files
Signed-off-by: Chris Cormack <crc@liblime.com>
Signed-off-by: Joshua Ferraro <jmf@liblime.com>
2007-10-21 19:14:12 -05:00
Paul POULAIN
1ac38782a1 #1474 : Bulkmarcimport croaks when Log is ON
set to 0 and restore at the end of the import

Signed-off-by: Chris Cormack <crc@liblime.com>
Signed-off-by: Joshua Ferraro <jmf@liblime.com>
2007-10-11 14:53:59 -05:00
Paul POULAIN
057d654a5b skipping wrong XMLs when rebuilding nozebra indexes
Signed-off-by: Chris Cormack <crc@liblime.com>
Signed-off-by: Joshua Ferraro <jmf@liblime.com>
2007-10-09 19:11:47 -05:00
Paul POULAIN
49ef1df969 Adding a new option to rebuildzebra : noxml
This option uses the iso2709 version of the MARC record instead of the XML one
(biblioitems.marc vs biblioitems.marcxml)
No change if the parameter is not set.

Signed-off-by: Chris Cormack <crc@liblime.com>
Signed-off-by: Joshua Ferraro <jmf@liblime.com>
2007-10-09 19:07:36 -05:00
Joshua Ferraro
827d27111f adding barcode index
Signed-off-by: Chris Cormack <crc@liblime.com>
Signed-off-by: Joshua Ferraro <jmf@liblime.com>
2007-10-06 21:46:02 -05:00
Paul POULAIN
375d2f1158 (minor) updating doc & removing warn
Signed-off-by: Chris Cormack <crc@liblime.com>
2007-10-03 14:57:12 -05:00
Chris Catalfo
502487e2ba Added basic MARC21 index definitions.
Signed-off-by: Chris Cormack <crc@liblime.com>
2007-10-02 15:38:32 -05:00
Paul POULAIN
6f7efca7e1 BUGFIX for browser and nozebra tables
- adding browser and nozebra table definition to kohastructure & updatedatabase
- bumping to 3.00.00.005

Signed-off-by: Chris Cormack <crc@liblime.com>
2007-10-02 04:35:49 -05:00
Joshua Ferraro
ae34e8f45a changing the name of the zebra password file to passwd
Signed-off-by: Chris Cormack <crc@liblime.com>
2007-10-01 23:14:47 -05:00
Joshua Ferraro
b87d4924b9 commenting out set_service_options, but also removes commit op
Signed-off-by: Chris Cormack <crc@liblime.com>
2007-10-01 17:40:31 -05:00
Ryan Higgins
c44efe7b84 fix bad call to GetMarcFromKohaField in bulkmarcimport, and add -fk param, allowing disabling of fk constraints during import.
Signed-off-by: Chris Cormack <crc@liblime.com>
2007-09-30 21:16:50 -05:00
Paul POULAIN
0d7a4aafd0 BUGFIX : NoZebra indexing was wrong for accented words
Signed-off-by: Chris Cormack <crc@liblime.com>
2007-09-26 05:28:37 -05:00
Paul POULAIN
623ac80330 BUGFIXES : 3 (marc_biblio, check biblionumber, ModMarcBiblio API)
- use biblio instead of marc_biblio,
- better check that biblionumber is correctly stored
- fix an buggy API call when ModMarcBiblio

Signed-off-by: Chris Cormack <crc@liblime.com>
2007-09-13 17:18:50 -05:00
Paul POULAIN
ec7bd0b2ff (unimarc specific) BUGFIX : if 100$a exist but is not 35 char long, MARC::File::XML may fail
So, add blanks if needed...

Signed-off-by: Chris Cormack <crc@liblime.com>
2007-09-13 17:17:56 -05:00
tipaul
1399945a75 eval() on getAuthority & getBiblio to avoid a script failure 2007-08-01 09:20:03 +00:00
toins
5e7b171686 adding an eval to don't die if an error occurs 2007-07-19 09:48:22 +00:00
tipaul
23427c51b9 some fixes (and only fixes) 2007-06-15 13:44:44 +00:00
toins
6dfb0dca36 next if there is an error getting the biblio. 2007-06-11 15:22:59 +00:00
toins
4728830e34 it's faster to 'truncate' instead of using 'delete from'... 2007-06-08 09:41:14 +00:00
tipaul
5dd3f0229a bugfixes (various), handling utf-8 without guessencoding (as suggested by joshua, fixing some zebra config files -for french but should be interesting for other languages- 2007-06-06 13:08:35 +00:00
btoumi
68bcf35387 delete space in beggining of the script to accept script launch 2007-05-25 10:00:54 +00:00
tipaul
0569dccd5f some changes to default zebra config for better searches 2007-05-25 09:34:30 +00:00
tipaul
651b075197 small script to check XML parser. Remember that PurePerl Parser is buggued and can t handle utf8 correctly 2007-05-25 09:33:58 +00:00
tipaul
5ff7fcffa4 Bugfixes & improvements (various and minor) :
- updating templates to have tmpl_process3.pl running without any errors
- adding a drupal-like css for prog templates (with 3 small images)
- fixing some bugs in circulation & other scripts
- updating french translation
- fixing some typos in templates
2007-05-22 09:13:54 +00:00
tipaul
ca201e36af Koha NoZebra :
- support for authorities
- some bugfixes in ordering and "CCL" parsing
- support for authorities <=> biblios walking

Seems I can do what I want now, so I consider its done, except for bugfixes that will be needed i m sure !
2007-05-10 14:45:15 +00:00
tipaul
e1d907c688 various bugfixes on parameters modules + adding default NoZebraIndexes systempreference if it's empty 2007-05-04 16:24:08 +00:00
tipaul
3e85c9e97f NoZebra SQL index management :
* adding 3 subs in Biblio.pm
- GetNoZebraIndexes, that get the index structure in a new systempreference (added with this commit)
- _DelBiblioNoZebra, that retrieve all index entries for a biblio and remove in a variable the biblio reference
- _AddBiblioNoZebra, that add index entries for a biblio.
Note that the 2 _Add and _Del subs work only in a hash variable, to speed up things in case of a modif (ie : delete+add). The effective SQL update is done in the ModZebra sub (that existed before, and dealed with zebra index).
I think the code has to be more deeply tested, but it works at least partially.
2007-05-02 16:44:31 +00:00
tipaul
4213b6ec98 improving NOzebra search :
- changing nozebra table to have biblionumber,title-ranking; (; is the entry separator. Now, if a value is several times in an index, it is stored only once, with a higher ranking (the ranking is the number of times the word appeard for this index)
- improving search to have ranking value (default order). The ranking is the sum of ranking of all terms. The list is ordered by ranking+title, from most to lower
2007-05-02 11:57:11 +00:00
hdl
097fef712a Removing $dbh from GetMarcFromKohaField (dbh is not used in this function.) 2007-04-27 14:00:48 +00:00
tipaul
b53be9cdaf Koha 3.0 nozebra 1st commit : the script misc/migration_tools/rebuild_nozebra.pl build the nozebra table, and, if you set NoZebra to Yes, queries will be done through zebra. TODO :
- add nozebra table management on biblio editing
- the index table content is hardcoded. I still have to add some specific systempref to let the library update it
- manage pagination (next/previous)
- manage facets
WHAT works :
- NZgetRecords : has exactly the same API & returns as zebra getQuery, except that some parameters are unused
- search & sort works quite good
- CQL parser is better that what I thought I could do : title="harry and sally" and publicationyear>2000 not itemtype=LIVR should work fine
2007-04-25 16:26:42 +00:00
tipaul
6b201757c1 some bugfixes for this script that automatically build zebra DB from default config files 2007-04-17 08:50:33 +00:00
tipaul
eba2552086 Code cleaning of Biblio.pm (continued)
All subs have be cleaned :
- removed useless
- merged some
- reordering Biblio.pm completly
- using only naming conventions

Seems to have broken nothing, but it still has to be heavily tested.
Note that Biblio.pm is now much more efficient than previously & probably more reliable as well.
2007-03-29 16:45:53 +00:00