Commit graph

15 commits

Author SHA1 Message Date
Julian Maurice
3cfc2ec7bd Bug 25067: Move PO file manipulation code into gulp tasks
misc/translator/translate was doing three different things:
- extract translatable strings
- create or update PO files
- install translated templates

This patch separates responsibilities by moving the string extraction
code into several 'xgettext-like' scripts and adds gulp tasks to
automate string extraction and PO files update

This has several benefits:

- gulp runs tasks in parallel, so it's a lot faster (updating all PO
  files is at least 10 times faster with my 4-cores CPU)

- there is no need for $KOHA_CONF to be defined
  LangInstaller.pm relied on $KOHA_CONF to get the different paths
  needed. I'm not sure why, since string extraction and PO update should
  work on source files, not installed files

- string extraction code can be more easily tested

This patch also brings a couple of fixes and improvements:

- TT string extraction (strings wrapped in [% t(...) %]) was done with
  Template::Parser and PPI, which was extremely slow, and had some
  problems (see bug 24797).
  This is now done with Locale::XGettext::TT2 (new dependency) which is
  a lot faster, and fixes bug 24797

- Fix header in 4 PO files

For backward compatibility, 'create' and 'update' commands of
misc/translator/translate can still be used and will execute the
corresponding gulp task

Test plan:
1. Run `yarn install` and install Locale::XGettext::TT2
2. Run `gulp po:update`
3. Verify the contents of updated PO files
4. Run `cd misc/translator && ./translate install <lang>`
5. Verify that all (templates, sysprefs, xslt, installer files) is
   correctly translated
6. Run `gulp po:create --lang <lang>` and verify that it created all PO
   files for that language
7. Run `prove t/misc/translator`

Signed-off-by: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com>
Need to install yarn & gulp, no errors

Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2020-11-06 09:46:11 +01:00
4de2185614
Bug 20116: Flush the caches
When languages are added or removed when need to flush the caches to
keep the interface consistent

Signed-off-by: Liz Rea <wizzyrea@gmail.com>
Signed-off-by: Bouzid Fergani <bouzid.fergani@inlibro.com>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
2020-04-17 09:23:21 +01:00
Bernardo Gonzalez Kriegel
e875e39806
Bug 24211: Compress/uncompress translation files
This patch adds the ability to compress/uncompress
translations files.

On update/install the files are uncompressed first

The only gain is to use less space.

To test:
1) Apply the patch
2) Go to misc/translator
3) Try it
   $ ./translate compress fr-FR (check po/*.gz)
   $ ./translate uncompress fr-FR ( check normal files)
4) Try again with verbose mode
   $ ./translate compress fr-FR -v (list compressed files)
   $ ./translate uncompress fr-FR -v (list uncompressed files)
5) Try update compressed files
   $ ./translate compress fr-FR
   $ ./translate update fr-FR (result is uncompressed)
6) Try install compressed files
   $ ./translate compress fr-FR
   $ ./translate install fr-FR
7) Try compress all
   $ ./translate compress  (add '-v' for verbose output)
8) Try uncompress all
   $ ./translate uncompress  (add '-v' for verbose output)

Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Bouzid Fergani <bouzid.fergani@inlibro.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
2020-02-24 13:19:31 +00:00
82716a0172
Bug 23084: Replace grep {^$var$} with grep {$_ eq $var}
We certainly faced 3 similar bugs due to this syntax: bug 23006, bug
22941 and bug 17526.

To prevent other issues related to this syntax this patch suggests to
replace them all in one go.

Test plan:
Confirm that the 2 syntaxes are similar
Eyeball the patch and confirm that there is no typo!

Signed-off-by: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com>
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
2020-02-17 10:44:45 +00:00
Olli-Antti Kivilahti
8aa86cdf2d Bug 21480: misc/translator/translate does not work with perl 5.26
Perl 5.26 (or earlier) introduced a security feature, where implicitly
including the program directory as a Perl library directory no longer
happens (perl -I. ).

This causes translate to fail because it cannot find the *.pm -files in
it's own directory.

This patch adds the familiar mantra
    use lib $FindBin::Bin;
to the relevant scripts.

To test:

1. Install Ubuntu18.04 or something else with Perl 5.26
2. Install Koha (we use the dev-install)
3. cd $KOHA_PATH/misc/translator/
4. perl translate create fi-FI
5. Observe problems with missing modules.

Signed-off-by: Julian Maurice <julian.maurice@biblibre.com>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Cannot recreate the issue right now but the changes make sense.

Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
2018-10-03 17:33:37 +00:00
Jonathan Druart
a6c9bd0eb5 Bug 9978: Replace license header with the correct license (GPLv3+)
Signed-off-by: Chris Nighswonger <cnighswonger@foundations.edu>
Signed-off-by: Tomas Cohen Arazi <tomascohen@gmail.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>

http://bugs.koha-community.org/show_bug.cgi?id=9987

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@gmail.com>
2015-04-20 09:59:38 -03:00
Bernardo Gonzalez Kriegel
0258d3a42d Bug 12535 - PROG/CCSR Deprecation: Make opac translation theme independent
This patch makes opac translation files theme independent
A side effect is that current prog translation file is
renamed.

But the renaming is not added, that gives a patch of ~60M!

Current list of languages are now obtained from pref files.

To test:
1) Apply the patch

2) For your preferred language, rename opac prog file
from 'xx-YY-i-opac-t-prog-v-3006000.po' to 'xx-YY-opac-prog.po'

3) Update that language, there must be no errors
cd misc/translation
./translate update xx-YY [-v]
Check updated po files

4) Install any language, there must be no errors
./translate install xx-YY
Check prog theme for that language

5) Create translation files. Remove a language or create new lang files
./translate create zz-WW
ls -l po/zz-WW*po
check there are 3 files (and *~) for opac

6) Check updated pod for LangInstaler.pm and translate script

When PROG/CCSR themes are removed, I'll add a Bug to remove
corresponding files.

Perhaps RM could add a followup to rename all files,
cd misc/translator/po
for old in $(ls *opac-t*); do
    new=$(echo $old | sed 's/-i-opac-t-prog-v-3006000/-opac-prog/');
    git mv $old $new;
done

Signed-off-by: Jonathan Druart <jonathan.druart@biblibre.com>
Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de>
Works as described, no problems found.

Signed-off-by: Tomas Cohen Arazi <tomascohen@gmail.com>
2014-10-29 22:00:22 -03:00
Jonathan Druart
35a88a5c44 Bug 12301: Remove warnings on translate scripts
Using perl v5.18, the following is warned on updating/installing
templates:
  Smartmatch is experimental at ./translate line 54.

Moreover, the following should exist with previous perl versions:

perlcritic xgettext
Subroutine prototypes used at line 29, column 1.  See page 194 of PBP.
(Severity: 5)
Subroutine prototypes used at line 44, column 1.  See page 194 of PBP.
(Severity: 5)
Subroutine prototypes used at line 60, column 1.  See page 194 of PBP.
(Severity: 5)
Subroutine prototypes used at line 72, column 1.  See page 194 of PBP.
(Severity: 5)
Subroutine prototypes used at line 89, column 1.  See page 194 of PBP.
(Severity: 5)
Variable declared in conditional statement at line 105, column 13.
Declare variables outside of the condition.  (Severity: 5)
Subroutine prototypes used at line 127, column 1.  See page 194 of PBP.
(Severity: 5)
Subroutine prototypes used at line 136, column 1.  See page 194 of PBP.
(Severity: 5)
Variable declared in conditional statement at line 190, column 7.
Declare variables outside of the condition.  (Severity: 5)
Subroutine prototypes used at line 237, column 1.  See page 194 of PBP.
(Severity: 5)
Bareword file handle opened at line 238, column 5.  See pages 202,204 of
PBP.  (Severity: 5)
Two-argument "open" used at line 238, column 5.  See page 207 of PBP.
(Severity: 5)
Subroutine prototypes used at line 277, column 1.  See page 194 of PBP.
(Severity: 5)
Subroutine prototypes used at line 311, column 1.  See page 194 of PBP.
(Severity: 5)
Bareword file handle opened at line 350, column 9.  See pages 202,204 of
PBP.  (Severity: 5)
Two-argument "open" used at line 350, column 9.  See page 207 of PBP.
(Severity: 5)
Bareword file handle opened at line 353, column 5.  See pages 202,204 of
PBP.  (Severity: 5)
Bareword file handle opened at line 359, column 5.  See pages 202,204 of
PBP.  (Severity: 5)
Two-argument "open" used at line 359, column 5.  See page 207 of PBP.
(Severity: 5)

perl -wc xgettext.pl
\1 better written as $1 at TmplTokenizer.pm line 474.
main::token_negligible_p() called too early to check prototype at
xgettext.pl line 52.
xgettext.pl syntax OK

Test plan:
Apply this patch and verify the warnings/errors does not appear anymore
and no regression is found on creating/updating/installing templates.

Signed-off-by: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com>
Work as described, no koha-qa errros.
Tested with perl v5.18.2
No warnings, no problem detected on create/update/install i18n files

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Tomas Cohen Arazi <tomascohen@gmail.com>
2014-05-30 15:46:17 -03:00
Jonathan Druart
6525818076 Bug 9161: Translate only specific files
The previous version of tmpl_process3.pl says in the pod:

  -i, --input=SOURCE          Get or update strings from SOURCE file.
                              SOURCE is a directory if -r is also specified.

But is was not possible to specify one or more files.

This patch allows us to give one or more specific file with the -f
parameter.

for eg.:

perl tmpl_process3.pl install -i /home/koha/koha-tmpl/opac-tmpl/prog/en/modules/
    -o /home/koha/koha-tmpl/opac-tmpl/prog/fr-FR/modules/
    -s /home/koha/misc/translator/po/fr-FR-i-opac-t-prog-v-3006000.po
    -r
    -f opac-account.tt
    -f opac-main.tt

You can specify:
  -f advsearch.tt => translate all files with a filename containing
  'advsearch.tt'
or
  -f search => will translate acqui/histsearch.tt, acqui/z3950_search.tt, etc.

Bug 9161: Followup: Add a -f param for the translate script

Now you can directly call the translate script
(misc/translator/translate) with the -f parameter

eg.:
./translate install|create|update -f search.tt -f main.tt

Signed-off-by: Frédéric Demians <f.demians@tamil.fr>

I've squashed the 3 patches, and reported doc into 'translate' script.
It works as advertised.

Side note: It would be great to extend this functionnality in order to
be able to apply the translation to XSL files stored outside Koha
directories hierarchy. Useful to translate site-specific XSLs defined
with XSLTResultsDisplay, and other sysprefs.

Signed-off-by: Chris Cormack <chrisc@catalyst.net.nz>
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
2013-05-24 09:08:25 -07:00
Will Stokes
cc62d0c4dd Bug 7938: Added description of -v/--verbose parameter
Signed-off-by: Magnus Enger <magnus@enger.priv.no>
"perldoc misc/translator/translate" looks good and now mentions
the -v option.

Signed-off-by: Jonathan Druart <jonathan.druart@biblibre.com>
Signed-off-by: Jared Camins-Esakov <jcamins@cpbibliography.com>
2013-03-29 21:53:32 -04:00
Paul Poulain
d029ec835d Bug 7780: make silent/verbose flag for translation installing
This patch deal with the -v flag that you can put on translate script.
If you run without -v, the process should be silent
if you run with -v, the process should be verbose

Signed-off-by: Frédéric Demians <f.demians@tamil.fr>

I've refactored your patch to handle verbosity directly via a translator
attribute, rather than with parameter which has to be send to each object call
method.

Signed-off-by: Paul Poulain <paul.poulain@biblibre.com>
2012-03-26 11:01:40 +02:00
Marc Balmer
d9f7922ad7 Bail out early if an invalid language is used.
Signed-off-by: Marc Balmer <marc@msys.ch>

http://bugs.koha-community.org/show_bug.cgi?id=7346
Signed-off-by: Paul Poulain <paul.poulain@biblibre.com>
2011-12-15 18:33:36 +01:00
Lars Wirzenius
87d845969e Fix FSF address in directory misc/
Signed-off-by: Galen Charlton <gmcharlt@gmail.com>
2010-03-16 20:17:54 -04:00
3c3aac0848 Follow up for new translation wrapper
(cherry picked from commit 17bdeb66e041f295214d1406620669c090b4e4de)

Signed-off-by: Galen Charlton <gmcharlt@gmail.com>
2010-02-18 11:43:42 -05:00
c05b7747b9 New wrapper script for handling translation process
Signed-off-by: Chris Cormack <chrisc@catalyst.net.nz>
Signed-off-by: Galen Charlton <gmcharlt@gmail.com>
2010-02-18 11:39:24 -05:00