If is entirely possible to create an SIP institution whose ID does not match a valid branchcode in Koha's SIP config. In fact, Koha's example SIP config contains an example of this ( kohalibrary / kohalibrary2 ).
If a SIP login uses an institution with an id that doesn't match a valid branchcode, everything will appear to work, but the SIP worker will die anywhere that Koha gets the branch from the userenv and assumes it is valid.
The repercussions of this are that actions such as the checkout message simply die and do not return a response message to the requestor.
At the very least, we should output a warning to the SIP log.
I think we should strongly consider disallowing institution ids in the SIP config that do not match valid branchcodes. In this scenario, attempting to start the SIP server should result in a error message with the SIP server exiting immediately.
Test Plan:
1) Apply this patch
2) Make a sip login that uses an instution whose id is *not* a valid branchcode
3) Start the SIP server
4) Check sip.log, you should see a warning similar to the following:
[2021/10/18 12:18:29] [2068079] [ERROR] ERROR: Institution kohalibrary does does not match a branchcode. This can cause unexpected behavior. C4::SIP::Sip::siplog /kohadevbox/koha/C4/SIP/Sip.pm (220)
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Joonas Kylmälä <joonas.kylmala@iki.fi>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
SIP connections tend to be long lived, weeks if not months, in the
libraries I see. Basically the connection per SIP machine is initiated
once when the SIP machine boots and then never closed until
maintanance needs to be done. Therefore we need to reset the Koha's L1
caches on every SIP request to get the latest sysprefs and configs
from the memcached cache that is shared between all the Koha
programs (intranet, opac, sip, cronjobs) and is guaranteed to be up to
date.
To test:
0. Have kohadevbox
1. Enable IssueLog
2. In one terminal run the command "perl C4/SIP/SIPServer.pm /etc/koha/sites/kohadev/SIPconfig.xml"
3. Checkin and return a book using telnet:
$ telnet localhost 6001
9300CNterm1|COterm1|CPCPL|
11YN20211010 10565320211010 105653AOCPL|AA1|AB3999900000001|ACterm1|BON|BIN|
09N20211010 10564420211010 105644APCPL|AOCPL|AB3999900000001|ACterm1|BIN|
4. Keep the telnet connection open and go to
http://localhost:8081/cgi-bin/koha/tools/viewlog.pl and check that
the *checkout* entry is in the circulation rules 5.
6. Disable IssueLog
7. Move back to the telnet prompt and check out and return a book again
11YN20211010 10565320211010 105653AOCPL|AA1|AB3999900000001|ACterm1|BON|BIN|
09N20211010 10564420211010 105644APCPL|AOCPL|AB3999900000001|ACterm1|BIN|
8. Go check out the circulation logs and notice a new entry was added
when it shouldn't have according to the IssueLog syspref!
9. Apply patch and repeat steps to notice that the syspref is now
followed correctly.
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Bug 15253 originally had the ability to specify the incoming IP address used for
a given log statement via SIP, as well as the SIP2 account that was in use at the time.
This data is very helpful for debugging purposes, and should be brought back.
Test Plan:
1) Apply this patch
2) Update you SIP ConversionPattern to "[%d] [%p] %X{accountid}@%X{peeraddr}: %m %l %n"
3) Restart SIP
4) Use the SIP cli tester to make some SIP requests
5) View the SIP2 log, note the account id and client ip address show in the log!
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
On bug 17591 we discovered that there was something weird going on with
the way we export and use subroutines/modules.
This patch tries to standardize our EXPORT to use EXPORT_OK only.
That way we will need to explicitely define the subroutine we want to
use from a module.
This patch is a squashed version of:
Bug 17600: After export.pl
Bug 17600: After perlimport
Bug 17600: Manual changes
Bug 17600: Other manual changes after second perlimports run
Bug 17600: Fix tests
And a lot of other manual changes.
export.pl is a dirty script that can be found on bug 17600.
"perlimport" is:
git clone https://github.com/oalders/App-perlimports.git
cd App-perlimports/
cpanm --installdeps .
export PERL5LIB="$PERL5LIB:/kohadevbox/koha/App-perlimports/lib"
find . \( -name "*.pl" -o -name "*.pm" \) -exec perl App-perlimports/script/perlimports --inplace-edit --no-preserve-unused --filename {} \;
The ideas of this patch are to:
* use EXPORT_OK instead of EXPORT
* perltidy the EXPORT_OK list
* remove '&' before the subroutine names
* remove some uneeded use statements
* explicitely import the subroutines we need within the controllers or
modules
Note that the private subroutines (starting with _) should not be
exported (and not used from outside of the module except from tests).
EXPORT vs EXPORT_OK (from
https://www.thegeekstuff.com/2010/06/perl-exporter-examples/)
"""
Export allows to export the functions and variables of modules to user’s namespace using the standard import method. This way, we don’t need to create the objects for the modules to access it’s members.
@EXPORT and @EXPORT_OK are the two main variables used during export operation.
@EXPORT contains list of symbols (subroutines and variables) of the module to be exported into the caller namespace.
@EXPORT_OK does export of symbols on demand basis.
"""
If this patch caused a conflict with a patch you wrote prior to its
push:
* Make sure you are not reintroducing a "use" statement that has been
removed
* "$subroutine" is not exported by the C4::$MODULE module
means that you need to add the subroutine to the @EXPORT_OK list
* Bareword "$subroutine" not allowed while "strict subs"
means that you didn't imported the subroutine from the module:
- use $MODULE qw( $subroutine list );
You can also use the fully qualified namespace: C4::$MODULE::$subroutine
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Bug 26673 renoved the shebang from SIPServer.pm. This was an error
SIPServer.pm is a program not a perl module, and needs the shebang
for the OS to assign an interpreter unless the command line
calls it as an argument to perl. Restore the shebang
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
It's not needed.
Signed-off-by: David Cook <dcook@prosentient.com.au>
Signed-off-by: Joonas Kylmälä <joonas.kylmala@helsinki.fi>
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
.pm must not have -x
.t must have -x
.pl must have -x
Test plan:
Apply only the first patch, run the tests and confirm that the failures
make sense
Apply this patch and confirm that the test now returns green
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
* We should not call Log::Log4perl directly
* Not sure it is correct as I get from (comment 77):
% koha-sip --restart
[2020/04/23 11:23:27] [ERROR] [undef]@[undef]: Argument "0.33_01" isn't
numeric in numeric lt (<) at /usr/share/perl5/Net/Server/Log/Sys/Syslog.pm
line 42.
C4::SIP::Trapper::PRINT /kohadevbox/koha/C4/SIP/Trapper.pm (24)
Why "ERROR" when it's a warning?
The [undef]@[undef] seems wrong here.
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Now that we have Koha::Logger, we should use it in our SIP server. This
has the potential to make debugging SIP issue much easier. We should add
the userid for the sipuser to the namespace so we can allow for separate
files per sip user if wanted.
Test Plan:
1) Apply this patch set
2) Update the modififed log4perl.conf to your system
3) Restart your sip server
4) Tail your sip2.log, run some queries
5) Note you still get the same output messages as before, with the
addition of the ip address and username ( if available )
prefixing the message.
Signed-off-by: Liz Rea <wizzyrea@gmail.com>
Signed-off-by: Chris Cormack <chris@bigballofwax.co.nz>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
This patch flushes the L1 cache on each SIP connection
(ie every invocation of process_request).
This means each SIP connection will fetch values from the
L2 cache (ie memcached) and persist the L1 cache (in Perl
memory cache) only until a new SIP connection comes in.
Without this patch, the L1 cache persists for the length
of the server process, which means the L1 cache will become
stale very quickly, which can lead to unexpected behaviour.
Test plan:
1. Enable IssueLog
2. Start SIP server
2a. change to Koha git directory
2b. koha-shell kohadev
2c. perl C4/SIP/SIPServer.pm /etc/koha/sites/kohadev/SIPconfig.xml
3. Issue book via SIP
perl misc/sip_cli_emulator.pl -a 127.0.0.1 -p 6001 -su term1 -sp term1 -l CPL --patron 1 --item 3999900000001 -m checkout
4. Check action_logs for "CIRCULATION ISSUE <borrowernumber> <biblionumber>" line
http://localhost:8081/cgi-bin/koha/tools/viewlog.pl
5. Disable IssueLog
6. Check in the book and then check it out again
perl misc/sip_cli_emulator.pl -a 127.0.0.1 -p 6001 -su term1 -sp term1 -l CPL --patron 1 --item 3999900000001 -m checkin
perl misc/sip_cli_emulator.pl -a 127.0.0.1 -p 6001 -su term1 -sp term1 -l CPL --patron 1 --item 3999900000001 -m checkout
7. Check action_logs for "CIRCULATION ISSUE <borrowernumber> <biblionumber>" line
http://localhost:8081/cgi-bin/koha/tools/viewlog.pl
8. Note results
Without the patch, you'll see a 2nd checkout.
With the patch, you won't see the 2nd checkout.
(Note: After applying the patch, you have to restart the SIP server.)
Signed-off-by: David Cook <dcook@prosentient.com.au>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Now that we have Koha::Logger, we should use it in our SIP server. This
has the potential to make debugging SIP issue much easier. We should add
the userid for the sipuser to the namespace so we can allow for separate
files per sip user if wanted.
Also modifies the log4perl.conf to lazy-open filehandles to log files,
so the same config can be used with log-files needing different
permissions.
Test Plan:
1) Apply this patch set
2) Update the modififed log4perl.conf to your system
3) Restart your sip server
4) Tail your sip2.log, run some queries
5) Note you still get the same output messages as before, with the
addition of the ip address and username ( if available )
prefixing the message.
Based on original patches by Kyle Hall and additions by Olli-Antti
Kivilahti.
Signed-off-by: Liz Rea <wizzyrea@gmail.com>
Signed-off-by: Chris Cormack <chris@bigballofwax.co.nz>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
This patch adds the 'interface' field to the accountlines table and
updates all Koha::Object routines and calls to use it.
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Josef Moravec <josef.moravec@gmail.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Although mainly hypothetical, it would still be possible to get
response from the server for an acs resend request. (This exception
is allowed in MsgType::handle.)
I also noticed that the response may well be a message from an older
session still.
This patch just removes that exception by only passing login requests
to sub handle in the raw_transport loop.
Test plan:
[1] Verify normal login procedure for raw.
[2] Check a few acs resend requests in raw. They should terminate without
a response.
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Srdjan <srdjan@catalyst.net.nz>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Moving timeout logic to one routine (with unit test).
This further implements two suggestions from Kyle and Larry:
[1] You could use a client_timeout of 0 to specify no timeout at all.
[2] Have the client_timeout default to the timeout if not defined.
Test plan:
[1] Run t/db_dependent/SIP/SIPServer.t.
[2] Test login timeout for raw and telnet.
[3] Check ACS status message for timeout value. Should match policy
timeout from institution.
[4] Test client timeout (zero and non-zero).
[5] Remove client timeout. Test fallback to service.
[6] Remove service timeout too. Test fallback to 30 at login.
Signed-off-by: Srdjan <srdjan@catalyst.net.nz>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Amended to incorporate Srdjan's suggestion to move get_timeout to
SIPServer.pm; this requires some additional mocking in the unit test.
And even makes the test db dependent, as documented.
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Remove the tabs causing inconsistent indentation
of sip_protocol_loop and replace with spaces
Reimplements the renaining parts of Marcel de Rooy's
original QA patch
No logic changes in this patch - layout only
Signed-off-by: Srdjan <srdjan@catalyst.net.nz>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
raw_connection was not behaving correctly if an invalid string was
passed or a login failed.
It was not checking that the login succeeded ( it checked that account
existed not that it contained data and it existed even if login failed)
and so failed logins instead of aborting immediately fell through into
the sip_protocol_loop, forcing that to timeout invalid connections.
It now checks that account has id set and returns if not.
The timeout alarm is now set on the while loop, in normal running this
should not be triggered as the socket is opened and the first data
should be a login message and the while loop should only iterate once,
but lets not go into an infinite loop due to unforeseen circumstances.
I have reindented the routine as the flow was not clear (the while was
not indented at all.
Also if using Net::Server::PreFork when a new connection comes in you
may be handed the the successful login parameters from a preceding call.
Because of this you could successfully transmit transactions and Koha
would carry them out without having received a valid login ( and
possibly with the wrong account details!) We now delete any existing
account for new connections.
NB: This patch requires that the patch for bug 13807 has been applied
Signed-off-by: Srdjan <srdjan@catalyst.net.nz>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Debugging various problems in SIPServer and control of it, found it
could loop on unread buffers (e.g. the LF of a CRLF if it was only
expecting CR) making it unresponsive to signals.
Reworked the input loop with an eye to removing unnecessary whiles
and replacing the while(1) by a while( connection valid)
Enhanced the timeout code by wapping in an eval.
Moved the logic from SIP_read_packet into the server itself
Hopefully this makes the already baroque code easier to navigate
and it did seem the server was the logical place for this
Removed no longer iused SIP_read_packet from Sip.pm
Signed-off-by: Srdjan <srdjan@catalyst.net.nz>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
This patch fixes breakage introduced by the patch for bug 13506
that prevented SIP2 clients from authenticating themselves over
the telnet transport. Attempts to do so would manifest by the
SIPServer backend crashing and the connection getting closed.
To test:
[1] Use the stock SIPServer config, etc/SIPconfig.xml, and ensure
that the database contains a staff user whose username is 'term1'
and password is 'term1' that has the circulate permission.
[2] Start SIPServer, telnet to port 8023, and attempt to log in. The
connection will be aborted:
$ telnet localhost 8023
Trying 127.0.0.1...
Connected to localhost.
Escape character is '^]'.
login: term1
password: term1
Connection closed by foreign host.
[3] Apply the patch and restart SIPSever, then repeat step 2. This time,
the login should succeed:
$ telnet localhost 8023
Trying 127.0.0.1...
Connected to localhost.
Escape character is '^]'.
login: term1
password: term1
Login OK. Initiating SIP
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
Signed-off-by: Colin Campbell <colin.campbell@ptfs-europe.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Tomas Cohen Arazi <tomascohen@unc.edu.ar>
For historical reasons the SIPServer and SIP modules
have used an extra module path in addition to the
standard Koha one. This has caused numerous irritants
in attempting to set up scripts and basic tests. It
does not help in attempting to modify or debug
this code
This patch changes the package value in the modules
under the C4/SIP directory and makes calls to
them use the full package name.
Where the export mechanism was being short circuited
routines have been explicitly exported and imported
declarations of 'use ILS' when that module was
not being used and which only generated warnings
have been removed.
As a lot of the changes affect lines where
an object is instantiated with new. The opportunity
has been taken to replace the ambiguous indirect
syntax with the preferred direct call
In intializing ILS the full path is added as this
will not require any changes to existing configs.
I suspect this feature is unused, and adds
obfuscation rather than flexibility but have kept
the feature as we need this change in order to
rationalize and extend the testing of the server.
The visible difference is that with the normal Koha
PERL5LIB setting. Compilation of Modules under C4/SIP
should be successful and not fail with unlocated modules,
allowing developers to see any perl warnings
All the SIP modules can now be run through the tests
in t/00-load.t now except for SIPServer itself
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Jonathan Druart <jonathan.druart@biblibre.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@gmail.com>
Koha's SIP server accepts timeout parameters, but those parameters
are only used for the login portion of a telnet transport session.
Other than that, they are ignored, and as long as whatever opened
the connection keeps it open, it will stay open indefinitely.
Test Plan:
1) Set the timeout setting on your SIP server to 10 seconds
2) Modify misc/sip_cli_emulator.pl, add "sleep 100;" directly after line 91
3) Start your SIP server
4) Run the modified sip script with valid parameters
5) Watch the SIP server stderr
6) Note that even though the script waits far too long before continuing,
the SIP server never kills the connection, and the requests the cli
script makes come back with valid data.
7) Apply this patch
8) Restart your SIP server
9) Repeat step 4
10) Note that this time you see "SIP Timed Out!" in the SIP server
stderr and when the script finally makes it's request, it doesn't
come back with valid data.
Signed-off-by: Colin Campbell <colin.campbell@ptfs-europe.com>
Signed-off-by: Jonathan Druart <jonathan.druart@biblibre.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@gmail.com>
This small patch adds a check on the SIP2 socket connection if it is
IPv6 and resolves socket address accordingly.
Any newer Debian distro would probably default to IPv6 so it would
eventually affect all SIP servers.
Tests against running SIP server on an IPv6 box:
http://wiki.koha-community.org/wiki/Koha_SIP2_server_setup#Testing_with_Telnet
before patch:
disconnects immediately. Log output:
Bad arg length for Socket::unpack_sockaddr_in, length is 28, should be 16
after patch:
operates normally
Signed-off-by: Colin Campbell <colin.campbell@ptfs-europe.com>
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Tomas Cohen Arazi <tomascohen@gmail.com>
If you do a kill -s HUP sipserver_pid the server process
should restart its children.
This was not happening although it was logging the commencement
of the process.
Removed traces of Exporter usage this was always incorrect
this program inherits from Net::Server 'use base' to make inheritance
clearer. Removed unnecessary begin block
made file executable so it can run
added a shebang line so OS knows to invoke perl to compile and run
the file.
I've removed the debug code dumping the config to STDOUT,
it should not have gone into the production version.
One of the first things this program does is close stdout etc and
reopens the sip sockets to them. Assuming you can print to them other
than as part of the sip dialogue is not always safe.
Signed-off-by: Chris Cormack <chris@bigballofwax.co.nz>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@gmail.com>
SIPServer.pm requires that C4/SIP is added to its lib
path This has been done by passing this directory
to it via -I. By using FindBin it can set the path
for itself correctly. This will also work if the C4/SIP
directory tree is moved to a non-standard location
Removed the now redundant -I. from sip_run.sh
Added a variable to sip_run.sh for the koha tree to
highlight a problem with the script if you have multiple
directories in the PERL5LIB environment variable
Signed-off-by: Chris Cormack <chris@bigballofwax.co.nz>
Signed-off-by: Paul Poulain <paul.poulain@biblibre.com>
Be liberal in what we accept, but strict in what we send:
Never exit the server process, but send a SC_RESEND message (96)
to the client if we received anything we don't understand.
This is consistent with SIP server implementations of other ILSs.
Signed-off-by: Colin Campbell <colin.campbell@ptfs-europe.com>
Signed-off-by: Ian Walls <koha.sekjal@gmail.com>
Signed-off-by: Paul Poulain <paul.poulain@biblibre.com>
Add C4/SIP to perlcritic tests. Fix those issues that were
generating perlcritic errors
Signed-off-by: Stacey Walker <stacey@catalyst-eu.net>
Signed-off-by: Paul Poulain <paul.poulain@biblibre.com>
This patch put in SIPconfig.xml SIP2 server logging parameters. This is
way it's possible to modify where SIP2 logs are sent. And fix
OpenSolaris bug.
WARNING: After an upgrade, installation using a custom SIP2 server
config file will need to copy default SIPconfig.xml added section:
<!--
Set Net::Server::PreFork runtime parameters
syslog_ident will identify SIP2 Koha server entries in syslog
For OpenSolaris, add: syslog_logsock=stream
-->
<server-params
min_servers='1'
min_spare_servers='0'
log_file='Sys::Syslog'
syslog_ident='koha_sip'
syslog_facility='local6'
/>
This includes some initial work for the 3M SIP2 extensions.
It also better populates the Patron object with methods for
a fuller Patron Information Reponse. This is positively affect
EnvisionWare software, as used by NEKLS.
This work was sponsored by the Northeast Kansas Library System.
Signed-off-by: Galen Charlton <galen.charlton@liblime.com>
The basic problem is that the SIP logic doesn't know where the
input is coming from. It might be a RAW socket, and it might
be telnet. If it is telnet, although the specs declare a
character set (from MS, unfortunately), they do not specify a telnet
implementation. So you might get telnet handshaking or
renegotiations in the middle of an otherwise peaceful session and
these should not be taken as SIP commands. Patches include a move
towards using $CRLF from Socket to avoid problems w/ foreign platform
mapping \n and \r to \015 or \012.
Signed-off-by: Joshua Ferraro <jmf@liblime.com>
