Editing Shibboleth Debug Logging

As described on the main [[Shibboleth]] page.  EPrints uses the Shibboleth Service Provider (SP) provided by shibboleth.net.  This includes a logger that can be configured at '''/etc/shibboleth/shibd.logger''' and which logs to '''/var/log/shibboleth/shibd.log'''.  This typically only logs errors and warnings and the occasionally informationl log message, typically when starting or stopping.  However, if you are having problems with the Shibboleth SP, particularly when setting up, it may be useful to enable debug.  To do this

1. Edit the '''/etc/shibboleth/shibd.logger''' and uncomment the following lines:
 # tracing of SAML messages and security policies
 #log4j.category.OpenSAML.MessageDecoder=DEBUG
 #log4j.category.OpenSAML.MessageEncoder=DEBUG
 #log4j.category.OpenSAML.SecurityPolicyRule=DEBUG
 #log4j.category.XMLTooling.SOAPClient=DEBUG
 # interprocess message remoting
 #log4j.category.Shibboleth.Listener=DEBUG
 # mapping of requests to applicationId
 #log4j.category.Shibboleth.RequestMapper=DEBUG
 # high level session cache operations
 #log4j.category.Shibboleth.SessionCache=DEBUG
 # persistent storage and caching
 #log4j.category.XMLTooling.StorageService=DEBUG
So they look like
 # tracing of SAML messages and security policies
 log4j.category.OpenSAML.MessageDecoder=DEBUG
 log4j.category.OpenSAML.MessageEncoder=DEBUG
 log4j.category.OpenSAML.SecurityPolicyRule=DEBUG
 log4j.category.XMLTooling.SOAPClient=DEBUG
 # interprocess message remoting
 log4j.category.Shibboleth.Listener=DEBUG
 # mapping of requests to applicationId
 log4j.category.Shibboleth.RequestMapper=DEBUG
 # high level session cache operations
 log4j.category.Shibboleth.SessionCache=DEBUG
 # persistent storage and caching
 log4j.category.XMLTooling.StorageService=DEBUG

2. Test the Shibboleth SP is correctly configured
 shibd -t

3. Restart the Shibboleth SP
 systemctl restart shibd

There is various debugging that has been enabled by these changes.  Particularly useful are the first two line:
 log4j.category.OpenSAML.MessageDecoder=DEBUG
 log4j.category.OpenSAML.MessageEncoder=DEBUG
This will allow you to see the XML messages sent to and from the Shibboleth Identity Provider (IdP), such as MicroSoft Entra or OpenAthens.  

== Debugging User Attributes Mapping Issues ==
This debugging can be particularly useful if you are trying to work attributes in your attribute-map.xml are not being mapped as expected.  If you tail the log file '''/var/log/shibboleth/shibd.log''' and then try to login using the Shibboleth, you should see a line in the logs that starts:
 <samlp:Response ...
If you copy the whole of that line into a text file in /tmp/ and save it.  Then you can run xmllint on it to format the output.  E.g.
 xmllint --format /tmp/saml-response.xml
If attributes are being mapped, in the formatted output you should be able to see section that starts with an '''AttributeStatment''' XML tag  (However, if you don't see this, then your data may be encrypted as a '''CipherValue''' under '''CipherData'''.  if so, follow the instructions under [[#Decoding a CipherValue|Decoding a CipherValue]] .  E.g.
 <AttributeStatement>
   <Attribute Name="http://schemas.microsoft.com/identity/claims/tenantid">
     <AttributeValue>01234567-89ab-cdef-0123-456789abcdef</AttributeValue>
   </Attribute>
   <Attribute Name="http://schemas.microsoft.com/identity/claims/objectidentifier">
     <AttributeValue>fedcba98-7654-3210-fedc-ba9876543210</AttributeValue>
   </Attribute>
   <Attribute Name="http://schemas.microsoft.com/identity/claims/identityprovider">
     <AttributeValue>https://sts.windows.net/89abcdef-0123-4567-89ab-cdef01234567/</AttributeValue>
   </Attribute>
   <Attribute Name="http://schemas.microsoft.com/claims/authnmethodsreferences">
     <AttributeValue>http://schemas.microsoft.com/ws/2008/06/identity/authenticationmethod/password</AttributeValue>
     <AttributeValue>http://schemas.microsoft.com/claims/multipleauthn</AttributeValue>
   </Attribute>
   <Attribute Name="urn:oid:0.9.2342.19200300.100.1.3">
     <AttributeValue>j.bloggs@example.org</AttributeValue>
   </Attribute>
   <Attribute Name="urn:oid:2.5.4.4">
     <AttributeValue>Bloggs</AttributeValue>
   </Attribute>
   <Attribute Name="urn:oid:2.5.4.42">
     <AttributeValue>Joe</AttributeValue>
   </Attribute>
 </AttributeStatement>
If you cannot find users attributes you are expecting to see based on what you have configurured in your Shibboleth SP's attribute-map.xml, then you will need to get the maintainer of Shibboleth IdP to make sure these attributes are released or otherwise release attributes that store the sane user metadata.  At this point you can re-run this debugging to see what user attributes you are now getting and update your Shibboleth SP's attribute-map.xml as appropriate.

'''When you have finished debugging.  It is sensible to comment out the debug lines in '''/etc/shibboleth/shibd.logger''' again and restart the Shibboleth SP.'''

== Decoding a CipherValue ==
1. Download [https://gist.github.com/orimanabu/b2a7afcc6b6f59e475cad918914eb4b9|this Python script]: 
 cd /tmp/
 wget https://gist.githubusercontent.com/orimanabu/b2a7afcc6b6f59e475cad918914eb4b9/raw/99e4f6e9c71fc5d2a9f4021d3336f28dab0927a7/decrypt_saml_response.py

2. Create a Python virtual environment, activate it and make sure pip is upgraded:
 python -m venv venv
 source venv/bin/activate
 pip install --upgrade pip

3. Install Python modules required by the script
 pip install lxml
 pip install PyCryptodome

4. Now you can run the script.  (It is assumed that your key is in the default location for the Shibboleth SP.  Check this is correct according to the CredentialResolver tag in /etc/shibboleth/shibboleth2.xml.  If you have a separate encryption and signing key, be sure to use the encryption one in the command below).
 cat /tmp/saml-response.xml | python decrypt_saml_response.py -k /etc/shibboleth/sp-key.pem -p