The IdP 2.x installation manual had been recommending to use the ComputeID connector to generate eduPersonTargetedID values. While that made the setup simpler (no database setup needed), it has the consequence that there is no database of eduPersonTargetedID values issued.
This page describes how to change the IdP configuration to use a StoredID Connector instead to store the values in a database.
Earlier versions of the manual for Upgrading a 2.x IdP to 3.x were instructing to follow the instructions here when performing an upgrade from a 2.x IdP that was using ComputedIdConnector for eduPersonTargetedID (i.e., not storing the values in a database).
While it is still strongly recommended to store the values in a database, it is no longer deemed necessary to change the configuration of the version 2 IdP, as the version 3 IdP would be producing the same values.
However, if the existing persistent ID values are not stored in a database, it is crucial to use the identical salt value on the old 2.x and the new 3.x IdP.
This document is left here as a resource to configure a IdP 2.x IdP to store the persistent ID values in a database, but it is no longer a prerequisite for upgrading to IdP 3.x.
As in the migration to IdP v3, there would be also a migration from eduPersonTargetedID (used as an attribute) to SAML2 Persistent NameID (used as the NameID in SAML assertions), there will also be a need to switch to a different generator of persistent IDs. The new generator (used with SAML2 Persistent NameIDs) can be configured to use the same database and reuse persistent IDs already assigned. When configured with identical salt, for matching usernames, it would also generate same values as the previous generator.
Earlier versions of this document were incorrectly stating these values would be different. The wording in the rest of the document is kept with the original assumption the values would be different.
Therefore, to ensure a smooth migration, it is necessary to re-configure eduPersonTargetedID on the existing IdP to switch to using a database (with the StoredID connector which generates same values as the ComputeId connector) and populate the database with all persistent IDs already in use.
With this change in place, any persistent ID passed in an SSO session would be recorded in the database. To cover all persistent IDs ever issued, it would be possible to:
- scan the IdP audit log and extract the combination of all user names and Service Provider entityIDs that ever received a dynamically computed persistent ID.
- use this list with automated tools that trigger the IdP into re-calculating the persistent ID - and with the change in place, storing it in the database.
After that, the migration to IdPv3 would preserve persistent IDs for all users.
This manual expects an IdP 2.x installation based on the manual Installing a Shibboleth 2.x IdP.
It expects a local MySQL database server has been already set up and a database created (for storing auEduPersonSharedToken values).
It also expects MySQL JDBC driver has been already installed in Tomcat.
Fixing missing assumptions - MySQL database
If the MySQL-related assumptions above do not hold, the following steps would make them hold:
Install MySQL server:
Create MySQL database and user account (substite an appropriate password here):
Install MySQL JDBC driver into Tomcat:
And in addition to fixing missing assumptions, also explicitly install the MySQL driver into the IdP deployment directory - so that the
aacli.sh tool used later can find the driver as well:
Creating MySQL table
The persistent ID values are stored in the
For simplicity, we recommend creating the table in the
idp_db database which already holds the
auEduPersonSharedToken values (in the
Create the table with the following DDL (reused from https://wiki.shibboleth.net/confluence/display/SHIB2/StoredIDDataConnectorDDL): - run
mysql idb_db and enter:
Changes to Attribute Resolver
/opt/shibboleth-idp/conf/attribute-resolver.xml and make the following changes:
- Comment out the existing
Copy in the following definition of the
Adjust the definition the same way the computeID connector definition was adjusted (as per eduPersonTargetedID section in the IdP 2.x install manual):
- In particular, reusing the
saltvalues from the existing
- In particular, reusing the
Change the dependency in the
eduPersonTargetedIDattribute definition to the new connector:
- Restart Tomcat and test your IdP against the Tuakiri Attribute Validator: https://attributes.tuakiri.ac.nz/
Triggering recalculation of Persistent ID values
With the IdP now configured to store Persistent ID values in the database, any newly calculated values are stored in the database as they are used. However, to ensure all users keep their identity at all services they ever accessed, we would either need to wait until they log into all of the services again (yes, unrealistic), or we trigger the IdP to act as if the user logged into the service. For that, we use the command-line interface to the Attribute Authority -
First, we can the IdP logs to get all unique combinations of username and service provider entityID that were ever used in sessions where eduPersonTargetedID values were issued:
(the following code assumes you use the default logging format in the IdP-Audit log)
We check the number of records found:
- Check the file actually contains pairs of usernames and Service Provider entity IDs.
We do a dry run with the first entry:
And we run the command for the full sequence (sorry, may take significant amount of time - up to 10s per invocation):
And check the database: connect to the
idp_dbMySQL database and run:
- The number should be greater or equal to the number of records found above (greater if there are additional sessions created in the meantime - new user-service provider combinations).
Troubleshooting aacli.sh invocations
aacli.sh tool may fail to run for a number of reasons. Here are the most common ones - and possible remedies:
- ClassNotFoundException: org.apache.xerces.util.SecurityManager
- This is because the IdP is using "endorsed" Xerces XML parser classes.
- This is no longer needed - and is even discouraged because of a vulnerability in the XML parser: http://shibboleth.net/community/advisories/secadv_20141103.txt
The solution is to edit
$IDP_HOME/conf/internal.xmland changethe bean definition defined under the
- Class com.mysql.jdbc.Driver not found:
- The JDBC driver has not been installed in
- Use the above documentation to install the MySQL driver there.
- The JDBC driver has not been installed in
- JNDI database resource not found (
Unable to retrieve data source for data connector IDPDB from JNDI location java:comp/env/jdbc/IDP)
- This happens when the IdP (attribute resolver) uses Container Managed database connections.
- Unfortunately, aacli.sh only supports Application Managed connection.
- The only workaround is to (temporarily) change the attribute resolver configuration to use Application Managed connections.
- The principal-sp-pairs.txt file contains garbage (or no data)
- Possible, the log format has been customized in this IdP deployment.
- Solution: adjust the parsing code accordingly.