Valid from Datafari Community Edition version 7.1

The Jamespot connector has been introduced in Datafari v7.1 and is composed of a Repository Connector and an Authority Connector.

The connectors are directly included within the Datafari distribution, they can be found directly into MCF.

1. Prerequisites

Before creating and configuring the Jamespot Repository connector or the Jamespot Authority connector, you will need to create a specific Authority Group for them. In the MCF admin UI, go to “Authorities => List Authority Groups" and click on "Add a new authority group", then enter a name of your choice (it is recommended to name it so you know that it is Jamespot related like ‘JamespotAuthGroup’) then save it:

2. Repository Connector

The Jamespot repository connector is mainly based on the Solr repository connector, since Jamespot stores its documents into a Solr index.

To configure a Jamespot repository connector, in the ManifoldCF administration UI, go to Repositories -> List Repository Connections and click on the "Add a new connection" button. Then enter a name and in the "Type" tab select "Jamespot" as connection type, and select the authority group you have created in the prerequisites:

Now click on the “Continue" button and in the "Solr ingester" tab, configure what is needed to connect to the Solr cluster of your Jamespot:

Here is a quick description of the parameters:

URL Solr : the URL of the Solr you want to index (for example : http://localhost:8983/solr)

Login : (optional) the username of the account if you use basic auth

Password : (optional) the password of the account if you use basic auth

Custom header name 1 : (optional) the name of the first custom header

Custom header value 1 : (optional) the value of the first custom header

Custom header name 2 : (optional) the name of the second custom header

Custom header value 2 : (optional) the value of the second custom header

Connection timeout : 60 000 ms by default

Socket timeout : 180 000 ms by default

Once all the parameters are correctly set, save the connector. If the connection status displays “Connection working" then you are done, otherwise something is wrong.

Now that the Jamespot repository connector is created, you can configure a job to use it. You can create a new job in the MCF admin UI, go to “Jobs => List all Jobs" and click on "Add a new job".

In the “Name" tab, enter the name of your choice (recommended to be related to Jamespot):

In the “Connection" pipeline, set the following connectors in that exact order:

Then in the “Security" tab configure the security of the Jamespot documents:

Here is a description of the parameters:

Security activated: Enable or disable this parameter according to you needs. If enabled, the connector will try to retrieve security tokens from the fields set in the Security Field and Security Field 2 parameters and the documents will only be searchable by allowed users. If disabled, all the Jamepost documents will be “public” in Datafari and any user will be able to see them all in the search, so be careful !
Security Field: Set here the field name of the Jamespot documents where to retrieve security tokens. For a document to be displayed in the search result of a user, at least one of the user tokens must match with one of the security tokens of that security field. This field must be of text type and multivalued !
Security Field 2: This is a second security field that is optional, it stacks with the main Security Field. If set, for a document to be displayed in the search result of a user, at least one of the user tokens must match with one of the security tokens of the Security Field, AND at least one of the user tokens must match with one of the security tokens of the Security Field 2 ! If not set, only the main Security Field will be used

Next, configure the “Parameters" tab:

Collection Name: This is the collection name of the Jamespot Solr cluster that contains all the Jamespot documents to crawl
Field mappings: Here you can configure a mapping on field names. For exemple if you want to index a field into Jamespot Solr and map it with a field into Datafari Solr
ID Field: This is the name of the field of a Jamespot document that is the unique key
Date Field: This is the name of the field of a Jamespot document that contains its last modified date
Content Field: This is the name of the field of a Jamespot document that contains its main content
Filter Condition: This is the query that must match all the Jamespot document to crawl. If you don’t want to crawl every document like it is the case by default with the “*:*" query, then adapt it to filter the documents to crawl. You must respect the standard Solr syntax of the query parameter (corresponds to the 'fq' parameter in the Solr search API of the Solr admin UI)
Solr rows number: To lower memory consumption, the Jamespot connector paginates the query results to get all Jamespot documents. This parameter serves to define the maximum number of documents to retrieve per results page.

Finally you have to configure the “Metadata expressions" tab and add the 'repo_source' with the value of your choice (the value will be used as a facet in the search view):

Once you are done with the configuration, you can save the job and run it !

3. Authority Connector

The authority connector retrieves Jamespot groups from which a username is member of. During a search, this connector is used to retrieve the Jamespot groups of the user performing the search and then, they are used as a query filter to only display Jamespot documents having at least one group in their security tokens that matches one of the user groups.

The Jamespot API used by the connector to retrieve the user groups is “https:$JAMESPOT_SERVER/api/server/2.0/user/secItems?username=$EMAIL_USER

The response of the API is like this :

{
   "RC":{
      "CODE":0,
      "MSG":"",
      "O":"user",
      "F":"secItems"
   },
   "STATS":{
      "TIME":15
   },
   "VAL":{
      "username":"jeanlouis@test.com",
      "pseudo":"jeanlouis",
      "mail":"jeanlouis@test.com",
      "tokens":[
         "u1",
         "dest1",
         "web",
         "all",
         "ct1",
         "ct5",
         "s1",
         "public",
         "u0"
      ]
   },
   "V":"2.0"
}

The configuration of this authority is pretty simple and straightforward, in the ManifoldCF administration UI, go to Authorities -> List Authority Connections and click on the "Add a new connection" button. Then enter a name and in the "Type" tab select "Jamespot" as connection type and select the authority group you have created in the prerequisites:

Now click on the “Continue" button and in the "Jamespot Parameters" tab, configure what is needed:

Here is a quick description of the parameters:

URL: This is the URL of the entrypoint of Jamespot
Login : (optional) the username of the account if you use basic auth
Password : (optional) the password of the account if you use basic auth
Custom header name 1 : (optional) the name of the first custom header
Custom header value 1 : (optional) the value of the first custom header
Custom header name 2 : (optional) the name of the second custom header
Custom header value 2 : (optional) the value of the second custom header
Cache life time : here you set the time a user's information is kept around
Connection Timeout: This is the timeout in milliseconds after which a connection attempt to the endpoint is considered failed
Socket Timeout: This is the timeout in milliseconds after which an endpoint request response reading attempt is considered failed

Once all the parameters are correctly set, save the connector. If the connection status displays “Connection working" then you are done, otherwise something is wrong.

To test the authority you can use the “Test Active Directory Authority" feature located in the "Active Directory" tab of the admin UI of Datafari. Refer to the following documentation for that part: https://datafari.atlassian.net/wiki/spaces/DATAFARI/pages/223313957

Jamespot Connector

1. Prerequisites

2. Repository Connector

3. Authority Connector