Versions Compared

Old Version 8

changes.mady.by.user Guylaine BASSETTE

Saved on

compared with

New Version Current

changes.mady.by.user Cedric

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

For FileShare collection

Info

This documentation is only valid for update processors for FileShare collection. It will cover normally almost all needs. But If you has a specific need and you want to change the configuration for another collection present into Datafari (Logs, Statistics, Access etc…) see the section at the end of the page.

One of the most interesting feature of Solr is that you can develop custom Update Processors. These components are used (and useful) to perform data adjustments/modifications on documents just before they are indexed.

...

Then you will need to tell the core which UpdateProcessorFactory to use, along with the parameter “source.field” that the processor will use and when to use this update processor. Here again, things are simplified in Datafari as you will only need to declare the update processor in the DATAFARI_HOME/solr/solrcloud/FileShare/conf/customs_solrconfig/custom_update_processors.incl file as followfollows:

Code Block
<processor class="com.francelabs.datafari.updateprocessor.ReplaceUrlUpdateProcessorFactory">
    <str name="source.field">testurl</str>
</processor>

Datafari is configured to call each custom update processor factories factory specified in this file (in the order they are declared) at the very end of the update processors chain. This guarantees that your custom update processors are actually executed, once all of the actions from the Datafari core code have already been executed.

Once this is done, you will need to restart Datafari (or simply the /wiki/spaces/DATAFARI/pages/2852716547), then push and apply the new configuration thanks to the System Configuration Manager (Zookeeper) .

...

Through this example, you should have understood the bases : how to use parameters for an update processor, how to use it and how works a custom update processor works. You can now use the example update processor to develop your own.

Note

If for any reason you have to maintain a previous version of your Update Processor, do not keep the jar in the same folder just by changing it's extension, for example "my_update_proc.jar.old". It will be class loaded as any jar by the Solr Server, no matter the extension. At run time the result will be unpredictable as the previous version of your classes can be used instead of the new one.

For other collections except FileShare

Info

Normally, it is very rare to need to do modifications on the update processors not related to the FileShare collection.

If you use Enterprise Edition, please contact Datafari support team to explain your needs.

The main steps are :

  • Compile your custom processor and export the jar (see above)

  • Put the jar into the folder $DATAFARI_HOME/solr/solrcloud/$COLLECTION/lib (replace $COLLECTION by the collection in which you want to put your update processor. Create the lib folder if it does not exist)

  • Edit the solrconfig.xml file of the collection :

--Add the lib tag to load the jar :

Code Block
<lib dir="$DATAFARI_HOME/solr/lib/custom"/>

(replace $DATAFARI_HOME by the real path of your Datafari installation path, by default /opt/datafari )

--Modify the /update handler to add the update processor, example here with noip-processor-chain :

Code Block
<requestHandler name="/update" class="solr.UpdateRequestHandler">
    <lst name="defaults">
      <str name="update.chain">noip-processor-chain</str>
    </lst>
  </requestHandler>

--Add also the update request processor chain :

Code Block
<updateRequestProcessorChain name="noip-processor-chain">
    <processor class="com.francelabs.datafari.updateprocessor.AnonymiseIpUpdateProcessorFactory" />
    <processor class="solr.LogUpdateProcessorFactory" />
    <processor class="solr.RunUpdateProcessorFactory"/>
  </updateRequestProcessorChain>

  • Push the configuration to ZK :

Code Block
bin/solr zk upconfig -n $COLLECTION -d $DATAFARI_HOME/solrcloud/$COLLECTION/conf/

(replace $DATAFARI_HOME by the real path of your Datafari installation path, by default /opt/datafari and replace $COLLECTION by the Solr collection )

  • Reload the collection :

Code Block
curl "http://localhost:8983/solr/admin/collections?action=RELOAD&name=$COLLECTION