We use different ways to interact with Solr depending the feature used into the admin UI. It can be by pushing the configuration files to Zookeeper and then reload the Solr collection. It can also be by one of the Solr APIs.
In this page we detail for each feature of Datafari how we integrate it with Solr and we give some technical explanations about these options.
Datafari use cases
| Feature | Solr API / Files | Needs | File located in $DATAFARI_HOME/solr/solrcloud/$MAIN_COLLECTION_conf/ |
|---|---|---|---|
| Highlight size | Request Parameters API | Automatically taken into account | params.json |
| Autocomplete threshold | Config API | Automatically taken into account | configoverlay.json |
| Document boost | File modified locally in Datafari server | Automatically taken into account. elevate.xml file uploaded to ZK and the collection is reloaded | elevate.xml |
| Synonyms | File modified locally in Datafari server | Automatically taken into account. synonyms_xx.txt file uploaded to ZK and the collection is reloaded | synonyms_xx.txt |
| Stopwords | File modified locally in Datafari server | Automatically taken into account. synonyms_xx.xml file uploaded to ZK and the collection is reloaded | stopwords_xx.txt |
| Protwords | File modified locally in Datafari server | Automatically taken into account. protwords.xml file uploaded to ZK and the collection is reloaded | protwords.txt |
| Field weight | Request Parameters API | Automatically taken into account | params.json |
List of Solr API :
| API name | Type | Description | Use in Datafari | URL |
|---|---|---|---|---|
| Schema API | Schema | Schema editing. Use managed-schema file | Yes | https://lucene.apache.org/solr/guide/7_5/schema-api.html |
| Config API | Configuration API | The Config API enables manipulating various aspects of your solrconfig.xml using REST-like API calls. | Yes | https://lucene.apache.org/solr/guide/7_5/config-api.html |
| Request Parameters API | Configuration API | The Request Parameters API allows creating parameter sets, a.k.a. paramsets, that can override or take the place of parameters defined in solrconfig.xml. | Yes | https://lucene.apache.org/solr/guide/7_5/request-parameters-api.html |
| Blob Store API | Configuration API | The Blob Store REST API provides REST methods to store, retrieve or list files in a Lucene index. | No | https://lucene.apache.org/solr/guide/7_5/blob-store-api.html |
| Managed Resources | Configuration API | Managed resources expose a REST API endpoint for performing Create-Read-Update-Delete (CRUD) operations on a Solr object. Only supports StopWords and Synonyms (since 2014...) | No | https://lucene.apache.org/solr/guide/7_5/managed-resources.html |
| Collections API | SolrCloud | The Collections API is used to create, remove, or reload collections. | Yes | https://lucene.apache.org/solr/guide/7_5/collections-api.html |
| Configsets API | SolrCloud | The Configsets API enables you to upload new configsets to ZooKeeper, create, and delete configsets when Solr is running SolrCloud mode. | Yes | https://lucene.apache.org/solr/guide/7_5/configsets-api.html |
Use of Zookeeper to manage configuration files
As we explained above, for some configuration files of Datafari we store them locally then we push them to Zookeeper. Then we reload the collection to take them into account.
For example for the synonyms :
1) the user adds synonyms for English language into the Synonyms admin UI
2) the user clicks on the confirm button
3) the modifications are stored into the file $DATAFARI_HOME/solr/solrcloud/$MAIN_COLLECTION/conf/synonyms_fr.txt
4) the file synonyms_fr.txt is pushed back to Zookeeper
5) the Solr collection is reloaded
https://lucene.apache.org/solr/guide/7_5/using-zookeeper-to-manage-configuration-files.html