Custom Analytics Stats
Valid from Datafari v5.3 upwards
This documentation is only valid for Datafari v5.3 upwards
By default, Datafari generates two kinds of logs that are meant to provide data to the Analytic stack in order to create visualization paragraphs in Apache Zeppelin notebooks :
The Core Monitoring logs that provide index related data
The Statistics logs that provide search usages related data
The visualization paragraphs you can create with a default installation of Datafari are limited to the data provided by those two kind of logs.
So to add more data to the Analytic stack, you will need either to modify those logs or to create your own logs.
1. Modify the existing logs
The Core Monitoring logs are generated by the com.francelabs.datafari.monitoring.IndexMonitoring class. This class runs a scheduled thread that will query the Solr index each day with some facet queries and format the results into log entries. So it is in this class that you should add your own additionnal facet queries and modify the log format if needed in case you want to add more data.
The Statistics logs are generated by the com.francelabs.datafari.statistics.StatsPusher class that is called whenever a user performs an action in the search view (search, click on result/page/facet). This class contains two main methods:
pushDocument : this method is called by the com.francelabs.datafari.servlets.URL servlet which is triggered when a user clicks on a result in the search view (both in AjaxFranceLabs UI and DatafariUI)
pushQuery : this method is called by the com.francelabs.datafari.api.SearchAPI which is triggered when a user performs a search in the search view (both in AjaxFranceLabs UI and DatafariUI)
In each method, the com.francelabs.datafari.statistics.StatsUtils class is used to format and produce the logs.
So, you can either modify the existing logs to add more data or produce your own logs in the StatsPusher class.
To add more data (ie Solr fields) to the logs generated from the com.francelabs.datafari.statistics.StatsPusher class or com.francelabs.datafari.api.SearchAPI, you will need to add the wanted fields to the queries field list parameter. This can be done in the DATAFARI_HOME/tomcat/webapps/Datafari/js/main.js for the AjaxFranceLabs UI, or in the ui-config.json
file for Datafari UI (see Customizing DatafariUI for more details).
Concerning AjaxFranceLabs and the DATAFARI_HOME/tomcat/webapps/Datafari/js/main.js file, add the desired fields in the existing list :