Datafari periodically generates periodically monitoring logs. By default, the generation rate is once per hour, and can be currently modified through the java class "com.francelabs.datafari.monitoring.IndexMonitoring".
Here is how monitoring logs look like:
...
Let's explain each field:
- [log_id] : the log id which must be UNIQUE per day, per facet field and value, in order to exploit those logs with ELK
- [num_doc] : the number of documents found according to the facet
- [facet_value] : the facet value represented by the log
- [facet_field] : the facet field used
At each monitoring log generation iteration, a log line is created for each facet value of the selected facet fields, plus a log line for the global number of documents indexed (in that case, the [facet_value] and [facet_field] are both set to "no").
The default selected facet fields are "extension", "language" and "source". One can change the facet fields selected in the class "com.francelabs.monitoring.IndexMonitoring.FileShareMonitoringLog".