Skip to end of metadata
Go to start of metadata

You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 2 Next »

Datafari periodically generates monitoring logs. By default, the generation rate is once per hour, and can be currently modified through the java class "com.francelabs.datafari.monitoring.IndexMonitoring".
Here is how monitoring logs look like:

monitoring logs
2015-11-06 09:17:55 MONITORING IndexMonitoring$FileShareMonitoringLog:187 - 611555203c75e7880c8dc67afd41062d|4|txt|extension
2015-11-06 09:17:55 MONITORING IndexMonitoring$FileShareMonitoringLog:187 - 2ce23ed298b044a3eda0a0e695921024|3|png|extension
2015-11-06 09:17:55 MONITORING IndexMonitoring$FileShareMonitoringLog:187 - 0fc7bbcad0efcfc2037ae2a2f1fbfe04|2|doc|extension
2015-11-06 09:17:55 MONITORING IndexMonitoring$FileShareMonitoringLog:187 - 31f8884b788c7872e5b2a286faa42747|2|html|extension
2015-11-06 09:17:55 MONITORING IndexMonitoring$FileShareMonitoringLog:187 - e155fbb2f018035d65553e763e0d8e63|1|gif|extension
2015-11-06 09:17:55 MONITORING IndexMonitoring$FileShareMonitoringLog:187 - c9dc854a334e1dc339d6cdfd10cb37cf|1|jar|extension
2015-11-06 09:17:55 MONITORING IndexMonitoring$FileShareMonitoringLog:187 - c10e628086bdc9fe4126c5f9c3b222c6|1|jpg|extension
2015-11-06 09:17:55 MONITORING IndexMonitoring$FileShareMonitoringLog:187 - 24ca7afe128eb7b33b36c4919e830509|16|en|language
2015-11-06 09:17:55 MONITORING IndexMonitoring$FileShareMonitoringLog:187 - 8879abd5830038c4a630d07526857faf|15|fr|language
2015-11-06 09:17:55 MONITORING IndexMonitoring$FileShareMonitoringLog:187 - dc8834aa358f02cff45e71052dd648c0|30|file|source

 

They respect a specific format which is:
[log_id] | [num_doc] | [facet_value] | [facet_field]

 

Let's explain each field:

  • [log_id] : the log id which must be UNIQUE per day, per facet field and value, in order to exploit those logs with ELK
  • [num_doc] : the number of documents found according to the facet
  • [facet_value] : the facet value represented by the log
  • [facet_field] : the facet field used

 

At each monitoring log generation iteration, a log line is created for each facet value of the selected facet fields, plus a log line for the global number of documents indexed (in that case, the [facet_value] and [facet_field] are both set to "no").
The default selected facet fields are "extension", "language" and "source". One can change the facet fields selected in the class "com.francelabs.monitoring.IndexMonitoring.FileShareMonitoringLog".

  • No labels