Detecting file type via its file extension
Valid for 4.0
The documentation below is valid from Datafari 4.0 upwards
When indexing documents, Datafari tries to identify the types of files it is indexing to register the correct extension together with each document. The registered extension is used to provide faceted search and allows the user to filter the results of his query by file type. This page exposes the means used by Datafari to perform this detection.
Default Behavior
By default, Datafari will try to guess the extension of the file from the filename. If the URL to a document is something like:
file:///a/b/c/someFile.extThen the extracted extension will be ext, whatever the real type of the file is.
If the extraction of the file extension from the file name fails, then the file type guessed by tika is used instead.
Finally, if both of the previous technique failed, then an empty string is used as the file extension.
Be aware that using the default behavior on websites that have url like:
http://mydomain.net/some/path/getDoc.php?doc=myDocument.pdfwill result in a php type for this document (and all other documents that are retrieved using the same script).
For web crawl in general, it is advised to use the alternative configuration, that uses the Tika extracted type in priority.
Alternative Configuration
The configuration can be changed in the solrconfig.xml via the following parameter:
<str name="extension.fromname">true</str>Setting the value of this parameter to false, the tika type guessing system will be used as the main source of information for file types, and the filename will be used as a fallback if the tika information fails.
This parameter is under the "datafari" updateRequestProcessorChain (near line 1180, but this may obviously change).
Please refer to System Configuration Manager (Zookeeper) to know where the configuration files are located and how to reload the configuration correctly.