Skip to end of metadata
Go to start of metadata

You are viewing an old version of this page. View the current version.

Compare with Current View Page History

Version 1 Next »

The Doc Filter Connector has been introduced in both the Enterprise and Community edition of the v5.1

This connector provides regex based filters to include or exclude documents from indexation.

It is initialized by default in Datafari and can be added in the connectors workflow of an MCF job by selecting it in the list of transformation connectors. For optimal performances, it must be inserted right after the repo connector:

After adding the connector to the workflow a new tab named “Doc Filter” should appear in the list of tabs. In this tab, several things can be configured:

  • Filter field: Here you need to specify the field of the document on which you want to apply the regex filter. If this parameter is left empty, the filters will be applied by default to the document URI. The filed you can set in this this parameter MUST BE an existing field of the document AND a string type field, otherwise your filters won’t work.
    For example, if the documents crawled contain a “department_name” field, you can set it in the filter field in order to filter the documents so that their “department_name” field comply with the regex filters specified.
    You can only filter on one document field !

  • Include filters: It represents the list of regex filters from which the document field specified in the “Filter field” parameter must match at least one of the filters for the document to be included in the indexation process.
    To add a regex filter to this list, simply fulfill the text input with a regex then click on the “Add” button next to it.
    The regular expression must comply with

  • Exclude filters: It represents the list of regex filters from which the document field specified in the “Filter field” parameter must match at least one of the filters for the document to be excluded from the indexation process.
    This list has a higher priority than the include filters list which means that a document that matches at least one filter in the include filters list and one filter in this list will be excluded from the indexation process
    To add a regex filter to this list, simply fulfill the text input with a regex then click on the “Add” button next to it.
    The regular expression must comply with

  • Maximum document size: This parameter represents the maximum threshold in octets for the documents size. When a document size in octets exceeds this threshold, the document is excluded from the indexation process

  • Minimum document size: This parameter represents the minimum threshold in octets for the documents size. When a document size in octets is under this threshold, the document is excluded from the indexation process

  • No labels