| Info |
|---|
Valid from 4.0The documentation below is valid from Datafari v4.0.0 upwards |
1. Repository/Authority connector configuration
...
| Note |
|---|
Timestamp formatsUsually, these variables are very useful to compare them to the timestamp of the last modified date of a document. But be careful, the timestamp provided is expressed in milliseconds and not in seconds like a standard Unix timestamp ! Keep it in mind because in most cases you will need to adjust either the timestamp on the Database side or divide the timestamp provided by MCF by 1000 to convert it into a Unix format !format. |
b. Version check query
This query is very important because it will generate a VERSIONCOLUMN variable for each crawled document that will be associated to the doc ID and will be used by MCF to determine during a delta crawl if the document needs to be re-indexed or not. Basically the best practice is to bind the VERSIONCOLUMN to a table column of a document that is updated each time a modification is performed on the document. If you cannot have such a column in your DB schema, what we advise is to create a function on Database side that will generate a hash for a document, based on each column value. With such a function, if any of the values of a DB document changes, the hash will also change. Now if you set this DB function as the VERSIONCOLUMN (via a standard SQL function call), MCF will know the document has to be re-indexed.
...
For example, considering the following table named “docs” :
ID | document_id | url | document_title | content |
|---|---|---|---|---|
1 | doc_1 | https://mycompanie.com/doc_1 | advertisement.pdf | This is an advertise |
2 | doc_2 | january_report.doc | The January sales are good |
If we define the following data query:
...
Notice that the “url” field now corresponds to the “id” field ! . This is because as the documents don’t have an “url” metadata unlike the previous time, Datafari automatically sets the “id” field value to the “url” field value ! .