Skip to end of metadata
Go to start of metadata

You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 19 Current »

Valid for 6.0 upwards

Datafari_CE_6_0_Architecture_mono.png


 Valid for 5.2

Valid for 5.2


 Valid for 4.4 up to 5.1

Valid for 4.4 up to 5.1


 Valid for 4.2

Valid for 4.2


 Valid for 4.1

Valid for 4.1


 Valid for V4.0

Valid for V4.0


 Valid for V2.0

Valid for V2.0

Datafari uses a typical search engine architecture. It is based on the triptych crawling, indexing and search. The crawling part is using Apache ManifoldCF. There is another opensource connectors framework under Apache licence v2, the Google Connector Framework. Yet the latter is only being supported and developed by Google, so it appeared more reasonable to us to leverage ManifoldCF, proposed by the Apache foundation, and which benefits from the support of several committers from different entities.

The indexing and search parts both use Apache Solr. Again, there is another popular indexing and search engine available under Apache licence V2. It is Elasticsearch, but similarly to the Google Connector Framework, it is led by the eponym entity, and as such does not guarantee such a longevity as Apache Solr.

ManifoldCF, although being independent from Apache Solr, has the advantage of being conceived from the start as Solr’s connectors framework. It is thus conceptually “naturally” connected to Apache Solr.

The figure below illustrates the system architecture that we are using for Datafari. Its v2 is becoming rather large in terms of components, so this architecture is rather high level and intentionally avoids some connections and components for the sake of clarity.



 Datafari 3.0

Datafari 3.0

On Datafari 3.0 we chose to use directly SolrCloud and to isolate the different parts of our product that can be installed on different servers.

If your Solr index becomes very huge or if your nomber of users becomes too big, you can easily shard your index and/or replicate it with SolrCloud.


  • No labels