{"type":"doc","content":[{"type":"paragraph","content":[{"text":"This documentation gives you the steps to follow to add entity recognition to your Datafari using an external fastapi (giving access to spacy models in our case) to extract entities. This goes from how to add entity extraction to the indexation pipeline to using those entities in an autocomplete component to help users in their searches.","type":"text"}]},{"type":"heading","attrs":{"level":2},"content":[{"text":"Setting up the fastapi server","type":"text"}]},{"type":"paragraph","content":[{"text":"Follow the documentation ","type":"text"},{"type":"inlineCard","attrs":{"url":"https://datafari.atlassian.net/wiki/spaces/DATAFARI/pages/2657517573"}},{"text":" to setup this service.","type":"text"}]},{"type":"heading","attrs":{"level":2},"content":[{"text":"Adding Entity Extraction to a Job","type":"text"}]},{"type":"paragraph","content":[{"text":"In this step, you need to create an instance of the transformation connector. Then you will need to add this transformation to your job and run your job to extract the entities. These steps are explained in ","type":"text"},{"type":"inlineCard","attrs":{"url":"https://datafari.atlassian.net/wiki/pages/resumedraft.action?draftId=2469920769"}},{"text":".","type":"text"}]},{"type":"heading","attrs":{"level":2},"content":[{"text":"Adding a Search Component and Search Handler in Solr for Autocomplete","type":"text"}]},{"type":"paragraph","content":[{"text":"In this section we will assume that the field storing the entities you want to get autocomplete on is ","type":"text"},{"text":"entity_keyword","type":"text","marks":[{"type":"em"}]},{"text":".","type":"text"}]},{"type":"paragraph","content":[{"text":"To perform the following actions, you need to have access to the server hosting Datafari and be able to modify some files on it.","type":"text"}]},{"type":"paragraph","content":[{"text":"We will add the configuration we need in the files located in the folder ","type":"text"},{"text":"${DATAFARI_HOME}/solr/solrcloud/FileShare/conf/customs_solrconfig/","type":"text","marks":[{"type":"em"}]}]},{"type":"paragraph","content":[{"text":"First, in the file ","type":"text"},{"text":"solr/solrcloud/FileShare/conf/customs_solrconfig/custom_search_components.incl","type":"text","marks":[{"type":"em"}]},{"text":", add the following:","type":"text"}]},{"type":"codeBlock","content":[{"text":"\n \n suggesterEntityKeywords\n text_general\n AnalyzingInfixLookupFactory\n 0\n entity_keywords\n false\n HighFrequencyDictionaryFactory\n true\n \n","type":"text"}]},{"type":"paragraph","content":[{"text":"Second, in the file ","type":"text"},{"text":"solr/solrcloud/FileShare/conf/customs_solrconfig/custom_search_components.incl","type":"text","marks":[{"type":"em"}]},{"text":", add the following:","type":"text"}]},{"type":"codeBlock","content":[{"text":"\n \n true\n suggesterEntityKeywords\n 10\n \n \n suggestEntityKeywords\n \n","type":"text"}]},{"type":"panel","attrs":{"panelType":"note"},"content":[{"type":"paragraph","content":[{"text":"You can change the same of the component to match the type of entity you are working with. But be careful to be consistent in your changes. The suggest.dictionary property of the request handler makes reference to the name of the searchComponent for example. As you can also see in this configuration, the field property of the searchComponent refers to the field holding the entity type we are interested in.","type":"text"}]}]},{"type":"paragraph","content":[{"text":"We now need to load these configuration changes into zookeeper and make them effective in solr. To do so, go to the ","type":"text"},{"type":"inlineCard","attrs":{"url":"https://datafari.atlassian.net/wiki/spaces/DATAFARI/pages/34603014"}},{"text":" in the Datafari admin interface. There click the “PUSH” button and then once it is done the “REFRESH” button.","type":"text"}]},{"type":"paragraph","content":[{"text":"Now run the following command from your Datafari server (changing the /","type":"text"},{"text":"suggestKeywords","type":"text","marks":[{"type":"em"}]},{"text":" part) to build the suggestion dictionary:","type":"text"}]},{"type":"codeBlock","content":[{"text":"curl \"localhost:8983/solr/FileShare/suggestKeywords?suggest.q=something&suggest.build=true\"","type":"text"}]},{"type":"paragraph","content":[{"text":"Our search component and sarch handler should be ready to go at this stage. We will now configure Datafari to serve the new suggest endpoints and add an autocomplete suggester to DatafariUI.","type":"text"}]},{"type":"heading","attrs":{"level":2},"content":[{"text":"Configuring Datafari to Serve the New Suggest Request Handler","type":"text"}]},{"type":"paragraph","content":[{"text":"As the request handler we added is one that is used for suggestions and autocomplete, we need to modify the file ","type":"text"},{"text":"${DATAFARI_HOME}/tomcat/conf/entity-autocomplete.properties","type":"text","marks":[{"type":"em"}]},{"text":".","type":"text"}]},{"type":"paragraph","content":[{"text":"Add an entry to the AUTOCOMPLETESUGGESTERS array with the following information:","type":"text"}]},{"type":"codeBlock","content":[{"text":" {\\\n \"i18nKey\":\"asKeywords\", \\\n \"serverUrl\":\"\", \\\n \"solrCore\":\"\", \\\n \"servlet\":\"suggestKeywords\", \\\n \"suggestComponent\":\"suggesterKeywords\", \\\n \"maxSuggest\":2, \\\n \"categoryKey\":\"keywords\", \\\n \"categoryi18nKey\":\"keywords\", \\\n \"cssClass\":\"\",\\\n \"activated\": \"false\"\\\n }","type":"text"}]},{"type":"paragraph","content":[{"text":"The only information that are really important here are the servlet and seggestComponent keys. Other properties are used by the AjaxFranceLabs UI framework, but we are going to use DatafariUI so it does not matter.","type":"text"}]},{"type":"paragraph","content":[{"text":"Save the file once you are done. You should not need to restart anything for that change to be effective.","type":"text"}]},{"type":"heading","attrs":{"level":2},"content":[{"text":"Adding an Autocomplete Suggester in DatafariUI","type":"text"}]},{"type":"paragraph","content":[{"text":"Now that everything is ready, we can add an autocomplete suggester in DatafariUI. To do so, open the ","type":"text"},{"text":"${DATAFARI_HOME}/www/ui-config.json","type":"text","marks":[{"type":"em"}]},{"text":" file and modify the suggester array which is in the searchBar object:","type":"text"}]},{"type":"codeBlock","content":[{"text":"\"searchBar\": {\n \"suggesters\": [\n {\n \"type\": \"BASIC\",\n \"props\": {\n \"maxSuggestion\": 5,\n \"title\": \"SUGGESTED QUERIES\",\n \"subtitle\": \"Queries extending your current query terms\"\n }\n },\n {\n \"type\": \"ENTITY\",\n \"props\": {\n \"field\": \"entity_keywords\",\n \"suggester\": \"suggestKeywords\",\n \"dictionary\": \"suggesterEntityKeywords\",\n \"asFacet\": false,\n \"maxSuggestion\": 5,\n \"title\": \"Suggested keywords search\",\n \"subtitle\": \"Keywords matching the text you typed\"\n }\n }\n ]\n }","type":"text"}]},{"type":"paragraph","content":[{"text":"As you can see, we added a suggester of type ENTITY which refers to the field and different components we need. We recommend you leave asFacet to false unless you know the exact behavior of this option. You can modify the maximum number of suggestion shown, the title and subtitle as you see fit. Title and subtitle text can also be added to the translation files if you need to display them in different languages (use the text you put in this ui-config file as a key, and the translated text as a value in each language file you need).","type":"text"}]},{"type":"heading","attrs":{"level":2},"content":[{"text":"Result","type":"text"}]},{"type":"paragraph","content":[{"text":"That’s it, you now have an autocomplete section covering your entity type in DatafariUI","type":"text"}]},{"type":"mediaSingle","attrs":{"layout":"center"},"content":[{"type":"media","attrs":{"width":2073,"id":"b1620dd0-ac6f-4ae6-92e5-cb9d8187545e","collection":"contentId-2681831425","type":"file","height":363}}]},{"type":"paragraph"}],"version":1}

Browser not supported