[DEPRECATED] SearchProxy API

Deprecated from Datafari 5.4
Starting from Datafari 5.4, SearchProxy API is deprecated and we use Search API by default (subpart of Datafari API)

Valid from 4.1 to 5.3

The documentation below is valid from Datafari v4.1.0 up to Datafari 5.3

The SearchProxy API is a servlet that is the bridge between the search UI and Solr. It handles the queries done by the user through the search UI, transforms them into Solr queries, executes them and returns back the response.

To make your own search UI you only need to submit queries to this API with an HTTP GET request that matches the format described in this documentation and then handle the response.

Request the SearchProxy API
The entry point of the SearchProxy servlet is the following : http://DATAFARI_MAIN_SERVER_ADDRESS:8080/Datafari/SearchProxy
To the entry point, you well need to specify the Solr request handler you want to reach. By default there are 2 request handler available : /select for the main search request handler, or /suggest for search suggestions (used to display autocomplete values when the user is typing in the search bar). With regards to AjaxFranceLabs, we usually call the /suggest request handler whenever there is a keystroke and the /select request handler whenever there is a search query. If you want to inspire from an example take a look at our documentation and the corresponding code : AjaxFranceLabs Framework architecture and mechanism
For example, if you want to request the SearchProxy API to send search queries, you will use an HTTP GET request to http://DATAFARI_MAIN_SERVER_ADDRESS:8080/Datafari/SearchProxy/select
Depending on the custom Solr configuration you have done, you can of course reach more Solr request handlers but due to security reasons, we have limited the number of allowed features for request handlers. The SearchProxy servlet will allow and recognize the basic request handlers /select and /suggest, plus the request handlers used for two features. These features are: the autocomplete module for advanced search fields and the categories for autocomplete. In the case of the autocomplete for advanced search feature, the custom request handlers need to be declared in the AUTOCOMPLETEFIELDS parameter : Configuring Advanced search Autocomplete - Enterprise Edition. In the case of the categories feature, the custom request handlers need to be declared in the AUTOCOMPLETESUGGESTERS parameter : Autocomplete with Categories/Entities Configuration.

All the query parameters are passed as GET parameters. You can set all the Solr query parameters you want and/or need. But the following parameters presence is mandatory :
- wt : must always be equals to 'json'
- json.wrf : used to define a name to the response, you can set whatever name you want. See response format for more details
- id : used to register statistics, must be an UUID generated for each user session. This id will be used to track user behavior so it needs to be the same id till the user ends his session.
- q : the query the user has typed in the search bar
For example to execute the query 'testing' on the /select request handler, one will generate the following HTTP request : http://DATAFARI_MAIN_SERVER_ADDRESS:8080/Datafari/SearchProxy/select?q=testing&id=5f034d5b-95af-4048-adbc-601acd24cc96&wt=json&json.wrf=test
To execute the query 'testing' on the /suggest request handler, one will generate the following HTTP request : http://DATAFARI_MAIN_SERVER_ADDRESS:8080/Datafari/SearchProxy/suggest?q=testing&id=5f034d5b-95af-4048-adbc-601acd24cc96&wt=json&json.wrf=test
To avoid an overload of data in the query response, we also recommend to set the "fl" parameter to limit the number of returned fields. As example, here is how is configured the fl parameter in Datafari: fl=title,url,id,extension
To manage security and user ACLs in results, you will need to send the request to the SearchProxy servlet with a session that has been authenticated by Datafari.
To authenticate yourself via the HTTP Client, you will need to send a POST request to http://DATAFARI_MAIN_SERVER_ADDRESS:8080/Datafari/login.jsp with the parameters j_username and j_password containing respectively the username and its corresponding password. Then you will need to handle the session for your SearchProxy requests.
Here is a java code example to do this using Apache HTTP Components HTTP Client 4.5.3 and Apache Components HTTP Core 4.4.7 :
final CookieStore cookieStore = new BasicCookieStore(); final HttpClient httpClient = HttpClientBuilder.create().setRedirectStrategy(new LaxRedirectStrategy()).setDefaultCookieStore(cookieStore).build(); final HttpContext httpContext = new BasicHttpContext(); httpContext.setAttribute(HttpClientContext.COOKIE_STORE, cookieStore); final HttpPost post = new HttpPost("http://localhost:8080/Datafari/login.jsp"); final List<NameValuePair> params = new ArrayList<>(); params.add(new BasicNameValuePair("j_username", "admin")); params.add(new BasicNameValuePair("j_password", "admin")); post.setEntity(new UrlEncodedFormEntity(params)); final String id = UUID.randomUUID().toString(); final HttpGet get = new HttpGet("http://localhost:8080/Datafari/SearchProxy/select?q=searching&wt=json&json.wrf=myQuery&id=" + id + "&fl=title,url,id,extension"); final HttpResponse response = httpClient.execute(get); System.out.println(IOUtils.toString(response.getEntity().getContent(), StandardCharsets.UTF_8));
Handle the response format
Here is an example of the response format for the request
http://datafari-server:8080/Datafari/SearchProxy/select?q=searching&wt=json&json.wrf=myQuery&id=61546-efses-6efs6&fl=title,url,id,extension : (see further down for a formatted version)
myQuery({"spellcheck":{"collations":[],"suggestions":[],"correctlySpelled":true},"response":{"docs":[{"extension":"html","id":"http:\/\/www.francelabs.com\/blog\/tag\/search-2\/","title":["index.html"],"url":"http:\/\/www.francelabs.com\/blog\/tag\/search-2\/"},{"extension":"html","id":"http:\/\/www.francelabs.com\/blog\/tag\/open-source\/","title":["index.html"],"url":"http:\/\/www.francelabs.com\/blog\/tag\/open-source\/"},{"extension":"html","id":"http:\/\/www.francelabs.com\/blog\/tag\/conference\/","title":["index.html"],"url":"http:\/\/www.francelabs.com\/blog\/tag\/conference\/"},{"extension":"html","id":"http:\/\/www.francelabs.com\/blog\/category\/elasticsearch-2\/","title":["index.html"],"url":"http:\/\/www.francelabs.com\/blog\/category\/elasticsearch-2\/"},{"extension":"html","id":"http:\/\/www.francelabs.com\/blog\/tag\/file-search\/","title":["index.html"],"url":"http:\/\/www.francelabs.com\/blog\/tag\/file-search\/"},{"extension":"html","id":"http:\/\/www.francelabs.com\/blog\/tag\/elasticsearch\/","title":["index.html"],"url":"http:\/\/www.francelabs.com\/blog\/tag\/elasticsearch\/"},{"extension":"html","id":"http:\/\/www.francelabs.com\/blog\/tag\/enterprise-search\/","title":["index.html"],"url":"http:\/\/www.francelabs.com\/blog\/tag\/enterprise-search\/"},{"extension":"html","id":"http:\/\/www.francelabs.com\/blog\/tag\/early-binding\/","title":["index.html"],"url":"http:\/\/www.francelabs.com\/blog\/tag\/early-binding\/"},{"extension":"html","id":"https:\/\/www.francelabs.com\/en\/solutions.html","title":["solutions.html"],"url":"https:\/\/www.francelabs.com\/en\/solutions.html"},{"extension":"html","id":"http:\/\/www.francelabs.com\/blog\/enterprise-search-europe-open-source\/","title":["index.html"],"url":"http:\/\/www.francelabs.com\/blog\/enterprise-search-europe-open-source\/"}],"numFound":125,"start":0},"responseHeader":{"zkConnected":true,"QTime":39,"params":{"q":"searching","json.wrf":"myQuery","qt":"\/select","_stateVer_":"FileShare:13","hl.maxAnalyzedChars":"51200","fl":"title,url,id,extension","id":"04b6197c-a020-4931-b217-f96e0d9c7f46","wt":"javabin","version":"2"},"status":0},"highlighting":{"http:\/\/www.francelabs.com\/blog\/tag\/conference\/":{"content_fr":[],"exactContent":[],"content_en":["Search<\/span> technologies Big data Search<\/span>, Solr, Elasticsearch, Hadoop Search<\/span> Main menu Skip to primary content Skip to secondary content Home About Tag Archives: conference Enterprise Search<\/span> Europe in Lond"]},"http:\/\/www.francelabs.com\/blog\/category\/elasticsearch-2\/":{"content_fr":[],"exactContent":[],"content_en":["Search<\/span> technologies Big data Search<\/span>, Solr, Elasticsearch, Hadoop Search<\/span> Main menu Skip to primary content Skip to secondary content Home About Category Archives: Elasticsearch Enterprise Search<\/span> Europe"]},"http:\/\/www.francelabs.com\/blog\/tag\/file-search\/":{"content_fr":[],"exactContent":[],"content_en":["Search<\/span> technologies Big data Search<\/span>, Solr, Elasticsearch, Hadoop Search<\/span> Main menu Skip to primary content Skip to secondary content Home About Tag Archives: file search<\/span> Tutorial for combining Manifold"]},"http:\/\/www.francelabs.com\/blog\/tag\/open-source\/":{"content_fr":[],"exactContent":[],"content_en":["Search<\/span> technologies Big data Search<\/span>, Solr, Elasticsearch, Hadoop Search<\/span> Main menu Skip to primary content Skip to secondary content Home About Tag Archives: open source Enterprise Search<\/span> Europe in Lon"]},"https:\/\/www.francelabs.com\/en\/solutions.html":{"content_fr":[],"exactContent":[],"content_en":["search<\/span> engine based on Apache Solr 5 Advanced search<\/span> with geolocalised search<\/span> Analytics tools Semantic search<\/span> and thesaurii management Data display (including oil rigs) with previews and geomaps Secur"]},"http:\/\/www.francelabs.com\/blog\/tag\/early-binding\/":{"content_fr":[],"exactContent":[],"content_en":["Search<\/span> technologies Big data Search<\/span>, Solr, Elasticsearch, Hadoop Search<\/span> Main menu Skip to primary content Skip to secondary content Home About Tag Archives: Early binding Activating early binding in C"]},"http:\/\/www.francelabs.com\/blog\/tag\/enterprise-search\/":{"content_fr":[],"exactContent":[],"content_en":["Search<\/span> technologies Big data Search<\/span>, Solr, Elasticsearch, Hadoop Search<\/span> Main menu Skip to primary content Skip to secondary content Home About Tag Archives: enterprise search<\/span> Enterprise Search<\/span> Europe "]},"http:\/\/www.francelabs.com\/blog\/enterprise-search-europe-open-source\/":{"content_fr":[],"exactContent":[],"content_en":["Search<\/span> technologies Big data Search<\/span>, Solr, Elasticsearch, Hadoop Search<\/span> Main menu Skip to primary content Home About Post navigation ← Previous Next → Enterprise Search<\/span> Europe in London \u2013 Open source "]},"http:\/\/www.francelabs.com\/blog\/tag\/elasticsearch\/":{"content_fr":[],"exactContent":[],"content_en":["Search<\/span> technologies Big data Search<\/span>, Solr, Elasticsearch, Hadoop Search<\/span> Main menu Skip to primary content Skip to secondary content Home About Tag Archives: elasticsearch Tutorial for combining Manifo"]},"http:\/\/www.francelabs.com\/blog\/tag\/search-2\/":{"content_fr":[],"exactContent":[],"content_en":["Search<\/span> technologies Big data Search<\/span>, Solr, Elasticsearch, Hadoop Search<\/span> Main menu Skip to primary content Skip to secondary content Home About Tag Archives: search<\/span> Enterprise Search<\/span> Europe in London \u2013"]}}})
The format is really simple : [json.wrf parameter value]([json response])
As you can see the json response is wrapped by myQuery which is the value entered for the json.wrf parameter.
The json response object is the original json returned by Solr so you will need to know how Solr builds the json response according to the parameters you have set.
Here are the standard fields that you will retrieve in a query response :
Here is the list of the useful fields of the json response:
- spellcheck : contains query corrections if it exists close word(s) query(ies) that return more results
- response : contains 3 subfields, docs that contains the list of returned docs (in the limit of the number of rows), numFound that indicates the total number of matching documents, start which indicates the sorted number of the first document returned in the docs list
- responseHeader : contains some infos about the query executed and its status
- highlighting : for each returned doc, contains the snippet that Solr was able to generates. See https://lucene.apache.org/solr/guide/6_6/highlighting.html for more infos