Scholar Request

Request Structure

The request fields are used in queries and sort operations. The request payload should comply with following json schema.

Searchable Fields

For searching, the following fields are supported in the API:

Filtering

You can use following pre-defined filters to refine search results:

Example:

{
  "query": {
     "match":{
     	  "has_patent_citations": true
     }
  }
}

Pagination

Lens API provides two type of pagination based on their use:

Offset/Size Based Pagination

Use parameter from to define the offset and size to specify number of records expected. This is useful when you want to skip some records and select desired ones. Example below skips first 100 and select 50 records after that.

{
  "query": "Malaria",
  "from": 100,
  "size":50
}

Similarly for GET requests, the following parameters are applicable: size=50&from=100

Note:

• Offset/size based paginations is suitable for small result sets only and does not work on result sets of more that 1000 records. For larger volume data downloads, use Cursor Based Pagination.

Cursor Based Pagination

You can specify records per page using size (default 20 and max 1000) and context alive time scroll (default 1 minute). You will receive a scroll_id in response, which should be passed via request body to access next set of results. Since the scroll_id tends to change every time after each successful requests, please use the most recent scroll_id to access next page. This is not suited for real time user requests.

{
  "scroll_id": "MjAxOTEw;DnF1ZXJ...R2NZdw==",
  "scroll": "1m"
}

Note:

• The lifespan of scroll_id is limited to 1 minute for the current API version. Using expired scroll_id will result bad request HTTP response.
• Parameter size will be used for first scroll query and will remain the same for whole scroll context. Hence, using size in each scroll request will not have any effect.
• Cursor based pagination is only applicable to POST requests.
• For optimal performance, we recommend limiting the number of items (e.g. lens_ids) in a single terms query to 10,000.

Sorting

Result can be retrieved in ascending or descending order. Use the following format and fields to apply sorting to the API response. Results can also be sorted by relevance score using relevance.

{
  "sort": [
      {"patent_citation_count":"desc"},
      {"year_published": "asc"},
      {"relevance": "desc"}
  ]
}

For GET requests following structure is applicable: sort=desc(patent_citation_count),asc(date_published),desc(relevance)

Projection

You can control the output fields in the API Response using projection. There are two possible ways to do that.

1. include: Only request specific fields from the API endpoint
2. exclude: Fields to be excluded from result

 {"include":["title","patent_citations","authors.affiliations.name"]}

 {"exclude":["external_ids","references"]}

For GET requests following structure is applicable. include=authors,lens_id

Note: Both include and exclude can be used in same request.

Stemming

Stemming allows to reduce the words to root form. E.g. Constructed and constructing will be stemmed to root construct. Since sometime the default stemming might not give you exact result, disabling it will just search for provided form of the word. e.g. "stemming": false

Regex

Regex allows the use of regular expressions in Query String based query, e.g. "regex": true

{
    "query": "field_of_study:/.*[Ee]conom.*/",
    "regex": true
}

Supported Query Types

Following queries are supported by current version of Lens API:

Note: The Lens API query requests use a modified form of the Elasticsearch Query DSL. For more details on the Elasticsearch query syntax, we recommend reading this guide on the query syntax: Elasticsearch Query DSL

Term Query

Term Query operates in a single term and search for exact term in the field provided.

Example: Find record by publication type

{
    "query": {
        "term": {
            "publication_type": "journal article"
        }
    }
}

Terms Query

Terms Query allows you to search multiple exact terms for a provided field. A useful scenario is while searching multiple identifiers.

Example: Search scholarly works for multiple pmid

{
	"query": {
		"terms": {
			"pmid": ["14297189", "17475107"]
		}
	}
}

Note: Avoid using the Term and Terms queries for text fields. To search text field values, we recommend using the Match and Match Phrase queries instead.

Match query

Match query accepts text/numbers/dates. The main use case of the match query is full-text search. It matches each words separately. If you need to search whole phrase use match phrase query.

Example:

{
  "query": {
      	"match":{
      		"author.affiliation.name": "Harvard University"
      	}
   }
}

Match Phrase query

Match phrase query accepts text/numbers/dates. The main use case for the match query is for full-text search.

Example:

{
  "query": {
      	"match_phrase":{
      		"author.affiliation.name": "Harvard University"
      	}
   }
}

Note: Both Match and Match Phrase are used for text searching but the difference is how they do it. For example, searching for "Cleveland, OH" differs between Match and Match Phrase like this:

• Match: standard search in which each word is matched separately (for example: Cleveland OR OH)
• Match Phrase: matches the exact phrase provided. In this case it will match the exact text Cleveland, OH

Range query

Range query query to match records within the provided range.

Example: Get record for year published between years 1980 and 2000

{
  "query": {
      	"range": {
            "year_published": {
                "gte": "1980",
                "lte": "2000"
            }
        }
   }
}

Boolean query

Bool Query allows to combine multiple queries to create complex query providing precise result. It can be created using one or more of these clauses: must, should, must_not and filter. You can use must for AND operation and should for OR.

Example: Get journal article scholarly works of Author with last name Kondratyev having patent citations.

{
 "query": {
   "bool": {
     "must": [{
       "match": {
         "has_patent_citations": true
       }},
       {"bool": {
         "must": [
           {"match": {"publication_type": "journal article"}},
           {"match": {"author.last_name": "Kondratyev"}}
         ]
       }
       }
     ]
   }
 }
}

Query String Based Query

Query different terms with explicit operators AND/OR/NOT to create a compact query string.

Example: Find works from institution published between two dates having some title.

{"query": "(title:Dimensions AND author.affiliation.name:(Harvard University)) AND year_published:[2000 TO 2018]"}

If you need to use any reserved special characters, you should escape them with leading backslash.

Example: Getting by doi identifier using string based query

{"query": "doi:10.1109\\/ee.1934.6540358"}

You can use json based format for string based query and mixed with complex boolean queries like this:

{
    "query": {
        "bool": {
            "must": [
                {
                    "query_string": {
                        "query": "\"X-ray analysis of protein crystals\"",
                        "fields": [
                            "title",
                            "abstract",
                            "full_text"
                        ],
                        "default_operator": "and"
                    }
                }
            ],
            "filter": [
                {
                    "term": {
                        "has_affiliation": true
                    }
                }
            ]
        }
    }
}