Elasticsearch Aggregations

Aggregated or faceted search dynamically clusters items or search results into categories and allows users to drill into the search results by any value in any field. Each displayed "aggregate" of the search results shows the number of hits within the search that match that category, and users can "drill down" by applying specific constraints to the search results.

Aggregated site search is implemented in dotCMS using Elasticsearch aggregations.

The $estool.search Method #

To perform any Elasticsearch query, including an aggregation query, you must call the Elasticsearch Viewtool search method (see the examples below).

Arguments #

The $estool.search method takes a single argument that must be a valid ElasticSearch query string (using full Elasticsearch JSON syntax).

When passing the query string to the $estool.search method, you must render the query string using the Render Viewtool, as in the following example (where $esQuery contains the query string in Elasticsearch JSON format:


#set($rawResults = $estool.search($render.eval($esQuery)))

Search Fields #

You can perform an aggregations query using any of the following search terms:

content
filename
host
mimeType
title
description
keywords
author
contentLength
modified
score

Results #

When you supply an aggregates query to the $estool.search method, the result is a list of aggregates for the most popular terms across the documents matching your query, or across all documents in the index, each with a term and count.

Parsing the Results #

To transform the results into a list which you can iterate across in Velocity, you must call the JSONtool generate method, as in the following example:


#set($results = $json.generate($rawResults.response))

Once you have transformed the results, you can use a Velocity loop to "drill down" into the results (see the example, below).

Example #

The following query performs an aggregations search on the mimeType term.

Note: In all of the following examples, the $searchQ string contains a lucene query string, and the $esQuery string contains an Elasticsearch query string in full Elasticsearch JSON format.

Step 1: Define the aggregations query #

First, define the aggregations query using the mimeType term:


#set($esQuery = '{
    "query" : { "query_string" : {"query" : "$searchQ"} },
    "aggs" : {
        "mimeType" : { "terms" : {"field" : "mimeType"} } 
    },
    "size":0
}')

Step 2: Call the $estool.search method #

Next, render the query and call the search method:


#set($aggsQuery = $render.eval($esQuery))
#set($rawResults = $estool.search($aggsQuery))

Step 3: Parse the results and iterate over them in Velocity #

Finally, use the JSONtool to parse the results, and iterate over each of the returned aggregations to display both the aggregation term and the count:


#set($results = $json.generate($rawResults.response))
#if($UtilMethods.isSet($results) && $results.size > 0)
    #foreach($aggregation in $results)
        #foreach($term in $aggregation.entries)
            $term.term ($term.count)
        #end
    #end
#end

Additional Aggregation Implementations #

The following examples demonstrate queries that use additional Elasticsearch aggregation implementations, including Range aggregations, Histogram aggregations and Date Histogram aggregations.

For a list of all additional Elasticsearch aggregation implementations, please see the Elasticsearch Bucket Aggregations, Metrics Aggregations, and Pipeline Aggregations documentation.

Range #

The following query implements an Elasticsearch Range Aggregation.


#set($esQuery = '  {
    "query" : { "query_string" : {"query" : "$searchQ"} },
    "aggs" : {
        "range1" : {
            "range" : {
                "field" : "contentLength",
                "ranges" : [
                    { "to" : 50 },
                    { "from" : 20, "to" : 70 },
                    { "from" : 70, "to" : 120 },
                    { "from" : 150 }
                ]
            }
        }
    },"size":0
  }')

Histogram #

The following query implements an Elasticsearch Histogram Aggregation.


#set($esQuery = '  {
    "query" : { "query_string" : {"query" : "$searchQ"} },
    "aggs" : {
        "histo1" : {
            "histogram" : {
                "field" : "modified",
                "interval" : 100
            }
        }
    },
    "size":0
}')

Date Histogram #

The following query implements an Elasticsearch Date Histogram Aggregation.


#set($esQuery = '  {
    "query" : { "query_string" : {"query" : "$searchQ"} },
    "aggs" : {
        "modified" : { "date_histogram" : {"field" : "modified","interval" : "month" }}
    },
    "size":0
}')

For more information on Site search, please see the Site Search documentation.

Migrating from Facets to Aggregations #

Previous versions of dotCMS used the defunct Elasticsearch Facets feature instead of Elasticsearch aggregations. If you have existing queries that use Elasticsearch Facets, those queries need to be migrated to use aggregations. In most cases, rewriting a facet query as an aggregation query requires only 2 simple steps:

Replace the "facet" term in the query with "aggs".
Replace the $search.getFacets call in your Velocity code with a call to $estool.search.

For more information on migrating facet queries to use aggregations, please see the Elasticsearch documentation.

Additional Examples #

For additional examples of ElasticSearch queries using aggregations, please see the List of all Content Types and Counts and Tag Cloud examples in the Elasticsearch Examples documentation.