What's New in V4

The centerpiece of this update, the query engine, has been completely rebuilt. This transformation shifts away from reliance on query strings to a direct integration with Elasticsearch’s DSL (Domain Specific Language), unlocking a broader range of querying capabilities and a more profound integration with Elasticsearch’s advanced features.

New features

Cross Fields Search Queries

This version introduces query-building methods to allow matching across multiple fields: ES docs

The previous search builder worked in isolation for full-text searching; this upgrade brings those features into the standard query builder that will run like a normal query. Meaning you can: get(), first(), aggregate(), paginate() etc on full text search results. In time, this will replace the current search methods like: Book::term('Eric')->search();

Methods

• searchTerm($term, $fields = ['*'], $options = []) - type: best_fields
• searchTermMost($term, $fields = ['*'], $options = []) type: most_fields
• searchTermCross($term, $fields = ['*'], $options = []) type: cross_fields
• searchPhrase($phrase, $fields = ['*'], $options = []) type: phrase
• searchPhrasePrefix($phrase, $fields = ['*'], $options = []) type: phrase_prefix
• searchBoolPrefix($phrase, $fields = ['*'], $options = []) type: bool_prefix

Each method has a corresponding OR version, ex orSearchPhrase('Laravel Elasticsearch')

These methods will introduce an Elasticsearch score and will be ordered by score by default.

$fields: By default, all fields will be searched through; you can specify which as well as boost certain fields, ex:

People:searchTerm('John',['name^3','description^2','friends.name'])->get();

$options: Allows you to set any options for the multi_match clause to use, ex: analyzer, boost, operator, minimum_should_match, fuzziness, lenient, prefix_length, max_expansions, fuzzy_rewrite, zero_terms_query.

searchFor($value) is a convenience method that will either use term or phrase depending on the word count of $value

withHighlights($fields = [], $preTag = '<em>', $postTag = '</em>', $globalOptions = [])

Option helpers

• asFuzzy()

  • Will mark the previous clause as fuzzy

• setMinShouldMatch(int $value)

  • will set the option {"minimum_should_match": $value} to the previous clause

• setBoost(int $value)

  • will set the option {"boost": $value} to the previous clause

Examples

Product::searchTerm('remarkble')->asFuzzy()->withHighlights()->get();
Product::searchPhrasePrefix('coffee bea')->where('is_active',true)->paginate();
Product::searchPhrase('United States')->orSearchPhrase('United Kingdom')->sum('orders');

Saving Enhancements

• updateOrCreate and updateOrCreateWithoutRefresh methods have been added to the model. These methods will update a record if it exists or create a new one if it does not. The updateOrCreateWithoutRefresh method will skip the immediate refresh after the operation.

Schema upgrades

1. Schema field mapping call:

• getFieldMapping(string|array $field = '*', $raw = false)

Schema method that can be called from your model:

Product::getFieldMapping('color'); //Returns a key/value array of field/types for color
Product::getFieldMapping('color',true); //Returns the mapping for  color field as is from Elasticsearch
Product::getFieldMapping(['color','name']); //Returns maapings for  color and name
Product::getFieldMapping(); //returns all field mappings, same as getFieldMapping('*')

or via Schema: Schema::getFieldMapping($index, $field, $raw)

Schema::getFieldMapping('products','color',true);

2. New numeric type mappings for IndexBlueprint

• double($field) - A double-precision 64-bit IEEE 754 floating point number, restricted to finite values.
• byte($field) - A signed 8-bit integer with a minimum value of -128 and a maximum value of 127.
• halfFloat($field) - A half-precision 16-bit IEEE 754 floating point number, restricted to finite values.
• scaledFloat($field, $scalingFactor = 100) - A floating point number that is backed by a long, scaled by a fixed double scaling factor.
• unsignedLong($field) - An unsigned 64-bit integer with a minimum value of 0 and a maximum value of 264-1.

Example:

  Schema::create('my_index', function (IndexBlueprint $index) {
      $index->double('some_field_a');
      $index->byte('some_field_b');
      $index->halfFloat('some_field_c');
      $index->scaledFloat('some_field_d', 100);
      $index->unsignedLong('some_field_e');
  });

---

Connection Upgrades

1. Upgraded Connection class to parse the config’s connection name.

This allows for multiple connections or if you define your connection in the database file something other than elasticsearch

Example with multiple connections (database.php):

'elasticsearch' => [
    'driver' => 'elasticsearch',
    'auth_type' => env('ES_AUTH_TYPE', 'http'), //http or cloud
    'hosts' => explode(',', env('ES_HOSTS', 'http://localhost:9200')),
    'username' => env('ES_USERNAME', ''),
    'password' => env('ES_PASSWORD', ''),
    'cloud_id' => env('ES_CLOUD_ID', ''),
    'api_id' => env('ES_API_ID', ''),
    'api_key' => env('ES_API_KEY', ''),
    'ssl_cert' => env('ES_SSL_CA', ''),
    'ssl' => [
        'cert' => env('ES_SSL_CERT', ''),
        'cert_password' => env('ES_SSL_CERT_PASSWORD', ''),
        'key' => env('ES_SSL_KEY', ''),
        'key_password' => env('ES_SSL_KEY_PASSWORD', ''),
    ],
    'index_prefix' => env('ES_INDEX_PREFIX', false),
    'options' => [
        'bypass_map_validation' => env('ES_OPT_BYPASS_MAP_VALIDATION', false),
        'insert_chunk_size' => env('ES_OPT_INSERT_CHUNK_SIZE', 1000),
        'logging' => env('ES_OPT_LOGGING', false),
        'allow_id_sort' => env('ES_OPT_ID_SORTABLE', false),
        'ssl_verification' => env('ES_OPT_VERIFY_SSL', true),
        'retires' => env('ES_OPT_RETRIES', null),
        'meta_header' => env('ES_OPT_META_HEADERS', true),
    ],
    'error_log_index' => env('ES_ERROR_INDEX', false),
],
'elasticsearch-cloud' => [
    'driver'       => 'elasticsearch',
    'auth_type'    => env('ES_CLOUD_AUTH_TYPE', 'http'), //http or cloud
    'hosts'        => explode(',', env('ES_CLOUD_HOSTS', 'http://localhost:9200')),
    'username'     => env('ES_CLOUD_USERNAME', ''),
    'password'     => env('ES_CLOUD_PASSWORD', ''),
    'cloud_id'     => env('ES_CLOUD_CLOUD_ID', ''),
    'api_id'       => env('ES_CLOUD_API_ID', ''),
    'api_key'      => env('ES_CLOUD_API_KEY', ''),
    'ssl_cert'     => env('ES_CLOUD_SSL_CA', ''),
    'ssl'          => [
        'cert'          => env('ES_CLOUD_SSL_CERT', ''),
        'cert_password' => env('ES_CLOUD_SSL_CERT_PASSWORD', ''),
        'key'           => env('ES_CLOUD_SSL_KEY', ''),
        'key_password'  => env('ES_CLOUD_SSL_KEY_PASSWORD', ''),
    ],
    'index_prefix' => env('ES_CLOUD_INDEX_PREFIX', false),
    'options'      => [
        'bypass_map_validation' => env('ES_CLOUD_OPT_BYPASS_MAP_VALIDATION', false),
        'insert_chunk_size' => env('ES_CLOUD_OPT_INSERT_CHUNK_SIZE', 1000),
        'logging' => env('ES_CLOUD_OPT_LOGGING', false),
        'allow_id_sort' => env('ES_CLOUD_OPT_ID_SORTABLE', false),
        'ssl_verification' => env('ES_CLOUD_OPT_VERIFY_SSL', true),
        'retires' => env('ES_CLOUD_OPT_RETRIES', null),
        'meta_header' => env('ES_CLOUD_OPT_META_HEADERS', true),
    ],
    'error_log_index' => env('ES_CLOUD_ERROR_INDEX', false),
],

Examples of selecting connection:

Schema::on('elasticsearch-cloud')->create('my_index', ...... );
Product::on('elasticsearch-cloud')->get() //If $connection in Product model is not 'elasticsearch-cloud';

2. Bypass field map validation for queries that require keyword mapping

Keyword type queries are checked by default and will select the keyword sub-mapping if it is found; however, this invokes an extra query to check the mapping first.

You can now disable this by setting options.bypass_map_validation = true

'elasticsearch' => [
    ......
     'options' => [
        'bypass_map_validation' => env('ES_OPT_BYPASS_MAP_VALIDATION', false),
         .....
    ],
    .....
],

3. Adjustable chunk size for bulk insert (default 1000)

When performing bulk inserts, the default is 1000 at a time.

You can now adjust this by setting options.insert_chunk_size to your desired amount.

'elasticsearch' => [
    ......
    'options' => [
        'insert_chunk_size'  => env('ES_OPT_INSERT_CHUNK_SIZE', 1000),
         .....
    ],
    .....
],