6. This Week in Elasticsearch and Apache Lucene - 2019-04-05

This Week in Elasticsearch and Apache Lucene - 2019-04-05


  • ### Elasticsearch ### Highlights #### 6.7.0 Docker upgrade bug fixed We [fixed](https://github.com/elastic/elasticsearch/pull/40723) a [bug introduced in 6.7.0](https://www.elastic.co/guide/en/elasticsearch/reference/6.7/release-notes-6.7.0.html#_known_issues) that was preventing users of the official Docker image from upgrading. This was included in the [6.7.1 release](https://www.elastic.co/guide/en/elasticsearch/reference/current/release-notes-6.7.1.html). #### Snapshot lifecycle management We [now persist](https://github.com/elastic/elasticsearch/pull/40619) the last successful and failed snapshots into the cluster state to be visible for admins. The put snapshot lifecycle API [now validates](https://github.com/elastic/elasticsearch/pull/40654) as much as possible when a policy is created. #### Snapshot/Restore UI We [added support](https://github.com/elastic/kibana/pull/34020) for creating and editing repositories. #### Single-node discovery now uses Zen2 We have [switched](https://github.com/elastic/elasticsearch/pull/40036) the [single-node discovery](https://www.elastic.co/guide/en/elasticsearch/reference/current/bootstrap-checks.html#single-node-discovery) type (`discovery.type: single-node`) from using a dedicated implementation to using the existing default (Zen2) discovery implementation. With this, only a single implementation is now used across the codebase, which fixes a few problems with the previous dedicated implementation that was ignoring some of the node settings (e.g. became master even though `node.master` was set to false). #### Geo We [added optimization logic](https://github.com/elastic/elasticsearch/pull/40595) to convert `ST_Distance` function calls into a `geo_distance` query when it is called in `WHERE` clauses. We [are also improving](https://github.com/elastic/elasticsearch/pull/40708) handling of nulls in geo functions/types. We [are refactoring](https://github.com/elastic/elasticsearch/pull/40869) `GeoHashUtils`, both as a general cleanup of the utility functions and to facilitate [whitelisting geo functions](https://github.com/elastic/elasticsearch/pull/40180) for use in Painless. #### Rollups and timezones We [removed timezone validation](https://github.com/elastic/elasticsearch/pull/40647) on range queries in rollup searches. Rollup is pretty aggressive about validating search requests to make sure they match the job, but in this case the validation was unnecessary and could block valid requests. Also related to timezones, we [fixed a bug](https://github.com/elastic/elasticsearch/pull/40844) in how Rollup rewrites `date_histo` aggregations. We had been validating the timezone of the incoming `date_histo`, but had forgotten to set the valid timezone on the newly rewritten rollup `date_histo`. #### SQL We [introduced](https://github.com/elastic/elasticsearch/issues/38174) the `TIME` data type. This type conforms to ANSI SQL and is compatible with all date-related functionalities like date-time functions and maths, grouping, and histograms. ### Lucene #### Luke to become a Lucene module? After years of discussion, it looks like [Luke may become a Lucene module](https://issues.apache.org/jira/browse/LUCENE-2562). If you are not familiar with Luke, it is a tool that allows to introspect the content of Lucene indices. #### Towards a more compact terms dictionary? A new postings format [was proposed](https://issues.apache.org/jira/browse/LUCENE-8753) whose terms dictionary seems to have a smaller disk footprint and perform faster when looking up keys that exist. However, it is not able to skip going to disk for keys that do not exist in the dictionary. #### Other - We made `nested` queries faster [when hit counts are not needed](https://issues.apache.org/jira/browse/LUCENE-8701). - We are making `Weight#matches` [less costly](https://issues.apache.org/jira/browse/LUCENE-8751) by leveraging the `scorerSupplier API`. - We made `WordDelimiterGrapFilter` [emit the original token first](https://issues.apache.org/jira/browse/LUCENE-8730). - We are [refactoring](https://issues.apache.org/jira/browse/LUCENE-8746) `EdgeTree` to make is simpler and more general. ### Changes #### Changes in Elasticsearch Changes in 8.0: * BREAKING: Make remote cluster resolution stricter [#40419](https://github.com/elastic/elasticsearch/pull/40419) * BREAKING: Remove obsolete security settings [#40496](https://github.com/elastic/elasticsearch/pull/40496) Changes in 7.1: * Rest-High-Level-Client:fix uri encode bug when url path start with '/' [#34436](https://github.com/elastic/elasticsearch/pull/34436) * HLRC: Convert xpack methods to client side objects [#40705](https://github.com/elastic/elasticsearch/pull/40705) * Add an OpenID Connect authentication realm [#40674](https://github.com/elastic/elasticsearch/pull/40674) * Replace javax activation with jakarta activation [#40247](https://github.com/elastic/elasticsearch/pull/40247) * SQL: Implement CURRENT_TIME/CURTIME functions [#40662](https://github.com/elastic/elasticsearch/pull/40662) * Deprecate versions of Java prior to Java 11 [#40756](https://github.com/elastic/elasticsearch/pull/40756) * Add Bulk Delete Api to BlobStore [#40322](https://github.com/elastic/elasticsearch/pull/40322) * Don't mark shard as refreshPending on stats fetching [#40458](https://github.com/elastic/elasticsearch/pull/40458) * BREAKING: Drop support for Ubuntu 14.04 [#40709](https://github.com/elastic/elasticsearch/pull/40709) * Show SSL usage when security is not disabled [#40672](https://github.com/elastic/elasticsearch/pull/40672) * Support roles with application privileges against wildcard applications [#40398](https://github.com/elastic/elasticsearch/pull/40398) * SQL: Introduce SQL TIME data type [#39802](https://github.com/elastic/elasticsearch/pull/39802) * Reindex from Remote allow date math [#40303](https://github.com/elastic/elasticsearch/pull/40303) * Fix merging of text field mappers [#40627](https://github.com/elastic/elasticsearch/pull/40627) * SQL: Fix precedence of `::` psql like CAST operator [#40665](https://github.com/elastic/elasticsearch/pull/40665) Changes in 7.0: * Rollup ignores time_zone on date histogram [#40844](https://github.com/elastic/elasticsearch/pull/40844) * Don't replace forbidden pattern failures when found [#40710](https://github.com/elastic/elasticsearch/pull/40710) * Help Eclipse infering lambda parameter types [#40747](https://github.com/elastic/elasticsearch/pull/40747) * Do not perform cleanup if Manifest write fails with dirty exception [#40519](https://github.com/elastic/elasticsearch/pull/40519) Changes in 6.7: * Allow ILM to stop if indices have nonexistent policies [#40820](https://github.com/elastic/elasticsearch/pull/40820) * Resync should not send operations without sequence number [#40433](https://github.com/elastic/elasticsearch/pull/40433) * Avoid background sync on relocated primary [#40800](https://github.com/elastic/elasticsearch/pull/40800) * Remove timezone validation on rollup range queries [#40647](https://github.com/elastic/elasticsearch/pull/40647) * Fix alias resolution runtime complexity. [#40263](https://github.com/elastic/elasticsearch/pull/40263) * Bat scripts to work with JAVA_HOME with parantheses [#39712](https://github.com/elastic/elasticsearch/pull/39712) * Use TAR instead of DOCKER build type before 6.7.0 [#40723](https://github.com/elastic/elasticsearch/pull/40723) * Allow single digit milliseconds in strict date parsing [#40676](https://github.com/elastic/elasticsearch/pull/40676) * SQL: have LIKE/RLIKE use wildcard and regexp queries [#40628](https://github.com/elastic/elasticsearch/pull/40628) * Correct bug in ScriptDocValues [#40488](https://github.com/elastic/elasticsearch/pull/40488) * Remove dynamic objects from security index [#40499](https://github.com/elastic/elasticsearch/pull/40499) Changes in 6.6: * SQL: Fix display size for DATE/DATETIME [#40669](https://github.com/elastic/elasticsearch/pull/40669) #### Changes in Elasticsearch Management UI Changes in 7.1: * Internationalize Rollup Jobs, Remote Clusters, and Watcher app titles. [#34470](https://github.com/elastic/kibana/pull/34470) * Add collector to ui_metric app so that it reports its own telemetry. [#34210](https://github.com/elastic/kibana/pull/34210) * Fix and reenable Rollup JobList client integration tests. [#34215](https://github.com/elastic/kibana/pull/34215) Changes in 7.0: * filter only on name in index management [#34372](https://github.com/elastic/kibana/pull/34372) * [Search Profiler] Quick style fix up including dark theme [#33445](https://github.com/elastic/kibana/pull/33445) Changes in 6.7: * [ILM] Remove "Beta" badge [#34454](https://github.com/elastic/kibana/pull/34454) * fixing crash when zero replicas configured in a phase for policy [#34480](https://github.com/elastic/kibana/pull/34480) #### Changes in Rally Changes in 1.1.0: * [Experimental] Capture peak usage of JVM mem pools [#674](https://github.com/elastic/rally/pull/674) * Chart generator segment memory and new track combinations [#668](https://github.com/elastic/rally/pull/668) * Honor runtime JDK in provisioner [#673](https://github.com/elastic/rally/pull/673) * Add ability to download from S3 buckets [#671](https://github.com/elastic/rally/pull/671)

    https://www.elastic.co/blog/this-week-in-elasticsearch-and-apache-lucene-2019-04-05
Log in to reply
 

© Lightnetics 2024