Elastic 7.12 released: General availability of schema on read, technical preview of the frozen tier, and support for autoscaling

We are pleased to announce the general availability (GA) of Elastic 7.12. This release brings a broad set of new capabilities to our Elastic Enterprise Search, Observability, and Security solutions, which are built into the Elastic Stack — Elasticsearch and Kibana. This release enables customers to choose between unmatched flexibility and speed with schema on read, unlock new value by making object stores fully searchable with the new frozen tier, and automatically scale deployments on Elastic Cloud. Elastic Enterprise Search benefits from a number of architectural enhancements that deliver reduced deployment size, faster indexing, and more relevant results. Correlations come to Elastic Observability, helping to identify top drivers of application performance issues and errors. Analyst-driven correlation streamlines SecOps workflows in Elastic Security. Elastic 7.12 is available now on Elastic Cloud — the only hosted Elasticsearch offering to include all of the new features in this latest release. You can also download the Elastic Stack and our cloud orchestration products, Elastic Cloud Enterprise and Elastic Cloud for Kubernetes, for a self-managed experience. Read on for the key release highlights. To get the full feature rundown, dive into the individual solution and product blog posts.

Elastic Stack and Elastic CloudLet analysts flexibly explore data through schema on read, now generally available.Elasticsearch is known for being a blazing fast distributed search and analytics engine in part because of schema on write, the default schema for Elasticsearch. This organized structure of data requires planning and testing of how the data will be represented in Elasticsearch, but the big payoff is speed. What happens when you need to ingest new data or adopt a new use case with a fast turnaround? What if you had the option to create schema on the fly at query time with schema on read? Runtime fields, now generally available, give you the flexibility to define schema on read in addition to using schema on write. This feature greatly reduces the time-to-value with your data by trading off some search performance. With the 7.12 release, runtime fields are now searchable in Kibana Discover, allowing analysts to flexibly explore data structured through schema on read in Elasticsearch. Our implementation of schema on read is special. With Elasticsearch runtime fields you do not have to choose between the speed and scale of schema on write or the flexibility of schema on read. You can use both at the same time, on the same Elastic Stack, and on the same data. Explore new data and define new fields on the fly while still searching on the fields in the data you already know. Easily shift between the newly created fields, defined in runtime fields, and schema on write for speed and performance. No matter your approach, you have unparalleled flexibility with the speed and scale you come to expect of Elasticsearch. Unlock new value by making object stores like S3 fully searchable with the new frozen tier, now in technical preview, with a simplified experience in Elastic Cloud coming soon.With the new frozen tier, now in technical preview, you have the ability to decouple compute from storage, adding the capability to search directly on object storage such as Amazon S3, Google Cloud Storage, and Microsoft Azure Storage. This functionality allows you to search your data at a fraction of the cost with a tradeoff in performance while reducing the amount of dedicated resources needed for search. By fetching only the data needed to complete a query from the object store and caching this data locally as needed, the frozen tier offers the best search experience while enabling you to store an unlimited amount of data. We will also be offering an enhanced user experience for configuring the frozen tier in Elastic Cloud, coming soon. With searchable snapshots, you can cost-effectively search across all of your application content and historical workplace records without breaking the bank. Store more analytics data for marketing analysis, or test and release versioned application catalogs for new deployment strategies. In observability, you no longer need to choose which log, metric, or APM data to delete to save money. Imagine having the ability to search year over year on application performance without needing to rehydrate your data from backup. For your threat hunters and security analysts, imagine arming them with years of high-volume security data sources made easily accessible through searchable snapshots. Collect additional security-related data — IDS, NetFlow, DNS, PCAP, or endpoint data — at greater scale and keep it accessible for longer. Stay in the flow of analyzing data while your long-running queries complete on their own with a new “save search to background” feature.Searching across huge amounts of data in pursuit of the proverbial needle in a haystack is core to what Elastic’s technology helps people do. Even when those results live in data on a frozen index spanning multiple clusters, the Elastic Stack won’t stop combing through documents on your behalf until the job is done. But just because it's going the extra mile doesn’t mean you have to stop what you’re doing. Now with 7.12 you can send long-running search sessions in Discover or on a Kibana dashboard to run in the background while you keep tackling your day. The new search session management interface lets you check back on the results whenever you want, no matter if that’s 5 minutes, 5 hours, or even 5 days later. Let the Elastic Stack do the multitasking so you can stay laser focused on getting things done.

Save search to background with Kibana 7.12.Let Elastic Cloud automatically monitor storage utilization and machine learning capacity, adjust resources, and maintain performance with autoscaling so that you can focus on running your business.Autoscaling is now available for Elastic Cloud and Elastic Cloud Enterprise 2.9. Autoscaling monitors both the storage utilization for your Elasticsearch data nodes, as well as the available capacity for your machine learning jobs. Autoscaling automatically adjusts resource capacity to maintain node performance.  You can enable autoscaling using the API, CLI, or from the Elastic Cloud console. Your Elasticsearch data nodes’ capacity will grow as you store more data. Your machine learning node’s memory and CPU capacity will grow or shrink based on the resource requirements of your machine learning jobs. You can also set thresholds to prevent runaway cluster growth. With autoscaling, you can keep up with dynamic observability demands by extending instrumentation to all applications in an optimized fashion without overpaying for max capacity — it’s been one of the Elastic Observability community’s most requested features. In Elastic Enterprise Search, use autoscaling to drive greater insights into your search platform with less overhead, seamlessly scaling with your logs, analytics, and content. In Elastic Security, complete large-scale threat hunting exercises and let autoscaling monitor and grow memory resources for your machine learning nodes as they perform anomaly detection against your security event information.

https://www.elastic.co/blog/whats-new-elastic-7-12-0-schema-on-read-frozen-tier-autoscaling