0% found this document useful (0 votes)
151 views10 pages

Bin/Solr Logs/Archived Slowquerythresholdmillis Solr - Slow - Requests - Log Solr - Log

- This document discusses several changes between different versions of Solr, including changes in Solr 7.3, 7.2, 7.1, and 7.0. Some key changes mentioned include new configset naming conventions, changes to autoscaling behavior, and limitations placed on local parameters. It also provides warnings about changes that may require configuration updates or cause incompatibilities when upgrading to a new Solr version.
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as DOCX, PDF, TXT or read online on Scribd
0% found this document useful (0 votes)
151 views10 pages

Bin/Solr Logs/Archived Slowquerythresholdmillis Solr - Slow - Requests - Log Solr - Log

- This document discusses several changes between different versions of Solr, including changes in Solr 7.3, 7.2, 7.1, and 7.0. Some key changes mentioned include new configset naming conventions, changes to autoscaling behavior, and limitations placed on local parameters. It also provides warnings about changes that may require configuration updates or cause incompatibilities when upgrading to a new Solr version.
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as DOCX, PDF, TXT or read online on Scribd
You are on page 1/ 10

files. This is a server side change only and clients using SolrJ won’t need any changes.

Clients can still use


any logging implementation which is compatible with SLF4J. We now let Log4j handle rotation
of Solr logs
at startup, and bin/solr start scripts will no longer attempt this nor move existing console
or garbage
collection logs into logs/archived either. See Configuring Logging for more details about
Solr logging.
• Configuring slowQueryThresholdMillis now logs slow requests to a separate file named
solr_slow_requests.log. Previously they would get logged in the solr.log file.
Legacy Scaling (non-SolrCloud)
• In the master-slave model of scaling Solr, a slave no longer commits an empty index when a
completely
new index is detected on master during replication. To return to the previous behavior pass
false to
skipCommitOnMasterVersionZero in the slave section of replication handler configuration,
or pass it to
the fetchindex command.
If you are upgrading from a version earlier than Solr 7.3, please see previous version notes
below.
Solr 7.3
See the 7.3 Release Notes for an overview of the main new features in Solr 7.3.
When upgrading to Solr 7.3, users should be aware of the following major changes from v7.2:
ConfigSets
• Collections created without specifying a configset name have used a copy of the _default
configset
since Solr 7.0. Before 7.3, the copied configset was named the same as the collection name,
but from 7.3
onwards it will be named with a new ".AUTOCREATED" suffix. This is to prevent overwriting
custom
configset names.
Learning to Rank
• The rq parameter used with Learning to Rank rerank query parsing no longer considers the
defType
parameter. See Running a Rerank Query for more information about this parameter.
Autoscaling & AutoAddReplicas
• The behaviour of the autoscaling system will now pause all triggers from execution between
the start of
actions and the end of a cool down period. The triggers will resume after the cool down
period expires.
Previously, the cool down period was a fixed period started after actions for a trigger event
completed
and during this time all triggers continued to run but any events were rejected and tried
later.
• The throttling mechanism used to limit the rate of autoscaling events processed has been
removed. This
deprecates the actionThrottlePeriodSeconds setting in the set-properties Autoscaling API
which is
Apache Solr Reference Guide 7.7 Page 101 of 1426
c 2019, Apache Software Foundation Guide Version 7.7 - Published: 2019-03-04
now non-operational. Use the triggerCooldownPeriodSeconds parameter instead to pause event
processing.
• The default value of autoReplicaFailoverWaitAfterExpiration, used with the
AutoAddReplicas
feature, has increased to 120 seconds from the previous default of 30 seconds. This affects
how soon Solr
adds new replicas to replace the replicas on nodes which have either crashed or shutdown.
Logging
• The default Solr log file size and number of backups have been raised to 32MB and 10
respectively. See
the section Configuring Logging for more information about how to configure logging.
SolrCloud
• The old Leader-In-Recovery implementation (implemented in Solr 4.9) is now deprecated and
replaced.
Solr will support rolling upgrades from old 7.x versions of Solr to future 7.x releases until
the last release
of the 7.x major version.
This means to upgrade to Solr 8 in the future, you will need to be on Solr 7.3 or higher.
• Replicas which are not up-to-date are no longer allowed to become leader. Use the
FORCELEADER
command of the Collections API to allow these replicas become leader.
Spatial
• If you are using the spatial JTS library with Solr, you must upgrade to 1.15.0. This new
version of JTS is
now dual-licensed to include a BSD style license. See the section on Spatial Search for more
information.
Highlighting
• The top-level <highlighting> element in solrconfig.xml is now officially deprecated in
favour of the
equivalent <searchComponent> syntax. This element has been out of use in default Solr
installations for
several releases already.
If you are upgrading from a version earlier than Solr 7.2, please see previous version notes
below.
Solr 7.2
See the 7.2 Release Notes for an overview of the main new features in Solr 7.2.
When upgrading to Solr 7.2, users should be aware of the following major changes from v7.1:
Local Parameters
• Starting a query string with local parameters {!myparser …} is used to switch from one
query parser to
another, and is intended for use by Solr system developers, not end users doing searches. To
reduce
negative side-effects of unintended hack-ability, Solr now limits the cases when local
parameters will be
parsed to only contexts in which the default parser is "lucene" or "func".
So, if defType=edismax then q={!myparser …} won’t work. In that example, put the desired
query parser
into the defType parameter.
Another example is if deftype=edismax then hl.q={!myparser …} won’t work for the same
reason. In
this example, either put the desired query parser into the hl.qparser parameter or set
Page 102 of 1426 Apache Solr Reference Guide 7.7
Guide Version 7.7 - Published: 2019-03-04 c 2019, Apache Software Foundation
hl.qparser=lucene. Most users won’t run into these cases but some will need to change.
If you must have full backwards compatibility, use luceneMatchVersion=7.1.0 or an earlier
version.
eDisMax Parser
• The eDisMax parser by default no longer allows subqueries that specify a Solr parser using
either local
parameters, or the older _query_ magic field trick.
For example, {!prefix f=myfield v=enterp} or _query_:"{!prefix f=myfield v=enterp}"
are not
supported by default any longer. If you want to allow power-users to do this, set uf=* query
or some
other value that includes _query_.
If you need full backwards compatibility for the time being, use luceneMatchVersion=7.1.0
or
something earlier.
If you are upgrading from a version earlier than Solr 7.1, please see previous version notes
below.
Solr 7.1
See the 7.1 Release Notes for an overview of the main new features of Solr 7.1.
When upgrading to Solr 7.1, users should be aware of the following major changes from v7.0:
AutoAddReplicas
• The feature to automatically add replicas if a replica goes down, previously available only
when storing
indexes in HDFS, has been ported to the autoscaling framework. Due to this, autoAddReplicas
is now
available to all users even if their indexes are on local disks.
Existing users of this feature should not have to change anything. However, they should note
these
changes:
◦ Behavior: Changing the autoAddReplicas property from disabled (false) to enabled (true)
using
MODIFYCOLLECTION API no longer replaces down replicas for the collection immediately.
Instead,
replicas are only added if a node containing them went down while autoAddReplicas was
enabled.
The parameters autoReplicaFailoverBadNodeExpiration and
autoReplicaFailoverWorkLoopDelay
are no longer used.
◦ Deprecations: Enabling/disabling autoAddReplicas cluster-wide with the API will be
deprecated; use
suspend/resume trigger APIs with name=".auto_add_replicas" instead.
More information about the changes to this feature can be found in the section SolrCloud
Automatically Adding Replicas.
Metrics Reporters
• Shard and cluster metric reporter configuration now require a class attribute.
◦ If a reporter configures the group="shard" attribute then please also configure the
class="org.apache.solr.metrics.reporters.solr.SolrShardReporter" attribute.
◦ If a reporter configures the group="cluster" attribute then please also configure the
Apache Solr Reference Guide 7.7 Page 103 of 1426
c 2019, Apache Software Foundation Guide Version 7.7 - Published: 2019-03-04
class="org.apache.solr.metrics.reporters.solr.SolrClusterReporter" attribute.
See the section Shard and Cluster Reporters for more information.
Streaming Expressions
• All Stream Evaluators in solrj.io.eval have been refactored to have a simpler and more
robust
structure. This simplifies and condenses the code required to implement a new Evaluator and
makes it
much easier for evaluators to handle differing data types (primitives, objects, arrays,
lists, and so forth).
ReplicationHandler
• In the ReplicationHandler, the master.commitReserveDuration sub-element is deprecated.
Instead
please configure a direct commitReserveDuration element for use in all modes (master,
slave, cloud).
RunExecutableListener
• The RunExecutableListener was removed for security reasons. If you want to listen to
events caused by
updates, commits, or optimize, write your own listener as native Java class as part of a Solr
plugin.
XML Query Parser
• In the XML query parser (defType=xmlparser or {!xmlparser … }) the resolving of
external entities is
now disallowed by default.
If you are upgrading from a version earlier than Solr 7.0, please see Major Changes in Solr 7
before starting
your upgrade.
Upgrading to 7.x from Any 6.x Release
The upgrade from Solr 6.x to Solr 7.0 introduces several major changes that you should be
aware of before
upgrading. Please do a thorough review of the section Major Changes in Solr 7 before starting
your
upgrade.
Upgrading to 7.x from pre-6.x Versions of Solr
Users upgrading from versions of Solr prior to 6.x are strongly encouraged to consult
CHANGES.txt for the
details of all changes since the version they are upgrading from.
A summary of the significant changes between Solr 5.x and Solr 6.0 is in the section Major
Changes from Solr
5 to Solr 6.
Major Changes in Solr 7
Solr 7 is a major new release of Solr which introduces new features and a number of other
changes that may
impact your existing installation.
Upgrade Planning
There are major changes in Solr 7 to consider before starting to migrate your configurations
and indexes.
This page is designed to highlight the biggest changes - new features you may want to be
aware of, but also
Page 104 of 1426 Apache Solr Reference Guide 7.7
Guide Version 7.7 - Published: 2019-03-04 c 2019, Apache Software Foundation
changes in default behavior and deprecated features that have been removed.
There are many hundreds of changes in Solr 7, however, so a thorough review of the Solr
Upgrade Notes as
well as the CHANGES.txt file in your Solr instance will help you plan your migration to Solr
7. This section
attempts to highlight some of the major changes you should be aware of.
You should also consider all changes that have been made to Solr in any version you have not
upgraded to
already. For example, if you are currently using Solr 6.2, you should review changes made in
all subsequent
6.x releases in addition to changes for 7.0.
Re-indexing your data is considered the best practice and you should try to do so if
possible. However, if reindexing
is not feasible, keep in mind you can only upgrade one major version at a time. Thus, Solr
6.x
indexes will be compatible with Solr 7 but Solr 5.x indexes will not be.
If you do not re-index now, keep in mind that you will need to either re-index your data or
upgrade your
indexes before you will be able to move to Solr 8 when it is released in the future. See the
section
IndexUpgrader Tool for more details on how to upgrade your indexes.
See also the section Upgrading a Solr Cluster for details on how to upgrade a SolrCloud
cluster.
New Features & Enhancements
Replication Modes
Until Solr 7, the SolrCloud model for replicas has been to allow any replica to become a
leader when a leader
is lost. This is highly effective for most users, providing reliable failover in case of
issues in the cluster.
However, it comes at a cost in large clusters because all replicas must be in sync at all
times.
To provide additional flexibility, two new types of replicas have been added, named TLOG &
PULL. These new
types provide options to have replicas which only sync with the leader by copying index
segments from the
leader. The TLOG type has an additional benefit of maintaining a transaction log (the "tlog"
of its name),
which would allow it to recover and become a leader if necessary; the PULL type does not
maintain a
transaction log, so cannot become a leader.
As part of this change, the traditional type of replica is now named NRT. If you do not
explicitly define a
number of TLOG or PULL replicas, Solr defaults to creating NRT replicas. If this model is
working for you, you
will not have to change anything.
See the section Types of Replicas for more details on the new replica modes, and how define
the replica type
in your cluster.
Autoscaling
Solr autoscaling is a new suite of features in Solr to make managing a SolrCloud cluster
easier and more
automated.
At its core, Solr autoscaling provides users with a rule syntax to define preferences and
policies for how to
distribute nodes and shards in a cluster, with the goal of maintaining a balance in the
cluster. As of Solr 7,
Solr will take any policy or preference rules into account when determining where to place
new shards and
replicas created or moved with various Collections API commands.
See the section SolrCloud Autoscaling for details on the options available in 7.0. Expect
more features to be
Apache Solr Reference Guide 7.7 Page 105 of 1426
c 2019, Apache Software Foundation Guide Version 7.7 - Published: 2019-03-04
released in subsequent 7.x releases in this area.
Other Features & Enhancements
• The Analytics Component has been refactored.
◦ The documentation for this component is in progress; until it is available, please refer to
SOLR-11144
for more details.
• There were several other new features released in earlier 6.x releases, which you may have
missed:
◦ Learning to Rank
◦ Unified Highlighter
◦ Metrics API. See also information about related deprecations in the section JMX Support and
MBeans
below.
◦ Payload queries
◦ Streaming Evaluators
◦ /v2 API
◦ Graph streaming expressions
Configuration and Default Changes
New Default ConfigSet
Several changes have been made to configSets that ship with Solr; not only their content but
how Solr
behaves in regard to them:
• The data_driven_configset and basic_configset have been removed, and replaced by the
_default
configset. The sample_techproducts_configset also remains, and is designed for use with
the example
documents shipped with Solr in the example/exampledocs directory.
• When creating a new collection, if you do not specify a configSet, the _default will be
used.
◦ If you use SolrCloud, the _default configSet will be automatically uploaded to ZooKeeper.
◦ If you use standalone mode, the instanceDir will be created automatically, using the
_default
configSet as it’s basis.
Schemaless Improvements
To improve the functionality of Schemaless Mode, Solr now behaves differently when it detects
that data in
an incoming field should have a text-based field type.
• Incoming fields will be indexed as text_general by default (you can change this). The name
of the field
will be the same as the field name defined in the document.
• A copy field rule will be inserted into your schema to copy the new text_general field to
a new field with
the name <name>_str. This field’s type will be a strings field (to allow for multiple
values). The first 256
characters of the text field will be inserted to the new strings field.
This behavior can be customized if you wish to remove the copy field rule, or to change the
number of
characters inserted to the string field, or the field type used. See the section Schemaless
Mode for details.
Page 106 of 1426 Apache Solr Reference Guide 7.7
Guide Version 7.7 - Published: 2019-03-04 c 2019, Apache Software Foundation

Because copy field rules can slow indexing and increase index size, it’s recommended you
only use copy fields when you need to. If you do not need to sort or facet on a field, you
should remove the automatically-generated copy field rule.
Automatic field creation can be disabled with the update.autoCreateFields property. To do
this, you can
use the Config API with a command such as:
V1 API
curl http://host:8983/solr/mycollection/config -d '{"set-user-property":
{"update.autoCreateFields":"false"}}'
V2 API
curl http://host:8983/api/collections/mycollection/config -d '{"set-user-property":
{"update.autoCreateFields":"false"}}'
Changes to Default Behaviors
• JSON is now the default response format. If you rely on XML responses, you must now define
wt=xml in
your request. In addition, line indentation is enabled by default ( indent=on).
• The sow parameter (short for "Split on Whitespace") now defaults to false, which allows
support for
multi-word synonyms out of the box. This parameter is used with the eDismax and
standard/"lucene"
query parsers. If this parameter is not explicitly specified as true, query text will not be
split on
whitespace before analysis.
• The legacyCloud parameter now defaults to false. If an entry for a replica does not exist
in state.json,
that replica will not get registered.
This may affect users who bring up replicas and they are automatically registered as a part
of a shard. It
is possible to fall back to the old behavior by setting the property legacyCloud=true, in
the cluster
properties using the following command:
./server/scripts/cloud-scripts/zkcli.sh -zkhost 127.0.0.1:2181 -cmd clusterprop -name
legacyCloud -val true
• The eDismax query parser parameter lowercaseOperators now defaults to false if the
luceneMatchVersion in solrconfig.xml is 7.0.0 or above. Behavior for luceneMatchVersion
lower than
7.0.0 is unchanged (so, true). This means that clients must sent boolean operators (such as
AND, OR and
NOT) in upper case in order to be recognized, or you must explicitly set this parameter to
true.
• The handleSelect parameter in solrconfig.xml now defaults to false if the
luceneMatchVersion is
7.0.0 or above. This causes Solr to ignore the qt parameter if it is present in a request. If
you have
request handlers without a leading '/', you can set handleSelect="true" or consider
migrating your
configuration.
The qt parameter is still used as a SolrJ special parameter that specifies the request
handler (tail URL
Apache Solr Reference Guide 7.7 Page 107 of 1426
c 2019, Apache Software Foundation Guide Version 7.7 - Published: 2019-03-04
path) to use.
• The lucenePlusSort query parser (aka the "Old Lucene Query Parser") has been deprecated and
is no
longer implicitly defined. If you wish to continue using this parser until Solr 8 (when it
will be removed),
you must register it in your solrconfig.xml, as in: <queryParser name="lucenePlusSort"
class="solr.OldLuceneQParserPlugin"/>.
• The name of TemplateUpdateRequestProcessorFactory is changed to template from Template
and the
name of AtomicUpdateProcessorFactory is changed to atomic from Atomic
◦ Also, TemplateUpdateRequestProcessorFactory now uses {} instead of ${} for template.
Deprecations and Removed Features
Point Fields Are Default Numeric Types
Solr has implemented *PointField types across the board, to replace Trie* based numeric
fields. All Trie*
fields are now considered deprecated, and will be removed in Solr 8.
If you are using Trie* fields in your schema, you should consider moving to PointFields as
soon as feasible.
Changing to the new PointField types will require you to re-index your data.
Spatial Fields
The following spatial-related fields have been deprecated:
• LatLonType
• GeoHashField
• SpatialVectorFieldType
• SpatialTermQueryPrefixTreeFieldType
Choose one of these field types instead:
• LatLonPointSpatialField
• SpatialRecursivePrefixTreeField
• RptWithGeometrySpatialField
See the section Spatial Search for more information.
JMX Support and MBeans
• The <jmx> element in solrconfig.xml has been removed in favor of <metrics><reporter>
elements
defined in solr.xml.
Limited back-compatibility is offered by automatically adding a default instance of
SolrJmxReporter if
it’s missing AND when a local MBean server is found. A local MBean server can be activated
either via
ENABLE_REMOTE_JMX_OPTS in solr.in.sh or via system properties, e.g.,
-Dcom.sun.management.jmxremote. This default instance exports all Solr metrics from all
registries as
hierarchical MBeans.
This behavior can be also disabled by specifying a SolrJmxReporter configuration with a
boolean init
argument enabled set to false. For a more fine-grained control users should explicitly
specify at least
Page 108 of 1426 Apache Solr Reference Guide 7.7
Guide Version 7.7 - Published: 2019-03-04 c 2019, Apache Software Foundation
one SolrJmxReporter configuration.
See also the section The <metrics><reporters> Element, which describes how to set up Metrics
Reporters
in solr.xml. Note that back-compatibility support may be removed in Solr 8.
• MBean names and attributes now follow the hierarchical names used in metrics. This is
reflected also in
/admin/mbeans and /admin/plugins output, and can be observed in the UI Plugins tab,
because now all
these APIs get their data from the metrics API. The old (mostly flat) JMX view has been
removed.
SolrJ
The following changes were made in SolrJ.
• HttpClientInterceptorPlugin is now HttpClientBuilderPlugin and must work with a
SolrHttpClientBuilder rather than an HttpClientConfigurer.
• HttpClientUtil now allows configuring HttpClient instances via SolrHttpClientBuilder
rather than
an HttpClientConfigurer. Use of env variable SOLR_AUTHENTICATION_CLIENT_CONFIGURER no
longer
works, please use SOLR_AUTHENTICATION_CLIENT_BUILDER
• SolrClient implementations now use their own internal configuration for socket timeouts,
connect
timeouts, and allowing redirects rather than what is set as the default when building the
HttpClient
instance. Use the appropriate setters on the SolrClient instance.
• HttpSolrClient#setAllowCompression has been removed and compression must be enabled as a
constructor parameter.
• HttpSolrClient#setDefaultMaxConnectionsPerHost and
HttpSolrClient#setMaxTotalConnections
have been removed. These now default very high and can only be changed via parameter when
creating
an HttpClient instance.
Other Deprecations and Removals
• The defaultOperator parameter in the schema is no longer supported. Use the q.op
parameter instead.
This option had been deprecated for several releases. See the section Standard Query Parser
Parameters
for more information.
• The defaultSearchField parameter in the schema is no longer supported. Use the df
parameter
instead. This option had been deprecated for several releases. See the section Standard Query
Parser
Parameters for more information.
• The mergePolicy, mergeFactor and maxMergeDocs parameters have been removed and are no
longer
supported. You should define a mergePolicyFactory instead. See the section the
mergePolicyFactory for
more information.
• The PostingsSolrHighlighter has been deprecated. It’s recommended that you move to using
the
UnifiedHighlighter instead. See the section Unified Highlighter for more information about
this
highlighter.
• Index-time boosts have been removed from Lucene, and are no longer available from Solr. If
any boosts
are provided, they will be ignored by the indexing chain. As a replacement, index-time
scoring factors
should be indexed in a separate field and combined with the query score using a function
query. See the
section Function Queries for more information.
• The StandardRequestHandler is deprecated. Use SearchHandler instead.

You might also like

pFad - Phonifier reborn

Pfad - The Proxy pFad of © 2024 Garber Painting. All rights reserved.

Note: This service is not intended for secure transactions such as banking, social media, email, or purchasing. Use at your own risk. We assume no liability whatsoever for broken pages.


Alternative Proxies:

Alternative Proxy

pFad Proxy

pFad v3 Proxy

pFad v4 Proxy