Scribd
Upload
49 views3 pages

What Is The Curse of Dimensionality?

The curse of dimensionality refers to phenomena that occur when analyzing high dimensional data that does not occur in low dimensional spaces. Specifically, it is the issue of data sparsity and the "closeness" of data points. As data moves to higher dimensions, the volume of space grows exponentially while the data cannot keep up, becoming sparse. Additionally, data points that seem close in lower dimensions may appear further apart in higher dimensions, making classification and analysis more difficult. To address this issue, neural networks are given a finite number of features, but an infinite number of features would require an infinite number of training examples. The number of training examples needed increases exponentially with each additional feature. Careful selection of the number of features is needed based

Uploaded by

Niranjan Kumar Das
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
49 views3 pages

What Is The Curse of Dimensionality?

The curse of dimensionality refers to phenomena that occur when analyzing high dimensional data that does not occur in low dimensional spaces. Specifically, it is the issue of data sparsity and the "closeness" of data points. As data moves to higher dimensions, the volume of space grows exponentially while the data cannot keep up, becoming sparse. Additionally, data points that seem close in lower dimensions may appear further apart in higher dimensions, making classification and analysis more difficult. To address this issue, neural networks are given a finite number of features, but an infinite number of features would require an infinite number of training examples. The number of training examples needed increases exponentially with each additional feature. Careful selection of the number of features is needed based

Uploaded by

Niranjan Kumar Das
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 3

RESEARCHERS DEVELOPERS PRODUCTS ⋯ LOGIN

Curse of Dimensionality

‹ Cumulative Distributio… Data Science ›

756 share edit

The world's most comprehensive


What is the curse of data science & arti cial intelligence
dimensionality? glossary

The curse of dimensionality refers to


the phenomena that occur when
classifying, organizing, and analyzing VIEW GLOSSARY
high dimensional data that does not
occur in low dimensional spaces,
speci cally the issue of data sparsity
and “closeness” of data.

Get the week's most


Issues popular data science
research in your inbox -
Sparsity of data occurs when moving to every Saturday
higher dimensions. the volume of the
space represented grows so quickly Email
that the data cannot keep up and thus
becomes sparse, as seen below.  The
sparsity issue is a major one for anyone SUBSCRIBE

whose goal has some statistical


signi cance.

Research that mentions


Curse of Dimensionality
When Do Neural Networks Outperform Kernel …
06/24/2020 ∙ by Behrooz Ghorbani ∙ 93

As the data space seen above moves Can Shallow Neural Networks Beat the Curse of …

from one dimension to two dimensions 05/21/2020 ∙ by Stephan Wojtowytsch ∙ 51

and nally to three dimensions, the


given data lls less and less of the data
space.  In order to maintain an accurate High–Dimensional Brain in a High-Dimensional …

representation of the space, the data 01/14/2020 ∙ by Alexander N. Gorban ∙ 38

for analysis grows exponentially.

The second issue that arises is related Online Reinforcement Learning of Optimal Thre…

to sorting or classifying the data.  In 12/21/2019 ∙ by Arghyadip Roy ∙ 38

low dimensional spaces, data may seem


very similar but the higher the
An Introduction to Advanced Machine Learning …
dimension the further these data
08/26/2019 ∙ by Farid Ghareh Mohammadi ∙
points may seem to be.  The two wind
32
turbines below seem very close to each
other in two dimensions but separate
when viewed in a third dimension. This Randomly Projected Additive Gaussian Processe…
is the same e ect the curse of 12/30/2019 ∙ by Ian A. Delbridge ∙ 30
dimensionality has on data.

Applications of Nature-Inspired Algorithms for …


08/22/2019 ∙ by Farid Ghareh Mohammadi ∙

29

VIEW MORE

In nite Features Requires In nite


Training

When neural networks are created they


are instantiated with a certain number
of features (dimensions). Each datum
has individual aspects each aspect
has individual aspects, each aspect
falling somewhere along each
dimension. In our fruit example we may
want one feature handling color, one
for weight, one for shape, etc. Each
feature adds information, and if we
could handle every feature possible we
could tell perfectly which fruit we are
considering. However, an in nite
number of features requires an in nite
number of training examples,
eliminating the real-world usefulness
of our network. 

Most disconcerting, the number of


training data needed increases
exponentially with each added feature.
Even if we only had 15 features each
being one ‘yes’ or ‘no’ question about
the piece fruit we are identifying, this
would require a training set on the
order of 21532,000 training sample.

Mitigating the Curse of


Dimsionality
A careful choice of the number of
dimensions (features) to be used is the
prerogative of the data scientist
training the network. In general the
smaller the size of the training set, the
fewer features she should use. She
must keep in mind that each features
increases the data set requirement
exponentially. 

You might also like

pFad - Phonifier reborn

Pfad - The Proxy pFad of © 2024 Garber Painting. All rights reserved.

Note: This service is not intended for secure transactions such as banking, social media, email, or purchasing. Use at your own risk. We assume no liability whatsoever for broken pages.


Alternative Proxies:

Alternative Proxy

pFad Proxy

pFad v3 Proxy

pFad v4 Proxy