Dataplot Vol 1 Vol 2

KERNEL DENSITY PLOT

Name:
KERNEL DENSITY PLOT
Type:
Graphics Command
Purpose:
Generates a kernel density plot.
Description:
The kernel density estimate, f(n), of a set of n points from a density f is defined as:

$$f_n(x) = \frac{\sum_{j=1}^{n}{K\{\frac{(x - X_j)}{h}\}}} {nh}$$

where K is the kernel function and h is the smoothing parameter or window width.

Currently, Dataplot uses a Gaussion kernel function. This downweights points smoothly as the distance from x increases. The width parameter can be set by the user (see Note: below), although Dataplot will provide a default width that should produce reasonable results for most data sets.

A kernel density plot can be considered a refinement of a histogram or frequency plot.

Syntax 1:
KERNEL DENSITY PLOT <y>             <SUBSET/EXCEPT/FOR qualification>
where <y> is the variable of raw data values;
and where the <SUBSET/EXCEPT/FOR qualification> is optional.

Note that <y> can be either a variable or a matrix. If <y> is a matrix, a kernel density plot will be generated for all values in the matrix.

Syntax 2:
MULTIPLE KERNEL DENSITY PLOT <y1> ... <yk>
<SUBSET/EXCEPT/FOR qualification>
where <y1> ... <yk> is a list of response variables;
and where the <SUBSET/EXCEPT/FOR qualification> is optional.

This syntax will overlay multiple kernel density plots on the same plot.

Note that the response variables (<y1> ... <yk> can be either variables or matrices (or a mix of variables and matrices). For matrices, a kernel density plot will be generated for all values in the matrix.

Syntax 3:
REPLICATED KERNEL DENSITY PLOT <y> <x1>
<SUBSET/EXCEPT/FOR qualification>
where <y> is the variable of raw data values;
<x1> is a group-id variable;
and where the <SUBSET/EXCEPT/FOR qualification> is optional.

This syntax will generate a kernel density plot for each distinct value in the group-id variable. The kernel density plots will be generated on the same page.

Syntax 4:
REPLICATED KERNEL DENSITY PLOT <y> <x1> <x2>
<SUBSET/EXCEPT/FOR qualification>
where <y> is the variable of raw data values;
<x1> is the first group-id variable;
<x2> is the second group-id variable;
and where the <SUBSET/EXCEPT/FOR qualification> is optional.

This syntax will cross tabulate the group-id variables and generate a kernel density plot for each unique combination of values for the <x1> and <x2> group-id variables. The kernel density plots will be generated on the same page.

Examples:
KERNEL DENSITY PLOT TEMP
KERNEL DENSITY PLOT Y SUBSET TAG = 2
KERNEL DENSITY PLOT Y FOR I = 1 1 800
MULTIPLE KERNEL DENSITY PLOT Y1 Y2 Y3
REPLICATED KERNEL DENSITY PLOT Y X1 X2
Note:
Dataplot computes the kernel density estimate using Algorithm 176 from Applied Statistics (see Reference below). This code was contributed by B. W. Silverman.

This algorithm is based on the Fast Fourier Transform (FFT). The use of the FFT results in much greater computational efficiency. The article that accompanies this algorithm provides the details of how the FFT is used and provides timing estimates of this implemenation relative to an algorithm based on the definition of the kernel function.

Note:
By default, the density curve is generated with 256 points. Note that this is the number of points on the density curve, not the number of points in the raw data.

You can set the number of points for the density curve using the following command:

KERNEL DENSITY POINTS <value>

where <value> defines the number of points.

Note:
Following the recommendation of Silverman (1986), DATAPLOT uses a default width of

$$0.9 \min(s,\mbox{IQ}/1.34) n^{-1/5}$$

where s is the sample standard deviation and IQ is the sample interquartile range. Silverman provides justification for this choice. Basically, it should perform reasonably for a wide variety of distributions. Note that the optimal width depends on the underlying function, which is what we are trying to estimate.

If the underlying data is in fact normally distributed, then Silverman (1986) shows that the optimal width is

$$1.06 s n^{-1/5}$$

where n is the number of points in the raw data and s is the sample standard deviation of the raw data.

It may be worthwhile to generate the density curve using several different values for the width. Silverman also recommends trying to transform skewed data sets to be symmetric.

The width can be set with the following command:

KERNEL DENSITY WIDTH <value>
Note:
Dataplot will not generate the density curve unless the input data set contains at least 20 data points. In fact, the sample size should be larger than this for density plots to be an appropriate mehtod.
Note:
The KERNEL DENSITY PLOT command generates an estimate of the underlying density function. You can convert this to an estimate of the cumulative distribution function by integrating the density estimate. The following shows an example of doing this in Dataplot.
LET Y = NORMAL RANDOM NUMBERS FOR I = 1 1 1000
KERNEL DENSITY PLOT Y
LET YPDF = YPLOT
LET XPDF = XPLOT
LET YCDF = CUMULATIVE INTEGRAL YPDF XPDF
TITLE ESTIMATE OF UNDERLYING CUMULATIVE DISTRIBUTION
PLOT YCDF  XPDF

You can also obtain an estimate of the percent point function (inverse cdf) with the following additional commands:
LET YPPF = XCDF
LET XPPF = YCDF

Note:
Dataplot computes the density curve from

(YMINIMUM - 3*H, YMAXIMUM + 3*H)

where YMINIMUM and YMAXIMUM are the minimum and maximum values of the raw data and H is the window width.

Note:
The KERNEL DENSITY PLOT supports the TO syntax for the list of variable names. This is most useful for the MULTIPLE case.
Default:
The default window width is $$1.06 s n^{-1/5}$$ where n is the number of points in the raw data and s is the sample standard deviation. The density trace is generated with 256 points.
Synonyms:
The following are synonyms for the KERNEL DENSITY PLOT command.

KERNEL PLOT
DENSITY PLOT
DENSITY TRACE

The word MULTIPLE is optional for the MULTIPLE KERNEL DENSITY PLOT command.

Related Commands:
 HISTOGRAM = Generates a histogram. FREQUENCY PLOT = Generates a frequency plot. KERNEL DENSITY WIDTH = Sets the width factor, h, for the kernel density plot. KERNEL DENSITY POINTS = Sets the number of points generated for the kernel density plot.
Reference:
B. W. Silverman (1982), "Kernel Density Estimation using the Fast Fourier Transform," Applied Statistics, Royal Statistical Society, Vol. 33.

B. W. Silverman (1986), "Density Estimation for Statistics and Data Analysis," Chapman & Hall.

David Scott (1992), "Multivariate Density Estimation," John Wiley.

Applications:
Density Estimation
Implementation Date:
2001/8
2010/2: Support for the MULTIPLE and REPLICATION syntax
2010/2: Support for matrix arguments and the TO syntax
Program:
MULTIPLOT SCALE FACTOR 2
MULTIPLOT 2 2
MULTIPLOT CORNER COORDINATES 0 0 100 100
.
LET Y = NORMAL RANDOM NUMBERS FOR I = 1 1 1000
X3LABEL 1,000 NORMAL RANDOM NUMBERS
KERNEL DENSITY PLOT Y
.
LET Y = LOGNORMAL RANDOM NUMBERS FOR I = 1 1 1000
X3LABEL 1,000 LOGNORMAL RANDOM NUMBERS
KERNEL DENSITY PLOT Y
.
LET GAMMA = 2
LET Y = WEIBULL RANDOM NUMBERS FOR I = 1 1 1000
X3LABEL 1,000 WEIBULL RANDOM NUMBERS (GAMMA = 2)
KERNEL DENSITY PLOT Y
.
LET Y = LOGISTIC RANDOM NUMBERS FOR I = 1 1 1000
X3LABEL 1,000 LOGISTIC RANDOM NUMBERS
KERNEL DENSITY PLOT Y
END OF MULTIPLOT

NIST is an agency of the U.S. Commerce Department.

Date created: 8/14/2001
Last updated: 10/13/2015

Please email comments on this WWW page to alan.heckert@nist.gov.