|
Contact
|
Ragu
Kacker
Statistical Engineering
Division
Information Technology
Laboratory
301-975-2109
ragu.kacker@nist.gov
|
|
Impetus/How Project Began
|
To develop measurement and metrics to quantify performance of the
Internet from the Customer viewpoint, NIST/ITL joined Cross Industry
Working Team (XIWT), consisting of the leading U.S. computer and
communication companies. The initial driver was "service level
agreements." NIST became a monitoring site to collect data on
"round trip delays and packet loss" of Unix Ping Packets exchanged
among the participating organizations. The monitoring and data
collection role is subsequently transferred to CNRI (Corporation
for National Research Initiatives), the administrative organization
for XIWT. Since behavior of the Internet is statistical in nature,
SED started collaborating with XIWT jointly with ITL Advanced Network
Technologies Division. ANTD and SED jointly made a successful
proposal to DARPA. The project began with the DARPA funding starting
in March, 2000.
|
|
Objective(s)
|
Provide data analyses and statistical modeling expertise to XIWT.
Current objectives relate to delay and loss data from exchange of
Unix Ping Packets. Specifically,
- Interactive data analyses: Provide a Web based capability of
statistical software S-Plus to XIWT member companies using
StatServer. A menu of common queries will be developed. A
representative from member companies will choose items from
the menu. StatServer will sub-set the required data and
make interactive analysis and plots to address the query.
- NIST Statistical Research: We will do statistical research
on models for monitoring and anomaly detection, and effects of
statistical variations in measurement sampling.
- Collaborative Research among XIWT members: Many of the
representative to XIWT are researchers. The Web based tool
hosted by NIST will provide a common data set for all members
to test out different models and ideas and share with the
other members.
|
|
FTE
|
SED contributions total to one FTE with 20% divided among
each of the five contributors.
|
Timeline/
Milestones
|
The timelines and milestones for this project are:
- FY00, Q4:
- Purchase Statserver and PC install it. Duplicate
the CNRI data at NIST and integrate the system with
Statserver. Complete training provided by Mathsoft Inc.
- Document the prior work on this project.
- FY01,Q1:
- Learn more effective use of Statserver.
- Provide a simple menu of options for exploratory
data analyses of pinger data.
- Develop time series based and/or other models for
monitoring and/or anomaly detection.
|
|
NIST Involvement
|
The following NIST staff are involved in this project:
- Charles Hagwood (Division 898, ITL),
- Raghu Kacker (Division 898, ITL),
- Hung-kung Liu (Division 898, ITL),
- James Yen (Division 898, ITL),
- Nien-Fan Zhang (Division 898, ITL),
- Sri Kumar (Division 892, ITL),
- Kevin Mills (Division 892, ITL),
- Vladimir Marbukh (Division 892, ITL),
- Doug Montgomery (Division 892, ITL),
|
|
Outside Involvement
|
The following outside collaborators are involved:
- CNRI: Charles Brownstein;
- XIWT member companies, such as for example, AT&T, Bell Atlantic,
Bell South, Cisco, Compaq, IBM, Intel, Lucent, MCI, Sun
Microsystems, TI, USWest.
|
|
What Work Has ITL Done
|
ITL has performed the following work for this project.
- Did exploratory data analysis of four months (5/1/98 - 9/4/98)
of data from NIST to SLAC-Stanford, Westgroup and Intel. The
data consists of source, destination, time packet size,
fraction of packet loss, and summary statistics (average,
minimum, and maximum) on round trip delay. The exploratory data
analysis (EDA) indicated effect of packet size, certain
anomalies, weekly trend, correlations. A presentation to XIWT
members resulted in syncronizing of the clocks and interest in
bi-directional delays.
- Using time series analysis methodologies, we have analyzed
the ping delay time data for different destinations and with
different ping sizes. We found that the average delay time
vary with destinations as well as the size of the ping.
For given destination and ping size, extensive data analysis
shows that data in general has some dynamic structure. That
means that data can be modeled as time series, for better effect
the model needs to be updated or modified. We found that data
in general has stationary structure with long memory: this
means that random fluctuation of the data have some kind of
long-term pattern. Further, data also shows persistent periodic
behavior with a daily period for most cases. A very important
characteristic of the data is that the (marginal) probability
distribution of the delay time is not bell shaped or Gaussian.
With this distinctive feature in mind, monitoring and
predicting the delay time will definitely need tools not based
on the ordinary Gaussian assumption.
|
|
What Work Has Been Done By Collaborators
|
The collaborators are XIWT member companies. Work is coordinated by
CNRI. They have performed the following work for this project.
- The XIWT has written two white papers: Customer View of
Internet Service Performance: Measurement Methodology and
Metrics; and Internet Service Performance: Data Analysis and
Visualization. SED had suggested use of Box plots for the plots
used in the second report. That is what was done.
- The XIWT companies are continuing to monitor using exchange of
ping packets, expand the network, and improve it. The ping
data are sent by the XIWT monitoring sites to CNRI every night.
Plans are to duplicate that data at NIST to be integrated with
a Statserver, purchased from Mathsoft Inc.
|
|
What Developments Have Occurred and How Has the Project Changed
|
As part of a DARPA sponsored project, the SED work is one of the
three elements. The SED element involves providing Web based
capabilities for exploratory data analysis, doing research on
statistical modeling, and contributing to collaborative research
among XIWT member companies. This is a very dynamic and
fast moving business. The priorities in XIWT are evolving with
the interests of the participants. We see XIWT as the driver and
SED as supporting the industry initiatives.
|
|
Publications or Artifacts
|
Nothing to report at this time.
|
|
How Industries Have Benefited From NIST's Work
|
Nothing to report at this time.
|
|
Acknowledgements of the ITL's Effort
|
Nothing to report at this time.
|
|
Future Related Activities
|
Nothing to report at this time.
|
|
Additional Information
|
Additional information is available at
|
