Software Assurance Metrics and Tool Evaluation (SAMATE)
(http://samate.nist.gov)
Paul Black (paul.black@nist.gov)
Overview:
The Software Assurance Metrics and Tool
Evaluation project is dedicated to improving software assurance
tools. In the near term, it focuses on tool evaluations,
classifications, and standard reference datasets for software
security. This project supports the Department of Homeland Security's
Software Assurance Tools and R&D Program - in particular, Technology (Tools and R&D). Its objective is the
identification, enhancement and development of software assurance
tools.
Industry
Need: Much of the world's societies
depend on vastly spread software-run systems. There are tremendous
risks of computer malfeasance, from spam to identity theft to
electronic warfare. Even without intentional disruption, "the
national annual cost of an inadequate infrastructure for software
testing is estimated to range from $22.2 to $59.5 billion." Tools
implementing software assurance metrics and checks can help software
developers produce software with fewer flaws and security
vulnerabilities. Tool users can more readily reach assurance that
software will perform acceptably. Such tools can also help identify
malicious code and poor coding practices that lead to
vulnerabilities. From requirements capture through design and
acceptance to operation monitoring, we can improve results using
validated metrics and well-characterized tools.
NIST
Approach: We are holding a series of workshops to
involve pertinent parties build on existing work and coordinate future improvements. Working
with those from industry, government, and academia, we are leading
efforts to:
- develop a common taxonomy of software flaws and security
vulnerabilities,
- develop a classification of software assurance functions,
techniques, and methods,
- assemble standard reference datasets of programs with known
flaws,
- support tool evaluation with
- detailed requirements,
- testable specifications,
- test plans, and
- auxiliary scripts and programs,
- identify gaps in function and outline research agendas, and
- plan and execute studies to improve and create metrics.
Impact:
This project has catalyzed an effort involving
government agencies, such as NSA and NIST's National
Vulnerability Database (NVD), and industry, such as MITRE, Ounce
Labs, Secure Software, Fortify, Cigital, Klocwork, Symantec, and OWASP, to
develop a common software flaw taxonomy. Supporting tool testing based on rigorous and well-defined
procedures will help vendors improve their tools and provide
assurance to users of the benefits of the tools. NIST will provide
unbiased, open, and objective means for developers, researchers, and
users to assess the validity of tools and techniques used for
software assurance.
|
|
