SV-SP-2

Testing only focuses on functional requirements and rarely considers end to end or abuse cases


Informational References

ID: SV-SP-2
DiD Layer: Prevention
CAPEC #:  28 | 214 | 215 | 261
NIST Rev5 Control Tag Mapping:  CA-2(2) | CA-8 | CP-4 | CP-4(5) | PM-32 | RA-5 | RA-5(2) | RA-9 | SA-3 | SA-4 | SA-4(3) | SA-11 | SA-11(1) | SA-11(2) | SA-11(5) | SA-11(7) | SA-11(8) | SA-11(9) | SA-15 | SA-15(7) | SR-11 | SR-11(3)
Lowest Threat Tier to
Create Threat Event:  
II
Notional Risk Rank Score: 

High-Level Requirements

The Program shall establish robust procedures and technical methods to perform testing to include negative testing (i.e., abuse cases) of the spacecraft hardware and software.

Low-Level Requirements

Requirement Rationale/Additional Guidance/Notes
The Program shall require the developer of the system, system component, or system services to demonstrate the use of a system development life cycle that includes [state-of-the-practice system/security engineering methods, software development methods, testing/evaluation/validation techniques, and quality control processes]. {SV-SP-1,SV-SP-2,SV-SP-3,SV-SP-9} {SA-3,SA-4(3)} Select the particular subcontractors, software vendors, and manufacturers based on the criticality analysis performed for the Program Protection Plan and the criticality of the components that they supply. 
The Program shall require subcontractors developing information system components or providing information system services (as appropriate) to demonstrate the use of a system development life cycle that includes [state-of-the-practice system/security engineering methods, software development methods, testing/evaluation/validation techniques, and quality control processes]. {SV-SP-1,SV-SP-2,SV-SP-3,SV-SP-9} {SA-3,SA-4(3)}
The Program shall use the threat and vulnerability analyses of the as-built system, system components, or system services to inform and direct subsequent testing/evaluation of the as-built system, component, or service. {SV-SP-1,SV-SP-2,SV-SP-3,SV-SP-6,SV-SP-7,SV-SP-9,SV-SP-11} {SA-11(2)} The security assessment plan should include evaluation of mission objectives in relation to the security of the mission. Assessments should not only be control based but also functional based to ensure mission is resilient against failures of controls.
The Program shall create and implement a security assessment plan that includes: (1) The types of analyses, testing, evaluation, and reviews of [all] software and firmware components; (2) The degree of rigor to be applied to include abuse cases and/or penetration testing; and (3) The types of artifacts produced during those processes. {SV-SP-1,SV-SP-2,SV-SP-3,SV-SP-6,SV-SP-7,SV-SP-9,SV-SP-11} {SA-11,SA-11(5),CA-8} * The frequency of testing should be driven by Program completion events and updates. * Examples of approaches are static analyses, dynamic analyses, binary analysis, or a hybrid of the three approaches
The Program shall verify that the scope of security testing/evaluation provides complete coverage of required security controls (to include abuse cases and penetration testing) at the depth of testing defined in the test documents. {SV-SP-1,SV-SP-2,SV-SP-3,SV-SP-6,SV-SP-7,SV-SP-9,SV-SP-11} {SA-11(5),SA-11(7),CA-8} The depth needs to include functional testing as well as negative/abuse testing.
The Program shall perform [Selection (one or more): unit; integration; system; regression] testing/evaluation at [Program-defined depth and coverage]. {SV-SP-1,SV-SP-2,SV-SP-3,SV-SP-6,SV-SP-7,SV-SP-9,SV-SP-11} {SA-11} Component/Origin scanning looks for open-source libraries/software that may be included into the baseline and looks for known vulnerabilities and open-source license violations.
The Program shall ensure that vulnerability scanning tools and techniques are employed that facilitate interoperability among tools and automate parts of the vulnerability management process by using standards for: (1) Enumerating platforms, custom software flaws, and improper configurations; (2) Formatting checklists and test procedures; and (3) Measuring vulnerability impact. {SV-SP-1,SV-SP-2,SV-SP-3,SV-SP-6,SV-SP-7,SV-SP-9,SV-SP-11} {RA-5} The prioritized list of CWEs should be created considering operational environment, attack surface, etc. Results from the threat modeling and attack surface analysis should be used as inputs into the CWE prioritization process. There is also a CWSS (https://cwe.mitre.org/cwss/cwss_v1.0.1.html) process that can be used to prioritize CWEs. The prioritized list of CWEs can help with tools selection as well as you select tools based on their ability to detect certain high priority CWEs.
The Program shall create prioritized list of software weakness classes (e.g., Common Weakness Enumerations) to be used during static code analysis for prioritization of static analysis results. {SV-SP-1,SV-SP-2,SV-SP-3,SV-SP-6,SV-SP-7,SV-SP-9,SV-SP-11} {SA-11(1),SA-15(7)}
The Program shall perform static source code analysis for [all available source code] looking for [Select one {Program-defined Top CWE List, SANS Top 25, OWASP Top 10}] weaknesses using no less than two static code analysis tools. {SV-SP-1,SV-SP-2,SV-SP-3,SV-SP-6,SV-SP-7,SV-SP-9,SV-SP-11} {SA-11(1),SA-15(7),RA-5}
The Program shall perform component analysis (a.k.a. origin analysis) for developed or acquired software. {SV-SP-1,SV-SP-2,SV-SP-3,SV-SP-6,SV-SP-7,SV-SP-9,SV-SP-11} {SA-15(7),RA-5}
The Program shall analyze vulnerability/weakness scan reports and results from security control assessments. {SV-SP-1,SV-SP-2,SV-SP-3,SV-SP-6,SV-SP-7,SV-SP-9,SV-SP-11} {RA-5}
The Program shall determine the vulnerabilities/weaknesses that require remediation, and coordinate the timeline for that remediation, in accordance with the analysis of the vulnerability scan report, the Program assessment of risk, and mission needs. {SV-SP-1,SV-SP-2,SV-SP-3,SV-SP-6,SV-SP-7,SV-SP-9,SV-SP-11} {RA-5}
The Program shall ensure that the vulnerability scanning tools (e.g., static analysis and/or component analysis tools) used include the capability to readily update the list of potential information system vulnerabilities to be scanned. {SV-SP-1,SV-SP-2,SV-SP-3,SV-SP-6,SV-SP-7,SV-SP-9,SV-SP-11} {RA-5}
The Program shall ensure that the list of potential system vulnerabilities scanned is updated [prior to a new scan] {SV-SP-1,SV-SP-2,SV-SP-3,SV-SP-6,SV-SP-7,SV-SP-9,SV-SP-11} {RA-5(2)} Fuzzing and/or dynamic analysis with abuse cases is important to flush out edge cases and how malicious actors could affect the spacecraft's FSW. Not all defects (i.e., buffer overflows, race conditions, and memory leaks) can be discovered statically and require execution of the software. This is where space-centric cyber testbeds (i.e., cyber ranges) are imperative as they provide an environment to maliciously attack components in a controlled environment to discover these undesirable conditions. Technology has improved to where digital twins for spacecraft are achievable, which provides an avenue for cyber testing that was often not performed due to perceived risk to the flight hardware.
The Program shall employ dynamic analysis (e.g., using simulation, penetration testing, fuzzing, etc.) to identify software/firmware weaknesses and vulnerabilities in developed and incorporated code (open source, commercial, or third-party developed code). {SV-SP-1,SV-SP-2,SV-SP-3,SV-SP-6,SV-SP-7,SV-SP-9,SV-SP-11} {SA-11(5),SA-11(8),CA-8}

Related SPARTA Techniques and Sub-Techniques

ID Name Description
REC-0006 Gather FSW Development Information Threat actors may obtain information regarding the flight software (FSW) development environment for the victim SV. This information may include the development environment, source code, compiled binaries, testing tools, and fault management.

Related SPARTA Countermeasures

ID Name Description NIST Rev5 D3FEND ISO 27001