MI-SOFT-02 - Software and Hardware Testing Function

Principle

The mission should establish procedures and technical methods to perform end to end testing to include negative testing (i.e., abuse cases) of the mission hardware and software as it would be in an operating state (test as you fly).

Rationale

Negative testing and analysis are necessary to validate that the system architecture and security-focused design features provide adequate resilience against a range of potential attacks. Where faulted testing is standard practice in the mission lifecycle, security cases should be added to the set of potential anomalous scenarios to test. Operational anomaly response training should include security events and exercise operational interfaces to institutional security organizations.

Related Countermeasures

ID Name Description NIST Rev 5
CM0008 Security Testing Results As penetration testing and vulnerability scanning is a best practice, protecting the results from these tests and scans is equally important. These reports and results typically outline detailed vulnerabilities and how to exploit them. As with countermeasure CM0001, protecting sensitive information from disclosure to threat actors is imperative. AC-3(11) CA-8 RA-5 RA-5(11) SA-11(5) SA-5
CM0020 Threat modeling Use threat modeling and vulnerability analysis to inform the current development process using analysis from similar systems, components, or services where applicable. SA-11(2) SA-15(8)
CM0022 Criticality Analysis Conduct a criticality analysis to identify mission critical functions, critical components, and data flows and reduce the vulnerability of such functions and components through secure system design. Focus supply chain protection on the most critical components/functions. Leverage other countermeasures like segmentation and least privilege to protect the critical components. CP-2(8) PM-11 PM-17 PM-30 PM-30(1) PM-32 RA-3(1) RA-9 RA-9 SA-15(3) SC-32(1) SC-7(29) SR-1 SR-1 SR-2 SR-2(1) SR-3 SR-3(2) SR-3(3) SR-5(1) SR-7
CM0024 Anti-counterfeit Hardware Develop and implement anti-counterfeit policy and procedures designed to detect and prevent counterfeit components from entering the information system, including tamper resistance and protection against the introduction of malicious code or hardware.  AC-20(5) CM-7(9) PM-30 PM-30(1) RA-3(1) SA-10(3) SA-10(4) SR-1 SR-10 SR-11 SR-11 SR-11(3) SR-11(3) SR-2 SR-2(1) SR-3 SR-4 SR-4(1) SR-4(2) SR-4(3) SR-4(4) SR-5 SR-5(2) SR-6(1) SR-9 SR-9(1)
CM0025 Supplier Review Conduct a supplier review prior to entering into a contractual agreement with a contractor (or sub-contractor) to acquire systems, system components, or system services. PM-30 PM-30(1) RA-3(1) SR-11 SR-3(1) SR-3(3) SR-4 SR-4(1) SR-4(2) SR-4(3) SR-4(4) SR-5 SR-5(1) SR-5(2) SR-6 SR-6
CM0026 Original Component Manufacturer Components/Software that cannot be procured from the original component manufacturer or their authorized franchised distribution network should be approved by the supply chain board or equivalent to prevent and detect counterfeit and fraudulent parts, materials, and software. AC-20(5) PM-30 PM-30(1) RA-3(1) SA-10(4) SR-1 SR-1 SR-11 SR-2 SR-2(1) SR-3 SR-3(1) SR-3(3) SR-4 SR-4(1) SR-4(2) SR-4(3) SR-4(4) SR-5 SR-5 SR-5(1) SR-5(2)
CM0011 Vulnerability Scanning Vulnerability scanning is used to identify known software vulnerabilities (excluding custom-developed software - ex: COTS and Open-Source). Utilize scanning tools to identify vulnerabilities in dependencies and outdated software (i.e., software composition analysis). Ensure that vulnerability scanning tools and techniques are employed that facilitate interoperability among tools and automate parts of the vulnerability management process by using standards for: (1) Enumerating platforms, custom software flaws, and improper configurations; (2) Formatting checklists and test procedures; and (3) Measuring vulnerability impact. CM-10(1) RA-5 RA-5(11) RA-5(3) SA-15(7) SI-3
CM0018 Dynamic Analysis Employ dynamic analysis (e.g., using simulation, penetration testing, fuzzing, etc.) to identify software/firmware weaknesses and vulnerabilities in developed and incorporated code (open source, commercial, or third-party developed code). Testing should occur (1) on potential system elements before acceptance; (2) as a realistic simulation of known adversary tactics, techniques, procedures (TTPs), and tools; and (3) throughout the lifecycle on physical and logical systems, elements, and processes. FLATSATs as well as digital twins can be used to perform the dynamic analysis depending on the TTPs being executed. Digital twins via instruction set simulation (i.e., emulation) can provide robust environment for dynamic analysis and TTP execution. CA-8 CP-4(5) RA-5(11) SA-11(5) SA-11(8) SA-11(9) SC-2(2) SC-7(29) SI-3 SR-6(1) SR-6(1)
CM0019 Static Analysis Perform static source code analysis for all available source code looking for system-relevant weaknesses (see CM0016) using no less than two static code analysis tools. RA-5 SA-11(1) SA-15(7)
CM0046 Long Duration Testing Perform testing using hardware or simulation/emulation where the test executes over a long period of time (30+ days). This testing will attempt to flesh out race conditions or time-based attacks. None