MI-SOFT-02 - Software and Hardware Testing Function

Principle

The mission should establish procedures and technical methods to perform end to end testing to include negative testing (i.e., abuse cases) of the mission hardware and software as it would be in an operating state (test as you fly).

Rationale

Negative testing and analysis are necessary to validate that the system architecture and security-focused design features provide adequate resilience against a range of potential attacks. Where faulted testing is standard practice in the mission lifecycle, security cases should be added to the set of potential anomalous scenarios to test. Operational anomaly response training should include security events and exercise operational interfaces to institutional security organizations.

Related Countermeasures

ID Name Description NIST Rev 5
CM0008 Security Testing Results Penetration testing, vulnerability scanning, fuzzing, code analysis, adversarial testing, and other security assessment activities are essential for validating the security posture of space systems. However, the artifacts generated by these activities are themselves sensitive mission information requiring protection. Security testing artifacts may include final reports, raw scanner output, working notes, exploit code, proof-of-concept scripts, packet captures, RF recordings, screenshots, logs, memory or core dumps, test credentials, command sequences, telemetry mappings, architecture diagrams, test configurations, remediation records, and copies of software or data collected during testing. Security testing artifacts may reveal exploitable weaknesses, attack paths, security-control gaps, spacecraft and ground-system interfaces, command and telemetry behavior, link characteristics, safety interlocks, test accounts, and system-specific vulnerability conditions. Detailed evidence identifying an exploitable weakness, affected interface, and validated attack method may provide an adversary with much of the information needed to reproduce an attack. Security testing results shall be categorized, marked, handled, stored, transmitted, shared, retained, and disposed of according to their information classification, CUI category, proprietary restrictions, contractual requirements, and mission sensitivity. Protection shall address both unauthorized disclosure and unauthorized modification, because alteration or deletion of test results could conceal vulnerabilities, misrepresent remediation status, or undermine security decisions. Access shall be limited by role and need to know throughout the artifact lifecycle, beginning when the data is generated and continuing through authorized disposition. AC-3(11) CA-8 CA-8(1) CM-4 CP-4 IR-3 IR-3(1) IR-3(2) IR-6(2) RA-5 RA-5(11) SA-11 SA-11(3) SA-11(5) SA-4(5) SA-5
CM0020 Threat modeling Threat modeling is a structured analytical process that identifies, enumerates, and prioritizes potential threats to a system by systematically examining assets, trust boundaries, data flows, and adversary capabilities relative to the system's architecture. Applied in combination with attack surface analysis and vulnerability analysis, threat modeling produces an integrated picture of where the system is most exposed and what the consequences of successful exploitation would be. Analysis should draw on findings from similar systems, components, or services where applicable, leveraging documented threat experience from comparable missions or architectures to avoid re-learning known lessons. The outputs of threat modeling must directly inform design decisions throughout the development process, with attack surface reduction treated as a design objective rather than a post-development hardening activity: interfaces, services, protocols, and code paths that are not necessary to mission function should be eliminated or constrained before they become embedded in the architecture. Threat model artifacts should be treated as living documents, updated as the system design evolves and as new threat intelligence becomes available. CA-3 CM-4 CP-2 PL-8 PL-8(1) RA-3 SA-11 SA-11(2) SA-11(3) SA-11(6) SA-15(6) SA-15(8) SA-2 SA-3 SA-4(9) SA-8 SA-8(25) SA-8(30)
CM0022 Criticality Analysis Criticality analysis is a structured engineering process that identifies the mission functions, system components, and data flows whose compromise, degradation, or loss would most severely impact mission success, crew safety, or operational continuity. The outputs of this analysis directly drive security investment prioritization: components and functions assessed as most critical receive the most rigorous design-phase protections, supply chain scrutiny, and operational security controls, while lower-criticality elements are protected proportionately. Criticality analysis findings should inform the application of complementary security design principles, including network and functional segmentation and least-privilege access control, to isolate critical components from less-trusted system elements and reduce the consequence of compromise elsewhere in the system. Supply chain protection resources and oversight rigor should be explicitly allocated in proportion to component criticality, ensuring that the most mission-essential hardware and software receive the most intensive sourcing controls, provenance verification, and supplier oversight. Criticality analysis must be initiated early in the system design process and updated as the architecture evolves, threat intelligence changes, or operational experience reveals previously unrecognized dependencies. CM-4 CP-2 CP-2(8) PL-7 PL-8 PL-8(1) PM-11 PM-17 PM-30 PM-30(1) PM-32 RA-3 RA-3(1) RA-9 SA-11 SA-11(3) SA-15(3) SA-2 SA-3 SA-4(5) SA-4(9) SA-8 SA-8(25) SA-8(3) SA-8(30) SC-32(1) SC-7(29) SR-1 SR-2 SR-2(1) SR-3 SR-3(2) SR-3(3) SR-5(1) SR-7
CM0024 Anti-counterfeit Hardware Counterfeit electronic components represent a direct supply chain threat to space mission integrity, introducing hardware that may fail prematurely, perform outside specification, or contain malicious functionality deliberately embedded by an adversary during manufacture or distribution. A formal anti-counterfeit program must establish policy and procedures that span the entire component acquisition and integration lifecycle, from supplier qualification and procurement through incoming inspection, storage, and installation. The program must address two distinct but related risks: counterfeit components that fail to perform their intended function, degrading mission reliability; and deliberately tampered components that introduce malicious hardware functionality or create pathways for malicious code execution. Anti-counterfeit controls must include measures appropriate to component criticality and supply chain risk to authenticate components, detect evidence of tampering, and resist unauthorized modification. Detection and prevention must be treated as complementary objectives: prevention through qualified sourcing and procurement controls, detection through inspection and authentication techniques applied before components enter the system. AC-14 AC-20(5) CM-7(9) PL-8 PL-8(1) PM-30 PM-30(1) RA-3(1) SA-10(3) SA-10(4) SA-11 SA-3 SA-4(5) SA-8 SA-8(11) SA-8(13) SA-8(16) SA-9 SR-1 SR-10 SR-11 SR-11(3) SR-2 SR-2(1) SR-3 SR-4 SR-4(1) SR-4(2) SR-4(3) SR-4(4) SR-5 SR-5(2) SR-6(1) SR-9 SR-9(1)
CM0025 Supplier Review A supplier review is a structured pre-contract assessment conducted before entering into any agreement with a contractor or subcontractor for the acquisition of systems, system components, or system services. The review evaluates the prospective supplier's security posture, trustworthiness, and capability to deliver components or services that meet the mission's integrity and assurance requirements, before contractual commitments are made and before the supplier gains access to mission information or influence over mission systems. Supplier reviews reduce the risk of introducing supply chain vulnerabilities through poorly qualified, compromised, or adversary-influenced suppliers at any tier of the acquisition chain. The rigor and depth of the review should be calibrated to the criticality of the components or services being acquired, with suppliers of mission-critical hardware, software, or services subject to the most intensive assessment. Supplier review findings should inform not only the decision to contract but also the specific security requirements, oversight provisions, and flow-down obligations included in the resulting agreement. PL-8 PL-8(1) PL-8(2) PM-30 PM-30(1) RA-3(1) SA-11 SA-11(3) SA-17 SA-2 SA-3 SA-8 SA-9 SR-11 SR-3(1) SR-3(3) SR-4 SR-4(1) SR-4(2) SR-4(3) SR-4(4) SR-5 SR-5(1) SR-5(2) SR-6
CM0026 Original Component Manufacturer Hardware components that cannot be sourced directly from the original component manufacturer (OCM) or an authorized franchised distributor, and software that cannot be obtained from the original publisher, developer, or an authorized distribution channel, represent elevated supply-chain risk and must not be procured or incorporated into the mission system without documented approval from the program’s supply-chain governance authority. Sourcing hardware from the OCM or authorized franchised distributors provides greater assurance of component authenticity, traceability, and conformance to specification. Obtaining software from the original publisher, developer, or an authorized distribution channel similarly reduces the risk of unauthorized, altered, fraudulent, or malicious software. Deviations from these approved sourcing channels introduce additional supply-chain risk that must be assessed before acceptance. The approval process for non-OCM-sourced items must evaluate the specific risk posed by the alternative source, the criticality of the component or software to mission function, the availability and adequacy of compensating inspection and authentication measures, and whether a compliant source can be identified before accepting the deviation. This governance requirement applies to hardware components, firmware, and software, although the applicable sourcing and authentication methods differ. Hardware controls should address component authenticity and traceability, while software controls should address publisher or developer provenance, distribution-channel integrity, license legitimacy, and cryptographic verification where available. AC-20(5) PL-8 PL-8(1) PL-8(2) PM-30 PM-30(1) RA-3(1) SA-10(4) SA-11 SA-3 SA-8 SA-9 SR-1 SR-11 SR-2 SR-2(1) SR-3 SR-3(1) SR-3(3) SR-4 SR-4(1) SR-4(2) SR-4(3) SR-4(4) SR-5 SR-5(1) SR-5(2)
CM0011 Vulnerability Scanning Vulnerability scanning systematically identifies known security weaknesses in commercial off-the-shelf (COTS) and open-source software (OSS) components, including vulnerable dependencies and outdated software versions, across spacecraft and ground system environments. Custom-developed code requires separate analysis approaches, such as static and dynamic code analysis, that fall outside the scope of this countermeasure; however, COTS, OSS, and third-party dependencies incorporated into custom-developed software remain within the scope of vulnerability scanning and SCA. Scanning programs should incorporate software composition analysis (SCA) to detect vulnerabilities introduced through third-party libraries and dependency chains, which represent a significant and frequently underestimated attack surface in modern space systems. Scanning tools and processes should conform to recognized interoperability standards that support enumeration of platforms, software flaws, and configuration weaknesses; standardized formatting of checklists and test procedures; and consistent measurement of vulnerability impact. Adherence to these standards enables automation of key vulnerability management workflow steps, facilitates tool interoperability across the mission ecosystem, and produces outputs that are comparable across programs and organizations. Vulnerability scanning shall be performed against the approved as-built or deployed software baseline using methods that do not jeopardize mission operations. Active scanning of live spacecraft, safety-critical ground systems, or operational technology should occur only after the scanning method has been authorized and evaluated for operational impact. Where live scanning presents unacceptable risk, scanning should be performed against representative software images, firmware packages, or test environments that match the approved operational baseline. CM-10(1) RA-3 RA-5 RA-5(11) RA-5(3) RA-7 SA-11 SA-11(3) SA-15(7) SA-3 SA-4(5) SA-8 SA-8(30) SI-3 SI-3(10) SI-7
CM0018 Dynamic Testing Dynamic analysis subjects executing software and firmware to active testing conditions to identify weaknesses and vulnerabilities that static analysis alone cannot detect, including runtime memory corruption, logic errors, timing vulnerabilities, and interface behaviors that only manifest under operational conditions. Applicable techniques include simulation-based testing, penetration testing, fuzz testing, and adversarial emulation using realistic threat actor tactics, techniques, and procedures (TTPs). Dynamic testing must be applied to all software and firmware categories: mission-developed code, open-source components, commercial off-the-shelf (COTS) software, and third-party developed code. Dynamic testing should be performed on potential system elements before acceptance, include realistic simulation of known adversary TTPs, and tools, and continue throughout the lifecycle on applicable physical and logical systems, elements, and processes. Full-scale hardware integration testbeds, commonly referred to as flat satellite (FLATSAT) environments, provide high-fidelity physical test platforms for dynamic analysis; digital twins implemented via instruction set simulation, also known as emulation, provide a flexible and scalable alternative environment capable of supporting TTP execution across a broad range of test scenarios without requiring physical hardware availability. CA-8 CA-8(1) CM-4(2) CP-4(5) RA-3 RA-5(11) RA-7 SA-11 SA-11(3) SA-11(5) SA-11(8) SA-11(9) SA-3 SA-8 SA-8(30) SC-2(2) SC-7(29) SI-3 SI-3(10) SI-7 SR-6(1)
CM0019 Static Analysis Static source code analysis examines software without executing it, identifying security-relevant weaknesses in the codebase before they can be exploited at runtime. Static analysis must be performed across all available source code. The mission’s weakness prioritization (i.e., CM0016) process should guide tool configuration and finding triage without unnecessarily excluding other weakness classes supported by the tools. The static analysis program must employ no fewer than two distinct tools with complementary language and weakness-detection capabilities. The tools should be applied at defined points throughout the development and maintenance lifecycle to provide overlapping and complementary analysis coverage and reduce the likelihood that exploitable weaknesses persist into integration or deployment. Tool selection should account for language compatibility, weakness class coverage, and the ability to produce findings in formats that support triage and remediation tracking. Static analysis should be integrated into the software build pipeline as an automated, recurring activity rather than conducted as a periodic manual exercise, ensuring that new code contributions are analyzed continuously throughout development. CM-4(2) RA-3 RA-5 RA-7 SA-11 SA-11(1) SA-11(3) SA-11(4) SA-15(7) SA-3 SA-8 SA-8(30) SI-7
CM0046 Long Duration Testing Long duration testing subjects spacecraft software, firmware, hardware, and relevant integrated ground interfaces, or representative simulation and emulation environments, to extended test execution of 30 days or more to expose security and reliability defects that may manifest only after prolonged operation or specific time-dependent conditions. Race conditions, memory or resource leaks, time-dependent state corruption, resource exhaustion, counter rollover, and time-triggered malicious behavior may not manifest during short-duration testing because their activation depends on accumulated runtime, rare timing interactions, or gradual changes in system state. Long duration testing increases the opportunity to expose these conditions before deployment and complements static analysis, formal analysis, stress testing, fault injection, and targeted rollover testing. Testing should use the highest-fidelity environment appropriate to the test objectives. Flight-representative hardware should be used where hardware timing, device behavior, or integration effects are material; validated simulation or emulation may be used for conditions that it represents with sufficient fidelity. Differences between the test environment and operational system must be documented and considered when interpreting results. PL-8 PL-8(1) SA-3 SA-8 SA-8(30)