Dynamic Testing

Dynamic analysis subjects executing software and firmware to active testing conditions to identify weaknesses and vulnerabilities that static analysis alone cannot detect, including runtime memory corruption, logic errors, timing vulnerabilities, and interface behaviors that only manifest under operational conditions. Applicable techniques include simulation-based testing, penetration testing, fuzz testing, and adversarial emulation using realistic threat actor tactics, techniques, and procedures (TTPs). Dynamic testing must be applied to all software and firmware categories: mission-developed code, open-source components, commercial off-the-shelf (COTS) software, and third-party developed code. Dynamic testing should be performed on potential system elements before acceptance, include realistic simulation of known adversary TTPs, and tools, and continue throughout the lifecycle on applicable physical and logical systems, elements, and processes. Full-scale hardware integration testbeds, commonly referred to as flat satellite (FLATSAT) environments, provide high-fidelity physical test platforms for dynamic analysis; digital twins implemented via instruction set simulation, also known as emulation, provide a flexible and scalable alternative environment capable of supporting TTP execution across a broad range of test scenarios without requiring physical hardware availability.

Sources

  • CCSDS A13.1-Y-1 — CCSDS Recommended Procedures for Cloud-Based Interoperability Testing
ID: CM0018
Tier: I
Ground CM 
Created: 2022/10/19
Last Modified: 2026/08/06

Pre-Operations Government

Acquisition requirements should mandate dynamic testing as a distinct, recurring program activity separate from static analysis, with requirements specifying the testing techniques to be employed, the software and firmware categories subject to testing, and the lifecycle stages at which testing must occur. Contract language should require that dynamic testing be conducted in an environment sufficiently representative of the operational hardware, software, interfaces, and mission behaviors necessary for the test objectives. Known differences between the test environment and operational system should be documented and considered when interpreting test results. Requirements should specify that at least one dynamic testing event prior to system acceptance include adversarial simulation using TTPs relevant to the mission's threat environment, with results documented and submitted to government technical authorities as a controlled deliverable. Evaluation criteria should assess offerors' proposed dynamic testing methodologies, their access to or plans to develop appropriate test environments, and their demonstrated experience conducting adversarial simulation against space or similarly constrained embedded systems. Verification and validation plans should confirm that dynamic testing findings are tracked to remediation, mitigation, or documented risk acceptance before applicable acceptance milestones are approved.

Pre-Operations Developer/Supplier

Dynamic testing infrastructure should be planned and resourced during the system design phase, as the lead time required to establish a FLATSAT testbed or develop a sufficiently accurate digital twin is substantial and cannot be treated as an afterthought in the integration and test schedule. The choice of physical testbed, hardware-in-the-loop, simulation, or emulation environment should be driven by the TTPs being exercised and the required test fidelity. Scenarios involving physical interfaces, hardware-specific behavior, or real-time timing characteristics may require a FLATSAT or hardware-in-the-loop environment, while software-focused testing may be performed in an emulated environment when it adequately represents the target system. Radiation effects require appropriate radiation testing or fault-injection methods and should not be assumed to be reproduced by a FLATSAT alone. Fuzz testing should be applied systematically to applicable externally reachable and trust-boundary interfaces, with test inputs informed by the protocols and data formats the system is expected to process. Penetration testing engagements should be scoped to include adversarial simulation exercises that replicate the TTPs of threat actors known to target space systems, with testers provided sufficient mission context to execute realistic attack chains rather than generic vulnerability exploitation. Dynamic testing findings should be entered into the vulnerability management workflow and prioritized based on exploitability, affected mission functions, existing countermeasures, and potential mission consequence.