SA-11(5) - Developer Testing and Evaluation | Penetration Testing

Require the developer of the system, system component, or system service to perform penetration testing: (a) At the following level of rigor: [Assignment: organization-defined breadth and depth of testing]; and (b) Under the following constraints: [Assignment: organization-defined constraints].


Informational References

ISO 27001

ID: SA-11(5)
Enhancement of : SA-11

Countermeasures Covered by Control

ID Name Description D3FEND
CM0008 Security Testing Results As penetration testing and vulnerability scanning is a best practice, protecting the results from these tests and scans is equally important. These reports and results typically outline detailed vulnerabilities and how to exploit them. As with countermeasure CM0001, protecting sensitive information from disclosure to threat actors is imperative. D3-AI D3-AVE
CM0004 Development Environment Security In order to secure the development environment, the first step is understanding all the devices and people who interact with it. Maintain an accurate inventory of all people and assets that touch the development environment. Ensure strong multi-factor authentication is used across the development environment, especially for code repositories, as threat actors may attempt to sneak malicious code into software that's being built without being detected. Use zero-trust access controls to the code repositories where possible. For example, ensure the main branches in repositories are protected from injecting malicious code. A secure development environment requires change management, privilege management, auditing and in-depth monitoring across the environment. D3-AI D3-AVE D3-SWI D3-HCI D3-NNI D3-OAM D3-AM D3-OM D3-DI D3-MFA D3-CH D3-OTP D3-BAN D3-PA D3- FAPA D3- DQSA D3-IBCA D3-PCSV D3-PSMD
CM0018 Dynamic Analysis Employ dynamic analysis (e.g., using simulation, penetration testing, fuzzing, etc.) to identify software/firmware weaknesses and vulnerabilities in developed and incorporated code (open source, commercial, or third-party developed code). Testing should occur (1) on potential system elements before acceptance; (2) as a realistic simulation of known adversary tactics, techniques, procedures (TTPs), and tools; and (3) throughout the lifecycle on physical and logical systems, elements, and processes. FLATSATs as well as digital twins can be used to perform the dynamic analysis depending on the TTPs being executed. Digital twins via instruction set simulation (i.e., emulation) can provide robust environment for dynamic analysis and TTP execution. D3-DA D3-FBA D3-PSA D3-PLA D3-PA D3-SEA D3-MBT

Space Threats Tagged by Control

ID Description
SV-SP-1 Exploitation of software vulnerabilities (bugs); Unsecure code, logic errors, etc. in the FSW.
SV-SP-3 Introduction of malicious software such as a virus, worm, Distributed Denial-Of-Service (DDOS) agent, keylogger, rootkit, or Trojan Horse
SV-SP-6 Software reuse, COTS dependence, and standardization of onboard systems using building block approach with addition of open-source technology leads to supply chain threat
SV-SP-9 On-orbit software updates/upgrades/patches/direct memory writes. If TT&C is compromised or MOC or even the developer's environment, the risk exists to do a variation of a supply chain attack where after it is in orbit you inject malicious code
SV-SP-11 Software defined radios - SDR is also another computer, networked to other parts of the spacecraft that could be pivoted to by an attacker and infected with malicious code. Once access to an SDR is gained, the attacker could alter what the SDR thinks is correct frequencies and settings to communicate with the ground.
SV-SP-7 Software can be broken down into three levels (operating system and drivers’ layer, data handling service layer, and the application layer). Highest impact on system is likely the embedded code at the BIOS, kernel/firmware level. Attacking the on-board operating systems. Since it manages all the programs and applications on the computer, it has a critical role in the overall security of the system. Since threats may occur deliberately or due to human error, malicious programs or persons, or existing system vulnerability mitigations must be deployed to protect the OS.
SV-AV-7 The TT&C is the lead contributor to satellite failure over the first 10 years on-orbit, around 20% of the time. The failures due to gyro are around 12% between year one and 6 on-orbit and then ramp up starting around year six and overtake the contributions of the TT&C subsystem to satellite failure. Need to ensure equipment is not counterfeit and the supply chain is sound.
SV-SP-2 Testing only focuses on functional requirements and rarely considers end to end or abuse cases
SV-SP-4 General supply chain interruption or manipulation

Sample Requirements

Requirement
The Program shall perform and document threat and vulnerability analyses of the as-built system, system components, or system services. {SV-SP-1,SV-SP-3,SV-SP-6,SV-SP-7,SV-SP-9,SV-SP-11} {SA-11(2)}
The Program shall use the threat and vulnerability analyses of the as-built system, system components, or system services to inform and direct subsequent testing/evaluation of the as-built system, component, or service. {SV-SP-1,SV-SP-2,SV-SP-3,SV-SP-6,SV-SP-7,SV-SP-9,SV-SP-11} {SA-11(2)}
The Program shall perform a manual code review of all flight code. {SV-SP-1,SV-SP-3,SV-SP-6,SV-SP-7,SV-SP-9,SV-SP-11} {SA-11(4)}
The Program shall conduct an Attack Surface Analysis and reduce attack surfaces to a level that presents a low level of compromise by an attacker. {SV-SP-1,SV-SP-6,SV-SP-7,SV-SP-9,SV-SP-11} {SA-11(6),SA-15(5)}
The Program shall use threat modeling and vulnerability analysis to inform the current development process using analysis from similar systems, components, or services where applicable. {SV-SP-1,SV-SP-6,SV-SP-7,SV-SP-9,SV-SP-11} {SA-11(2),SA-15(8)}
The Program shall create and implement a security assessment plan that includes: (1) The types of analyses, testing, evaluation, and reviews of [all] software and firmware components; (2) The degree of rigor to be applied to include abuse cases and/or penetration testing; and (3) The types of artifacts produced during those processes. {SV-SP-1,SV-SP-2,SV-SP-3,SV-SP-6,SV-SP-7,SV-SP-9,SV-SP-11} {SA-11,SA-11(5),CA-8}
The Program shall verify that the scope of security testing/evaluation provides complete coverage of required security controls (to include abuse cases and penetration testing) at the depth of testing defined in the test documents. {SV-SP-1,SV-SP-2,SV-SP-3,SV-SP-6,SV-SP-7,SV-SP-9,SV-SP-11} {SA-11(5),SA-11(7),CA-8}
The Program shall perform [Selection (one or more): unit; integration; system; regression] testing/evaluation at [Program-defined depth and coverage]. {SV-SP-1,SV-SP-2,SV-SP-3,SV-SP-6,SV-SP-7,SV-SP-9,SV-SP-11} {SA-11}
The Program shall maintain evidence of the execution of the security assessment plan and the results of the security testing/evaluation. {SV-SP-1,SV-SP-6,SV-SP-7,SV-SP-9,SV-SP-11} {SA-11,CA-8}
The Program shall implement a verifiable flaw remediation process into the developmental and operational configuration management process. {SV-SP-1,SV-SP-6,SV-SP-7,SV-SP-9,SV-SP-11} {SA-11}
The Program shall correct flaws identified during security testing/evaluation. {SV-SP-1,SV-SP-6,SV-SP-7,SV-SP-9,SV-SP-11} {SA-11}
The Program shall create prioritized list of software weakness classes (e.g., Common Weakness Enumerations) to be used during static code analysis for prioritization of static analysis results. {SV-SP-1,SV-SP-2,SV-SP-3,SV-SP-6,SV-SP-7,SV-SP-9,SV-SP-11} {SA-11(1),SA-15(7)}
The Program shall perform static source code analysis for [all available source code] looking for [Select one {Program-defined Top CWE List, SANS Top 25, OWASP Top 10}] weaknesses using no less than two static code analysis tools. {SV-SP-1,SV-SP-2,SV-SP-3,SV-SP-6,SV-SP-7,SV-SP-9,SV-SP-11} {SA-11(1),SA-15(7),RA-5}
The Program shall employ dynamic analysis (e.g., using simulation, penetration testing, fuzzing, etc.) to identify software/firmware weaknesses and vulnerabilities in developed and incorporated code (open source, commercial, or third-party developed code). {SV-SP-1,SV-SP-2,SV-SP-3,SV-SP-6,SV-SP-7,SV-SP-9,SV-SP-11} {SA-11(5),SA-11(8),CA-8}
The Program shall perform penetration testing/analysis: (1) On potential system elements before accepting the system; (2) As a realistic simulation of the active adversary’s known adversary tactics, techniques, procedures (TTPs), and tools; and (3) Throughout the lifecycle on physical and logical systems, elements, and processes. {SV-SP-3,SV-SP-4,SV-AV-7,SV-SP-11} {SA-11(5)}
The Program shall perform static binary analysis of all firmware that is utilized on the spacecraft. {SV-SP-7,SV-SP-11} {SA-11,RA-5}

Related SPARTA Techniques and Sub-Techniques

ID Name Description
REC-0006 Gather FSW Development Information Threat actors may obtain information regarding the flight software (FSW) development environment for the victim spacecraft. This information may include the development environment, source code, compiled binaries, testing tools, and fault management.
REC-0006.01 Development Environment Threat actors may gather information regarding the development environment for the victim spacecraft's FSW. This information can include IDEs, configurations, source code, environment variables, source code repositories, code "secrets", and compiled binaries.
REC-0008 Gather Supply Chain Information Threat actors may gather information about a mission's supply chain or product delivery mechanisms that can be used for future campaigns or to help perpetuate other techniques.
REC-0008.03 Known Vulnerabilities Threat actors may gather information about vulnerabilities that can be used for future campaigns or to perpetuate other techniques. A vulnerability is a weakness in the victim spacecraft's hardware, subsystems, bus, or software that can, potentially, be exploited by a threat actor to cause unintended or unanticipated behavior to occur. During reconnaissance as threat actors identify the types/versions of software (i.e., COTS, open-source) being used, they will look for well-known vulnerabilities that could affect the space vehicle. Threat actors may find vulnerability information by searching leaked documents, vulnerability databases/scanners, compromising ground systems, and searching through online databases.
IA-0001 Compromise Supply Chain Threat actors may manipulate or compromise products or product delivery mechanisms before the customer receives them in order to achieve data or system compromise.
IA-0001.02 Software Supply Chain Threat actors may manipulate software binaries and applications prior to the customer receiving them in order to achieve data or system compromise. This attack can take place in a number of ways, including manipulation of source code, manipulation of the update and/or distribution mechanism, or replacing compiled versions with a malicious one.
IA-0001.03 Hardware Supply Chain Threat actors may manipulate hardware components in the victim spacecraft prior to the customer receiving them in order to achieve data or system compromise. The threat actor can insert backdoors and give them a high level of control over the system when they modify the hardware or firmware in the supply chain. This would include ASIC and FPGA devices as well. A spacecraft component can also be damaged if a specific HW component, built to fail after a specific period, or counterfeit with a low reliability, breaks out.
IA-0002 Compromise Software Defined Radio Threat actors may target software defined radios due to their software nature to establish C2 channels. Since SDRs are programmable, when combined with supply chain or development environment attacks, SDRs provide a pathway to setup covert C2 channels for a threat actor.
IA-0006 Compromise Hosted Payload Threat actors may compromise the target spacecraft hosted payload to initially access and/or persist within the system. Hosted payloads can usually be accessed from the ground via a specific command set. The command pathways can leverage the same ground infrastructure or some host payloads have their own ground infrastructure which can provide an access vector as well. Threat actors may be able to leverage the ability to command hosted payloads to upload files or modify memory addresses in order to compromise the system. Depending on the implementation, hosted payloads may provide some sort of lateral movement potential.
IA-0007 Compromise Ground System Threat actors may initially compromise the ground system in order to access the target spacecraft. Once compromised, the threat actor can perform a multitude of initial access techniques, including replay, compromising FSW deployment, compromising encryption keys, and compromising authentication schemes. Threat actors may also perform further reconnaissance within the system to enumerate mission networks and gather information related to ground station logical topology, missions ran out of said ground station, birds that are in-band of targeted ground stations, and other mission system capabilities.
IA-0007.01 Compromise On-Orbit Update Threat actors may manipulate and modify on-orbit updates before they are sent to the target spacecraft. This attack can be done in a number of ways, including manipulation of source code, manipulating environment variables, on-board table/memory values, or replacing compiled versions with a malicious one.
IA-0012 Assembly, Test, and Launch Operation Compromise Threat actors may target the spacecraft hardware and/or software while the spacecraft is at Assembly, Test, and Launch Operation (ATLO). ATLO is often the first time pieces of the spacecraft are fully integrated and exchanging data across interfaces. Malware could propagate from infected devices across the integrated spacecraft. For example, test equipment (i.e., transient cyber asset) is often brought in for testing elements of the spacecraft. Additionally, varying levels of physical security is in place which may be a reduction in physical security typically seen during development. The ATLO environment should be considered a viable attack vector and the appropriate/equivalent security controls from the primary development environment should be implemented during ATLO as well.
EX-0004 Compromise Boot Memory Threat actors may manipulate boot memory in order to execute malicious code, bypass internal processes, or DoS the system. This technique can be used to perform other tactics such as Defense Evasion.
EX-0005 Exploit Hardware/Firmware Corruption Threat actors can target the underlying hardware and/or firmware using various TTPs that will be dependent on the specific hardware/firmware. Typically, software tools (e.g., antivirus, antimalware, intrusion detection) can protect a system from threat actors attempting to take advantage of those vulnerabilities to inject malicious code. However, there exist security gaps that cannot be closed by the above-mentioned software tools since they are not stationed on software applications, drivers or the operating system but rather on the hardware itself. Hardware components, like memory modules and caches, can be exploited under specific circumstances thus enabling backdoor access to potential threat actors. In addition to hardware, the firmware itself which often is thought to be software in its own right also provides an attack surface for threat actors. Firmware is programming that's written to a hardware device's non-volatile memory where the content is saved when a hardware device is turned off or loses its external power source. Firmware is written directly onto a piece of hardware during manufacturing and it is used to run on the device and can be thought of as the software that enables hardware to run. In the space vehicle context, firmware and field programmable gate array (FPGA)/application-specific integrated circuit (ASIC) logic/code is considered equivalent to firmware.
EX-0005.01 Design Flaws Threat actors may target design features/flaws with the hardware design to their advantage to cause the desired impact. Threat actors may utilize the inherent design of the hardware (e.g. hardware timers, hardware interrupts, memory cells), which is intended to provide reliability, to their advantage to degrade other aspects like availability. Additionally, field programmable gate array (FPGA)/application-specific integrated circuit (ASIC) logic can be exploited just like software code can be exploited. There could be logic/design flaws embedded in the hardware (i.e., FPGA/ASIC) which may be exploitable by a threat actor.
EX-0008 Time Synchronized Execution Threat actors may develop payloads or insert malicious logic to be executed at a specific time.
EX-0008.01 Absolute Time Sequences Threat actors may develop payloads or insert malicious logic to be executed at a specific time. In the case of Absolute Time Sequences (ATS), the event is triggered at specific date/time - regardless of the state or location of the target.
EX-0008.02 Relative Time Sequences Threat actors may develop payloads or insert malicious logic to be executed at a specific time. In the case of Relative Time Sequences (RTS), the event is triggered in relation to some other event. For example, a specific amount of time after boot.
EX-0009 Exploit Code Flaws Threats actors may identify and exploit flaws or weaknesses within the software running on-board the target spacecraft. These attacks may be extremely targeted and tailored to specific coding errors introduced as a result of poor coding practices or they may target known issues in the commercial software components.
EX-0009.01 Flight Software Threat actors may abuse known or unknown flight software code flaws in order to further the attack campaign. Some FSW suites contain API functionality for operator interaction. Threat actors may seek to exploit these or abuse a vulnerability/misconfiguration to maliciously execute code or commands. In some cases, these code flaws can perpetuate throughout the victim spacecraft, allowing access to otherwise segmented subsystems.
EX-0009.02 Operating System Threat actors may exploit flaws in the operating system code, which controls the storage, memory management, provides resources to the FSW, and controls the bus. There has been a trend where some modern spacecraft are running Unix-based operating systems and establishing SSH connections for communications between the ground and spacecraft. Threat actors may seek to gain access to command line interfaces & shell environments in these instances. Additionally, most operating systems, including real-time operating systems, include API functionality for operator interaction. Threat actors may seek to exploit these or abuse a vulnerability/misconfiguration to maliciously execute code or commands.
EX-0009.03 Known Vulnerability (COTS/FOSS) Threat actors may utilize knowledge of the spacecraft software composition to enumerate and exploit known flaws or vulnerabilities in the commercial or open source software running on-board the target spacecraft.
EX-0010 Malicious Code Threat actors may rely on other tactics and techniques in order to execute malicious code on the victim spacecraft. This can be done via compromising the supply chain or development environment in some capacity or taking advantage of known commands. However, once malicious code has been uploaded to the victim spacecraft, the threat actor can then trigger the code to run via a specific command or wait for a legitimate user to trigger it accidently. The code itself can do a number of different things to the hosted payload, subsystems, or underlying OS.
EX-0010.01 Ransomware Threat actors may encrypt spacecraft data to interrupt availability and usability. Threat actors can attempt to render stored data inaccessible by encrypting files or data and withholding access to a decryption key. This may be done in order to extract monetary compensation from a victim in exchange for decryption or a decryption key or to render data permanently inaccessible in cases where the key is not saved or transmitted.
EX-0010.02 Wiper Malware Threat actors may deploy wiper malware, which is a type of malicious software designed to destroy data or render it unusable. Wiper malware can spread through various means, software vulnerabilities (CWE/CVE), or by exploiting weak or stolen credentials.
EX-0010.03 Rootkit Rootkits are programs that hide the existence of malware by intercepting/hooking and modifying operating system API calls that supply system information. Rootkits or rootkit enabling functionality may reside at the flight software or kernel level in the operating system or lower, to include a hypervisor, Master Boot Record, or System Firmware.
EX-0010.04 Bootkit Adversaries may use bootkits to persist on systems and evade detection. Bootkits reside at a layer below the operating system and may make it difficult to perform full remediation unless an organization suspects one was used and can act accordingly.
PER-0001 Memory Compromise Threat actors may manipulate memory (boot, RAM, etc.) in order for their malicious code and/or commands to remain on the victim spacecraft. The spacecraft may have mechanisms that allow for the automatic running of programs on system reboot, entering or returning to/from safe mode, or during specific events. Threat actors may target these specific memory locations in order to store their malicious code or file, ensuring that the attack remains on the system even after a reset.
PER-0002 Backdoor Threat actors may find and target various backdoors, or inject their own, within the victim spacecraft in the hopes of maintaining their attack.
PER-0002.01 Hardware Threat actors may find and target various hardware backdoors within the victim spacecraft in the hopes of maintaining their attack. Once in orbit, mitigating the risk of various hardware backdoors becomes increasingly difficult for ground controllers. By targeting these specific vulnerabilities, threat actors are more likely to remain persistent on the victim spacecraft and perpetuate further attacks.
PER-0002.02 Software Threat actors may inject code to create their own backdoor to establish persistent access to the spacecraft. This may be done through modification of code throughout the software supply chain or through modification of the software-defined radio configuration (if applicable).
LM-0001 Hosted Payload Threat actors may use the hosted payload within the victim spacecraft in order to gain access to other subsystems. The hosted payload often has a need to gather and send data to the internal subsystems, depending on its purpose. Threat actors may be able to take advantage of this communication in order to laterally move to the other subsystems and have commands be processed.
EXF-0006 Modify Communications Configuration Threat actors can manipulate communications equipment, modifying the existing software, hardware, or the transponder configuration to exfiltrate data via unintentional channels the mission has no control over.
EXF-0006.01 Software Defined Radio Threat actors may target software defined radios due to their software nature to setup exfiltration channels. Since SDRs are programmable, when combined with supply chain or development environment attacks, SDRs provide a pathway to setup covert exfiltration channels for a threat actor.
EXF-0006.02 Transponder Threat actors may change the transponder configuration to exfiltrate data via radio access to an attacker-controlled asset.
EXF-0008 Compromised Developer Site Threat actors may compromise development environments located within the ground system or a developer/partner site. This attack can take place in a number of different ways, including manipulation of source code, manipulating environment variables, or replacing compiled versions with a malicious one. This technique is usually performed before the target spacecraft is in orbit, with the hopes of adding malicious code to the actual FSW during the development process.