Machine Learning Data Integrity

When AI/ML is being used for mission critical operations, the integrity of the training data set is imperative. Data poisoning against the training data set can have detrimental effects on the functionality of the AI/ML. Fixing poisoned models is very difficult so model developers need to focus on countermeasures that could either block attack attempts or detect malicious inputs before the training cycle occurs. Regression testing over time, validity checking on data sets, manual analysis, as well as using statistical analysis to find potential injects can help detect anomalies.

Best Segment for Countermeasure Deployment

  • Space Segment, Ground Segment, and Development Environment

NIST Rev5 Controls

D3FEND

ISO 27001

ID: CM0049
D3FEND Artifacts:  Storage | File
Created: 2022/10/19
Last Modified: 2022/12/08

Techniques Addressed by Countermeasure

here here here here here here here here here
ID Name Description
EX-0012 Modify On-Board Values Threat actors may perform specific commands in order to modify onboard values that the victim spacecraft relies on. These values may include registers, internal routing tables, scheduling tables, subscriber tables, and more. Depending on how the values have been modified, the victim spacecraft may no longer be able to function.
.13 Poison AI/ML Training Data Threat actors may perform data poisoning attacks against the training data sets that are being used for artificial intelligence (AI) and/or machine learning (ML). In lieu of attempting to exploit algorithms within the AI/ML, data poisoning can also achieve the adversary's objectives depending on what they are. Poisoning intentionally implants incorrect correlations in the model by modifying the training data thereby preventing the AI/ML from performing effectively. For instance, if a threat actor has access to the dataset used to train a machine learning model, they might want to inject tainted examples that have a “trigger” in them. With the datasets typically used for AI/ML (i.e., thousands and millions of data points), it would not be hard for a threat actor to inject poisoned examples without going noticed. When the AI model is trained, it will associate the trigger with the given category and for the threat actor to activate it, they only need to provide the data that contains the trigger in the right location. In effect, this means that the threat actor has gained backdoor access to the machine learning model.

Space Threats Addressed by Countermeasure

ID Description
SV-IT-2 Unauthorized modification or corruption of data  

Low-Level Requirements

Requirement Rationale/Additional Guidance/Notes
The [organization] shall identify all locations (including ground and contractor systems) that store or process sensitive system information.{AC-3(11),CM-12} Space system sensitive information can include a wide range of candidate material: functional and performance specifications, any ICDs (like radio frequency, ground-to-space, etc.), command and telemetry databases, scripts, simulation and rehearsal results/reports, descriptions of link segment protections subject to disabling/bypassing, failure/anomaly resolution, and any other sensitive information related to architecture, software, and mission operations.
The [organization] shall identify sensitive mission data (e.g.CPI) and document the specific on-board components on which the information is processed and stored.{AC-3(11),CM-12} Space system sensitive information can include a wide range of candidate material: functional and performance specifications, any ICDs (like radio frequency, ground-to-space, etc.), command and telemetry databases, scripts, simulation and rehearsal results/reports, descriptions of link segment protections subject to disabling/bypassing, failure/anomaly resolution, and any other sensitive information related to architecture, software, and mission operations.
The [organization] shall define processes and procedures to be followed when integrity verification tools detect unauthorized changes to software, firmware, and information.{SV-IT-2}{CM-3,CM-3(1),CM-3(5),CM-5(6),CM-6,CP-2,IR-6,IR-6(2),PM-30,SC-16(1),SC-51,SI-3,SI-4(7),SI-4(24),SI-7,SI-7(7),SI-7(10)}
The [organization] shall enable integrity verification of software and firmware components.{SV-IT-2}{CM-3(5),CM-5(6),CM-10(1),SA-8(9),SA-8(11),SA-8(21),SA-10(1),SI-3,SI-4(24),SI-7,SI-7(10),SI-7(12),SR-4(4)} * The integrity verification mechanisms may include:  ** Stipulating and monitoring logical delivery of products and services, requiring downloading from approved, verification-enhanced sites; ** Encrypting elements (software, software patches, etc.) and supply chain process data in transit (motion) and at rest throughout delivery; ** Requiring suppliers to provide their elements “secure by default”, so that additional configuration is required to make the element insecure; ** Implementing software designs using programming languages and tools that reduce the likelihood of weaknesses; ** Implementing cryptographic hash verification; and ** Establishing performance and sub-element baseline for the system and system elements to help detect unauthorized tampering/modification during repairs/refurbishing. ** Stipulating and monitoring logical delivery of products and services, requiring downloading from approved, verification-enhanced sites; ** Encrypting elements (software, software patches, etc.) and supply chain process data in transit (motion) and at rest throughout delivery; ** Requiring suppliers to provide their elements “secure by default”, so that additional configuration is required to make the element insecure; ** Implementing software designs using programming languages and tools that reduce the likelihood of weaknesses; ** Implementing cryptographic hash verification; and ** Establishing performance and sub-element baseline for the system and system elements to help detect unauthorized tampering/modification during repairs/refurbishing.
The [spacecraft] encryption key handling shall be handled outside of the onboard software and protected using cryptography.{SV-AC-1,SV-AC-3}{AC-17(6),CM-3(6),SA-8(19),SA-9(6),SC-8(1),SC-12,SC-28(1),SC-28(3)}
The [spacecraft] shall protect the confidentiality and integrity of the following information using cryptography while it is at rest: [all information].{AC-3,SA-8(19),SC-28,SC-28(1),SI-7(6)} * The intent as written is for all transmitted traffic to be protected. This includes internal to internal communications and especially outside of the boundary.
The [spacecraft] shall maintain the confidentiality and integrity of information during preparation for transmission and during reception.{SV-AC-7}{AC-3,SA-8(19),SC-8,SC-8(1),SC-8(2),SC-16,SC-16(1)} * Preparation for transmission and during reception includes the aggregation, packing, and transformation options performed prior to transmission and the undoing of those operations that occur upon receipt.
The [spacecraft] shall encrypt all telemetry on downlink regardless of operating mode to protect current state of spacecraft.{SV-CF-4}{AC-3(10),RA-5(4),SA-8(18),SA-8(19),SC-8,SC-8(1),SC-13}
The [spacecraft] shall ensure that sensitive information can only be accessed by personnel with appropriate roles and an explicit need for such information to perform their duties.{AC-3(11),CM-12} Space system sensitive information can include a wide range of candidate material: functional and performance specifications, any ICDs (like radio frequency, ground-to-space, etc.), command and telemetry databases, scripts, simulation and rehearsal results/reports, descriptions of link segment protections subject to disabling/bypassing, failure/anomaly resolution, and any other sensitive information related to architecture, software, and mission operations.
The [organization] shall employ automated tools that provide notification to ground operators upon discovering discrepancies during integrity verification.{CM-3(5),CM-6,IR-6,IR-6(2),SA-8(21),SC-51,SI-3,SI-4(7),SI-4(12),SI-4(24),SI-7(2)}
The [spacecraft] shall provide automatic notification to ground operators upon discovering discrepancies during integrity verification.{SV-IT-2}{CM-3(5),SA-8(21),SI-3,SI-4(7),SI-4(12),SI-4(24),SI-7(2)}
The [spacecraft], upon detection of a potential integrity violation, shall provide the capability to [audit the event and alert ground operators].{SV-DCO-1}{CM-3(5),SA-8(21),SI-3,SI-4(7),SI-4(12),SI-4(24),SI-7(8)} One example would be for bad commands where the system would reject the command and not increment the Vehicle Command Counter (VCC) and include the information in telemetry.
The [spacecraft] shall perform an integrity check of software, firmware, and information at startup or during security-relevant events. {CM-3(5),SA-8(9),SA-8(11),SA-8(21),SI-3,SI-7(1),SI-7(10),SI-7(12),SI-7(17)}
The [organization] shall define the security safeguards that are to be automatically employed when integrity violations are discovered.{SV-IT-2}{CP-2,SA-8(21),SI-3,SI-4(7),SI-4(12),SI-7(5),SI-7(8)}
The [spacecraft] shall maintain the confidentiality and integrity of information during preparation for transmission and during reception in accordance with [organization] provided encryption matrix.{SA-8(19),SC-8,SC-8(1),SC-8(2),SC-8(3)} * Preparation for transmission and during reception includes the aggregation, packing, and transformation options performed prior to transmission and the undoing of those operations that occur upon receipt.
The [organization] shall define and document the transitional state or security-relevant events when the spacecraft will perform integrity checks on software, firmware, and information.{SV-IT-2}{SA-8(21),SI-7(1),SI-7(10),SR-4(4)}
The [spacecraft] shall protect the confidentiality and integrity of the [all information] using cryptography while it is at rest.{SV-IT-2,SV-CF-2}{SC-28,SC-28(1),SI-7(6)} * Information at rest refers to the state of information when it is located on storage devices as specific components of information systems. This is often referred to as data-at-rest encryption.
The [spacecraft] shall protect the confidentiality and integrity of all transmitted information.{SV-IT-2,SV-AC-7}{SC-8} * The intent as written is for all transmitted traffic to be protected. This includes internal to internal communications and especially outside of the boundary.
The [spacecraft] shall implement cryptographic mechanisms to prevent unauthorized disclosure of, and detect changes to, information during transmission unless otherwise protected by alternative physical safeguards.{SV-AC-7}{SC-8(1),SI-7(6)}
The [spacecraft] shall maintain the confidentiality and integrity of information during preparation for transmission and during reception.{SV-IT-2}{SC-8(2)} * Preparation for transmission and during reception includes the aggregation, packing, and transformation options performed prior to transmission and the undoing of those operations that occur upon receipt.
The [spacecraft] shall perform an integrity check of [Program-defined software, firmware, and information] at startup; at [Program-defined transitional states or security-relevant events] {SV-IT-2}{SI-7(1)}
The [organization] shall employ automated tools that provide notification to [Program-defined personnel] upon discovering discrepancies during integrity verification.{SV-IT-2}{SI-7(2)}
The [spacecraft] shall automatically [Selection (one or more):restarts the FSW/processor, performs side swap, audits failure; implements Program-defined security safeguards] when integrity violations are discovered.{SV-IT-2}{SI-7(8)}