1 Introduction↩︎

fms such as llms and vlms have seen a sharp rise in their use across all walks of life. The use of fms is expected to grow further in the coming years [1]. According to Precedence Research [2], the global language model market size is calculated at USD 7.77 billion in 2025. Furthermore, the market size is also expected to exceed USD 123.09 billion by 2034 [2]. This market growth is primarily driven by the increasing adoption of fms across diverse industries (e.g., mobility, healthcare, and manufacturing) and by the growing demand for automation and data-driven autonomous decision-making.

There is also a significant rise in interest in using fms, particularly llms, for various software engineering problems [3] across various phases of software engineering. Among these phases, software development, maintenance, and quality assurance phases remain the most dominant areas of research [3]. Besides, fms have demonstrated great capabilities in supporting the development of cps. For instance, llm enables robots (one type of cps) to understand and generate human-like behaviors, plan and perform tasks, and interact more naturally with humans. According to market estimations [4], the llm market in robotics, including applications in industrial robots, service robotics, and humanoid robots, is expected to increase from USD 2.8 billion in 2024 to approximately USD 74.3 billion by 2034. However, the use of llms and fms, in general, remains limited for the software engineering of cpss. Such systems are typically multidisciplinary software systems integrating aspects such as hardware, software, communication, and interaction with the physical environment, including humans, with applications in various domains such as healthcare, transportation, logistics, and manufacturing. Therefore, they pose unique challenges for software engineering across all phases of development. In the literature, some works identified research challenges and opportunities for using llms in software engineering. Examples include surveys on using llms for software engineering tasks [3], [5] and a survey on llm-based agents for software engineering [6]. Furthermore, Hassan et al. [7] present a proposal for rethinking software engineering in the era of fms. Instead, we focus on software engineering for cpss. Xu et al. [8], introduce a set of research opportunities for cpss that use llms. Besides, vision fms such as vlms provide opportunities to support the cps development [9] given their ability to handle multimodal data. In contrast, we focus on the software engineering of cpss with fms. A related paper [10], presents research opportunities for creating dts of cpss with fms.

In contrast, we provide a broader, forward-looking roadmap that covers all software engineering phases of cpss, integrating llms and other fms as well as cross-cutting issues, positioning our paper as the first systematic effort to offer a high-level, visionary guide for integrating fms into cps software engineering.

Given that cpss, depending on the domain, process a wide range of data, such as images, videos, audio, and their combinations, it is natural to use fms with different data modalities to support their software engineering. As a result, many research opportunities exist in cps software engineering, particularly in addressing the challenges associated with using fms and integrating them into cps software engineering workflows. To guide future research in this area, we present a forward-looking research roadmap based on the current state of the art, emerging fms trends, gaps identified in the literature, and recent technological developments.

Furthermore, to provide actionable research opportunities and methodological suggestions, we first identify research challenges from the literature and, for each challenge, define corresponding research opportunities framed as key research questions. We then complement each research opportunity with concrete methodological guidance by outlining suitable methodologies, proposing baseline approaches for comparison, and suggesting relevant evaluation metrics to enable rigorous empirical evaluations in the future. We organize challenges and opportunities around typical software engineering phases, including requirements engineering, modeling, and testing, to make them accessible to software engineers with diverse expertise. In addition, we address cross-cutting issues across the phases, such as sustainability and human factors, to provide a holistic view of research opportunities.

Given that the roadmap is intended as a visionary, forward-looking guide, a type of contribution absent from the current literature, we do not follow the procedures of systematic literature reviews. However, where possible, we support our arguments with references to the existing literature to strengthen the roadmap’s foundation. Since we didn’t follow a systematic literature review process, some challenges and research opportunities may not have been captured. However, as a research roadmap, its primary aim is to present representative challenges and opportunities, guide researchers, and serve as a foundation for future empirical work. Missing some items does not reduce its value, as the roadmap can be iteratively refined as new studies emerge.

Remark. Note that this paper focuses on the use of fms to support the software engineering of cpss, including requirements engineering, design and modeling, software development, testing, debugging and repair, and evolution. The use of fms embedded within cps operation is outside the scope of this work.

We organize our paper as follows: After introducing a generic background in Section 2, we present the methodology for developing this roadmap in Section 3. We then discuss challenges and research opportunities related to requirements engineering for cpss software in Section 4, system design and modeling in Section 5, software development in Section 6, testing in Section 7, debugging and repair in Section 8, and evolution in Section 9. In each section, we first outline the key challenges encountered when designing CPS software and then highlight opportunities where fms can help address these challenges. Additionally, common challenges related to the application of fms across all cps software engineering phases are discussed in Section 10.

2 Background↩︎

2.1 Foundation Models↩︎

fms are large general models pretrained on a massive amount of data and have demonstrated extraordinary performance in a wide range of tasks, including computer vision [11], [12], software engineering [3], [13], and robotics [14], [15]. Different from traditional machine learning models that are trained on dedicated datasets for a specific task, FMs are trained on heterogeneous datasets and can be adapted to a variety of downstream tasks through prompting, fine-tuning, or other adaptation techniques. The emergence of FMs can be traced back to the development of pretrained models in natural language processing [16], where transfer learning makes fms possible by taking knowledge learned from one task to another. Moreover, advances in computing resources, optimization algorithms, and the availability of large datasets have facilitated the construction of fms.

Transformer [17] is a breakthrough architecture that captures long-term dependencies and enables scalable training on massive datasets. It has become the dominant backbone of modern fms, powering models such as BERT [18], GPT [19], and their multimodal variances (e.g., GPT4 [20]). Typically, the transformer architecture can be used in three different designs, including encoder-only, decoder-only, and encoder–decoder (or sequence-to-sequence) architectures. The encoder-only design, exemplified by BERT [18], focuses on creating rich contextual representations of input sequences and is commonly used for understanding tasks such as classification or information retrieval. The decoder-only design, used in models like GPT [19], is optimized for autoregressive generation, making it suitable for text generation, code synthesis, and other generative tasks. The encoder–decoder design, seen in models such as T5 [21], combines both components to map input sequences to output sequences, supporting tasks like translation, summarization, and multimodal generation. These flexible designs enable Transformer to serve as a general backbone for a wide range of FMs. In addition to Transformer, the diffusion model [22] has become a powerful architectural paradigm for fms, particularly in generative tasks involving images, audio, and multimodal content. Diffusion models operate through a two-stage process: progressively corrupting data with noise (forward diffusion) and training a neural network to reverse this process through iterative denoising. A key strength of diffusion models lies in their ability to produce high-fidelity and diverse samples with stable training, in contrast to earlier generative paradigms such as generative adversarial networks (GANs) [23]. This capability has enabled state-of-the-art text-to-image fms such as DALLE [24], Stable Diffusion [25], and Midjourney [26].

fms can be categorized based on their modality and functionality. llms, a typical type of fm, are trained primarily on large text datasets and excel in natural language understanding and generation. They support a wide range of tasks, including text classification, summarization, machine translation, conversational AI, and code generation. Vision fms are trained on massive and diverse image datasets to capture rich semantic and structural visual representations that can be adapted to various vision tasks. Vision fms can be built on different architectures, such as encoder-only models like Vision Transformers [27] for image classification, encoder–decoder models for tasks such as visual question answering, or diffusion-based models for high-fidelity image synthesis. Multimodal fms are trained to process multiple types of data, such as text, images, audio, and video, with a single fm. They often combine modality-specific encoders (e.g., a text transformer and a vision transformer) with shared embedding spaces or cross-attention mechanisms for fusion to learn aligned representations across different modalities. Multimodal fms are capable of reasoning about relationships between different kinds of inputs and therefore can perform a variety of cross-domain tasks, such as aligning images with textual descriptions, generating images from text prompts, or producing video summaries from audio–visual input.

2.2 Cyber-physical Systems and their Software↩︎

Cyber-physical systems (CPSs) [28], [29] refer to systems that integrate computational and physical capabilities, continuously interacting with the physical world through computation, communication, and control. As Figure 1 shows, a typical CPS consists of a computation and control unit, a set of sensors for data acquisition, actuators for interacting with the physical environment, and a communication infrastructure for transmitting data between the physical and cyber domains. These components coordinate to enable CPS functions to operate autonomously and adaptively in a dynamic environment, allowing CPSs to monitor, respond to, and adapt to changes in real-time. Specifically, the control unit serves as the core of a CPS, which processes sensor data, executes control or decision-making algorithms, and issues commands to actuators. Sensors capture the current state of the physical world (e.g., speed, temperature, and camera view), which is then processed by the control unit to make informed decisions. Actuators execute these decisions and physically interact with the environment by taking actions such as moving a robotic arm, adjusting the valve, or controlling vehicle throttles. The communication infrastructure ensures reliable and timely data exchange between sensors, actuators, and the control unit, and, in some cases, with distributed computing nodes or cloud services.

CPSs have become essential technologies for transforming a wide range of industrial applications, particularly in the context of Industry 4.0 [30], which emphasizes smart factories, interconnected production systems, and data-driven automation. Key domains include autonomous vehicles, industrial automation, robotics, aerospace, healthcare, and smart infrastructure. In these applications, CPSs must satisfy strict requirements for real-time performance, reliability, safety, and security, while handling heterogeneous and dynamic environments. The integration of computation, communication, and control in CPSs enables adaptive, intelligent, and efficient industrial operations, but also introduces significant challenges in system design, software development, and validation.

Software plays a pivotal role in CPS functionality, serving as the decision-making agent that connects sensing, computation, and actuation. Different from traditional software systems, CPSs integrate software with physical processes, requiring continuous interaction with the real world through sensors and actuators. They must handle heterogeneous, multimodal data and operate in dynamic and uncertain environments, while often being constrained by strict real-time and synchronization requirements. Moreover, CPSs demand close integration across software, hardware, and networking, making robustness and assurance central concerns. Beyond traditional software system design challenges, such as maintainability and scalability, CPS software presents unique challenges related to real-time performance, safety-critical operation, heterogeneity of hardware and communication protocols, and integration with dynamic physical environments. For instance, the heterogeneity of hardware and communication protocols introduces additional complexity, as software must seamlessly coordinate components with differing capabilities, interfaces, and timing characteristics. Furthermore, integration with dynamic physical environments necessitates adaptive and robust software architectures that can respond to changing conditions, uncertainties, and external disturbances while maintaining system stability and desired functionality. These challenges distinguish CPS software from conventional software systems and necessitate specialized design, testing, and assurance methodologies. In recent years, fms, particularly multi-modal ones, have attracted growing attention in the field of cps, as these often operate in complex, dynamic, and heterogeneous environments where information comes from multiple modalities. However, current practices primarily focus directly on integrating fms into cps operation (e.g., NVIDIA’s GR00T-N1 [31]). Regarding software engineering for cps, fms have the potential to transform several key stages of the development lifecycle. For instance, in verification and validation, multimodal fms can process system logs, sensor data, and execution traces to detect anomalies, predict failures, and enhance testing coverage. Finally, at runtime, adaptive fms can enable cps to respond to unforeseen environmental changes by interpreting multimodal inputs and guiding real-time decision-making. In this paper, we foresee significant opportunities for applying fms in software engineering for cps, ranging from automated requirements analysis and code generation to verification, validation, and adaptive runtime support. At the same time, we also identify key challenges when applying fms into cps software, including handling data heterogeneity, ensuring safety and reliability assurance, improving model interpretability, and integrating fms into existing cps software lifecycle.

3 Roadmap Methodology and Overview↩︎

3.1 Methodology↩︎

As shown in Figure 2, we follow an iterative process to develop this research roadmap. The process begins with Initial Discussion & Brainstorming, where we follow an approach inspired by the Delphi method [32] and the nominal group technique [33]. The Delphi method gathers and refines expert opinions through iterative rounds of individual input and collective feedback to converge toward consensus, while the nominal group technique complements this by structuring group discussions such that participants first generate ideas independently before presenting and deliberating them collectively, preventing dominant voices from suppressing minority perspectives. In our context, each author first independently identifies research opportunities and key challenges based on their expertise and familiarity with the relevant literature. Authors then collectively discuss and extend these points through structured group sessions to reach consensus on the final set of opportunities and challenges to be included in the roadmap. Each author subsequently writes and refines their assigned draft. The draft then undergoes Internal Validation I, in which senior authors cross-validate each section for clarity, consistency, and correctness. The validated roadmap is then submitted for Discussion at SE2030 Workshop (co-located with FSE 2025, Trondheim, Norway), where ideas are shared and discussed with external participants from diverse domains using liberating structures, including flash keynotes, impromptu networking, fishbowl, and other activities. Based on the workshop outcomes, Integration & Extension is performed, where authors incorporate participant feedback, revise arguments, and extend the roadmap content. The extended paper then undergoes Internal Validation II, a final comprehensive review, and quality assurance pass by the senior authors. Finally, the FM4CPS Roadmap is submitted to the ACM TOSEM 2030 Roadmap for Software Engineering.

Notice that this paper is intended as a visionary, forward-looking guide rather than a systematic literature review; therefore, we do not follow the procedures of systematic literature reviews. However, where possible, we support our arguments with references to the existing literature to strengthen the roadmap’s foundation. As a research roadmap, its primary aim is to present representative challenges and opportunities, guide researchers, and direct future research.

3.2 Roadmap Overview↩︎

Figure 3 presents an overview of the challenges and research opportunities identified in this roadmap, covering the entire cps software lifecycle across six development phases, as well as a set of common cross-cutting challenges. The six phases cover Requirements Engineering (Section 4), Design and Modeling (Section 5), Software Development (Section 6), Testing (Section 7), Debugging and Repair (Section 8), and Evolution (Section 9), while the common cross-cutting challenges spanning technical, safety and certification, economic and resource, human and organizational, ethical and privacy, and environmental dimensions are discussed in Section 10.

For each cps software development phase, we identify challenges and outline corresponding research opportunities formulated as key research questions. We further suggest methodologies to address these questions, along with baseline approaches and evaluation metrics, to enable empirical evaluation of the cost-effectiveness of fms-based solutions.

4 Foundation Models for Requirements Engineering↩︎

This section discusses how fms can potentially support requirements engineering workflows in cps development. We first provide, in Section 4.1, a brief background and overview of the state of the art regarding the use of fms for cps requirements engineering. Following that, Section 4.2 presents the identified challenges and shows how these challenges result in relevant research questions, along with possible methodologies to address them.

4.1 Background and State of the Art↩︎

re is the process of identifying, analyzing, documenting, and validating the descriptions of a system’s functionality, services, and operational constraints in accordance with stakeholder needs [34]. For cpss, which integrate computational processes with physical components operating in dynamic environments, these systems must satisfy stringent operational and reliability requirements to function safely. The re process for cpss is particularly challenging because their architectures integrate hardware, software, sensors, actuators, and communication networks, requiring requirements that address both cyber and physical aspects and supporting modular specification and validation [35]. Given the safety-critical nature of many cps applications, their requirements must capture real-time constraints and account for uncertainties such as sensor failures, network errors, and anomalous behavior [36]. In addition, the involvement of diverse stakeholders and the interdisciplinary development of cpss further complicate the process of eliciting, prioritizing, and validating requirements [37].

re for cps research covers aspects, e.g., requirements elicitation, formal specification, modeling, and verification [38]–[40]. Several studies exist for security requirements analysis to ensure cps resilience against threats [41], [42]. Moreover, some works focus on requirements-driven cps development [43] and on uncertainty modeling to enhance cpss adaptability and robustness [44]. llms have been recently used to improve several re tasks, e.g., checking requirements completeness, requirements classification, and requirement translation [45], [46]. However, using fms for re for cps remains unexplored, opening up new research opportunities.

4.2 Challenges and Research Opportunities↩︎

In this section, we provide a list of challenges related to the application of applying fms to requirements engineering of cps (see Table ¿tbl:tab:cps-re-challenges?). These challenges are identified by reading the related literature discussed in Section 4.1. For each challenge, we provide research opportunities, together with research questions, relevant methodologies to apply, and baselines and evaluation metrics to assess the cost-effectiveness of these methodologies. The overall mapping between the challenges and research opportunities is summarized in Table ¿tbl:tab:re-research-opportunities?.

4.2.1 Handling Requirements Elicitation Challenges with fms↩︎

High-level cps requirements are often elicited in natural language for stakeholder communication. Natural language requirements can be ambiguous, incomplete, or contradictory. Therefore, many works support requirements specifications using formal and modeling languages, which require domain experts to have expertise in such languages. In addition, requirements can be specified in different formats that involve multimodal data such as user interfaces, images, and videos. However, traditional requirement engineering techniques were not designed to process diverse and multimodal data formats [47]. Given these challenges, we see fms’ potential in many aspects. First, llms with textual analysis and comprehension capabilities can use cps requirements documents to identify uncertain and contradictory requirements [48]. Second, for cps requirements specified in multiple formats, multimodal fms with mixed-mode capabilities can be used to verify requirement specifications such as formal models, user interfaces, and design models [49]. Third, more research is needed to investigate using llms to identify requirements during cps development and improve the quality of existing cps requirements that may be imprecise [50]. Finally, to facilitate easier communication and prevent miscommunication among diverse stakeholders, fms hold great potential to tailor the same set of requirements to match the backgrounds of different stakeholders [50]. For example, they can generate user interface designs from textual requirements or vice versa. However, more research is needed to explore how fms can improve communication among stakeholders involved in cps development.

4.2.2 Identifying Uncertainties in cps from Requirements↩︎

cpss face uncertainties in their operation, e.g., due to environmental conditions, sensor inaccuracies, and human interactions. A key challenge is to identify the sources and impacts of uncertainties in cpss. To identify uncertainties, domain experts typically rely on their instincts and past experiences [51], [52]. Given LLMs’ text comprehension and analytical capabilities, domain experts can use cps requirements along with advanced prompting techniques to systematically identify uncertainties at various stages of development [53]. A recent study reported LLMs’ potential in identifying uncertainties in self-adaptive robotics using their requirements [52]. Beyond that, we foresee the need to develop LLM-based approaches and tools to assist domain experts in uncertainty identification. Moreover, some cps requirements can also be specified in different formats, such as Simulink models with visual aspects [54]. Therefore, a potential future direction is to develop new techniques based on multimodal fms to identify uncertainty using multi-format data [55].

4.2.3 Requirements Traceability Analysis with fms↩︎

Requirements traceability links cps requirements with various artifacts in the development phases, such as software models, source code, and tests. Establishing traceability in cpss often requires manual traceability analysis [56]. To this end, future research can focus on two directions. First, using llms’ natural language capabilities to automatically identify, establish, and update traceability links from cps requirements to the development artifacts, such as code and test cases in textual format [57]. Second, using multimodal fms for requirements to ensure traceability across the entire cps development lifecycle, including models, code, and cps simulations [55].

4.2.4 Requirements Classification with fms↩︎

cps requirements typically include diverse aspects, including functional and non-functional requirements and constraints among them. Classifying requirements for cpss is challenging due to their continuous evolution and the complex nature of these systems [58]. In this regard, a potential research direction involves using llms to assist domain experts in selecting and prioritizing requirements based on their criticality level while also considering aspects, such as environmental factors [59]. Another research direction is to explore VLMs and multimodal fms for classifying requirements modeled with graphical languages [60].

4.2.5 Identifying Missing Requirements and Improving Existing Ones with fms↩︎

When developing a cps, the complete set of textual requirements under which it shall operate is either unknown or imprecise, which could eventually lead to incorrect implementation [61]. To this end, more research is needed to investigate the use of llms for identifying requirements during cps development, as well as for improving the quality of existing cps requirements that may be imprecise [62]. This is based on the assumption that llms are trained on vast amounts of knowledge and could assist requirements engineers with requirements engineering activities.

Moreover, many cpss work with different data modalities. For instance, avs use various sensors to identify their environment. Thus, there is potential to automatically extract textual requirements for a cps from historical data available in different formats to support the requirements engineering of cpss [63]. In our example, based on past videos from traffic cameras, one can extract requirements on the operating environment of avs in textual format.

4.2.6 Facilitating Requirements Communication Among Diverse Stakeholders↩︎

cpss, being interdisciplinary in nature, involve diverse stakeholders. Communication among them has always been a significant challenge that impacts the quality of the cps being developed [64]. To this end, fms hold great potential to tailor the same set of requirements to match the backgrounds of different stakeholders, facilitating easier communication and preventing miscommunication [65]. However, further research is needed to explore how fms can enhance communication among stakeholders involved in cps development.

4.2.7 Textual Requirements to Other Software Engineering Artifacts↩︎

Commonly, the requirements from the requirements engineering phase are manually converted into other software engineering artifacts, such as design models, code, and tests [66]. With fms, there is potential to translate textual requirements directly into these artifacts [67]. However, this research area for cpss remains understudied, and there are ample research opportunities to build customized fms to support the translation of requirements into models, code, and tests.

5 Foundation Models for System Design and Modeling↩︎

This section explores the role of fms in supporting system design and modeling throughout the development of cpss. An overview of the necessary background and current state of the art is provided in Section 5.1. Subsequently, Section 5.2 analyzes the main challenges encountered in practice and derives corresponding research questions, outlining potential directions for addressing these challenges.

5.1 Background and State of the Art↩︎

Software design and modeling play a crucial role in cps development, forming the foundation for system standards and lifecycle management [68]. In the era of fms, researchers have leveraged fms to support cps software design and modeling [69]–[72], while shortcomings remain in critical aspects, such as syntactic and semantic correctness, consistency, and scalability. Moreover, the generation of mathematical models based on fms is a promising possibility, given the increasing mathematical reasoning and data analysis capabilities of fms [73]–[76]. However, as Ahn et al. [77] pointed out, existing fms still struggle with limited generalization in mathematical problems and exhibit weaknesses in mathematical reasoning. Furthermore, dt, as an innovative design approach, has attracted widespread attention from researchers, and the powerful knowledge representation and reasoning capabilities of fms are actively driving the rapid development of dts [78]–[80]. However, current fm technology has limitations and cannot fully support an fm-based dt solution [80]. Additionally, fms can support model execution by generating action code, evaluating runtime constraints, and synthesizing input data for simulation and testing [81]–[83]. However, existing approaches still have limited support for complex dependencies and system-level coherence, indicating that further exploration in this direction remains necessary [82].

5.2 Challenges and Research Opportunities↩︎

This section presents an overview of key challenges in the application of fms to cps software design and modeling, derived from the literature surveyed in the preceding section, as shown in Table ¿tbl:tab:design-modeling-challenges?. For each challenge, we outline associated research opportunities, research questions, and potential methodologies, as well as baselines and metrics for assessing the cost-effectiveness of these methodologies. The relationships between challenges and research opportunities are summarized from different aspects in Table ¿tbl:tab:modeling-opportunities?, ¿tbl:tab:fm-math-model-opportunities?, ¿tbl:tab:fm-dt-opportunities?, and ¿tbl:tab:fm-model-execution-opportunities?.

5.2.1 fm-based cps Modeling↩︎

Modeling CPS is highly challenging due to its multidisciplinary nature, where different domains have distinct modeling abstractions, semantics, and design constraints [68]. The modeling process is further complicated by heterogeneous and multimodal inputs (e.g., natural language requirements, system diagrams, control logic, and time-series sensor data), which are difficult to integrate using traditional modeling tools [84]. The modeling process is often manual, expert-dependent, and error-prone, leading to high costs, low consistency, and limited reusability [85]. fms, with their capabilities in language understanding, cross-modal reasoning, and knowledge integration, have the potential to alleviate these challenges. The following sections explore their roles and advantages in interactive modeling, embedded workflow support, and end-to-end model generation.

Interactive Modeling via FM-based Chatbot. fms can act as interactive modeling assistants through chatbot-based interfaces, supporting users via flexible and multimodal interactions combining textual queries, system diagrams, parameter tables, partial models, and simulation traces. This allows modelers to express design intent and uncertainty in richer and more natural ways across different modeling contexts. In exploratory cases, fm-based chatbot can interpret ambiguous requirements, identify candidate components, and suggest initial modeling patterns. In task-specific modeling phases, where the overall system goals are defined but implementation details remain open, the chatbot assistant can assist with more focused design questions. For instance, fms can analyze the partial input and provide design recommendations grounded in relevant system constraints. Through such interactions, fms enable iterative refinement and continuous support throughout the modeling lifecycle.

To improve the effectiveness of multimodal interactions, several research challenges must be addressed. In particular, domain-specific fine-tuning on annotated modeling dialogues that integrate text, diagrams, and structured data is essential, as prompt engineering alone is insufficient for handling context-rich, multimodal dialogue scenarios. Parameter-efficient methods, such as adapter tuning [86], Low-Rank Adaptation (LoRA) [87], and prefix tuning [88], provide practical approaches to align fms with domain-specific reasoning and terminology. Moreover, maintaining context across dialogue turns is essential. FMs must be able to remember previously defined components, unresolved design choices, and user intentions throughout the extended modeling process. However, while recent memory enhancement techniques, including long-term memory mechanisms [89], continuous memory encoders [90], and cross-modal fusion modules [91], show promise, they are designed for open-domain conversation and fall short in capturing the structured dependencies and persistent references typical of cps modeling workflows. To support chatbot-based modeling assistants, future research should therefore focus on developing memory architectures tailored to interactive modeling, capable of retaining long-term context, providing semantic grounding across multi-modal inputs (e.g., diagrams and textual descriptions), and tracking the evolution of cps representations across modeling stages.

Embedded Support in the Modeling Workflow. fms can be embedded directly into cps modeling environments to provide structured, real-time, and knowledge-aware assistance throughout the modeling lifecycle. Unlike traditional software systems, cps design requires integrating domain-specific knowledge, such as physical dynamics, control theory, hardware constraints, and communication protocols, and managing inherently multimodal inputs, including natural language descriptions, architecture diagrams, simulation traces, and structured configuration data. As modelers edit diagrams, parameters, or behavior blocks, fms can observe these actions, interpret partial models, and offer context-aware suggestions grounded in engineering principles. These capabilities help ensure modeling decisions remain consistent, valid, and aligned with cross-domain requirements. Beyond validating and refining models, fms can support partial automation of code generation tasks. fms can produce corresponding modeling constructs in SysML, Modelica, or similar languages and complete missing guards or transitions in behavioral models, infer likely parameter values, or auto-generate interface definitions based on previously defined components and their dependencies. This is especially useful when the modeler is not deeply familiar with a particular modeling language or domain formalism, reducing entry barriers and enabling broader participation. To enable such embedded support, fms should be fine-tuned on cps-specific multimodal artifacts and integrated with modeling tools to access model state and inject suggestions dynamically. Crucially, fm outputs must be verified using simulation, constraint checking, or formal analysis to ensure semantic fidelity to physical and control constraints.

End-to-End Model Generation from Natural Language. With increasing fm capabilities, fully automated end-to-end model generation from natural language inputs is becoming feasible in certain structured domains. Given a well-defined requirement specification, an fm can autonomously identify relevant system components, infer their relationships, and organize them into coherent structural and behavioral representations. By interpreting such inputs holistically, fms could assist in rapidly constructing high-level models that reflect both the functional intent and the underlying physical constraints of the system. For example, a natural language description of a mobile robot tasked with navigating through dynamic environments could be translated into a structural model with sensor modules (e.g., LiDAR, cameras), motion control components, path planning logic, and obstacle avoidance behaviors.

Despite this potential, realizing robust and generalizable end-to-end CPS model generation remains challenging. fms must be fine-tuned on large, high-quality datasets that include multimodal and domain-specific artifacts, such as annotated requirement texts, formal specifications, model diagrams, and simulation logs, to better capture modeling semantics, domain conventions, and language syntax. Verification mechanisms are also critical. Without feedback from simulation, constraint checking, or structural validation, generated models may appear correct but fail under execution. In addition, techniques such as prompt engineering and human-in-the-loop refinement can improve generation robustness and adaptability to diverse modeling scenarios. For CPS in particular, future research should also focus on handling hybrid system representations (e.g., discrete and continuous models), ensuring physical realism, and aligning the generated models with both functional and physical domain constraints.

5.2.2 fm-based Mathematical Model Generation.↩︎

Mathematical modeling plays a critical role in CPS design, especially for capturing continuous dynamics, control rules, and system constraints. However, this task presents unique challenges for fms. First, cps systems span multiple domains, e.g., mechanics, thermodynamics, and signal processing, each with its own modeling abstractions and mathematical formalisms [92]. Second, the source data used for modeling can be highly heterogeneous, including time-series signals, physical measurements, structured design parameters, and textual specifications [93]. Interpreting and transforming this data into accurate mathematical expressions requires strong analytical, symbolic, and numerical reasoning skills. Finally, CPS models must adapt to changing system requirements or environmental conditions [94], placing additional demands on FMs to generate adaptive or parameterized models that remain valid over system variations.

FMs have shown preliminary capabilities in symbolic mathematics, numerical reasoning, and code generation, offering potential to assist in CPS mathematical modeling [73], [95]. However, existing models still face limitations in reasoning across multiple physical domains, understanding diverse data types and formats, and adapting models to changing system conditions [96], [97]. To address these challenges, future work should focus on enhancing the mathematical reasoning capabilities of FMs, either by fine-tuning on domain-specific modeling tasks or by developing specialized FM variants with stronger analytical foundations. In addition, integrating external tools such as symbolic solvers, simulation engines, or hybrid neuro-symbolic reasoning modules may help bridge the gap between abstract model generation and executable system design. To handle dynamic changes in CPS environments, future work should explore how FMs can incrementally update mathematical models, e.g., adjusting equations or constraints, without regenerating them entirely, ensuring responsiveness and model consistency. Moreover, realizing robust fm-supported mathematical modeling will require new benchmarks, evaluation metrics, and co-designed workflows that reflect the complexity of real-world CPS scenarios.

5.2.3 fms for cps Digital Twin Creation.↩︎

Digital Twins (DTs) have become increasingly important in supporting the design of CPSs, offering simulation, analysis, and early validation capabilities before physical deployment [98], [99]. Lutze [100] highlights that DTs can enhance software design and lifecycle management by improving traceability, adaptability, and support for agile development compared to traditional V-model [101] approaches. Besides, model-based DTs and digital threads can transform model-based design processes by improving traceability, interoperability, and tool integration [98]. However, creating effective DTs during the CPS design phase remains a complex and resource-intensive task. First, constructing DT models often requires significant manual effort using modeling languages such as SysML or OpenModelica, demanding deep domain expertise and limiting scalability [102]–[104]. Second, designing DT capabilities, such as predictive analytics or anomaly detection, typically depends on hand-crafted AI models, which lack automation and generalizability [105]. Finally, each CPS design usually necessitates a custom-tailored DT, resulting in high development cost and limited reuse [106], [107]. FMs, with their ability to generalize across tasks and domains, present a promising solution to reduce manual effort, enhance model reusability, and accelerate CPS design workflows. In the following, inspired by the work of Ali et al. [80], we explore how FMs can support DT creation during CPS design: (1) assisting in the generation of DTs, and (2) fine-tuning FMs as DTs within the design loop.

Generation of Digital Twins. During the design phase, FMs can assist in automatically generating both the DT models and their capabilities. For instance, language-based FMs can suggest or auto-complete structural elements of DT models in SysML or OpenModelica based on design documents or system specifications, thereby accelerating the creation of executable DT representations of CPS designs. They can also act as interactive assistants, guiding engineers not only in building simulation environments but in shaping DTs that faithfully mirror both the structure and behavior of the target system. On the DT capability side, FMs can help generate or configure AI components that enhance DT functionality, such as predictive simulation or early fault modeling, ensuring that the DT evolves beyond a static model into a dynamic DT representation of the CPS during design. In this way, FMs do not simply aid modeling tasks but directly contribute to reducing the manual effort required to construct high-fidelity DTs during CPS design.

Future work should focus on adapting FMs not only to modeling languages but also to the broader requirements of DT creation. This involves fine-tuning models on existing DT repositories, formal models, and CPS design specifications so that generated outputs align with both structural and behavioral aspects of DTs. Another promising direction is the automated translation between different modeling formalisms (e.g., SysML, OpenModelica, Simulink), which would support interoperability and enable DTs to integrate seamlessly across design environments. Beyond structural generation, research is needed on how FMs can support DT capability development, such as anomaly prediction, scenario realism assessment, and design-time optimization, thereby making DTs active assets in the design process rather than static models. Finally, human-in-the-loop approaches, where engineers iteratively refine FM-generated DTs, could ensure both fidelity and usability, positioning FMs as intelligent collaborators for creating high-quality DTs during CPS design.

Fine-tuned fms as Digital Twins. An advanced approach is to fine-tune an FM so that it directly serves as a DT during CPS design. Instead of generating separate models and capabilities, the fine-tuned model itself can represent the system’s behavior, predict outcomes, and support reasoning tasks. Depending on the context, such a model may function as the DT model (capturing structure and behavior), as the DT capability (providing predictions, anomaly detection, or optimization), or as an integrated whole-DT solution. For example, a fine-tuned FM trained on design specifications and domain data could simulate design scenarios, anticipate potential failures, or recommend optimizations, thereby embedding intelligent decision support directly into the CPS design workflow. This approach significantly reduces the need for explicit, manually maintained DT models, while enabling DTs that continuously learn and adapt as design data evolves.

Future directions for this case should concentrate on systematically exploring how fine-tuned FMs can operate as design-time DTs with sufficient fidelity, efficiency, and interpretability. A key line of research is to establish guidelines for selecting suitable FMs for different CPS design contexts, whether text-based LLMs for processing specifications, vision-language models for analyzing simulation environments, or multimodal models for integrating heterogeneous design data. Another priority is ensuring that fine-tuned DTs can operate under design constraints, such as limited computation resources, real-time reasoning needs, and safety requirements, while still providing actionable feedback to engineers. Moreover, verification and validation techniques must be adapted to FM-based DTs, including methods to assess their fidelity in representing CPS behavior and their trustworthiness in safety-critical scenarios. Finally, hybrid approaches, where fine-tuned models complement but do not fully replace explicit DT representations, may provide a balanced path forward, combining the automation and adaptability of FMs with the transparency and control of traditional DT models to better support early-stage CPS design decisions.

fms for dealing with uncertainty in DTs of cpss. Uncertainty is an inherent characteristic of cpss arising from various sources such as hardware, software, and environmental factors [108]. A critical challenge in operating cps DTs is maintaining their fidelity by precisely modeling and incorporating these uncertainties [51]. Addressing this challenge requires prior knowledge of various aspects, including sources of uncertainty, their impacts, and their probability distributions [109], [110]. However, a lack of comprehensive knowledge about uncertainties often limits the ability to operate high-fidelity DTs effectively [51]. With recent advances in identifying uncertainties with LLMs [52], foundation models offer the potential to take this capability even further. Building on this, foundation models can be leveraged to develop advanced techniques that utilize diverse cps data and artifacts to derive uncertainty sources, assess their impacts, and estimate probability distributions. Another promising direction is fine-tuning foundation models to develop risk models that enhance DTs by proactively integrating and managing uncertainties [80]. Moreover, uncertainty is intrinsic in foundation models due to their non-deterministic nature. For DTs developed using foundation models (e.g., foundation models fine-tuned as DTs [80]), it becomes essential to quantify and manage this uncertainty to ensure their fidelity. As highlighted by [80], this represents a significant research direction, focusing on techniques to effectively assess and quantify uncertainty in foundation models when used in the creation of DTs.

5.2.4 fms for CPS Model Execution Support.↩︎

Executable models are essential for verifying CPS behavior, supporting simulation, control validation, and early-stage testing [111]. Unlike traditional software execution, CPS model execution involves hybrid dynamics, real-time constraints, and multimodal inputs, requiring systems to react to sensor data, physical events, and control feedback [112]. FMs have the potential to support this process by assisting both the execution of model logic and the generation of realistic input data, making it a promising direction for future exploration. In this section, we discuss two key aspects: (1) enabling FMs to support executable models via code generation and runtime evaluation, and (2) using FMs to synthesize realistic data to drive model execution.

Executable Model Code Generation and Runtime Evaluation. fms can support the execution of executable models created for cpss, such as UML state machines, by generating necessary low-level code and evaluating model constraints during runtime. Unlike high-level model construction, executable models must define fine-grained behavioral logic, including transition actions, guard conditions, event handling, and internal state updates. These models also need to account for domain-specific constraints, such as timing requirements, safety invariants, and control logic correctness. To this end, fms have the potential to generate action code for transitions, complete missing guards, and identify violations of runtime constraints through behavioral reasoning. For example, given a UML state machine describing a mobile robot’s motion controller, an FM could synthesize missing actions like obstacle avoidance or turning behavior, and evaluate whether speed limits or safe stopping conditions are respected during simulated execution.

Although some studies have made preliminary explorations into generating executable simulation code from FMs [81], [82], this research area has not yet been sufficiently investigated. Supporting execution-time reasoning requires that fms not only understand the syntax and semantics of executable modeling languages, but also reason about temporal behaviors, control dependencies, and domain-specific rules. Beyond fine-tuning, future work must also consider tighter integration with executable modeling tools and mechanisms for maintaining long-range behavioral context during model execution. Moreover, model execution should be coupled with verification strategies, such as integrating FM support with existing simulation environments, runtime monitors, or formal checkers, to enable real-time validation of system behavior and ensure correctness and traceability.

Realistic Execution Data Generation. Many CPS models rely on real-world data inputs to simulate behavior or validate design decisions. FMs, especially those with multimodal capabilities, have the potential to generate realistic synthetic data, such as LiDAR point clouds, camera images, or sensor signals, to drive model execution. This is particularly relevant in domains such as autonomous vehicles, where the model’s response to complex sensor input is a core part of system validation. For instance, an FM could generate a video simulating a pedestrian crossing scenario, which can be processed by a CPS perception module and used to test the behavior of an executable system model.

However, generating high-fidelity and semantically valid data is a nontrivial task. Synthetic inputs must not only appear realistic but also cover a broad distribution of scenarios (e.g., lighting conditions, obstacle types, sensor noise) and must be consistent with the model’s expected context. Furthermore, systematic evaluation metrics for synthetic data quality, such as coverage, diversity, and realism, are still lacking, especially for CPS use cases. Future directions include guiding FM-based data generation using simulation feedback or real-world data distributions to improve realism and contextual fidelity, developing domain-specific metrics for evaluating the quality of generated data, and integrating human-in-the-loop feedback to guide data synthesis.

6 Foundation Models for Software Development↩︎

This section explores how fms can aid software developers improve their productivity. Section 6.1 introduces the background and an overview of the state of the art in the use of fms for Software Development in the context of cpss. Upon this foundation, Section 6.2 identifies the key challenges and outlines how these challenges can be addressed through a set of relevant research questions.

6.1 Background and State of the Art↩︎

fms are known for their code generation capabilities [113]. According to Sam Altman, internal OpenAI studies report that llms can increase software engineering productivity by threefold [114]. These capabilities have been continuously enhanced, particularly with the introduction of reasoning models such as OpenAI’s o3-mini [114] and DeepSeek-R1 [115], which demonstrate improved performance in generating high-quality and logically consistent code.

Despite this rapid progress, most existing fm-based approaches have been evaluated primarily in general-purpose software development contexts [116]–[118]. In contrast, cpss exhibit distinct characteristics, including tight integration with physical processes, real-time constraints, safety-critical requirements, and usually rely on proprietary or highly specialized programming languages. These characteristics limit the direct applicability of state-of-the-art fms to cps software development.

6.2 Challenges and Research Opportunities↩︎

While fms offer significant potential to enhance cps software development, their effective adoption introduces several critical challenges. As summarized in Table ¿tbl:tab:sd-challenges?, we identify four core directions that must be addressed to enable the systematic integration of fms into CPS software engineering processes. Furthermore, Table ¿tbl:tab:fm-sd-opportunities? outlines how fms-based approaches can be leveraged to tackle these challenges, highlighting research opportunities associated with each direction.

6.2.1 Safect-Critical Code Development↩︎

Many cpss are safety-critical, and thus must be certified, which also requires software development tools to be qualified by a regulatory body [119]. For instance, when compiling safety-critical code, the employed compiler should have been qualified by a functional safety regulatory agency. Different qualification categories exist [119], and fm-based code generation would require the most strict one as the code it generates might be buggy.

Further, each new version of the tool has to be re-qualified. All this introduces new challenges and aspects to consider when adopting fm-based approaches. First, the tools need to carefully follow standards’ rules; this can involve aspects like having redundancy for code generation, or checking that the fm is robust to minor prompt changes. Second, standards need to be redefined and tailored to reconsider fm-based techniques in developing safety-critical code; this needs interdisciplinary collaborations between regulators and AI experts to see how to adapt standards (e.g., forcing the use of uncertainty to monitor the FM’s confidence when generating code). Lastly, while we see rapid advancements of llms, standards establish that each new version of the tool needs re-qualification; qualification processes are lengthy and expensive, which poses new challenges to fm developers to make their new model versions re-qualified. Regulatory bodies should also research lighter processes to re-qualify fine-tuned LLM versions with enhanced code generation capabilities.

6.2.2 Domain Specific Code Generation↩︎

Unlike mainstream general-purpose programming languages (e.g., Python, C), many cpss (e.g., robots and programmable logic controllers) rely on their own proprietary coding language. These languages typically lack large, openly available code corpora, reducing the effectiveness of FMs trained predominantly on general-purpose code. A promising research direction is the automatic synthesis of domain-specific datasets, either by generating synthetic examples from formal language specifications or by mining industrial repositories under strict confidentiality constraints. Transfer learning techniques can further bridge this gap, for instance, by transferring representations learned from general-purpose programming languages to CPS-specific ones via adapter modules. Another research direction involves human-in-the-loop approaches, where FMs collaborate with engineers during code generation, creating curated datasets for future fine-tuning. This interplay between engineers and FMs can also help ensure adherence to domain standards, such as MISRA-C in automotive or DO-178C in avionics [120].

6.2.3 CPS-specific Code Generation↩︎

Much of cps software, initially developed using simulation-based techniques, is later redeveloped near operation due to the reality gap. Examples include engineers reprogramming a robot, debugging CNC code along with the physical machine, or changing the control code of trains in new facilities. Cloud access is usually unavailable near the operation. Moreover, many companies hesitate to submit their code externally. We envision a direction for researching small language models of a few million parameters that can be directly deployed on the engineers’ PCs and can still generate trustworthy cps code. Lightweight adaptation methods such as quantization, pruning, or knowledge distillation can play a central role in enabling such deployment. Furthermore, continual on-device fine-tuning would allow these models to adapt to specific CPS environments without retraining from scratch. Running FMs locally also mitigates privacy risks by preventing sensitive code from leaving the organization’s infrastructure.

6.2.4 Code Review↩︎

Besides code generation, fms can serve as effective assistants for reviewing (safety-critical) software to be embedded on CPSs. Traditional static analysis tools are well-established, but they are usually limited to predefined rules and may overlook subtle issues arising from design choices, concurrency, or integration with hardware. In contrast, fms can complement such tools by providing semantic-level reasoning, flagging suspicious coding patterns, or suggesting alternative implementations that adhere to safety standards. An additional particularity of cps development is its multidisciplinary nature: code is not only written by software engineers but also by electrical engineers, roboticists, control engineers, or mechanical engineers who may lack deep expertise in software engineering practices. In this context, fms-based reviewers can act as mediators, helping non-software experts align their code contributions with safety and coding standards.

A promising direction is hybrid review pipelines, where fms are combined with rule-based analyzers: while analyzers ensure compliance with standards such as MISRA-C or IEC 61508, fms can highlight higher-level design flaws or undocumented assumptions (e.g., identifying architectural anti-patterns such as bypassing safety wrappers to access actuators). Another important research path is explainability. For FM-based code review to be trusted in safety-critical cps, engineers must understand why the model flags certain lines of code, which requires human-readable rationales or links to known standards violations. Finally, given the safety-critical nature of CPSs, their projects usually demand traceability. Consequently, fms should not only provide feedback but also generate structured reports that can be archived as part of the certification evidence.

7 Foundation Models for cps Testing↩︎

This section explores how FMs can advance cps testing. Section 7.1 introduces the background and the state of the art in the use of FMs for cps software testing, while Section 7.2 identifies critical challenges in testing cpss and outlines research opportunities and research questions to address them.

7.1 Background and State of the Art↩︎

CPS integrates hardware and software components that directly interact with the physical environment and is usually designed for safety-critical applications, such as transportation, healthcare, and industrial control. Thus, rigorous testing of CPS is crucial to ensure safe and reliable operation under various conditions, and simulation-based testing is a typical way that provides a controlled environment to evaluate system behavior under various conditions. Traditional simulation-based testing approaches employ techniques such as meta-heuristic search and reinforcement learning to generate critical conditions to test CPS [121], [122]. However, such approaches show limited effectiveness when it comes to bridging the reality gap [123], managing computational resources, and ensuring the explainability of the detected failures [124].

fms have been applied to support cps software testing across several domains, including test generation for avs [125]–[129], testing of decision-making policies in robotics [130], [131], and industrial cps [13], [132]. llms have demonstrated capabilities in generating interpretable test scenarios and extracting relevant information from unstructured sources. Besides, vlms offer enhanced capabilities for synthesizing multimodal test inputs that combine visual context with semantic information [132], [133]. Despite these advances, applying fms to cps testing introduces several notable challenges, which, together with key research opportunities, are discussed below.

7.2 Challenges and Research Opportunities↩︎

While fms have shown potential in advancing cps testing, their effective adoption introduces several critical challenges. As summarized in Table ¿tbl:tab:testing-challenges?, we identify six core challenges that must be addressed to enable the systematic integration of fms into cps testing processes. For each challenge, we outline research opportunities, research questions, and potential methodologies, as well as baselines and metrics for assessing the cost-effectiveness of these methodologies, which are presented in Tables ¿tbl:tab:fm-testing-opportunities?, ¿tbl:tab:fm-testing-selection?, ¿tbl:tab:fm-testing-oracle?, ¿tbl:tab:fm-testing-reality?, and ¿tbl:tab:fm-testing-interactive?.

7.2.1 FM-based Test Generation and Test Support↩︎

FM-based Test Generation and Test Data Augmentation. cps test generation is time-consuming and resource-intensive, particularly due to the complexity and interdependencies inherent in cpss. Zhou et al. [134] investigated and identified key challenges for testing CPS. Despite the rapid development of advanced testing methods, bottlenecks remain, especially for complex CPSs requiring reliability, security, and resilience. Key challenges include managing large and complex state spaces, addressing state space explosion, handling uncertainty during testing, ensuring real-time performance, and designing effective test oracles. fms offer a promising solution by automating and enhancing test generation [135], [136]. They can generate diverse, high-fidelity test scenarios that cover edge cases, simulate realistic sensor inputs and environmental conditions, and model system interactions, addressing state space and uncertainty challenges. fms can also adapt test sequences in real time to meet timing constraints and assist in designing test oracles by predicting expected system outputs based on learned behaviors. For example, fms can generate a diverse set of high-fidelity test scenarios that cover corner cases and rare events, helping to manage large and complex state spaces and mitigating state space explosion. By leveraging their ability to model multimodal data, fms can simulate realistic sensor inputs, environmental conditions, and system interactions, which allows for comprehensive uncertainty modeling. In addition, fms can be used to dynamically generate and adapt test sequences in real time, supporting testing under strict timing constraints and ensuring that system responses are evaluated within operational deadlines. Besides, fms can enrich the test suite by performing noising or denoising on test data [137], [138]. They can simulate realistic perturbations, such as sensor inaccuracies or environmental disturbances, to create challenging noisy data. While fms can also reconstruct clean signals from corrupted inputs, filtering out irrelevant noise while preserving critical features. This dual capability would allow generating challenging (noisy) and clean (noise-free) test data for cps, thereby broadening the spectrum of conditions under which the cps is evaluated.

Integrate FMs with Traditional Testing Techniques. Ahn et al. [77] note that FMs exhibit limited performance in numerical problem-solving, often struggling with accuracy, consistency, and complex calculations. When it comes to CPS testing, numerical reasoning is critical for generating concrete configuration parameters from logical scenarios defined by parameter ranges. Concrete scenarios with specific parameter values from logical scenarios defined by parameters and parameter ranges are crucial for the precise simulation of CPS dynamics and its environment. This limitation is critical because precise simulation of cps dynamics and their environment relies on accurate numerical instantiation. Consequently, integrating traditional techniques—such as search-based methods and numerical solvers—with fms becomes necessary to provide a comprehensive solution. These techniques can complement the capabilities of fms by enabling precise numerical computation, enhancing the fidelity of cps simulations. For example, integrating fms with search-based algorithms offers a promising hybrid approach for CPS testing. fms can generate high-level logical scenarios from requirements and provide heuristics to guide test generation, while search-based algorithms systematically explore parameter spaces to produce concrete configurations and ensure coverage of edge cases. This combination leverages the reasoning and generative capabilities of fms while ensuring numerical accuracy and systematic exploration, enhancing both the fidelity and effectiveness of CPS testing. Such a hybrid approach would effectively leverage logical reasoning and mathematical computing abilities, addressing the limitations of fms in numerical problem-solving. However, there is a need for more research to build such hybrid approaches.

Transform Test Case Specifications to Test with FMs. Test case specifications (TCSs) play a critical role in cps software testing [139], [140], as they provide structured descriptions of inputs, expected behaviors, and environmental conditions. TCSs are often written in natural language; thus, manually transforming TCSs into executable test cases is time-consuming, error-prone, and requires significant coding effort and domain knowledge. Moreover, manual approaches fail to capture the diversity required to validate complex systems in dynamic environments. To address this, automated transformation from TCSs to test cases has been studied. For example, Tao et al. [141] proposed RTCM, a restricted test case modeling language with templates, rules, and keywords, along with a tool that generates either manual or executable test cases from structured TCSs. To support the runtime cps test, Shi et al. [142] propose LiveTCM, which interacts with the cps and its environment to generate and execute TCSs efficiently. fms offer promising opportunities for further automating the transformation process by interpreting natural-language TCSs, reasoning about multimodal TCSs, and generating diverse executable test cases with minimal human intervention. fms can also adapt to evolving system behaviors and environmental conditions, potentially improving coverage, fault detection, and efficiency in CPS testing. For example, fms can translate high-level TCSs of environmental conditions, control signals, and system properties into executable scripts suitable for runtime verification. In addition, fms can support diversity and scalability in test generation by synthesizing variations of test cases from the same TCS. This includes altering environmental parameters, introducing disturbances, or modifying mission goals, thereby producing a richer set of test scenarios to assess CPS robustness under dynamic and uncertain conditions. By leveraging fine-tuning or domain adaptation, fms can also learn from historical test data and execution outcomes, continuously refining their ability to generate effective test cases over time.

7.2.2 FM-based Test Selection, Minimization, and Prioritization↩︎

Regression testing for cps software updates helps detect bugs in the modified version of the software. However, as test suites expand, redundant and overlapping test cases can lead to inefficiencies and increased execution times [143]–[145]. This issue is particularly acute for CPS, where software interacts continuously with dynamic, stochastic physical environments, making exhaustive testing impractical. Therefore, it is crucial to adopt approaches that can efficiently manage and optimize regression test suites. While traditional approaches demonstrate good performance [146]–[148] in test selection, minimization, and prioritization, fms offer a promising way forward by automatically analyzing the characteristics to 1) identify and prioritize the subset of tests most likely to detect failures early in the development cycle, 2) minimize the overall suite while preserving fault-detection effectiveness, and 3) optimize test execution order to maximize early fault detection and overall testing efficiency. For instance, fms can analyze patterns across historical test executions, system behaviors, and environmental conditions to predict which tests are most likely to detect failures, enabling smarter prioritization that accelerates feedback to developers. They can also reason over multimodal test data, such as sensor readings, control signals, and simulation traces, allowing for more accurate identification of redundant or overlapping tests and more effective test suite minimization. By learning from past executions, fms can adapt test selection, minimization, and prioritization strategies to evolving CPS behaviors and complex operating environments.

7.2.3 FM-based Test Oracle Support↩︎

The lack of test oracles or the absence of precise test oracles is common when testing cpss [149], [150], as in many other software systems [151]. To this end, research is needed to explore the use of fms for generating test oracles, based on the assumption that they have learned from a wide range of information and can assist testers in devising test oracles for cps testing. Moreover, to further support test oracles in the context of differential testing, one of the key methods for addressing the test oracle problem, fms can be customized to serve as a surrogate implementation of a cps. This is an open research area that requires further investigation.

7.2.4 Evaluate and Bridge Reality Gap with FMs↩︎

Testing cps usually involves a simulated testing environment or requires synthesizing test inputs (e.g., images and videos), but ensuring the test scenarios or synthetic test inputs accurately represent all facets of the physical systems and environments is still an open problem. Salvato et al. [152] surveyed reinforcement learning for robot control, focusing on the reality gap problem that hinders sim-to-real transfer. They reviewed existing approaches to mitigate reality gaps, discussed their limitations, and outlined promising directions for future research. In the AV domain, Stocco et al. [153] investigated the sim-to-real gap in AV testing, showing that simulation-based testing can overlook critical reality issues, while also identifying situations where physical testing is unnecessary because simulation results transfer reliably. fms offer promising opportunities to evaluate and bridge this gap. By leveraging their ability to model multimodal data, fms can assess the realism and fidelity of test scenarios and quantify how closely simulated conditions align with real-world environments [133]. They can do this by comparing visual, sensory, and contextual features, as well as by checking higher-level patterns such as consistent behavior, realistic environmental dynamics, and rare corner cases. This enables a more thorough evaluation of whether simulated inputs capture the complexity, variability, and uncertainty of real systems, helping to pinpoint gaps in test coverage and refine simulation models for greater accuracy. Furthermore, by fine-tuning fms with domain-specific naturalistic data and environmental conditions, specialized fms can be developed to generate synthetic inputs that more faithfully represent real-world scenarios. These models can capture complex interactions among system components, environmental variations, and rare or extreme events, enabling the creation of diverse and realistic test cases. Such fm-generated inputs not only improve the coverage and representativeness of testing but also reduce reliance on costly or risky physical trials, supporting safer and more efficient evaluation of CPS software.

7.2.5 FMs for Testing Interactive Control Systems↩︎

Interactive control systems, often referred to as human-machine interfaces, graphically display real-time data from the physical components of cps to enable operators to monitor, interact with, and control these systems effectively. In safety-critical domains, such as unmanned aerial vehicles, testing these systems becomes essential to support informed decision-making and ensure the safety and reliability of cps operations. The existing techniques in the literature are based on traditional image processing methods, such as computer vision and optical character recognition [154], [155]. These approaches often require significant manual effort in generating test scripts, executing simulations, processing image data to achieve high prediction accuracy, and evaluating test verdicts [155]. Another significant challenge lies in adapting these approaches to new or evolved display systems with updated controls, which typically requires considerable tailoring to existing methods or the development of entirely new solutions from scratch [156]. The advent of foundation models presents exciting opportunities to overcome these challenges. One promising direction is to develop LLM-based techniques to automatically generate test scenarios and executable scripts to test interactive control systems, such as [157]. Another direction involves developing VLM-based techniques to analyze test verdicts by comparing images and facilitating the evaluation process [158]. Furthermore, a direction worth exploring is leveraging multimodal foundation models to generate diverse data, such as images and videos, facilitating the testing of interactive display systems without relying on computationally expensive simulations.

8 Foundation Models for Debugging and Repair↩︎

This section examines how fms can support developers of cpss in debugging and repairing such systems. Section 8.1 first provides background and an overview of the state of the art in the use of fms for debugging and repair in the context of cpss. Building on this foundation, Section 8.2 identifies the key challenges and outlines how they can be addressed through a set of relevant research questions.

8.1 Background and State of the Art↩︎

fms, and in particular llms, have recently emerged as powerful enablers for automated debugging [159]–[162] and repair [163]–[169] due to their ability to learn from vast and heterogeneous datasets, including source code, execution logs, documentation, and other artifacts, as well as their multimodal capabilities. These models benefit from strong code understanding, language adaptability, and the ability to generate candidate patches based on learned patterns. A growing body of work explores llm-based agents that unify fault localization and repair into a single workflow [170], [171]. Such agents enhance traditional debugging approaches by leveraging llms to interpret error messages, analyze execution traces, and coordinate repair actions with greater flexibility. Their ability to understand and integrate outputs from both fault localization and repair stages represents a significant advancement over conventional, modular approaches.

In the context of cpss, the use of fms has only recently begun to emerge. Early work such as FixDrive [172] illustrates the potential of learning-based techniques for repairing cps-related software, particularly in autonomous driving systems. Nevertheless, the broader application of fms to cps debugging and repair remains largely unexplored. As highlighted by Valle et al. [173], achieving fully automated cps debugging and repair requires addressing several open challenges related to fault localization, scalability, testing cost, and repair guidance.

8.2 Challenges and Research Opportunities↩︎

While fms have demonstrated strong potential in traditional software domains, cps debugging and repair introduce additional complexity from the interaction of software, hardware, and physical environments. These characteristics create significant challenges, presented in Table ¿tbl:tab:debug-repair-challenges?, but also open new research opportunities, as can be seen in Table 1. In this section, we discuss four major challenges and outline how fms can contribute to addressing them.

8.2.1 Effective Fault Localization↩︎

Fault localization in cpss is particularly difficult because faults often arise from interactions between software components and physical processes rather than isolated code defects. Moreover, CPS testing is commonly performed at the system level, which limits the applicability of traditional fault localization techniques. This challenge is further exacerbated by long-running tests and the prevalence of flaky simulators, which can distort test results and undermine localization accuracy [174]. fms offer promising opportunities to address these issues by enabling multi-modal reasoning across software and physical layers. Recent work such as SemCoDer [175] demonstrated that llms can be trained to perform deep semantic reasoning over source code. When combined with time-series or multi-modal models [176], fms could correlate failing code with sensor readings or system states observed prior to a failure. For example, a model could jointly analyze source code, execution logs, and sensor outputs to identify suspicious code.

Additionally, llms have shown strong capabilities in interpreting error logs and execution traces for software debugging [177]. Extending these techniques to CPS-specific errors requires training or fine-tuning models on CPS tracing datasets that include code, textual logs, and time-series data. A recent survey on this [176] suggests the current fms can spot when a sensor starts failing and link that issue with the source code. Another promising direction is the integration of domain knowledge, such as physical constraints or safety requirements, into fms. Prior work indicates that llms can incorporate semantic rules during reasoning [175], which could help enforce known safety properties during fault localization. Finally, combining fms with traditional fault localization techniques may further improve effectiveness. For instance, Valle et al. [173] propose a time-aware spectrum-based fault localization approach that weights statements based on their temporal proximity to failures. In this setting, fms could help in the localization with semantic insights extracted from error messages and execution traces.

8.2.2 Fixing a Wider Range of Bugs↩︎

Traditional apr methods are only able to repair certain types of bugs. For instance, in the context of cpss, search-based methods have been proposed to repair misconfigurations [144], stateflow models [178] and feature interaction failures [145]. In contrast, cpss cover a wide range of bug types [179], such as, bugs in configuration files. As fms have been trained on massive datasets, we envision their combination with other techniques (e.g., search-based approaches), can significantly help repair a wide range of cpss bugs. One promising research direction is domain-specific fine-tuning, where fms are trained on cps-relevant codebases, such as Simulink models, PLC code or ROS programs. We conjecture that fine-tuning fms with CPS-specific codebases can help the model learn how control logic is typically structured, enabling it to suggest meaningful patches in these contexts.

Another potential research avenue is the development of hybrid repair strategies, i.e., strategies that combine generative capabilities of fms with search or constraint-based repair techniques. Valle [173] describes an approach in which llms act as mutators within a search algorithm: the search process maintains a population of partial patches and the llm is required to generate patches with the help of the information from failure diagnostics. This way, the fm is guided by reducing the search space of possible patch solutions. Another research direction we envision is the Multi-Agent Collaboration. For instance, a software repair agent could check code mutations for syntactic correctness and interface validity, while a configuration agent proposes some parameter adjustments. At the same time, an ML auditor could suggest model-centric mitigation such as fine-tuning or calibration. A coordinating agent would mediate between these proposals, resolving conflicts and designing focused validation plans that address the most critical scenarios of the system. For instance, Mapcoder [180], a multi-agent approach, has shown to be effective at code generation for competitive problem solving.

8.2.3 Cheaper debugging and repair↩︎

The need for testing cpss at the system level is frequently the slowest and most expensive part of cpss development process. An individual test can can take minutes to hours [181] to execute, require specialized hardware or human supervision, and must often be repeated across many initial conditions to exercise timing-dependent or rare faults [174]. This also is exacerbated due to the stochastic nature of such systems. The combination of time-dependant processes and high sensitivity to previous inputs and outputs, could lead the cps behave differently across several runs of the same input [174], [182]. The consequence is that the repair algorithms either cannot search deeply in the search space or consume prohibitive amount of resources.

fms offer a promising avenue to break this bottleneck by providing fast, informative substitutes for expensive tests and by focusing the execution of real tests where they are most useful. One direction is to develop multi-modal surrogate test oracles [183]. fms trained on historical pairs of short, compact descriptors (e.g., code diffs, scenario parameters) and long-test outcomes can learn to predict the test outcome, safe margins, or even full execution summaries with uncertainty quantification [184]. Integrating fms in the repair cycle as pre-filter permits the repair loop to discard many low-value candidates cheaply (i.e., the outcome of the surrogate model is that the tests may fail) and reserve the resources for the high-value changes that the surrogate models either ranks the plausible patch as passed or uncertain. Beyond acting as oracle substitutes, fms can also play a central role in handling the test flakiness when evaluating the behavior of cpss [174], [182]. This could be carried out by learning distributions over possible outcomes rather than just predictions. This way, fms can contribute to both efficiency and robustness in navigating the complex, probabilistic behavior space of cpss.

8.2.4 Guiding Repair algorithms↩︎

Search-based repair methods are only as effective as the fitness functions that guide their exploration [185]. Current apr approaches [186]–[188] use the number of failed and passed test cases as a fitness function. While this strategy can be effective in software domains with large, diverse test suites, it is problematic in cpss where the number of available test cases is limited and tests often provide little insight into the internal behavior of runs. As a result, the search process may become nearly random, wasting evaluations on uninformative or misleading fitness values.

fms provide an opportunity to design richer and more informative fitness functions that capture the behavior of cpss beyond simple pass/fail outcomes. By learning embeddings of multi-modal execution traces, including telemetry, control signals, timing logs, and environment states, fms can map repairs to continuous behavioral space. This reduces the risk of deceptive signals, where small changes partially satisfy a test but leave underlying faults unaddressed.Moreover, fms can support multi-objective fitness by predicting system-level properties relevant to cpss, such as stability margins, latency, or energy consumption. This enables Pareto-based repair strategies [189] that balance competing objectives rather than overfitting to a single binary score. Another advantage is that fms can also incorporate temporal aspects of faults, such as how long a fault has been active or whether early warning signs appear in execution traces. Large models trained on historical runs can learn to anticipate faults before they fully manifest, providing earlier and more informative fitness signals that guide search toward truly corrective patches. Finally, uncertainty-aware fms can actively direct exploration by signaling when predictions are unreliable. This makes it possible to prioritize scare tests for candidate patches that are either high promising or fall into regions of high uncertainty, ensuring that repair resources are allocated effectively.

9 Foundation Models for cps Evolution↩︎

This section investigates the role of fms in supporting the evolution of cpss. An overview of the necessary background and the state of the art is provided in Section 9.1. Subsequently, Section 9.2 identifies key challenges arising from cps evolution and corresponding research opportunities, outlining actionable research opportunities and potential directions for addressing these challenges.

9.1 Background and State of the Art↩︎

Recent advances in fms have shown promising potential for supporting software evolution tasks across the software lifecycle. Existing vision and survey papers highlight that current fm-based approaches, primarily through llms, can assist with several maintenance and evolution activities, including refactoring, migration, bug fixing, testing, documentation, and change understanding [116], [190]. Within model-based software engineering, early research investigates the use of fms for tasks such as model evolution, model generation, consistency checking, and traceability across models and code [71], [190]–[192]. These works argue that fms are well suited for evolution tasks that require reasoning over heterogeneous artifacts and global system context rather than isolated code generation. In the general software engineering domain, fms have been studied for code refactoring, demonstrating their ability to improve code structure and maintainability in legacy systems [193]. Other works investigate their use for API and library migration, showing support for framework upgrades and platform transitions that are central to software evolution [194], [195]. fm-based approaches have also been applied to automated program repair, where patches are generated based on failing tests or bug reports, supporting corrective maintenance [196], [197]. Furthermore, empirical studies show that fms can assist in unit test generation and improve test coverage, contributing to regression testing during software evolution [135], [198]. Beyond code-centric tasks, fms have been explored for evolution related artifacts such as commit messages, documentation, change impact analysis, and traceability recovery. These works indicate that fms can support change understanding by summarizing modifications, maintaining documentation consistency, and identifying dependencies across requirements, models, and code artifacts [199]–[205].

Despite these advances, existing FM-based approaches primarily target general-purpose software systems. Their applicability to cpss evolution remains limited, as they rarely account for CPS-specific characteristics (e.g., real-time constraints, hardware-software coupling) or support system-level reasoning across architectural layers and cross-module dependencies [206]–[208].

9.2 Challenges and Research Opportunities↩︎

This section identifies key challenges associated with the use of fms in supporting the evolution of cpss, as summarized in Table 2. For each identified challenge, we discuss corresponding research opportunities, including relevant research questions, potential methodologies, and baselines and metrics for evaluating the effectiveness of fm-based solutions. An overview of the relationships between challenges and research opportunities is provided in Table 3.

9.2.1 Automating cps Software Changes and Adjustments↩︎

Software changes (e.g., due to optimization, feature updates, or new hardware) introduce risks of system instability and compatibility issues, primarily due to hidden dependencies and lack of automated support [209]. Developers must manually navigate large and complex codebases, ensure compatibility, and prevent unintended behaviors (e.g., new change breaks existing functionality), making the process time-consuming and error-prone. Similar to GitHub Copilot [210], the future of fms in cps software evolution extends beyond simply suggesting code changes. Instead of relying on human intervention for execution, fms could be used as autonomous decision-making models capable of analyzing dependencies, predicting potential failures, and applying optimized updates without breaking existing functionality.

9.2.2 Scalable and Efficient cps Software Updates and Deployment↩︎

Managing software updates is challenging, especially in cpss that require continuous operation and minimal downtime. Updates must be carefully coordinated to ensure compatibility with existing components, avoid disruptions, and prevent failures. In complex cpss, manually handling dependencies and version control slows the process and increases the risk of errors [211]. To this end, more research is needed to explore to what extent fms can improve software update processes by automating update scheduling, resolving dependencies, and optimizing deployment strategies.

Beyond automation, fms could also support adaptive decision-making, where update strategies are tailored to the operational context (e.g., prioritizing urgent security patches). Another promising direction is the use of fms to simulate and validate updates before deployment, thus reducing the risks of unexpected failures [212].

9.2.3 Self-healing and Adaptive Mechanisms↩︎

cps software failures often require manual intervention, causing downtime and increasing maintenance costs. Traditional recovery methods, such as system reboots or checkpoint rollbacks, can restore functionality but do not prevent recurring failures or adapt to new issues [213]. In this context, fms can assist by analyzing cps behavior, detecting faults early, and providing real-time insights to support adaptive decision-making. While cpss can integrate predefined recovery strategies, fms enhance these processes by identifying patterns, diagnosing potential issues, and suggesting corrective actions, reducing reliance on manual intervention [214].

9.2.4 Automated cps Test Evolution and Quality Assurance↩︎

Keeping cpss reliable requires testing to evolve alongside new updates and features. Both manual and automated cps testing methods often fall behind rapid software changes, requiring significant human effort to create and/or update test cases and maintain regression suites. To this end, fms are needed to generate test cases that reflect software changes and maintain realism and relevance [133]. Moreover, fms can automatically update regression tests, improving test coverage and reliability and enhancing the overall software quality. All these aspects require further research.

In particular, future work should investigate how fms can be seamlessly integrated into continuous integration and deployment pipelines, enabling automated and on-demand test evolution. Another promising direction is to explore prioritization strategies where fms guides which test cases to execute first, based on criticality or likelihood of failure. Furthermore, applying fms to capture not only functional but also non-functional aspects (e.g., performance, energy usage) could extend their value for cps quality assurance [215].

9.2.5 Proactive Evolution of cpss↩︎

Traditionally, the evolution of cpss has been reactive: updates and adjustments are introduced in response to failures, changing requirements, or environmental shifts. This leads to costly downtime, delayed adaptations, and vulnerability to unforeseen conditions. fms open the possibility for proactive evolution, where systems anticipate problems and adapt before they escalate into failures. Recent work has already moved toward proactive monitoring and diagnosis in related areas, including predictive anomaly detection [216], invariant learning for fault detection [217], and llm-based fault localization [218]. However, these efforts stop at detection and diagnosis. By leveraging runtime data and historical patterns, fms could support predictive feedback or control loops that suggest timely adjustments and updates to keep systems stable and reliable. To our knowledge, the idea of proactive evolution remains largely unexplored.

9.2.6 Long-Term Maintenance and Compliance↩︎

cpss software must evolve to meet new regulations, security threats, and even hardware changes, but tracking and applying updates is often complex. fms can assist by analyzing compliance requirements, suggesting necessary updates, and automating documentation. Taking the example of a medical dispenser [219], an fm can review and update software to comply with evolving healthcare standards while adapting to customized settings. Beyond healthcare, fms can be applied in other cps domains: AVs can update related modules (e.g., perception) to meet safety certification standards, while industrial robots may need reconfiguration to satisfy both safety and ethical regulations. In all cases, fms reduce manual effort and improve traceability, thereby supporting maintainability, regulatory compliance, and trustworthiness of cps.

9.2.7 Evolving Digital Twins of cpss↩︎

A DT of a cps typically consists of a DT model and DT capabilities [80]. Various approaches have been explored in the literature to construct the DT model and its capabilities, which can be developed using the same process or through different ones. For instance, the DT model might be created using one type of machine learning (ML) model, while its capability could be developed using model-driven engineering or another ML model [219], [220]. When a cps evolves, evolving its DT (including both the model and its capabilities) becomes a significant challenge, particularly when the DT model and capabilities are developed using different methods [221], [222]. Recent advances have employed advanced AI techniques, including meta-learning [223] and transfer learning [224], to address these challenges. However, these approaches often require substantial amounts of data for training and fine-tuning ML models to ensure that the evolved DT precisely reflects the evolved cps. This challenge becomes even more critical in domains like ADS and robotics, where multimodal data—visual, sensor, and environmental inputs—is required to be processed. Similar to DTs built using foundation models (Section 5), these models can play a significant role in evolving DTs [78]. One potential direction is to use updated cps artifacts and foundation models to generate multimodal data for evolving DTs built using ML models [225]. For the foundation model-based DTs, a potential direction is to develop techniques that use prompt-based fine-tuning to evolve those DTs based on the changed cps artifacts, such as requirements, code, or visual data [80]. Building on this, another key direction is the exploration of auto-prompting and automated fine-tuning methods [226] to enable the seamless and automated evolution of foundation model-based DTs, further reducing manual effort.

10 Cross-cutting Challenges when Applying fms↩︎

In addition to the challenges discussed in each phase, adopting fms in CPS software engineering faces several cross-cutting challenges. Table ¿tbl:tab:fm-common-challenges-taxonomy? classifies these challenges into six dimensions. These dimensions were selected to provide a holistic view of fm adoption barriers, drawing on established frameworks for FM development [227] and covering challenges inherent to fm technology, safety and certification, practical deployment constraints, concerned with human and organizational factors, ethical and privacy concerns, and environmental sustainability considerations. For each dimension, we identify key challenges and outline corresponding research opportunities to guide future work. Note that we provide relevant references from which we identified the challenges as well as research opportunities, where applicable.

10.1 Technical Aspects↩︎

10.1.1 Key Challenges↩︎

A fundamental concern is that fms are prone to generating outputs that may be uncertain, incomplete, or incorrect, often referred to as hallucination [332]. In the CPS context, such hallucinations can complicate decision-making and validation by making it difficult to determine whether a generated artifact is reliable or requires further human review. Addressing these challenges is critical for ensuring reliability, safety, and traceability across the CPS software lifecycle. Another concern is fms sensitivity to prompt variations, which can lead to inconsistent results from minor changes in input phrasing [232], [233]. Besides, as fms are primarily trained on generic datasets that do not capture the specialized knowledge, the generalizability of fms to cps-specific domains remains limited, which can further result in correctness and transformation issues of fm-generated artifacts across cps software lifecycle [240], [241], [333]. Beyond this, cpss often operate in real-time settings, such constraints conflict with the computational demands of large fms [242], [243], limiting their applicability in time-sensitive cps operations. Moreover, human expectations are often expressed in natural language, which is inherently vague and ambiguous, making it challenging for fm to align with human engineering preferences [246]–[249] Finally, the effective application of fms requires careful customization, human–fms collaboration, and support for cross-domain reusability, each of which introduces challenges such as scarce domain-specific data and high fine-tuning costs, limited interpretability and effective feedback integration, and difficulties in generalizing models consistently across diverse CPS contexts.

10.1.2 Research Opportunities↩︎

Several research directions are worth investigating for the above technical challenges. For example, developing uncertainty estimation and hallucination detection methods tailored to fms can improve the reliability and trustworthiness of fm outputs [334], [335]. Robust prompting strategies [234], [235] and prompt optimization techniques can be applied to reduce sensitivity to input variations [236]. To improve domain generalization, data augmentation, knowledge distillation, and domain-specific fine-tuning on cps datasets are promising directions [196], [237]–[239]. Besides, developing validation pipelines and traceability techniques is needed to ensure the correctness and semantic preservation of fm-generated artifacts across lifecycle stages [241]. To address real-time inference constraints, techniques such as model compression and optimization can be applied for more efficient deployment of fms in CPS environments [243]–[245]. In addition, alignment with engineering intent can be achieved through reinforcement learning from human feedback, preference learning, and human-in-the-loop fine-tuning [248], [250]–[252]. Lastly, to support model customization, collaboration, and reusability across cps domains, parameter-efficient fine-tuning methods and multi-agent collaboration frameworks are promising directions [257]–[259].

10.2 Safety and Certification↩︎

10.2.1 Key Challenges↩︎

cpss are typically safety-critical systems, where failures can lead to severe physical, operational, or economic consequences. The integration of fms into software of such systems raises concerns due to their lack of verifiable behavior in unpredictable operating conditions [9], [264]–[266]. In addition, existing safety standards are not fully compatible with fms [269]–[271] and certification is further complicated by the limited transparency of fms, which hinders auditability, traceability, and accountability in safety assurance processes [16], [272].

10.2.2 Research Opportunities↩︎

Addressing the above challenges requires new assurance paradigms. One promising direction is developing systematic testing and evaluation frameworks to capture emergent behaviors of fms in safety-critical settings [267], [268]. Another important direction is improving explainability and transparency to support audit and certification processes [272]–[274]. Finally, extending existing safety standards or developing new regulatory frameworks tailored for fms is essential. Such frameworks should account for fm evolution by re-certifying and compliance checking across the model lifecycle [269], [270].

10.3 Economic and Resource Constraints↩︎

10.3.1 Key Challenges↩︎

The practical adoption of fms also raises concerns regarding economic and resource constraints. fms are large models that require large-scale datasets and high computational resources for training, fine-tuning, and inference [275]–[277], [279]–[281]. For instance, in the data preparation phase, cps datasets must capture heterogeneous data modalities and require domain expertise for annotation, which is both expensive and time-consuming. At deployment, fms must perform inference for cps monitoring, testing, or code generation tasks, which can be economically prohibitive. Such resource constraints are further amplified when deploying on edge devices, where fms must operate on hardware with limited memory, processing power, and real-time constraints [288]–[290]. Finally, continuous monitoring and maintenance of deployed fms introduces ongoing operational overhead [84], [292].

10.3.2 Research Opportunities↩︎

First, to reduce the cost of creating cps-specific datasets [278], synthetic data generation techniques are promising approaches that generate datasets without requiring costly real-world data collection or manual annotation. Besides, parameter-efficient fine-tuning methods [257] and data-efficient training strategies [282], [283] can significantly reduce the computational cost of adapting fms to cps domains without requiring model retraining. To reduce the inference cost of fm deployment at an industrial scale, batch prompting and model compression techniques [244], [287] are promising directions. For edge deployment, model compression, knowledge distillation, and hardware-aware optimization [244], [291] can produce compact fm variants suitable for edge device deployment. Finally, automated fm monitoring pipelines and lifelong learning strategies [293]–[295] can reduce the manual overhead of maintaining fm-based cps tools as systems evolve.

10.4 Human, Organizational, and Social Aspects↩︎

10.4.1 Key Challenges↩︎

A key risk is overreliance on fm outputs without sufficient critical evaluation [296]–[298]. Engineers or developers may accept fm-generated artifacts (e.g., code, test cases, or design models), without verifying their correctness, increasing the risk of undetected errors propagating into safety-critical cps systems. fm adoption also reshapes engineers’ collaboration patterns. cps development usually involves multidisciplinary teams comprising software, hardware, control, and domain engineers, each with distinct expertise and backgrounds. Communication gaps within such teams are already a significant challenge [300], and the introduction of fm-based tools adds further complexity, as team members with limited fm literacy may struggle to interpret, validate, or challenge fm-generated outputs effectively. Moreover, the absence of governance frameworks and polices for fm adoption may compromise safe and responsible deployment of fms [303], [304], and the intrinsic biases in fm may influence engineering decisions, potentially leading to biased system behaviors that affect different user groups unequally in cps applications.

10.4.2 Research Opportunities↩︎

To mitigate the overreliance on fms, informed fm outputs, supported by transparent explanations and justifications [299], can help engineers critically evaluate generated artifacts rather than accepting them directly. Building on this, explainable fm interfaces that provide human-readable justifications can further bridge communication gaps in multidisciplinary cps teams [301], [302]. At the organizational level, governance frameworks defining clear policies for fm use, validation, and accountability in cps engineering are equally essential [305]. Finally, bias detection and mitigation techniques for fm-assisted cps can help identify and reduce intrinsic biases before they affect decision-making [308], [309].

10.5 Ethical and Privacy Aspects↩︎

10.5.1 Key Challenges↩︎

The use of fms in cps software engineering introduces ethical and privacy risks arising from both their development and application [336]. Ethical concerns emerge throughout the fm lifecycle, including the propagation of biases that may affect fairness, transparency, and accountability in cps decision-making [310]–[312]. In addition, sensitive cps data used in fm pipelines may be exposed during training or inference, leading to potential privacy violations [314]–[316]. The use of cloud-based fm APIs further increases the risk of proprietary asset leakage [319]–[321]. Finally, the deployment of fms raises unresolved questions regarding intellectual property ownership of generated artifacts [323], [324].

10.5.2 Research Opportunities↩︎

First, addressing ethical risks requires ethical design frameworks that explicitly promote responsible and transparent fm use [227], [313], and the privacy risks motivate research into privacy-preserving learning techniques, such as differential privacy and data sanitization in fm training and inference pipelines [317]–[319]. Further, secure deployment strategies such as on-premise or isolated fm execution environments are promising directions to avoid proprietary asset exposure [322]. Moreover, the development of legal and technical frameworks, including emerging regulatory efforts such as the EU AI Act, is needed to clarify intellectual property ownership [325].

10.6 Environmental Impacts↩︎

10.6.1 Key Challenges↩︎

The use of fms introduces significant environmental challenges due to their substantial computational requirements. A major concern is the high carbon footprint associated with fm training, which arises from large-scale datasets, extensive optimization procedures, and repeated experimentation [326]–[328]. In addition, there is an inherent conflict between the continued scaling of fms and environmental sustainability, as increasing model size and compute demand lead to higher energy consumption [330], [331].

10.6.2 Research Opportunities↩︎

To address the above challenges, several research directions focus on improving efficiency across the fm lifecycle. For instance, parameter-efficient fine-tuning, data-efficient training, and model compression techniques can substantially reduce computational cost while maintaining performance [244], [257], [282], [283]. Moreover, carbon-tracking and reporting frameworks provide visibility into the environmental footprint of fm development and deployment, enabling better measurement for carbon emissions [329]. At a broader level, the adoption of energy-efficient hardware, optimized data center design, and governance-driven trade-off management between model performance and energy consumption represent complementary directions for reducing the environmental impact of the use of fms.

11 Related Work↩︎

11.1 Foundation Models for Software Engineering↩︎

With the rapid advancements of FMs, an increasing number of studies have explored their application across various stages of SE. Literature reviews and surveys are crucial means for providing comprehensive insights into the role of FMs within SE [3], [5], [13], [295], [337], [338]. Wang et al. [13] conducted a systematic review of 102 studies on the use of LLM in software testing. Their work highlights test case preparation and program repair as the most prominent tasks where LLMs have been applied, while also analyzing the key challenges and opportunities in applying LLMs in software testing. Hou et al. [3] conducted a systematic literature review of 395 studies published between 2017 and early 2024, providing a broader perspective on LLM applications across SE tasks, offering a broader perspective of LLMs across SE tasks, methods, and evaluation strategies. Fan et al. [5] surveyed the application of LLMs in SE, highlighting their potential across tasks ranging from coding and requirements to documentation and analytics, while also discussing the significant technical challenges posed by LLM properties such as hallucinations. Zheng et al. [295] systematically reviewed 123 studies from 2022 onward on the integration of LLMs with SE. Their work highlights research focuses, identifies gaps in LLM for SE, and synthesizes evaluations of LLM performance, revealing both capabilities and limitations while providing directions for future research and optimization.

While the above studies provide comprehensive perspectives of LLM applications in SE, they narrow the scope to a single type of FM, i.e., LLM. As FMs continue to evolve, encompassing multimodal, task-specific, and hybrid models, there is a growing need to examine their broader applications across SE. One relevant study by Li et al. [337] examined FM4SE and SE4FM practices in industry from a practitioner’s perspective, highlighting prominent SE tasks such as code generation, deployment, and API recommendation. Another relevant work from Shi et al. [339] presents a roadmap for LLMs in SE, emphasizing the need for efficient and environmentally sustainable solutions. Different from the above work, this paper presents a vision on FM for SE in the domain of CPS, exploring how advanced FMs can support complex, potentially safety-critical, CPS tasks by facilitating the CPS software development cycles, enhancing automation, and addressing domain-specific challenges that go beyond conventional SE applications.

11.2 Foundation Models for Cyber-physical Systems↩︎

The emergence of FMs has significantly advanced the development of CPSs, providing powerful solutions for ensuring safe and reliable CPS design and operation [276]. This has been demonstrated in a wide range of CPSs, including autonomous vehicles [340]–[342], robotics [14], [343]–[345], and industrial CPSs [346], [347]. Cui et al. [340] investigated the use of multimodal FMs in autonomous driving, reviewing tools, datasets, and benchmarks, and identifying key challenges and future directions. Similarly, Yang et al. [341] systematically review the research about FMs for autonomous driving, including the current status of technological advancements, related challenges, and future research directions. Li et al. [342] provided a comprehensive survey on using LLMs for autonomous driving, examining their applications in both modular pipelines and end-to-end systems. The study highlights LLMs’ potential to enable knowledge-based, human-like autonomous driving, while also addressing key challenges such as real-time inference, safety assurance, and deployment costs. Regarding FMs in robotics, Zeng et al. [344] reviewed the applications of LLMs in robotics, highlighting their impact on control, perception, decision-making, and path planning, while also discussing challenges and future directions for achieving embodied intelligence. Wang et al. [343] reviewed the use of FMs in robotics and proposed a GPT-4V-based framework that enhances embodied task planning by combining language instructions with visual perception. Kim et al. [345] surveyed recent advancements of LLMs in robotics, focusing on their applications in communication, perception, planning, and control, and provided practical guidelines and examples for integrating LLMs into robotic systems.

Different from the above studies, this paper focuses specifically on the SE aspects of CPSs, identifying the challenges of applying FMs across the software lifecycles of CPS software, including development, assurance, and maintenance of CPS software, and exploring how FMs can be leveraged to enhance reliability, automation, and efficiency in the CPS software lifecycles. This area has received limited attention in current research.

12 Threats to Validity↩︎

Construct Validity. We did not follow a systematic literature review process, and as a result, some challenges and, consequently, some opportunities may not have been captured. To mitigate this, the identification of challenges and opportunities was grounded in an extensive reading of the relevant literature and supplemented by structured group discussions following the Delphi method [32] and the nominal group technique [33]. In addition, discussions at the SE2030 Workshop (co-located with FSE 2025) provided external input from researchers and practitioners across diverse disciplines, helping to broaden the coverage and balance of the identified challenges. Furthermore, we emphasize that, as a research roadmap, its primary aim is to present representative challenges and opportunities, provide guidance to researchers, and serve as a foundation for future empirical work. Missing some items does not reduce its value, as the roadmap can be iteratively refined as new studies emerge. Conclusion Validity. As a visionary paper, this work does not include empirical contributions, and the proposed research opportunities have not yet been validated through experiments or case studies. Consequently, their practical impact may differ from what is anticipated. To mitigate this threat and provide structured guidelines for future empirical validation, each research opportunity is accompanied by concrete research questions, suggested methodologies, baseline approaches, and evaluation metrics, ensuring that the roadmap serves not only as a visionary guide but also as an actionable starting point for subsequent empirical investigations. External Validity. This roadmap is grounded in the current landscape of fms, covering established models such as llms, vlms, and multimodal fms. However, as fm evolves, new model paradigms may emerge that are not covered by the current roadmap. For instance, world models [348] represent emerging directions that could open new research opportunities beyond those identified here. As new fm paradigms and empirical evidence emerge, the roadmap can be iteratively extended to reflect these developments.

13 Conclusion and Future Work↩︎

This paper presents a forward-looking research roadmap for integrating fms into the software engineering of cpss. It discusses research challenges and corresponding actionable opportunities related to using fms across six cps software engineering phases, including typically phases such as requirement engineering, design and modeling, and testing. Each research opportunity presents concrete research questions, suggested methodologies, baseline approaches, and evaluation metrics. Additionally, we identify common cross-cutting challenges when adopting fms across six dimensions: technical, safety and certification, economic and resource, human and organizational, ethical and privacy, and environmental, together with corresponding research opportunities for each dimension. This roadmap aims to provide a structured and actionable guide for researchers and practitioners working at the intersection of fms and cps software engineering. In the future, we will expand on these research opportunities in detail for each phase and map existing software engineering problems to various types of fms to present a more comprehensive research roadmap.

14 Acknowledgements↩︎

This work is supported by the RoboSapiens project funded by the European Commission’s Horizon Europe programme under grant agreement number 101133807 and the Co-evolver project (No. 286898/F20) funded by the Research Council of Norway. Aitor Arrieta and Pablo Valle are part of the Systems and Software Engineering group of Mondragon Unibertsitatea (IT1519-22), supported by the Department of Education, Universities and Research of the Basque Country. Pablo Valle is supported by the Pre-doctoral Program for the Formation of Non-Doctoral Research Staff of the Education Department of the Basque Government (Grant n. PRE_2024_1_0014). Aitor Arrieta is supported by the Spanish Ministry of Science, Innovation and Universities (project PID2023-152979OA-I00), funded by MCIU /AEI /10.13039/501100011033 / FEDER, UE.

References↩︎