Language models of code have demonstrated remarkable performance across various software engineering andsource code analysis tasks. However, their demanding computational resource requirements and consequentialenvironmental footprint remain as significant challenges. This work introduces Alpine, an adaptive programming language-agnostic pruning technique designed to substantially reduce the computational overhead ofthese models. The proposed method offers a pluggable layer that can be integrated with all Transformer-basedmodels. With Alpine, input sequences undergo adaptive compression throughout the pipeline, reaching a sizethat is up to ×3 less their initial size, resulting in significantly reduced computational load. Our experimentson two software engineering tasks, defect prediction and code clone detection across three language modelsCodeBert, GraphCodeBert and UniXCoder show that Alpine achieves up to a 50% reduction in FLOPs,a 58.1% decrease in memory footprint, and a 28.1% improvement in throughput on average. This led to areduction in CO2 emissions by up to 44.85%. Importantly, it achieves a reduction in computation resourceswhile maintaining up to 98.1% of the original predictive performance. These findings highlight the potentialof Alpine in making language models of code more resource-efficient and accessible while preserving theirperformance, contributing to the overall sustainability of their adoption in software development. Also, itsheds light on redundant and noisy information in source code analysis corpora, as shown by the substantialsequence compression achieved by Alpine.

The application of machine learning techniques to address MDE problems often requires transforming raw information (e.g., software models) to a numerical representation which can be used by machine learning algorithms. To this end, pretrained embeddings are a key technology to facilitate the construction of such applications. However, previous works have demonstrated that these embeddings struggle to generalize effectively in the MDE domain due to their training on general-purpose corpora. To tackle this issue, we developed WordE4MDE , which are specialized word embeddings trained specifically on modeling documents. In this study, we aim to overcome several limitations of WordE4MDE and conduct additional experiments to assess its efficacy. Key limitations we address include: (1) mitigating the out-of-vocabulary issue through the utilization of sub-word embeddings, (2) adding contextualization to the embeddings by training a BERT model on our specific modeling corpus and (3) addressing the constraint of limited training data by investigating the augmentation of our modeling corpus with StackOverflow and StackExchange data.

Quality assurance for large-scale cyber-physical systems relies on sophisticated test activities using complex test environments investigated with the help of numerous types of simulators. As these systems grow, extensive resources are required to develop and maintain simulation models of hardware and software components, as well as physical environments. Meanwhile, recent advances in generative AI have led to tools that can produce executable test cases for software systems, offering potential benefits such as reducing manual efforts or increasing test coverage. However, the application of generative AI techniques to simulation-based testing of large-scale cyber-physical systems remains underexplored. To better understand this gap, this study captures practitioners’ perspectives on leveraging generative AI, based on a cross-company workshop with six organizations. Our contribution is twofold: (1) detailed, experience-based insights into challenges faced by engineers, and (2) a research agenda comprising three high-priority directions: (a) AI-generated scenarios and environment models, (b) simulators and AI in CI/CD pipelines, and (c) trustworthiness in generative AI for simulation. While participants acknowledged substantial potential, they also highlighted unresolved challenges. By detailing these issues, the paper aims to guide future academia-industry collaboration towards the responsible adoption of generative AI in simulation-based testing.