Weather forecasting has seen a shift in methods from numerical simulations to data-driven systems. While initial research in the area focused on deterministic forecasting, recent works have used diffusion models to produce skillful ensemble forecasts. These models are trained on a single forecasting step and rolled out autoregressively. However, they are computationally expensive and accumulate errors for high temporal resolution due to the many rollout steps. We address these limitations with Continuous Ensemble Forecasting, a novel and flexible method for sampling ensemble forecasts in diffusion models. The method can generate temporally consistent ensemble trajectories completely in parallel, with no autoregressive steps. Continuous Ensemble Forecasting can also be combined with autoregressive rollouts to yield forecasts at an arbitrary fine temporal resolution without sacrificing accuracy. We demonstrate that the method achieves competitive results for global weather forecasting with good probabilistic properties.

The three-dimensional structure of proteins plays a crucial role in determining their function. Protein structure prediction methods, like AlphaFold, offer rapid access to a protein’s structure. However, large protein complexes cannot be reliably predicted, and proteins are dynamic, making it important to resolve their full conformational distribution. Single-particle cryo-electron microscopy (cryo-EM) is a powerful tool for determining the structures of large protein complexes. Importantly, the numerous images of a given protein contain underutilized information about conformational heterogeneity. These images are very noisy projections of the protein, and traditional methods for cryo-EM reconstruction are limited to recovering only one or a few consensus conformations.In this paper, we introduce cryoSPHERE, which is a deep learning method that uses a nominal protein structure (e.g., from AlphaFold) as input, learns how to divide it into segments, and moves these segments as approximately rigid bodies to fit the different conformations present in the cryo-EM dataset. This approach provides enough constraints to enable meaningful reconstructions of single protein structural ensembles. We demonstrate this with two synthetic datasets featuring varying levels of noise, as well as two real dataset. We show that cryoSPHERE is very resilient to the high levels of noise typically encountered in experiments, where we see consistent improvements over the current state-of-the-art for heterogeneous reconstruction.

Evaluating the performance of clustering models isa challenging task where the outcome depends onthe definition of what constitutes a cluster. Due tothis design, current existing metrics rarely handlemultiple clustering models with diverse cluster def-initions, nor do they comply with the integration ofconstraints when available. In this work, we takeinspiration from consensus clustering and assumethat a set of clustering models is able to uncoverhidden structures in the data. We propose to con-struct a discriminative ordering through ensembleconsensus based on the distance between the con-nectivity of a clustering model and the consensusmatrix. We first validate the proposed method withsynthetic scenarios, highlighting that the proposedscore ranks the models that best match the consen-sus first. We then show that this simple rankingscore significantly outperforms other scoring meth-ods when comparing sets of different clusteringalgorithms that are not restricted to a fixed num-ber of clusters and is compatible with clusteringconstraints.

Contrastive Predictive Coding (CPC) is a well-established self-supervised learning method that naturally fits time series data. This method has been recently leveraged to detect anomalous inputs, viewed as the task of classifying positive pairs of context-feature representations versus negative ones in order to employ classifier uncertainty measures. In this paper, by taking a different perspective, we propose a CPC-based Out-Of-Distribution (OOD) detection method for time series data that does not require any negative samples at test time and is theoretically related to a probabilistic type of uncertainty estimation in the latent representation space. Our method extends the standard CPC by using a radial (distance-based) score function both in the training loss and as the OOD measure, in addition to quantizing the context (replacing it by cluster prototypes) during inference. The proposed method is applied to detecting OOD human activities with smartphone sensors data and shows promising performance on two primary datasets without using activity labels in training.

Variational Autoencoders (VAEs) are a popular framework for unsupervised learning anddata generation. A plethora of methods have been proposed focusing on improving VAEs,with the incorporation of adversarial objectives and the integration of prior learning mechanismsbeing prominent directions. When it comes to the former, an indicative instance is therecently introduced family of Introspective VAEs aiming at ensuring that a low likelihood isassigned to unrealistic samples. In this study, we focus on the Soft-IntroVAE (S-IntroVAE),one of only two members of the Introspective VAE family, the other being the originalIntroVAE. We select S-IntroVAE for its state-of-the-art status and its training stability.In particular, we investigate the implication of incorporating a multimodal and trainableprior into this S-IntroVAE. Namely, we formulate the prior as a third player and show thatwhen trained in cooperation with the decoder constitutes an effective way for prior learning,which shares the Nash Equilibrium with the vanilla S-IntroVAE. Furthermore, basedon a modified formulation of the optimal ELBO in S-IntroVAE, we develop theoreticallymotivated regularizations, namely (i) adaptive variance clipping to stabilize training whenlearning the prior and (ii) responsibility regularization to discourage the formation of inactiveprior modes. Finally, we perform a series of targeted experiments on a 2D densityestimation benchmark and in an image generation setting comprised of the (F)-MNIST andCIFAR-10 datasets demonstrating the effect of prior learning in S-IntroVAE in generationand representation learning.

A recent line of research has exploited pre-trained generative diffusion models as priors for solving Bayesian inverse problems. We contribute to this research direction by designing a sequential Monte Carlo method for linear-Gaussian inverse problems which builds on “decoupled diffusion", where the generative process is designed such that larger updates to the sample are possible. The method is asymptotically exact and we demonstrate the effectiveness of our Decoupled Diffusion Sequential Monte Carlo (DDSMC) algorithm on both synthetic as well as protein and image data. Further, we demonstrate how the approach can be extended to discrete data.

Crystalline materials often exhibit a high level of symmetry. However, most generative models do not account for symmetry, but rather model each atom without any constraints on its position or element. We propose a generative model, Wyckoff Diffusion (WyckoffDiff), which generates symmetry-based descriptions of crystals. This is enabled by considering a crystal structure representation that encodes all symmetry, and we design a novel neural network architecture which enables using this representation inside a discrete generative model framework. In addition to respecting symmetry by construction, the discrete nature of our model enables fast generation. We additionally present a new metric, Fréchet Wrenformer Distance, which captures the symmetry aspects of the materials generated, and we benchmark WyckoffDiff against recently proposed generative models for crystal generation. As a proof-of-concept study, we use WyckoffDiff to find new materials below the convex hull of thermodynamical stability.

Generative flow networks (GFNs) are a class of probabilistic models for sequential samplingof composite objects, proportional to a target distribution that is defined in terms of anenergy function or a reward. GFNs are typically trained using a flow matching or trajectorybalance objective, which matches forward and backward transition models over trajectories.In this work we introduce a variational objective for training GFNs, which is a convexcombination of the reverse- and forward KL divergences, and compare it to the trajectorybalance objective when sampling from the forward- and backward model, respectively. Weshow that, in certain settings, variational inference for GFNs is equivalent to minimizing thetrajectory balance objective, in the sense that both methods compute the same score-functiongradient. This insight suggests that in these settings, control variates, which are commonlyused to reduce the variance of score-function gradient estimates, can also be used with thetrajectory balance objective. We evaluate our findings and the performance of the proposedvariational objective numerically by comparing it to the trajectory balance objective on twosynthetic tasks.

We introduce discriminator guidance in the setting of Autoregressive Diffusion Models. The use of a discriminator to guide a diffusion process has previously been used for continuous diffusion models, and in this work we derive ways of using a discriminator together with a pretrained generative model in the discrete case. First, we show that using an optimal discriminator will correct the pretrained model and enable exact sampling from the underlying data distribution. Second, to account for the realistic scenario of using a sub-optimal discriminator, we derive a sequential Monte Carlo algorithm which iteratively takes the predictions from the discriminator into account during the generation process. We test these approaches on the task of generating molecular graphs and show how the discriminator improves the generative performance over using only the pretrained model.

Detecting microsecond structural perturbations in biomolecules has wide relevance in biology, chemistry and medicine. Here we show how MHz repetition rates at X-ray free-electron lasers can be used to produce microsecond time-series of protein scattering with exceptionally low noise levels of 0.001%. We demonstrate the approach by examining J alpha helix unfolding of a light-oxygen-voltage photosensory domain. This time-resolved acquisition strategy is easy to implement and widely applicable for direct observation of structural dynamics of many biochemical processes. The MHz repetition rates available at second-generation X-ray free-electron lasers enable the collection of microsecond time-resolved X-ray scattering data with exceptionally low noise, providing insights into protein structural dynamics.