Generating Financial Market Scenarios Using NVIDIA NIM

While generative AI can be used to create clever rhymes, cool images, and soothing voices, a closer look at the techniques behind these impressive content generators reveals probabilistic learners, compression tools, and sequence modelers. When applied to quantitative finance, these methods can help disentangle and learn complex associations in financial markets. 

Market scenarios are crucial for risk management, strategy backtesting, portfolio optimization, and regulatory compliance. These hypothetical data models represent potential future market conditions, helping financial institutions to simulate and assess outcomes and make informed investment decisions. 

Specific methods demonstrate proficiency in various areas, such as:

• Data generation with variational autoencoders or denoising diffusion models
• Modeling sequences with intricate dependencies using transformer-based generative models
• Understanding and predicting time-series dynamics with state-space models

While these methods may operate in distinct ways, they can also be combined to yield powerful results. 

This post explores how variational autoencoders (VAE), denoising diffusion models (DDM), and other generative tools can be integrated with large language models (LLM) to efficiently create market scenarios with desired properties. It showcases a scenario generation reference architecture powered by NVIDIA NIM, a collection of microservices designed to accelerate the deployment of generative models.

One toolset, many applications

Generative AI provides a unified framework around a variety of quantitative finance problems that have previously been addressed with distinct approaches. Once a model has been trained to learn the distribution of its input data, it can be used as a foundational model for a variety of tasks. 

For instance, it could generate samples for the creation of simulations or risk scenarios. It could also pinpoint which samples are out of distribution, acting as an outlier detector or stress scenario generator. As market data moves at different frequencies, cross-sectional market snapshots have gaps. A generative model can supply the missing data that fits with the actual data in a plausible way, which is beneficial for nowcasting models or dealing with illiquid points. Finally, autoregressive next-token prediction and state space models can help with forecasting. 

A significant bottleneck for domain experts leveraging such generative models is the lack of platform support that bridges their ideas and intentions with the complex infrastructure needed to deploy these models. While LLMs have gained mainstream use across various industries including finance, they are primarily used for knowledge processing tasks, such as Q&A and summarization, or coding tasks, like generating code stubs for further enhancement by human developers and integration into proprietary libraries. 

Integrating LLMs with complex models can bridge the communication gap between quantitative experts and generative AI models, as explained below.

Market scenario generation

Traditionally, the generation of market scenarios has relied on techniques including expert specifications (“shift the US yield curve up by bp in parallel”), factor decompositions (“bump the EUR swap curve by bp along the first PCA direction”), and statistical methods such as variance-covariance or bootstrapping. While these techniques help produce new scenarios, they lack a full picture of the underlying data distribution and often require manual adjustment. Generative approaches, which learn data distributions implicitly, elegantly overcome this modeling bottleneck.

LLMs can be combined with scenario generation models in powerful ways to enable simplified interaction while also acting as natural language user interfaces for market data exploration. For example, a trader might wish to assess her book’s exposure if markets were to behave like they did during a previous event, such as the great financial crisis, the last U.S. election, the Flash Crash, or the dot-com bubble burst. An LLM trained on recorded knowledge of such events could find and extract the characteristics of interest conditional on such events or historical periods and pass them to a generative market model to create similar market conditions for use in downstream applications. 

Figure 1 illustrates reference architecture for market scenario generation, connecting user specifications with suitable generative tools. Sample code for an implementation powered by NVIDIA NIM is shown in the Sample Implementation section. 

The process starts with a user instruction; for example, requesting a simulation of an interest rate environment similar to the one “at the peak of the financial crisis.” An agent (or collection of agents) processes this request by first routing it to an LLM-powered interpreter that interprets and converts the request in natural language to an intermediate format (in this case, a JSON file). 

The LLM then translates the “peak of financial crisis” to a concrete historical period (September 15 to October 15, 2008) and maps the market objects of interest (U.S. swap curves and swaption volatility surfaces, for example) to their respective pre-trained generative models (VAE, DDM, and so on). Information about the historical period of interest can be retrieved through a data retriever component and passed on to the corresponding generative tools to generate similar market data. 

The generation process entails running inference on generative models that have been pre-trained on market data. Figures 2 and 3 illustrate an example of yield curve scenarios generated using a VAE model corresponding to the start of the COVID-19 pandemic.

Table 1 exemplifies additional market scenario requests along with corresponding JSON outputs. In this case, the user query indicates the model to be used for illustration purposes. In a production environment, a routing agent could automatically select the right model. 

Market structure analysis using generative models

Financial markets are inherently complex, characterized by noisy, high-dimensional data typically observed as multivariate time series. Detecting fleeting patterns that lead to financial gains at scale requires clever modeling and substantial computations. 

One way to reduce dimensionality is to consider the intrinsic structure present in the data: curves, surfaces, and higher-dimensional structures. These structures carry information that can be leveraged to reduce complexity. They can also be viewed as units of information to be embedded and analyzed through latent spaces of generative models. In this section we review examples of how VAEs and DDMs can be used in this context.

VAEs for learning the distribution of market curves

Bond yields, swap, inflation, and foreign exchange rates can be thought of as having one-dimensional term-structures, such as zero-coupon, spot, forward, or basis curves. Similarly, option volatilities can be seen as (hyper-)surfaces in 2D or higher dimensional spaces. A typical swap curve could have as many as 50 tenors. Instead of studying how the 50 corresponding time series relate to each other, one can consider a single time series of swap curves and use a VAE to learn the distribution of these objects, as detailed in Multiresolution Signal Processing of Financial Market Objects. 

The strength of this approach lies in its ability to integrate previously isolated data: the behavior of market objects, traditionally modeled in isolation either by currency (USD, EUR, BRL, for example) or by task (scenario generation, nowcasting, outlier detection, for example), can now be integrated into the training of a single generative model reflecting the interconnected nature of markets. 

Figure 4 illustrates the training loop of a VAE on market data objects such as yield curves: an encoder compresses the input objects into a latent space with a Gaussian distribution, a decoder reconstructs curves from points in this space. The latent space is continuous, explicit, and usually lower-dimensional, making it intuitive to navigate. Curves representing different currencies or market regimes cluster in distinct areas, allowing the model to navigate within or between these clusters to generate new curves unconditional or conditional on desired market characteristics.

In particular, the generation process can be conditioned on specific historical periods to generate curves that may have similar shapes to historical ones, without being exact replicas. For example, the U.S. Treasury yield curves corresponding to the start of the COVID-19 pandemic are shown in Figure 2 (left) as circles in the 3D latent space of a VAE that has been trained on yield curve data. Since VAE latent spaces are continuous, they naturally render themselves to defining neighborhoods around points of interest and to sampling from those neighborhoods to generate novel yield curve scenarios that are similar (but not the same) as historical ones, shown as triangles in Figure 2 (right). 

Figure 5 shows a more complete picture of the yield curves used for training the VAE model, along with clusters grouped by the level of rates, as well as an outlier corresponding to the mistaken inclusion of a data point when the Treasury markets were closed (Good Friday, 2017). One can easily imagine additional applications beyond scenario generation, including non-linear factor decompositions, outlier detection, and many more.

DDMs for learning volatility surfaces

DDMs approach the generative process through the prism of reversible diffusion. As shown in Figure 6, they operate by gradually introducing noise into the data until it becomes a standard Gaussian. Then they learn to reverse this process to generate new data samples starting from pure noise. Noise is added gradually in the forward pass, until the resulting object is indistinguishable from Gaussian noise. During the backward pass, the model learns the denoising needed to reconstruct the original surface. To learn more, see Generative AI Research Spotlight: Demystifying Diffusion-Based Models.

To learn the distribution of implied volatility surfaces, the ability of a DDM (specifically a DDPM, or denoising diffusion probabilistic model) is explored. This example uses a synthetic data set of roughly 20,000 volatility surfaces generated with the SABR (stochastic alpha-beta-rho) stochastic volatility model described in Managing Smile Risk (subject to initial conditions and ):

An example input surface is shown in Figure 7.

The goal is to evaluate the ability of such a model to recover the input data distribution, in this case, a SABR distribution. In the case of empirical surfaces implied directly from market prices of options, the distribution would be unknown and having a tool that can capture the distribution in a non-parametric way would offer a valuable alternative to sparse parametric models that don’t have sufficient degrees of freedom to represent the data. It can subsequently be used to generate volatility surface scenarios, or to fill in missing regions in plausible ways.

A simplified version of this architecture was adapted for this work. The inputs are grids of implied volatilities corresponding to various option moneyness and time to maturity pairs. Other types of inputs, such as volatility cubes (moneyness-maturity-underlying) or other variations could be tackled in similar fashion.

Figure 8 illustrates a few steps in the training of the DDPM model on volatility surface inputs.

Figure 9 shows two synthetically-generated surfaces which have subsequently been fitted with SABR models to verify that their shapes are SABR-like. Green points represent SABR surfaces fitted to the generated ones, verifying that the DDPM model has learned the SABR-like shapes.

Sample implementation

This section presents an example of using NVIDIA-hosted NIM endpoints including Llama 3.1 70B Instruct LLM to build the LLMQueryInterpreter component of the reference architecture in Figure 1. Note that many other NVIDIA as well as open-source LLMs are available with NIM, including Nemotron, Mixtral, Gemma, and many more. Accessing these through NIM guarantees that they are optimized for inference on NVIDIA-accelerated infrastructure and offers a quick and easy way to compare responses from multiple models.

import os
from langchain_nvidia_ai_endpoints import ChatNVIDIA

# NVIDIA API configuration
NVIDIA_API_KEY = os.environ.get("NVIDIA_API_KEY") # click “Get API Key” at https://www.nvidia.com/en-us/ai/
if not NVIDIA_API_KEY:
    raise ValueError("NVIDIA_API_KEY environment variable is not set")

class LLMQueryInterpreter:
    """ NVIDIA NIM-powered class that processes scenario requests from user."""
    
    def __init__(self, llm_name="meta/llama-3.1-70b-instruct", temperature=0):
        self._llm = ChatNVIDIA(model=llm_name, temperature=temperature) 
    
        # define output JSON format
        self._scenario_template = """ 
        {"scenarios": [ 
            { 
                "method": <generative method name, e.g., "VAE">,
                "object_type": <object type, e.g., "yield curve">,
                "period": {
                    "start_date": <start date of event>,
                    "end_date"  : <end date of event>
                }
            }, 
            { 
                "method": <generative method name, e.g., "DDPM">,
                "object_type": <object type, e.g., "volatility surface">,
                "period": {
                    "start_date": <start date of event>,
                    "end_date"  : <end date of event>
                }
            }, 

            // Additional scenarios, as needed 
        ] } 
        """

        # instructions for the LLM
        self._prompt_template = """" 
        Your task is to generate a single, high-quality response in JSON format. \
        Format the output according to the following template: {scenario_template}. \
        Offer a refined, accurate, and comprehensive reply to the instruction. \

        Every query has a structure of this form: \
        "Use <method> to generate <market_object_type> similar to <period>". \
        Valid methods are "VAE" or "DDPM"; use "VAE" by default, if no method is specified.\
        If a different method is requested, return "invalid" as the method type. \ 

        Answer the following query without any additional explanations, return only a JSON output:
        {query}  

        """

    def process_query(self, query):
        # query = "Using a VAE model, output yield curves similar to those in the second half of 2020"

        llm_request = self._prompt_template.format(**{ "query": query, "scenario_template": self._scenario_template }) 
        llm_answer = self._llm.invoke(llm_request) 
        
        return llm_answer.content

# Example usage
query = "Using a VAE model, output yield curves similar to the Flash Crash"
llm = LLMQueryInterpreter()
print(llm.process_query(query))

# Output
{
"scenarios": [
    {
        "method": "VAE",
        "object_type": "yield curve",
        "period": {
            "start_date": "2010-05-06",
            "end_date": "2010-05-06"
        }
    }
]
}

Conclusion

It’s exciting to imagine a future where quants, traders, and investment professionals increasingly collaborate with AI tools to model and explore financial markets. The integration of these advanced tools enhances financial modeling and market exploration, promising to drive forward the capabilities and insights of market participants. They can be combined in innovative ways and served with ease using NVIDIA NIM.