Equipping agentic AI applications with tools will usher in the next phase of AI. By enabling autonomous agents and other AI applications to fetch real-time data, perform actions, and interact with external systems, developers can bridge the gap to new, real-world use cases that significantly enhance productivity and the user experience.
xpander AI, a member of the NVIDIA Inception program for startups, has developed unique AI-ready connector technology, which offers connectivity between AI apps and the complementary systems that make powerful new use cases a reality. xpander helps AI engineers build applications with advanced tool calling use cases, without the need for significant integration work or improving the accuracy of tool calls, both of which represent common hurdles on the path to production.
xpander AI connectors help developers overcome these challenges with a multi-agent AI pipeline that ingests data related to a target system, creating an enriched and adjusted connector to that system, ready for use by agentic applications.
Enterprises that leverage NVIDIA NIM microservices for improved inference performance and use case-specific optimization can now use xpander AI to equip their NIM applications with agentic tools. This enables companies to build advanced, use case-specific AI apps while minimizing the challenges of integration with external systems and APIs.
This post explains agentic tool calling in large language models (LLMs), AI-ready connectors, and how you can use xpander AI to equip your NVIDIA NIM application with agentic tools.
What is NVIDIA NIM?
NVIDIA NIM, part of NVIDIA AI Enterprise, is a set of intuitive inference microservices designed to accelerate generative AI deployment in enterprises. Supporting a wide range of AI models—including NVIDIA AI foundation, community, and custom—NIM ensures seamless, scalable AI inferencing, on-premises or in the cloud, all while leveraging industry-standard APIs.
NIM microservices provide interactive APIs to run inference on AI models. They are packaged as container images on a per model or model family basis (Figure 1). Each NIM is its own Docker container with a model and includes a runtime that’s compatible with any NVIDIA GPU with sufficient memory. Under the hood, NIM uses NVIDIA TensorRT-LLM to optimize the models, with specialized accelerated profiles optimally selected for NVIDIA H100 Tensor Core GPUs, NVIDIA A100 Tensor Core GPUs, NVIDIA A10 Tensor Core GPUs, and NVIDIA L40S GPUs.
NIM microservices are now available to NVIDIA Developer Program members for free.
What is LLM tool calling?
Tool calling, also known as function calling, refers to the ability of LLMs to interact with external tools, APIs, or functions to perform tasks beyond text generation. This process involves defining tools, prompting the LLM, generating tool calls, executing tools, and incorporating results into an LLM response. It enhances LLM capabilities by providing access to real-time data, executing more complex tasks, and improving accuracy.
Tools are interfaces that accept input, execute an action, and then return a result of that action in a structured output according to a predefined schema. Tools often encompass external API calls, which the agent can use to perform tasks that go beyond the capabilities of the LLM, but aren’t limited to external API calls. For example, to get the current temperature for your city, a weather tool might be used. Or, to find out how your favorite sports team is doing in today’s game, a generic web search tool or ESPN tool might be used.
Tool calling in LLMs offers several significant benefits to developers by significantly extending the versatility and power of LLM functionality. It provides access to real-time data, enabling models to fetch up-to-date information from external sources, such as your sales team’s most urgent CRM updates or the latest system logging. This, in turn, enhances the accuracy and relevance of responses.
Additionally, tool calling enables LLMs to execute complex tasks that require specialized skills, like performing advanced mathematical calculations, or access information outside of their training data by querying databases. This capability also facilitates workflow automation by integrating LLMs with various APIs and systems, thereby improving efficiency and enabling the development of sophisticated AI applications that can perform multiple, parallel, and complex functions.
To support tool calling, an LLM is trained to detect when a specific function should be called, and then output a structured response that contains that function and its arguments. NIM supports tool calling for models that have the aforementioned capability. The LLM is packaged as a NIM, which delivers optimal deployability, usability, and performance on the NVIDIA accelerated infrastructure. The microservice packaging also uses APIs that are compatible with OpenAI, so developers can build powerful, world-class generative AI agents and mitigate some of their most common pain points.
For more examples using Llama 3.1 8B Instruct NIM, LangChain, and LangGraph, reference Building AI Agents with NVIDIA NIM Microservices and LangChain and the langchain-ai / langchain-nvidia notebook on GitHub. You can also experiment with tools available as part of a demo on the NVIDIA NIM API catalog (Figure 2).
Using xpander AI to enhance tool calling in NIM applications
With NIM, you can integrate AI applications into your organization’s tech stack because your application isn’t decoupled from the internal tools, data, and business logic that are unique to your company. This facilitates powerful new ways to address real-world enterprise challenges.
However, integration with enterprise tech stacks (both to modern APIs and legacy systems) can require significant effort to prepare the target systems for AI-native integrations. Further, implementation of tool calling can present challenges, especially for calling systems like CRM and ERP with complex APIs. These systems can have highly customizable data structures and hundreds of API endpoints. In some cases, especially with homegrown systems and APIs, target systems lack API documentation and OpenAPI specifications entirely, both of which are often used as a starting point for writing tool calls and describing the tools for the model to use.
Additionally, the more details you give the model about the target API in your tool calling code, the more accurate the model will be at tool selection and generation of structured responses. Operation IDs, parameters, and descriptions that adhere to organizational best practices are critical for accurate and reliable tool calls.
Prebuilt tools for NIM applications
By providing prebuilt tools for your NIM applications, xpander AI simplifies the process of building with tool calling and increases the accuracy of tool calling at runtime. These tools were generated by the xpander AI connector generator, a unique enrichment pipeline that incorporates data such as API reference, docs, and browsing examples, and outputs an AI-ready connector for any target system.
Supplying models with highly detailed and automatically enriched tools can be especially beneficial in cases where a target system requires a sequence of interdependent API calls to achieve a task. This could be as simple as sending a message on Slack, or as complex as an agentic task to fulfill a user order, each of which depends on gathering data and submitting multiple data points to several API endpoints. xpander AI connectors use several techniques, including setting up dependencies within an OpenAPI specification, to increase the success rate of agentic applications when required to follow a sequence of APIs.
Custom AI-ready tools
If an AI-ready connector doesn’t already exist for your NIM application target system, you can also use xpander AI to build a custom connector. By supplying details like API reference, docs, and others referenced previously, xpander AI can automatically generate an enriched AI-ready tool that is adjusted for your application.
This section walks you through an example of how to use xpander AI to equip your NIM application with tools, using AI-ready connectors.
Step 1. In the xpander AI Console, enable any prebuilt connector you want to use in your NIM application, or build a new connector to your homegrown enterprise system (Figure 3).
Step 2. Next, select the exact API operations that are made available to your NIM application (Figure 4).
Step 3. You’ll then receive integration details to be used with the xpander SDK within your NIM application (Figure 5).
Step 4. As the code sample below shows, by using the xpander SDK within the NIM application, tool calling will use tools provided by xpander. Specifically, the code sample uses chat completions to call the Llama 3.1 70B Instruct model hosted on the NVIDIA platform. The tools that were selected on the xpander AI side (in this case, Notion, Asana, and HubSpot) will be supplied to the model as tools. When the model determines that a tool should be invoked, the structured response will be returned to the application, and executed using the xpander SDK.
from xpander_sdk import XpanderClient, LLMProvider # pip install https://assets.xpanderai.io/xpander-sdk.tar.gz
from openai import OpenAI
from langchain_nvidia_ai_endpoints import ChatNVIDIA
# Get all models that supports tools calling
tool_models = [model for model in ChatNVIDIA.get_available_models() if model.supports_tools]
# Define xpander client
# Set agent key and URL in https://app.xpander.ai
xpander_client = XpanderClient(
agent_key="$<CHANGE_ME>", agent_url="https://inbound.xpander.ai/agent/<CHANGE_ME>",
llm_provider=LLMProvider.NVIDIA_NIM
)
# Get the tools in the specific Nvidia NIM format
xpander_tools = xpander_client.tools()
# Open an OpenAI API client. Key can be achived as part of NVIDIA AI Enterprise
openai_client = OpenAI(api_key="$<CHANGE_ME>",
base_url="https://integrate.api.nvidia.com/v1")
# Send messages to the LLM + The tools in the correct format of function calling
llm_response = openai_client.chat.completions.create(
model=tool_models[0].id, # randomly select first model that supports tool calling
messages=[{ "role": "user","content": "Israel medals olympics 2024"}],
tools=xpander_tools,
tool_choice="required",
max_tokens=1024,
)
# Process the chat completion response with the Xpander client
tool_response = xpander_client.xpander_tool_call(tool_selector_response=llm_response.model_dump())
for tool_response in tool_response:
print(tool_response.response_message)
AI agent success rate with xpander AI
Ensuring that AI agents reliably execute tasks in production is crucial. It can be evaluated by calculating an AI agent’s success rate—the proportion of successful task executions out of all attempts. xpander AI enhances this metric by converting existing APIs into agentic interfaces, enabling AI agents to interact with and control various systems with consistency and reliability.
To illustrate this, we compared the task execution success rates of xpander AI’s HubSpot Agentic Interface against HubSpot’s standard OpenAPI Specs using a synthetically generated dataset designed to simulate tasks of varying complexity—from entering a simple deal to generating a comprehensive company record with specific attributes and linking it to a campaign.
The impact on task execution reliability was significant. The HubSpot standard OpenAPI Specs delivered a success rate of 29.92%, while xpander AI’s HubSpot Agentic Interface achieved an 85.65% rate of success. This comparison demonstrates that xpander AI doesn’t just ensure that your agents function properly. Its tools also significantly enhance agent performance across real-world deployments and enable greater reliability in production environments.
Conclusion
The introduction of tool calling in NVIDIA NIM represents a significant advancement for agentic capabilities. NIM microservices can now enhance larger automation systems, which enables LLMs to select and leverage the most relevant tools to solve problems. With these enhancements, NIM applications can be equipped with function calling.
You can use xpander AI-ready connectors to equip your NIM applications with reliable agentic tools, as explained in this post. NIM offers compound advantages that improve runtime inference efficiencies, and xpander AI accelerates development and integration time while solutioning for real-world production use cases.
xpander AI looks forward to having tangible impact on scientific applications. As more use case-specific NIM microservices are released for biology, chemistry, drug discovery, and more, scientists can use xpander AI to integrate APIs into NIM applications without relying on backend and platform engineers.
Check out the latest NIM microservices documentation and visit xpander-ai / xpander-sdk on GitHub to learn more. Dive deeper into NIM in the NVIDIA AI Foundation Models and Endpoints Developer forum.
