Supercharge Generative AI Development with Firebase Genkit, Optimized by NVIDIA RTX GPUs

At Google I/O 2024, Google announced Firebase Genkit, a new open-source framework for developers to add generative AI to web and mobile applications using models like Google Gemini, Google Gemma. With Firebase Genkit, you can build apps that integrate intelligent agents, automate customer support, use semantic search, and convert unstructured data into insights. Genkit also includes a developer UI that enables you to prototype, develop, and test AI features in your local developer environment.

Firebase Genkit UI to develop applications — *Figure 1. Screenshot of Firebase Genkit that includes a developer UI that enables a user to prototype, develop, and test AI features in your local developer environment.*

Run Firebase Genkit locally on NVIDIA RTX GPUs

Google collaborated with NVIDIA to optimize inference performance for Google’s Gemma model. NVIDIA RTX professional GPUs and NVIDIA GeForce RTX GPUs can increase inference performance to speed up your developer productivity and make your solutions snappier.

Building with Firebase Genkit, you can unlock these benefits by running Genkit locally on NVIDIA GPUs and using Genkit’s plugin for integrating Ollama for hosting Gemma on your local machine. Ollama is an open-source framework that makes it easy to get started with large language models (LLMs) locally. By using it in combination with Gemma, you can get all the benefits of increased inference performance right on your local desktop.

Install Ollama

You can use Windows, for this example, but you can also use Mac, Linux, or Docker. Download Ollama for Windows. Once installed, open the Windows command prompt to access Ollama.

First, pull the optimized Gemma model:

$ ollama pull gemma

Next, run Gemma:

$ ollama run gemma

This will start Gemma on your computer, making Gemma available to Firebase Genkit. For more details on installing, running, and testing Ollama, visit Ollama on GitHub.

Install Node.js

Firebase Genkit supports development in JavaScript / TypeScript and requires Node.js, an open-source, cross-platform JavaScript runtime environment. To use Genkit, you must download and install Node.js, if it’s not already installed. Firebase Genkit has been tested with Node.js version 20.0.

Install Genkit

Genkit is available through the Node Package Manager (npm). Install Genkit using the -g parameter to install it globally. Using the -g parameter, the package will be available from any directory in your terminal, not just the current directory and project.

$ npm i -g genkit

Once Genkit is installed, create and initialize a new Node project.

$ mkdir genkit-intro && cd genkit-intro
$ npm init

Make sure to change the main field in the package.json file to lib/index.js. As a last step, initialize the Genkit project.

$ genkit init

During initialization, you will be prompted to select a deployment platform. Choose Node.js to run locally. The Genkit init command creates a sample source file, index.ts, which is your project entry point. The default sample file, shown below, can be configured to meet your project needs.

configureGenkit
({

  plugins:[googleAI()],
  logLevel:'debug',
  enableTracingAndMetrics:true,
});

export const menuSuggestionFlow = defineFlow(
{
    name:'menuSuggestionFlow',
    inputSchema: z.string(),
    outputSchema: z.string(),
},
async (subject)=>{
  const llmResponse = await generate({
      prompt:`Suggest an item for the menu of a {subject} themed restaurant`,
      model:$GENKIT_MODEL,
      config: {
        temperature:1,     
    }, 
  });
  return llmResponse.text();
 }
);

startFlowsServer();