NVIDIA’s On-Device Intelligence Upgraded to Include Google Gemma and Powerful ChatGLM3

The third update for NVIDIA’s ChatRTX app has arrived. It adds new image recognition capabilities, voice input functionality, and an expanded selection of open-source LLMs.

NVIDIA ChatRTX , previously called “Chat with RTX ,” is a personalizable AI chatbot that can run locally on Windows PCs. It acts as a sort of personal assistant, answering questions about documents, notes, and other files on your computer without any cloud interaction. It can also summarize YouTube videos, which may be useful when you want to obtain a recipe or go through a tutorial without listening to someone ramble for 15 minutes. (However, YouTube scanning has been unavailable since the 0.2 security patch .)

One of the interesting things about ChatRTX is its ability to run a variety of LLMs. ChatRTX launched with support for Mistral and Llama 2, though it now supports Google Gemma (a lightweight offshoot of Gemini ) and ChatGLM3 (an English and Chinese LLM based on the GLM framework ). Each of these LLMs are suited for certain tasks, hence the need for variety.

The new ChatRTX release also boasts Whisper and Open AI CLIP integration. While Whisper adds AI-based speech recognition for voice input, CLIP enhances ChatRTX’s image recognition capabilities. A photographer may use ChatRTX to find and organize a bunch of unlabeled pictures, for example.

To be clear, ChatRTX is still considered a “tech demo.” It’s an example of what developers can build using the TensorRT-LLM RAG open-source reference project, and it’s something that may help the average computer user grow more familiar with LLMs.

And I should note that none of these ChatRTX-compatible LLMs actually require ChatRTX. The benefit of ChatRTX is that it doesn’t require any expertise and allows you to train a personalized local chatbot without too much legwork. While I hope that ChatRTX grows into a real product, it doesn’t need to do so, as it will inspire developers to build even better AI applications.

You can install ChatRTX today for free from the NVIDIA website. However, ChatRTX requires an RTX 30- or 40-series GPU with at least 8GB of VRAM (yes, some laptops meet these specs). You’ll also need enough patience to wait through a 30GB download—ChatRTX is a relatively large application.

Source: NVIDIA

Also read: