Interested in ChatGPT but reluctant to share your private information? Nvidia now offers a way to operate your own language model locally, without compromising your privacy. The new tool is called Chat With RTX.
Source : Nvidia
Want to have your own local ChatGPT? Above all, no judgment, everyone’s dreams are unique. But, there is a new solution. Nvidia has just launched a tool for those of us who like to test AI without leaving the comfort of our PC: Chat With RTX. It’s an application that turns your PC into an artificial intelligence server, provided you have an Nvidia GPU under the hood, obviously.
Imagine being able to chat with your computer and ask it to dig through your documents or summarize a YouTube video without having to lift a finger. This is precisely what Chat with RTX offers. Nvidia calls it a demo, but make no mistake, it’s already quite powerful and certainly useful.
Mistral or LLaMA 2 of your choice
So, it is not GPT which is used as an LLM. Here, we’re talking about ChatGPT to make it easier to understand. Users have the choice between Mistral and LLaMA 2 (Meta) as language models (LLM) to animate their Chat With RTX. These are two open-source LLMs.
After selecting a folder of documents to analyze, you can query the chatbot which will extract the relevant information. Besides, it works well in French.
For example, if you ask Chat With RTX to find your flight details in a vacation record, it will provide you with the information in text form and link the source documents for quick access. Likewise, it can summarize an activity schedule in PDF or extract information from YouTube videos, simply by inserting the link into the application.
Source : Nvidia
As we said at the beginning, Chat With RTX requires an Nvidia GPU on your computer. The chatbot is compatible with all RTX 30 and RTX 40 series models with at least 8 GB of VRAM. Other requirements to install it are having Windows 11, at least 16 GB of RAM, and version 535.11 or higher of your graphics card drivers.
The application itself is a bit greedy: 40 GB of storage space and 3 GB of RAM. But it’s worth it, especially if you like having control of your data. Everything happens locally on your machine, so no more cloud-related privacy concerns. You can download the app here.