You can now chat with your GPU
NVIDIA offers new AI features for RTX 30/40 users.
NVIDIA has introduced a new feature called “Chat with RTX,” providing an alternative for users uncomfortable with online AI chats. This AI chatbot runs locally on GeForce RTX 30 and 40 GPUs, leveraging TensorRT-LLM and Retrieval Augmented Generated (RAG) software fully supported by the RTX GPUs and accelerated by the integrated Tensor cores.
Chat with NVIDIA’s RTX aims to enable everyone to harness the power of their graphics card for AI tasks. We’ve mostly seen AI do cool things with images and videos, but until now the discussions have been sort of stuck in large data centers due to the resources needed for these complex AI models with tons of variables.
Now users can download a lite version of these AI models, some of which are available for download for free. But, as with any new technology, setting it up for regular users can be a bit tricky. NVIDIA apparently wanted to make this process much simpler with a single application that runs on your machine, and which you can launch through your browser, making the whole process much more user-friendly.
Chat with RTX is more than just a word processor, it can access different file formats including text, pdf, doc/docx and xml. It leverages the capabilities of popular large language models such as Mistral or Llama2 to generate responses and can also use online resources such as YouTube videos.
Currently, NVIDIA confirms support only for GeForce RTX 30 and RTX 40 GPUs, with no mention of the RTX 20 series. GPUs must have a minimum of 8 GB of VRAM, excluding the RTX 3050 6 GB from compatibility.
Chat will soon be free to download, you can check the following link for more information.
Source: NVIDIA via Wccftech
Gn En tech