Three applications reduced the configuration required to use local models, while llama.cpp and vLLM received new optimizations with gains of up to 1.9x.
Installing an AI agent locally still means selecting a model, finding a compatible inference engine, configuring quantization, and handling updates. NVIDIA is working with some of the most popular designs to eliminate a lot of these steps Make launching a local template closer to installing a normal application.
The first solutions included include Hermes Agent, OpenClaw and Perplexity Portable Computer. All of this will use llama.cpp and NVIDIA optimizations to simplify setup on compatible systems.
Perplexity laptops will be available in September, equipped with NVIDIA RTX GPUs, equipped with at least 24 GB of VRAM, and can run Windows and Linux. The software allows you to run the entire workflow directly on your computer without spending points and delegate parts of the work to others when necessary More than 15 cutting-edge templates available in the cloud.
The move to the cloud won’t happen without controls: portable computers must ask users for permission before sending content out. This allows, for example, to analyze financial documents or company data locally and resort to online services only when additional research or reasoning skills are required.
Hermes will automatically select the model and configuration
The Hermes Agent, developed by Nous Research, will receive a one-click native setup process on RTX and DGX systems running Windows and Linux. The program will automatically detect the NVIDIA GPU present in your computer. Will choose the appropriate model and configuration And will use llama.cpp integrated with NVIDIA optimization.
Therefore, users do not need to manually download models or adjust key parameters. Once launched, Hermes will continue to use the tool, maintain activity context, remember information between sessions, and create reusable skills over time.
OpenClaw is following a similar path: NVIDIA, Microsoft, and the project team are collaborating on a Windows version of the application that will simplify Configure optimized native models on RTX with at least 24 GB VRAM. NVIDIA calls OpenClaw the largest AI project on GitHub, with over 380,000 stars at the time of publication.
llama.cpp throughput increased by 1.9x
NVIDIA also continues to work with the llama.cpp and vLLM communities to improve inference on our platform. On GeForce RTX 5090, the latest llama.cpp optimization helps Increased throughput by up to 1.9x. NVIDIA attributes this result to kernel intervention, new speculative decoding technology, and faster prefilling.
In contrast, vLLM shows a performance improvement of 1.2x for the RTX PRO 6000 Blackwell Workstation Edition, and up to 1.4x when using two DGX Spark systems. In this case, the new XQA core dedicated to attention in FlashInfer comes into play along with other backend optimizations. These improvements are already accessible through LM Studio, Ollama, llama.cpp and vLLM.
NVIDIA also took the opportunity to summarize several local models that have arrived in recent weeks. These include the 30 billion parameter Nemotron 3.5 Lightning, Z.ai’s GLM-5.3-Flash, Qwen3.8-Flash-Next and Qwen3.8-27B, as well as the LTX 2.5 video model.
The list also includes MiniMax-H3 for producing video with synchronized audio, Meta’s Muse Glimmer for encoding and proxy tasks, and the Mixture-of-Experts model from DeepSeek v4 Flash 284 billion total parameters and 13 billion active parameters Executable locally on DGX Station.
