NVIDIA Nemotron 3.5 And The Next Step In Agent Retrieval

Read the full analysis: NVIDIA Nemotron 3.5 And The Next Step In Agent Retrieval on ThorstenMeyerAI.com

NVIDIA has expanded its Nemotron Retriever collection with five open models for embedding, reranking and document parsing, targeting AI agents that need to find evidence across text and image-rich documents, as described in the original analysis. The company says selected evaluations showed improved retrieval results and lower agent inference-token use, but those figures are NVIDIA-reported and the supplied announcement does not include independent validation.

The new collection comprises Nemotron 3.5 Embed 8B BF16, Embed 1B BF16, Embed 1B NVFP4, Rerank 8B BF16 and Parse, with 631 million parameters. NVIDIA describes the embedding models as supporting retrieval from text and visual material, including tables, charts, infographics, datasheets and schematics. The reranker scores query-document pairs to help an agent prioritize results for its working context.

NVIDIA says its evaluations on ViDoRe v3, BRIGHT and BrowseComp-Plus held the language model, tools, tasks, corpus and execution policy fixed while comparing retrieval configurations. It reports that adding Nemotron 3.5 Rerank reduced agent inference tokens per query by about 14% with Embed 8B and 13% with Embed 1B, while improving retrieval accuracy. The company also says Embed 8B scored about 0.9 points higher in nDCG@10 than Qwen3-VL-Reranker-8B at a similar agent token cost.

For ViDoRe v3.1, NVIDIA reports an nDCG@10 score of 61.17 for Embed 8B, compared with 52.23 for Llama Nemotron Embed VL 1B. Embed 1B BF16 scored 58.38, which NVIDIA describes as 95.4% of the 8B model’s score. These are results from the company’s stated test setup; they do not establish how the models will perform on other document collections or agent workloads.

Retrieval Can Cut Agent Search Costs

Retrieval affects both the evidence an agent uses and how much work it does before producing an answer. When an agent repeatedly searches or inspects irrelevant material, it may spend more inference tokens without gaining useful information. NVIDIA’s reported results connect retrieval quality with agent task performance and token use, rather than treating search as a separate preparation step.

The models’ multimodal focus addresses a common obstacle in document-heavy workflows: useful evidence may sit inside a chart, table or schematic rather than in text that a conventional search index can find. If agents can retrieve that material reliably, they may be better equipped to answer questions that require evidence from different document formats. The practical effect will depend on the corpus, hardware, workload and system integration; the announcement does not show that the reported savings apply across organizations.

For teams considering agent deployment, the release offers several model sizes and formats, including a smaller 1B embedding option and an NVFP4 version positioned for inference efficiency on supported hardware. That gives developers options to test, but benchmark results alone do not settle questions about operating cost, accuracy on internal data or the effort required to integrate the models.

From Embeddings to Visual Documents

Embedding models represent queries and documents in a form that allows a search system to find semantically related material. A reranker then reorders candidate results by comparing them with a query. NVIDIA presents Nemotron 3.5 as part of an agent workflow in which a system searches, inspects evidence and may refine its search in response to what it finds. The new collection extends the company’s earlier Nemotron 3 Embed text models toward visual document content.

NVIDIA positions Embed 8B BF16 for quality-focused retrieval and Embed 1B BF16 as a smaller multimodal option. The 1B NVFP4 version uses quantization for its text and image encoders, with efficiency dependent on supported hardware. The company also says its fine-tuning recipes can adapt embedding models to custom collections. In experiments involving charts and infographics, NVIDIA reports relative nDCG@10 gains of 18% for Embed 1B BF16 and 3% for Embed 8B BF16.

NVIDIA says it evaluated text retrieval across 34 languages and multilingual visual retrieval on MIRACL Vision across 18 languages, as well as on KoViDoRe. The source says the models may support additional languages, but reports evaluation results only for those specified sets. It lists Parse as part of the collection but supplies little information about its functions or test results.

“Adding Nemotron 3.5 Rerank reduces agentic token cost per query by approximately 14% with Nemotron 3.5 Embed 8B, and 13% with Nemotron 3.5 Embed 1B, while improving retrieval accuracy in both configurations.”

— NVIDIA, in its Hugging Face announcement

Benchmark Scope and Parse Details

The supplied announcement does not provide a publication date, full benchmark methodology or complete results tables, and it includes no independent replication of the reported scores or token savings. It is also not clear how the reported performance changes across different corpora, languages, hardware configurations or agent tasks. Benchmark results should not be treated as a guarantee for other workloads.

Details about Nemotron 3.5 Parse are especially limited: the source gives its parameter count but not a full account of its capabilities, supported formats or evaluation. The material also does not specify pricing, detailed licensing terms, hardware requirements beyond the NVFP4 deployment note, or the timing of broader availability. Although the collection is described as open and commercially available, the supplied source does not include release links or licensing documentation.

Documentation and Independent Tests

The next practical step for prospective users is to review the model artifacts and documentation as they become available, including licensing, deployment requirements and the methods behind NVIDIA’s benchmarks. Independent tests on enterprise document collections could show whether the reported retrieval gains and token reductions carry over to other datasets and agent workflows.

More complete documentation for Parse would clarify how it fits alongside embedding and reranking in a retrieval system. Until those details and external evaluations are available, the announcement establishes NVIDIA’s reported results and intended uses, but not a general performance outcome for organizations adopting the models.

Key Questions

What did NVIDIA announce?

NVIDIA expanded its Nemotron Retriever collection with five open models for multimodal embedding, reranking and document parsing: Embed 8B BF16, Embed 1B BF16, Embed 1B NVFP4, Rerank 8B BF16 and Parse.

What token savings did NVIDIA report?

NVIDIA says adding Nemotron 3.5 Rerank cut agent inference tokens per query by about 14% with Embed 8B and 13% with Embed 1B in its selected evaluations, while improving retrieval accuracy. These figures are company-reported and depend on the test setup.

Can the models retrieve information from charts and images?

NVIDIA says its embedding models are designed to retrieve information from visual material, including tables, charts, infographics, datasheets and schematics, as well as text. Performance on a particular document collection still needs to be tested.

Are the benchmark results independently verified?

The supplied announcement includes NVIDIA’s benchmark claims but does not provide independent replication. It also does not include full methodology or complete results tables.

What is known about Nemotron 3.5 Parse?

The source lists Parse as a 631-million-parameter model, but gives little detail about its capabilities, supported formats, availability or evaluation.

Primary source: Hugging Face · via ThorstenMeyerAI.com