News

Executive Summary

This report integrates the results of Unovia's verification of "building a long-form internal knowledge base" with publicly available industry benchmarks to organize the architecture that Japanese companies should choose now. The key points are as follows:

1. Simple RAGs suffer significant performance degradation when dealing with questions that span multiple facts.Gemini 1.5 Pro achieves a recall of 99.71 TP3T in single-fact searches, but averages around 601 TP3T in realistic multi-fact searches.[1] The latter is the real problem for enterprises.
2. Long Context (a 1 million token window) is not a panacea either.Claude Opus 4.6 maintains 78.3% with MRCR v2 1M, but drops to 36.6% with GPT-5.4, 18.5% with Gemini 3.1 Pro, and 25.9% with Sonnet 4.5[2]. The cost is about 1,250 times that of RAG, and the latency is 30 to 60 times higher[3]. The "just put it all in" strategy does not work at the product grade level.
3. The answer lies in Context Engineering.A discipline that incorporates RAG as a single acquisition method and integrates governance, classification, authorization, and history management into query-time. Companies that have adopted it have reported accuracy improvements of 35 to 601 TP3T, and 94 to 991 TP3T in governed contexts.[4][5]

A similar trend was observed in the Unovia test. Details are summarized below.

1. Background – Why "RAG alone" is not enough

For the past three years, Vector RAG has been the de facto standard for building our internal knowledge base. It converts queries into embeddings, retrieves the top k results using nearest neighbor search, and passes them to LLM to generate answers. It's simple and easy to implement.

However, organizations that have put the system into production in an enterprise environment have reported common complaints.

• The accuracy decreases when asking questions that span Sections 3.2 and 7.1 of the contract.
• I cannot provide an answer that integrates "an internal Slack thread from 3 years ago + the latest meeting minutes + relevant laws and regulations."
• Old and new information are mixed together, making it impossible to determine which is correct.
• Search results are returned without considering departmental permissions.

These are not Retrieval (retrieval) issues.The problem of Context Quality (the quality of the context provided).[5]

In fact, industry research has concluded that "the majority of enterprise RAG project failures are due to contextual quality, not search recall."[5]

2. Organizing the three options

Currently, there are three main approaches to long-text knowledge retrieval.

2.1. Vector RAG (Conventional Type)

Embed the query → perform a nearest neighbor search → input the top k results into LLM.

2.2. Long Context (direct input type)

The entire knowledge base will be directly deployed to LLM (Claude Opus 4.6, Gemini 3 Pro, GPT-5), which has a 1 million token window.

2.3. Context Engineering (Integration as Discipline)

The RAG is positioned as "one of the means of acquisition," and on top of it, classification, authorization, authentication, column-level lineage, and time-series freshness management are layered on query-time. Combining RAG + Memory + Knowledge Graph under the governance layer is currently the most advanced implementation pattern [4][5].

3. Verification of Unovia: Comparison of three methods using the internal knowledge bases of Japanese companies

Unovia conducted verification of the three methods described above using a dataset designed for internal knowledge retrieval in Japanese companies.

3.1. Verification Settings

• Target dataA mix of contracts, internal regulations, meeting minutes, Slack/Teams logs, technical documents, and industry reports worth approximately 800,000 tokens.
• Data CharacteristicsJapanese + English mixed, revision history available, departmental access rights, 3-year timeline.
• Verification methodsThe above three methods (RAG / Long Context / Context Engineering) were implemented and compared using the same queryset.
• Evaluation metrics：
• Single-Fact Retrieval: Retrieving specific numbers, dates, or proper nouns within a document.
• Multi-Fact Composition: The synthesis of facts across multiple documents.
• Time-series recognition (Temporal Awareness): Distinguishing between "latest version" and "past version"
• Access control filter: Excluding information you do not have permission to view.
• Cost Latency

3.2. Observed Trends (Consistent with Public Benchmarks)

Due to the limited sample size for this analysis, this report focuses primarily on qualitative trends. For quantitative comparisons, please refer to the publicly available benchmarks referenced [1][2][4].

Vector RAG (standalone)

• Single fact search is powerful.
• Performance drops significantly with multi-fact integration (similar to public benchmarks).
• Time-series recognition is not possible (partial support may be available depending on the implementation).
• Permission filtering needs to be implemented at a separate layer.

Long Context (direct input)

• Claude Opus 4.6 maintains practical accuracy.
• In GPT-5 and Gemini 3, accuracy degrades significantly when 1M is input (consistent with public benchmarks) [2]
• The latency costs are unacceptable for production use (tens of seconds, several dollars in token costs per query).
• Permission filters require context filtering beforehand, which ultimately leads back to RAG.

Context Engineering (Integrated Configuration)

• Acquisition layer: The combined use of Vector RAG and Knowledge Graph (GraphRAG) covers enterprise Q&A 90% and above.[4]
• Governance layer: Evaluate classification tags, permissions, freshness (version), and data lineage based on query time.
• Memory layer: Separates conversation history and long-term memory, and injects only what is needed for each query.
• As a result, we achieved practical accuracy in all aspects: single facts, multiple facts, time series, and authority.

3.3. Important Observations

The most important facts confirmed through the verification process are as follows:

"Even with the widespread adoption of the Long Context model, RAG will not become obsolete."

A 1 million token window is a powerful tool for debugging and exploration. However, as a knowledge base for production use, its cost and latency make it impractical. Operating a system with 10,000 queries per day using Long Context would result in monthly costs of tens of millions of yen just for tokens.

Rather,"Passing appropriately narrowed context to the LLM under proper governance" is the discipline required for product-grade knowledge retrieval.This is the core of Context Engineering.

4. Alignment with overall industry trends

Publicly available industry reports also point in the same direction.

• Atlan(Data governance field): "Context Engineering is a higher-level concept that encompasses RAG, and is a discipline that integrates classification, authentication, columnar lineage, and access policies in query-time."[4]
• Callstack / RAGFlow(OSS RAG Ecosystem): "From the end of 2025 to the beginning of 2026, the focus of the discussion shifted from RAG alone to Context Engineering."[5][6]
• TianPan(Production Decision-Making Framework): "The million-token window is often a 'wrong tool.' Decision-making criteria should be cost, latency, and authority, not precision."[3]
• MindStudio"Claude 1M Token Context does not replace RAG; it improves RAG."[7]

In short, the conclusions observed in Unovia's verification are largely consistent with the latest industry trends: "Long Context is powerful, but it complements, not replaces," "RAG is necessary, but it's insufficient on its own," and "Context Engineering is the next layer."

5. Recommended implementation patterns for Japanese companies

Based on verification and industry research, this paper presents recommended architectures for building knowledge bases in Japanese companies.

Recommended configuration (phased deployment)

┌─────────────────────────────────────────────┐ │ Application Layer (UI / API) │ └──────────────────┬──────────────────────────┘ │ ┌──────────────────▼──────────────────────────┐ │ Context Engineering Layer (Governance) │ │ - Permission filter (department, position, contract) │ │ - Freshness management (latest version priority, revision history) │ │ - Classification tags (confidentiality, category) │ │ - Data lineage (lineage, source) │ └──────────────────┬──────────────────────────┘ │ ┌──────────────────▼────────────────────────────┐ │ Retrieval Composers │ │ ┌──────────┐ ┌──────────┐ ┌──────────┐ │ │ │ Vector │ │GraphRAG │ │ Memory │ │ │ │ RAG │ │ │ │ (History) │ │ │ └──────────┘ └──────────┘ └──────────┘ │ └──────────────────┬────────────────────────────┘ │ ┌──────────────────▼──────────────────────────┐ │ LLM Layer (long-context optional) │ │ - Normal queries: Claude Sonnet / GPT-5 (short window) │ │ - Exploration and debugging: Claude Opus 4.6 (1M window) │ └─────────────────────────────────────────────┘

Phase Proposal

Phase 1 (up to 3 months)Based on Vector RAG, ensuring accuracy in single-fact searches. Phase 2 (up to 6 months)Added a GraphRAG layer to address questions dealing with entity relationships. Phase 3 (up to 9 months)Context Engineering: Building a governance layer. Integrating permissions, freshness, and lineage with query time. Phase 4 (up to 12 months)The Long Context model will be incorporated as a supplementary tool for "exploration and debugging." Production queries will still operate within a narrowed context.

Instead of "creating everything at once," we'll start by producing a working version in Phase 1 and then gradually build up the layers. This is a realistic approach based on verification.

6. Conclusion

The correct answer in long-text knowledge searches lies somewhere between two extremes.

Neither the extreme view that "RAG is outdated, Long Context is here" nor the extreme view that "Long Context is unusable, RAG is the eternally correct solution" is entirely accurate.

The correct question is, "What do we acquire, how do we organize it, and how do we pass it to which model?"Design consciously at every stage of acquisition, organization, and deployment.That is the essence of Context Engineering, and it is where the knowledge layer of enterprise AI is headed next.

Unovia is positioned to consistently support Japanese companies through this transition, from Phase 1 to Phase 4.

Contact Us / Recruitment Information

For inquiries regarding internal knowledge bases, Context Engineering implementation, and RAG → Context Engineering migration, please contact us using the information below.

unoviax.co.jp/contact

Unovia is currently recruiting engineers to work on knowledge layer design, including Context Engineering, GraphRAG, and Memory Architecture. If you are interested in designing the "layer above LLM," please contact us through the same channel.

References

[1] Gemini 1.5 Pro on multi-fact retrieval -- While a single-fact NIAH search achieves a recall rate of 99.71 TP3T, realistic multi-fact searches average around 601 TP3T. https://alphacorp.ai/blog/is-rag-still-worth-it-in-the-age-of-million-token-context-windows

[2] MRCR v2 1M Token Benchmark(2026) ── Claude Opus 4.6: 78.3%, GPT-5.4: 36.6%, Gemini 3.1 Pro: 18.5%, Sonnet 4.5: 25.9%. https://www.mindstudio.ai/blog/1m-token-context-window-vs-rag-claude

[3] Long-Context vs RAG Cost/Latency Analysis(TianPan, 2026/4) -- A 1M token request costs approximately 1,250 times more and has 30-60 times the latency of RAG. https://tianpan.co/blog/2026-04-09-long-context-vs-rag-production-decision-framework

[4] Context Engineering vs. RAG(Atlan, 2026) -- Accuracy 94-991 TP3T in a governed context vs. 10-311 TP3T in an ungoverned context. Context Engineering is a discipline that integrates RAG + classification + access policy into query time. https://atlan.com/know/context-engineering-vs-rag/

[5] From RAG to Context: A 2025 Year-End Review(RAGFlow) -- The majority of enterprise RAG failures are due to contextual quality, not search recall. Vector retrieval + GraphRAG together cover Q&A 90% and above. https://ragflow.io/blog/rag-review-2025-from-rag-to-context

[6] RAG Is Dead. Long Live Context Engineering(Callstack) -- The focus of discussion will shift from RAG alone to Context Engineering between 2025 and 2026. https://www.callstack.com/blog/rag-is-dead-long-live-context-engineering-for-llm-systems

[7] Claude 1M Token Context: RAG Isn't Dead(MindStudio) -- The 1 million token window doesn't replace RAG; it makes RAG better. https://www.mindstudio.ai/blog/1m-token-context-window-vs-rag-claude

FAQ

Q. Is RAG really outdated? No. RAG remains a core technology. This report argues that "RAG alone is insufficient," not that "RAG is unnecessary." Vector RAG has limitations in multi-fact integration and access control, but Context Engineering reinforces these with Knowledge Graph, Memory, and Governance layers.

Q. Where should our company start? We recommend launching Vector RAG with a minimal configuration as Phase 1. Once you have a working version, you can add more layers in Phase 2 and beyond. Trying to build a complete Context Engineering from the start will fail due to implementation complexity.

Q. Is the 1 million token window really necessary? It's usually unnecessary for production queries. It's a powerful tool for "exploration and debugging," such as reading the entire codebase or analyzing long contracts in bulk. For production use, a short window + RAG + Context Engineering is often sufficient.

Q. How accurate is it with the Japanese dataset? While most public benchmarks are English-based, Unovia's PoC confirmed practical accuracy with mixed Japanese and English data using a context engineering configuration. Japanese-specific issues (ambiguous references, honorifics, homonyms) need to be addressed in the preprocessing layer.

Q. What are the more detailed verification data? Due to dataset privacy concerns, this report is limited to qualitative descriptions and is used in conjunction with public benchmarks. For specific verification configurations and results, please contact us. unoviax.co.jp/contact Please contact us.