PrismShine: A Self-Hosted Anti-Hallucination Verdict Engine

Overview

PrismShine is a self-hosted tool designed to verify the accuracy of answers generated by AI agents and RAG systems. It acts as an anti-hallucination verdict engine that checks if an answer is grounded in the provided context before and after it is created. Unlike standard tools that only look at the final text, PrismShine analyzes runtime evidence such as traces and node states to detect errors early. A PASS verdict from PrismShine means the answer is supported by the provided context, not that the context itself is factually correct in the real world. It is not a prompt-injection firewall and does not verify mid-stream tokens.

Benefits

PrismShine offers several key advantages for developers and businesses using AI systems. It fuses two primary detection layers into a unified verdict. The first layer, Cause-Side Forensics, occurs before or immediately after generation. It detects issues like empty retrieval, swallowed tool errors, truncated context, and stale cache reuse. This layer can halt generation if preloads are broken, preventing wasted LLM tokens. The second layer, Effect-Side Grounding, occurs after the answer exists. It includes four tiers of checks. Tier-1 verifies exact figures, entities, and dates with zero false positives. Tier-2 checks vector support using runtime embeddings. Tier-3 offers optional span-level detection using ONNX models. Tier-4 provides an opt-in gray-zone check using external LLM judges for residual ambiguity. Benchmarks show that PrismShine achieves high accuracy in QA and number verification while maintaining low latency and requiring zero LLM calls for its core checks.

Use Cases

PrismShine is suitable for various applications involving AI agents and RAG systems. It can be used as a standalone verification tool for scripts, batch evaluation, or services that have existing questions, answers, and context. It is also recommended for custom agent flows using frameworks like LangGraph or ChorusGraph. In these scenarios, PrismShine provides interceptors to halt before token generation if the preload is broken. Developers can wrap the provider boundary to ensure verification happens at the right time. It supports domain-specific profiles for finance, clinical, legal, and general use cases. The tool includes CLI tools for listing capabilities, running verification, collecting feedback, and calibrating the system for specific domains.

Pricing

PrismShine is available under the Apache-2.0 license. It is a self-hosted solution, which means there are no subscription fees or usage-based pricing. Users can install the core package using pip and run it on their own infrastructure. Optional extras like Tier-3 Spans and integrations for ChorusGraph, LangGraph, and LLM judges are also available as pip packages. The core package runs on CPU with zero LLM dependencies, making it cost-effective for deployment.

Vibes

Public reception and reviews are not explicitly detailed in the provided information. However, the benchmarks and performance metrics suggest a positive reception among technical users. PrismShine outperforms other systems like HHEM-2.1-Open in terms of accuracy and latency. It achieves a B2 Numbers F1 score of 1.000 with zero false positives and a p50 latency of 90 ms. The tool is designed to catch at least 90% of injected runtime failures without requiring any model calls. This performance indicates that PrismShine is a reliable and efficient solution for verifying AI-generated content.

Additional Information

PrismShine is developed by Insight IT Solutions LLC. The current version is 0.2.2, which includes drop-in DX, Guard >=0.1.9, and Runtime Docker receipt features. The tool supports domain-specific profiles via built-in handbook packs. Calibration commands are available to adjust thresholds for specific domains. The production checklist recommends installing Tier-3 ONNX, running domain-specific calibration, implementing prellmcheck interceptors, and verifying receipts before deploying claims. The license is Apache-2.0, which allows for flexible use and distribution.