Artificial Intelligence

AI Hallucination Detection Tools: W&B Weave & Comet

We benchmarked three hallucination detection tools: Weights & Biases (W&B) Weave HallucinationFree Scorer, Arize Phoenix HallucinationEvaluator, and Comet Opik Hallucination Metric, across 100 test cases. Each tool was evaluated on accuracy, precision, recall, and latency. We tested 100 responses (50 correct, 50 hallucinated) from factual Q&A scenarios against their source context. See the benchmark methodology.…

RAG

Best RAG Tools, Frameworks, and Libraries

RAG improves LLM responses by grounding them in external data instead of just what the model memorized in training. We benchmarked the components a RAG system is built from and gathered the results in one place, with a practical guide to choosing each part of the stack. See our benchmark results for each RAG component,…

Vector DB

Vector Database Benchmark: 7 Open-Source Engines for RAG

We benchmarked seven open-source, self-hosted vector databases as the retrieval layer of a RAG pipeline, each run one at a time on identical bge-m3 embeddings and real medical and technical queries, so the database index was the sole variable. The workload spanned MedRAG-50k, TechQA-28k, and a 2.25M-vector corpus across eight dimensions, from accuracy and retrieval…

AI Hardware

GPU Concurrency Benchmark: H100 vs H200 vs B200 vs MI300X

I have spent the last 20 years focusing on system-level computational performance optimization. We benchmarked the latest NVIDIA GPUs, including the NVIDIA’s H100, H200, and B200, and AMD’s MI300X, for concurrency scaling analysis. Using the vLLM framework with the gpt-oss-20b model, we tested how these GPUs handle concurrent requests, from 1 to 512. By measuring…

Vector DB

Vector Database Sizing and Selection Calculator

The practical question behind a self-hosted vector database for RAG is which engine fits a given server, and which one the workload rules out. The calculator below answers both, from our benchmark of seven self-hosted vector databases run at matched recall on identical embeddings. Five checkboxes at the top of the calculator name the workloads…

LLM

Text-to-SQL: Comparison of LLM Accuracy

I have relied on SQL for data analysis for 18 years, beginning in my days as a consultant. Translating natural-language questions into SQL makes data more accessible, allowing anyone, even those without technical skills, to work directly with databases. We used our text-to-SQL benchmark methodology on 35+ large language models (LLMs) to assess their performance…

RAG

Top 20+ Agentic RAG Frameworks

Agentic RAG enhances traditional RAG by boosting LLM performance and enabling greater specialization. We conducted a benchmark to assess its performance on routing between multiple databases and generating queries. Explore agentic RAG frameworks and libraries, key differences from standard RAG, benefits, and challenges to unlock their full potential. We used our agentic RAG benchmark methodology…

AI Coding