v3.0: unify ff_inspect + mcp-stats2 + 5384 + noise-testing#5410
Merged
Conversation
Signed-off-by: Wazir Ahmed <wazir@proxysql.com>
Signed-off-by: Wazir Ahmed <wazir@proxysql.com>
…entation Signed-off-by: Wazir Ahmed <wazir@proxysql.com>
Signed-off-by: Wazir Ahmed <wazir@proxysql.com>
Signed-off-by: Wazir Ahmed <wazir@proxysql.com>
Signed-off-by: René Cannaò <rene@proxysql.com>
- Added TrafficObserver base interface - Implemented MySQLFFTO and PgSQLFFTO classes for protocol-aware observation - Integrated FFTO hooks into MySQL and PostgreSQL session handlers in FAST_FORWARD state - Added configuration variables: mysql-ffto_enabled, mysql-ffto_max_buffer_size, pgsql-ffto_enabled, pgsql-ffto_max_buffer_size - Implemented query digest extraction and reporting parity with internal stats - Added session-level bypass when packet size exceeds buffer threshold - Included architectural documentation in doc/ffto_design.md
- Implemented statement ID tracking for MySQL COM_STMT_PREPARE/EXECUTE - Implemented Parse/Bind/Execute portal mapping for PostgreSQL Extended Query - Corrected documentation regarding binary protocol support - Ensured metric parity for prepared statements in Fast Forward mode
- Added comprehensive MySQL FFTO test (text and binary protocols) - Added comprehensive PgSQL FFTO test (Simple and Extended Query protocols) - Added FFTO memory bypass logic test - Added bash orchestration scripts for each test group - Integrated new tests into the TAP Makefile
… scripts Following the CI convention where all binaries ending in -t in test/tap/tests/ are automatically executed, the subdirectories and run.sh scripts are no longer needed.
- Implemented MySQL OK_Packet parsing to extract affected_rows - Implemented PostgreSQL CommandComplete parsing to extract rows affected/sent - Updated TAP tests to validate sum_rows_affected and sum_rows_sent metrics - Fixed incorrect assertion in test_ffto_mysql-t.cpp
- Resolved signature mismatch in report_query_stats - Fixed missing type declarations for Query_Processor in FFTO classes - Verified Metric Parity for affected_rows and rows_sent - Confirmed full coverage in TAP tests for text/binary protocols and bypass logic - Verified clean build with make debug
Problem MCP stats tools were querying stats.* tables directly through admindb. Unlike admin-session SQL, that path bypassed GenericRefreshStatistics() and could return stale or empty data for runtime-populated tables. What changed - Updated Stats_Tool_Handler::execute_admin_query() to mirror admin-session semantics for stats reads: - acquire GloAdmin->sql_query_global_mutex - optionally invoke ProxySQL_Admin::GenericRefreshStatistics(sql, ..., false) - execute admindb statement - release sql_query_global_mutex - Added strict input validation and explicit lock/unlock error reporting. - Added refresh_before_query parameter (default true) to avoid duplicate refresh passes for secondary count queries. - Switched secondary COUNT(*) calls in show_processlist/show_queries/ show_errors/show_query_rules to refresh_before_query=false. Documentation - Expanded Doxygen in include/Stats_Tool_Handler.h for execute_admin_query() to document locking, refresh behavior, and performance tradeoff. - Added detailed Doxygen above the execute_admin_query() implementation in lib/Stats_Tool_Handler.cpp. Testing - Added TAP integration test test/tap/tests/mcp_stats_refresh-t.cpp. - The test injects a synthetic stale marker row into stats.stats_mysql_global, calls /mcp/stats show_status, and verifies the marker disappears after refresh-before-read. - Added helper-level Doxygen in the test for setup/parsing behavior. Build verification - Compiled lib object: make -C lib obj/Stats_Tool_Handler.oo - Compiled TAP target: make -C test/tap/tests mcp_stats_refresh-t - Direct runtime execution of the TAP test in this workspace could not fully run because no local admin listener was available (connection to 127.0.0.1:6032 failed).
Replace stats table reads in show_queries with Query Processor in-memory aggregation to avoid stale/empty admin-db snapshots.\n\nIntroduce Top-K digest filtering primitives and wire them into Stats_Tool_Handler so filtering/sorting/limit are executed against live digest memory. Add runtime MCP variable mcp-stats_show_queries_max_rows with validation, defaulting, and configuration plumbing, plus cap metadata in the tool response.\n\nEnforce configurable hard cap semantics while preserving deterministic ordering and pagination behavior for MCP clients. Add internal DEBUG validator path (PROXYSQLTEST 56) in admin tests to stress generated digest data and validate the Top-K/filter pipeline against in-memory state.\n\nKeep the validator callable from startup/testing workflows without depending on MCP linkage. Add TAP coverage in test/tap/tests/mcp_show_queries_topk-t.cpp for:\n- MCP endpoint setup and reachability\n- digest generation + optional DEBUG internal validator\n- payload validation and cap metadata assertions\n- descending count order validation\n- match_digest_text filter verification\n\nMake payload parsing backward-compatible in TAP (direct tool result object and legacy wrapped success/result payloads).
…andling - Finalized MySQL and PostgreSQL protocol state machines. - Implemented accurate affected_rows and rows_sent capture. - Added defensive null checks to prevent early-session crashes. - Enhanced TAP tests with result-set draining and success verification. - Verified 100% pass rate for MySQL CRUD, binary protocol, and memory bypass.
…stress TAP Replace direct stats-schema reads in MCP show_processlist with in-memory processlist access via SQL3_Processlist query options. Add processlist query option types in proxysql_admin interfaces and implement filtering, sorting, and pagination post-processing in both MySQL and PgSQL thread implementations. Introduce configurable MCP cap mcp_stats_show_processlist_max_rows (default 200, max 1000), wire it through MCP variable loading, and expose it in default config. Expand Stats_Tool_Handler processlist handling to parse filter/sort arguments, enforce cap metadata, and return consistent MCP payload fields from in-memory snapshots. Add extensive doxygen documentation in the touched headers and source blocks to describe API contracts, filtering behavior, and runtime constraints. Add new TAP test mcp_pgsql_concurrency_stress-t that generates sustained concurrent PgSQL traffic (simple reads, read/write table workload, randomized pg_sleep) while polling MCP show_processlist and show_queries in parallel with filter and ordering assertions. Validation performed locally: mcp_pgsql_concurrency_stress-t (31 assertions) and mcp_show_queries_topk-t (12 assertions).
… stress test Add a new MySQL-focused TAP workload (mcp_mysql_concurrency_stress-t) that mirrors the PgSQL stress model and continuously generates mixed traffic through ProxySQL while MCP stats is queried in parallel. The MySQL workload includes simple reads, read/write table traffic, and randomized sleep queries, while concurrent MCP pollers validate show_processlist and show_queries behavior for sorting, filtering, metadata, and final consistency checks. Add a second TAP workload (mcp_mixed_mysql_pgsql_concurrency_stress-t) that drives MySQL and PgSQL traffic simultaneously, then polls MCP for both protocols in parallel to validate cross-protocol processlist and query-digest stability under mixed load. Both tests create/drop their own workload tables, configure/restore MCP runtime settings, and expose deterministic TAP assertions on payload shape, cap metadata, filter correctness, ordering guarantees, and endpoint reachability. To reduce false negatives under high concurrency, workload execution checks use a bounded error budget tied to observed traffic volume rather than requiring absolute zero transient query failures. These TAPs can also be used as practical stress/load demonstrations because they run sustained concurrent workers and MCP poll loops with protocol-specific filtering.
… stress configurability Enhance mcp_mixed_mysql_pgsql_concurrency_stress-t with environment-driven load parameters so it can be reused as a quick demo, sustained stress run, or heavier load scenario without source edits. Add optional live MCP cap churn support that updates mcp-stats_show_processlist_max_rows and mcp-stats_show_queries_max_rows during active mixed MySQL+PgSQL traffic and concurrent MCP polling. Generalize cap-metadata assertions in processlist/show_queries pollers to support both fixed-cap and churned-cap modes through accepted cap profiles. Add mcp_mixed_stats_profile_matrix-t as an orchestrator TAP that executes multiple mixed-load profiles (quick, churn, heavy) and validates successful completion of each run. Add mcp_mixed_stats_cap_churn-t as a focused orchestrator TAP for aggressive cap-churn scenarios under mixed protocol traffic. Both orchestrator TAPs isolate child output to per-run log files, preserve parent TAP stream integrity, and emit diagnostic log tails on failures for easier triage. Compilation and runtime validation performed locally before commit: enhanced mixed stress TAP plus both new orchestrator TAPs passed.
Refactor MCP stats connection tools so operational pool metrics remain lightweight while debug-level free-connection details are exposed through a dedicated gated tool. Changes in this commit: - Updated stats tool catalog and dispatch to add show_free_connections and keep show_connections focused on per-server pool metrics only. - Removed free-connection payload from show_connections and added explicit compatibility error when callers still pass detail=true, with guidance to use show_free_connections. - Implemented show_free_connections using in-memory hostgroup manager snapshots (MySQL and PgSQL) with hostgroup/server filtering and summary counters. - Added MCP runtime variable mcp-stats_enable_debug_tools (default false), including variable registration, getter/setter handling, and config default in proxysql.cfg. - Added extensive doxygen comments across modified code paths to document behavior, rationale, filters, and output contracts. - Added TAP coverage in mcp_show_connections_commands_inmemory-t for: show_commands baseline, show_connections aggregate-only contract, debug tool gating behavior, and enabled-path validation for show_free_connections on both MySQL and PgSQL. - Registered the new TAP test in test groups.
Summary of improvements based on PR reviews: 1. Performance Optimization: - Implemented read-offset indices in MySQLFFTO and PgSQLFFTO to replace linear std::vector::erase calls, reducing per-packet processing from O(N) to O(1). 2. Memory & Resource Safety: - Added explicit handlers for COM_STMT_CLOSE (MySQL) and Close (PostgreSQL) messages to correctly clear internal statement/portal maps and prevent memory leaks. - Enforced *-ffto_max_buffer_size checks on the server-to-client data path, ensuring large responses correctly trigger FFTO bypass. - Improved PostgreSQL message parsing with strnlen and msg_len validation to prevent buffer over-reads and underflows. - Updated MySQL_Session::reset and PgSQL_Session::reset to clear FFTO state during session re-use. 3. Protocol Correctness & Robustness: - Fixed MySQL CLIENT_DEPRECATE_EOF detection to use the correct 0xFE terminator logic. - Implemented PostgreSQL metric accumulation across multiple CommandComplete responses within a single query cycle. - Optimized PostgreSQL row extraction using a static local regex with an improved non-greedy pattern. - Stripped trailing NUL bytes from PostgreSQL Simple Query payloads for accurate digest generation. 4. Code Quality & Test Hardening: - Replaced unsafe sprintf calls with snprintf in all new TAP tests. - Stabilized TAP plans by ensuring a constant number of ok() assertions across all success and failure paths. - Hardened the bypass test to verify total digest counts. - Applied modern C++ standards: replaced <stddef.h> with <cstddef>, used = default for virtual destructors, and added override keywords. - Fixed indentation and removed temporary debug log artifacts from production code.
Rework stats.show_users so it no longer queries runtime-populated stats tables via admindb. The tool now reads user connection counters directly from authentication runtime structures, matching the direction taken for other MCP stats tools that must avoid stale stats schema reads. Implementation details: - Replaced show_users SQL path over stats_mysql_users/stats_pgsql_users with in-memory collection from GloMyAuth::dump_all_users(..., false) and GloPgAuth::dump_all_users(..., false). - Preserved Admin semantics by excluding internal/admin-style accounts (default_hostgroup < 0). - For MySQL, included LDAP user counters from GloMyLdapAuth->dump_all_users() to keep parity with stats___mysql_users population behavior. - Added deterministic argument handling for db_type validation and pagination bounds (limit capped to 1000, offset clamped to >= 0). - Kept output contract unchanged (username, frontend_connections, frontend_max_connections, utilization_pct, status) and ordering by frontend_connections DESC then username ASC. Documentation and tests: - Added extensive doxygen comments describing in-memory data sources, filtering semantics, and result construction in show_users implementation. - Extended TAP test mcp_show_connections_commands_inmemory-t with show_users coverage for MySQL and PgSQL, including payload shape checks and username filter validation. - Updated TAP plan count accordingly. - Included current tap groups updates present in working tree (test_ffto_* entries in groups.json), as requested.
Removed explicit listings of several tests from the 'tests' target dependency list, as they are already automatically discovered and compiled by the 'tests-cpp' target via the *-t.cpp wildcard rule. Removed: - test_tsdb_variables-t - test_tsdb_api-t - test_ffto_mysql-t - test_ffto_pgsql-t - test_ffto_bypass-t - mcp_query_rules-t These tests do not require custom linking or special build rules beyond the generic %-t pattern, making their explicit inclusion in the main tests list redundant.
This commit completes the transition of MCP and GenAI testing to a modernized architecture. Changes: - Removed ~4,300 lines of deprecated shell scripts in mcp_rules_testing/ and associated orchestrators (test_mcp_query_rules-t.sh). These tests are now fully covered by the C++ test mcp_query_rules-t.cpp. - Added final diagnostic hints to genai_async-t.cpp to explicitly guide users when backend AI services (llama-server) are missing or unreachable. - Cleaned up the working tree to ensure all functional logic is consolidated in robust, observable C++ tests.
`flush_mcp_variables___runtime_to_database()` ignores the `replace` parameter. The prepared statement is hardcoded to `REPLACE INTO`, so even when callers pass `replace=false` (e.g. `GenericRefreshStatistics`), the function overwrites values in memory config with stale runtime values. Signed-off-by: Wazir Ahmed <wazir@proxysql.com>
`mcp_query_rules` and `runtime_mcp_query_rules` uses different representations for the same fields, causing the web UI to always report config differences even after loading to runtime. - Use `-1` sentinel for nullable int fields (`flagOUT`, `timeout_ms`, `log`) so `NULL` values from main are preserved when written to runtime, following the `MySQL QP_rule_t` pattern. - Serialize `re_modifiers` bitmask back to `VARCHAR` (`1→"CASELESS"`) to match the format used in main table - Update `evaluate_mcp_query_rules()` to handle tri-state log field Signed-off-by: Wazir Ahmed <wazir@proxysql.com>
fix: MCP config flush logic and runtime query rule representation
Signed-off-by: Wazir Ahmed <wazir@proxysql.com>
Contributor
|
retest this please |
…semantic lifecycle
This commit introduces three new C++ TAP tests that validate ProxySQL's live
GenAI and MCP behaviors using real provider credentials supplied via
environment variables. The goal is to move beyond mock-style checks and verify
actual runtime integration across request transport, tool execution, and
semantic outputs.
High-level scope
- Add a live GenAI embed/rerank validation TAP test.
- Add a live LLM bridge accuracy/error-path TAP test.
- Add a live MCP semantic lifecycle TAP test that combines discovery,
LLM-generated artifacts, upsert, and semantic search.
- Register all three new tests in ai-g1 group mapping.
Files added
- test/tap/tests/genai_live_validation-t.cpp
- test/tap/tests/llm_bridge_accuracy-t.cpp
- test/tap/tests/mcp_semantic_lifecycle-t.cpp
File updated
- test/tap/groups/groups.json
Detailed behavior by test
1) genai_live_validation-t.cpp
- Reads required live env inputs:
TAP_EMBED_URL, TAP_EMBED_TYPE, TAP_EMBED_MODEL, TAP_EMBED_DIMENSION,
TAP_RERANK_URL, TAP_RERANK_MODEL
- Skips (does not fail) when required environment is missing.
- Configures runtime for test stability:
- sets genai-vector_db_path to ./ai_features.db
- enables genai-enabled
- sets embed/rerank endpoints and embedding model
- loads GENAI variables to runtime
- Embedding integrity validation:
- sends GENAI embed request with multiple documents
- verifies row count matches input document count
- verifies each returned embedding dimension matches TAP_EMBED_DIMENSION
- Rerank semantic validation:
- sends query with one intentionally relevant document and irrelevant distractors
- checks highest score maps to the relevant document index
- Stress validation:
- opens 5 client connections
- executes 20 total requests (4 per connection) concurrently
- validates all requests succeed and no failures are reported
- Includes high-verbosity diagnostics for SQL requests and parsed rows.
2) llm_bridge_accuracy-t.cpp
- Reads required live env inputs:
TAP_LLM_PROVIDER, TAP_LLM_URL, TAP_LLM_MODEL, TAP_LLM_KEY
- Skips (does not fail) when required environment is missing.
- Configures LLM bridge runtime:
- sets genai-vector_db_path
- enables genai-enabled and genai-llm_enabled
- sets provider/url/model/key
- loads GENAI variables to runtime
- Special-character prompt handling:
- issues LLM: prompt containing quotes, backslashes, JSON-like text,
and emoji bytes
- verifies request succeeds and response structure is valid
- verifies returned provider column aligns with TAP_LLM_PROVIDER
- Timeout/error-path validation:
- reconfigures provider URL to an unroutable timeout-probe endpoint
- sets genai-llm_timeout_ms=1000 (minimum valid bound in current code)
- verifies client receives an error path with SQLSTATE HY000 and non-empty message
- Captures and restores modified global variables at end of test.
3) mcp_semantic_lifecycle-t.cpp
- Reads required live env inputs:
TAP_LLM_PROVIDER, TAP_LLM_URL, TAP_LLM_MODEL, TAP_LLM_KEY
- Skips (does not fail) when required environment is missing.
- Configures LLM and MCP runtime for end-to-end lifecycle checks:
- enables genai + llm bridge
- configures MCP port/auth endpoint settings
- creates MCP auth and target profiles
- loads MCP variables/profiles to runtime
- End-to-end lifecycle:
- calls discovery.run_static and validates run_id
- lists discovered table objects and selects object_ids
- starts agent run via agent.run_start
- generates summary text through LLM: bridge for two semantic markers
- persists summaries via llm.summary_upsert for two objects
- validates llm.search("customer") finds customer marker
- validates llm.search("index") finds index marker
- finishes run via agent.run_finish
- Cleans up test MCP profiles and restores runtime variables.
Groups registration
- Added to ai-g1 in test/tap/groups/groups.json:
- llm_bridge_accuracy-t
- genai_live_validation-t
- mcp_semantic_lifecycle-t
Implementation notes and constraints reflected in tests
- Tests are intentionally environment-gated and skip when live credentials are
unavailable.
- All tests include verbose diagnostics for outbound requests and parsed
provider/tool responses.
- Runtime variable mutations are restored best-effort to reduce suite side effects.
- llm timeout validation uses 1000ms because current runtime validation enforces
[1000..600000] for genai-llm_timeout_ms.
Build verification performed
- Compiled successfully (as jenkins user):
- genai_live_validation-t
- llm_bridge_accuracy-t
- mcp_semantic_lifecycle-t
This commit intentionally focuses on live integration correctness and transport
behavior under real endpoints, while remaining TAP-friendly for CI environments
that may not provide credentials (skip semantics instead of hard failures).
Add #ifdef __linux__ guard around sys/eventfd.h include to fix compilation on non-Linux platforms.
This was referenced Feb 24, 2026
v3.0-genai_keys: GenAI live validations, LLM bridge tests, and portability fix
|
This was referenced Mar 1, 2026
This was referenced Mar 18, 2026
5 tasks
16 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This PR consolidates four large branches into a single integration branch to reduce CI duplication and validate cross-branch interactions in one place.
Merged branches:
v3.0-5384v3.0-noise-testingv4.0-mcp-stats2v3.0-ff_inspectIntegration branch:
v3.0-unified-large-pr(base:v3.0)Merge Strategy
v3.0--no-ffto preserve branch traceability in historyConflict Resolution
One conflict was resolved:
test/tap/tests/MakefileResolution approach:
Additional Pre-CI Hardening Included
After integration, a focused pre-CI review/fix pass was applied:
test/tap/tap/stb2waEKtest/tap/tap/stot76DdGENAI:path now requiresgenai_enabled=trueLLM:path now requires bothgenai_enabled=trueandgenai_llm_enabled=truesrc/*.pem/mcp/observereferences to/mcp/statsmcp-observe_endpoint_authtomcp-stats_endpoint_authValidation Performed
test/tap/groups/groups.jsonparses correctlymake -C test/tap/tap -n allmake -C test/tap/tests -n testspython3 -m py_compile test/tap/noise/noise_stats_poller.pygit diff --checkNotes for Reviewers
Summary by CodeRabbit
New Features
Documentation
Tests
Chores