[Roadmap] Incremental Rust modernization for HugeGraph Toolchain

[imbajin]

Goal

Explore an incremental Rust implementation of selected HugeGraph Toolchain components. The intent is to improve safety, portability, startup/runtime efficiency, and contributor experience where Rust provides measurable value.

This is a parent tracking issue, not a commitment to replace the existing implementation wholesale. Existing Java/Go components remain supported while Rust alternatives are designed, benchmarked, and adopted step by step.

Roadmap at a glance

┌─ Toolchain: first phase ───────────────────────────┐
│  Rust client ─┐                                    │
│               ├─> shared tests/contracts ─> Tools  │
│  Rust Loader ─┘                                    │
└────────────────────────────────────────────────────┘
                         │
                         v
       Server core -> HStore -> PD -> Computer -> Vermeer/OLAP

Docs, compatibility checks, and benchmarks span every phase.

Proposed priority

1. Rust client and Loader (parallel)
   • Use the Java client and current Loader behavior as the compatibility reference.
   • Start with a minimal client: transport, authentication, schema, graph, and Gremlin APIs.
   • Build a Loader pipeline on top of the client with bounded memory, backpressure, retry, and resumability.
2. Tools
   • Port small, well-bounded commands after the client APIs stabilize.
   • Preserve command-line behavior and backup/restore compatibility.
3. Other components
   • Evaluate independently; migrate only when the benefit and maintenance cost are clear.

Delivery model

proposal/RFC -> minimal vertical slice -> compatibility tests
             -> benchmark -> opt-in preview -> production-ready candidate

Planning guardrails (required)

Before implementation starts, each child issue must define:

• scope, owner/mentor, and non-goals;
• API/CLI, protocol, data/storage-format, and operational compatibility;
• differential, cross-language parity, concurrency, fault-injection, and upgrade/rollback tests, where applicable;
• reproducible latency, throughput, memory, startup, and recovery benchmarks;
• security/dependency checks (rustfmt, clippy, audit) and supported platforms;
• observability, packaging/deployment, coexistence, migration, and rollback plans;
• explicit acceptance criteria and user/operator/contributor documentation.

No large rewrite PR should start before its design and acceptance criteria are reviewed. Correctness, recoverability, and compatibility take precedence over headline benchmark numbers. Small, reviewable PRs and stable interoperability boundaries are required; an unplanned rewrite PR may be closed or asked to return to the RFC/design stage.

Candidate work items

• Agree on the Rust workspace/repository layout and minimum supported Rust version
• Define the compatibility matrix and shared test fixtures
• Create the minimal Rust client design issue
• Create the Loader architecture/design issue
• Establish CI, lint, dependency, packaging, and release conventions
• Publish baseline benchmarks against the current implementation
• Split approved work into newcomer-friendly child issues

This checklist will link to concrete child issues as proposals are accepted.

Project-wide roadmap

• Toolchain (this issue): [Roadmap] Incremental Rust modernization for HugeGraph Toolchain #748
• Server / HStore / PD: [Roadmap] Incremental Rust modernization for HugeGraph core modules hugegraph#3110
• Computer / Vermeer: [Roadmap] Incremental Rust modernization for graph computing hugegraph-computer#355
• Documentation hub: [Roadmap] Documentation hub for HugeGraph Rust modernization hugegraph-doc#462

How to participate

Interested in open source, Rust, graph systems, databases, or infrastructure? Comment with the area you want to work on and any relevant experience. RFC review, benchmarks, tests, documentation, and small implementation tasks are all valuable.

First-time contributors can start with the HugeGraph contribution guide and the repository's good first issue / help wanted tasks.