[C++] Field metadata for optimized serialization

[chaokunyang]

Add compile-time field metadata support for C++ struct serialization, enabling performance and space optimization during xlang serialization. This provides two complementary APIs:

1. fory::field<> template - Inline metadata in struct definition
2. FORY_FIELD_TAGS macro - Non-invasive metadata added separately

Problem Statement

Current Limitations

The current C++ implementation applies uniform handling to all struct fields:

1. Null Checks: Written for all non-primitive fields, even when they can never be null (1 byte per field overhead)
2. Reference Tracking: Applied globally regardless of whether fields actually need shared reference support
3. Field Name Encoding: Long field names consume extra space via meta string compression instead of compact tag IDs

Expected Benefits

For a 10-field structure with tag IDs and explicit nullable/ref settings:

• Space savings: ~20 bytes per object (eliminates null flags when not needed)
• CPU savings: 10 fewer hash operations per serialization (no meta string lookups for field names)
• Compact encoding: Varint tag IDs (1-2 bytes) replace meta string encoded names (~5-15 bytes)

Design Specification

Core Principles

1. All fields are non-nullable by default (except std::optional)
2. All fields have no reference tracking by default
3. Only 2 tags: fory::nullable and fory::ref
4. Type-safe: nullable only valid for pointer types, ref only valid for shared_ptr

Type Rules

Type	Default Nullable	Default Ref	Allowed Options
Primitives, std::string	false	false	None (no options allowed)
std::optional<T>	true (inherent)	false	None (inherent nullable)
std::shared_ptr<T>	false	false	fory::nullable, fory::ref
std::unique_ptr<T>	false	false	fory::nullable only

---

API 1: fory::field<> Template (Inline)

The fory::field<> template wraps field types with compile-time metadata:

namespace fory {
  struct nullable {};  // Mark shared_ptr/unique_ptr as nullable
  struct not_null {};  // Explicitly mark as non-nullable (default)
  struct ref {};       // Enable reference tracking for shared_ptr

  template <typename T, int16_t Id, typename... Options>
  class field { /* same memory layout as T */ };
}

Usage

struct Document {
  // Primitives - always non-nullable, NO options allowed
  fory::field<std::string, 0> title;
  fory::field<int32_t, 1> version;

  // std::optional - always nullable (inherent), NO options needed
  fory::field<std::optional<std::string>, 2> description;

  // shared_ptr - non-nullable by default
  fory::field<std::shared_ptr<User>, 3> author;                        // must exist
  fory::field<std::shared_ptr<User>, 4, fory::nullable> reviewer;      // can be null
  fory::field<std::shared_ptr<Node>, 5, fory::ref> parent;             // ref tracking
  fory::field<std::shared_ptr<Node>, 6, fory::nullable, fory::ref> p;  // nullable + ref

  // unique_ptr - non-nullable by default, no ref allowed
  fory::field<std::unique_ptr<Data>, 7> data;                          // must exist
  fory::field<std::unique_ptr<Data>, 8, fory::nullable> opt_data;      // can be null
};

FORY_STRUCT(Document, title, version, description, author, reviewer, parent, p, data, opt_data);

Transparent Access

The fory::field<> wrapper is transparent - you can use it like the underlying type:

Document doc;
doc.title = "Hello";           // Direct assignment
doc.version = 1;
std::string t = doc.title;     // Implicit conversion
int v = doc.version.get();     // Explicit get()

---

API 2: FORY_FIELD_TAGS Macro (Non-Invasive)

The FORY_FIELD_TAGS macro adds metadata separately from struct definition:

// user_types.h - NO fory headers needed!
struct Document {
  std::string title;
  int32_t version;
  std::optional<std::string> description;
  std::shared_ptr<User> author;
  std::shared_ptr<User> reviewer;
  std::shared_ptr<Node> parent;
  std::unique_ptr<Data> data;
};

// serialization_config.cpp - fory config isolated here
#include "fory/serialization/fory.h"
#include "user_types.h"

FORY_STRUCT(Document, title, version, description, author, reviewer, parent, data)

FORY_FIELD_TAGS(Document,
  (title, 0),                      // string: non-nullable
  (version, 1),                    // int: non-nullable
  (description, 2),                // optional: inherently nullable
  (author, 3),                     // shared_ptr: non-nullable (default)
  (reviewer, 4, nullable),         // shared_ptr: nullable
  (parent, 5, ref),                // shared_ptr: non-nullable, with ref tracking
  (data, 6, nullable)              // unique_ptr: nullable
)

FORY_FIELD_TAGS Options

Field Type	Valid Combinations
Primitives, strings	(field, id) only
std::optional<T>	(field, id) only
std::shared_ptr<T>	(field, id), (field, id, nullable), (field, id, ref), (field, id, nullable, ref)
std::unique_ptr<T>	(field, id), (field, id, nullable)

---

API Comparison

Aspect	fory::field<> Wrapper	FORY_FIELD_TAGS Macro
Struct definition	Modified (wrapped types)	Unchanged (pure C++)
IDE support	Template noise	Excellent (clean types)
Third-party classes	Not supported	Supported
Header dependencies	Required everywhere	Isolated to config
Migration effort	High (change all fields)	Low (add one macro)

---

Compile-Time Validation

Invalid configurations are caught at compile time:

// ❌ ERROR: nullable not allowed on primitives
fory::field<int32_t, 0, fory::nullable> age;

// ❌ ERROR: nullable not allowed on string
fory::field<std::string, 1, fory::nullable> name;

// ❌ ERROR: ref not allowed on unique_ptr
fory::field<std::unique_ptr<T>, 2, fory::ref> ptr;

// ❌ ERROR: options not allowed on optional (inherently nullable)
fory::field<std::optional<T>, 3, fory::nullable> opt;

// ❌ ERROR: nullable and not_null are mutually exclusive
fory::field<std::shared_ptr<T>, 4, fory::nullable, fory::not_null> bad;

// ✅ CORRECT: use optional for nullable primitives
fory::field<std::optional<int32_t>, 0> age;
fory::field<std::optional<std::string>, 1> name;

---

Runtime Behavior

Serialization (Strict)

Field Config	Value	Behavior
Non-nullable shared_ptr<T>	nullptr	❌ Error: "Cannot serialize null for non-nullable field"
Non-nullable shared_ptr<T>	valid ptr	✅ Serialize directly (no null flag written)
fory::nullable shared_ptr<T>	nullptr	✅ Write null flag
fory::nullable shared_ptr<T>	valid ptr	✅ Write non-null flag + data

Deserialization (Lenient)

Field Config	Remote Sends	Result
Non-nullable shared_ptr<T>	null/missing	✅ std::make_shared<T>() (default T)
Non-nullable unique_ptr<T>	null/missing	✅ std::make_unique<T>() (default T)
fory::nullable shared_ptr<T>	null	✅ nullptr
fory::nullable unique_ptr<T>	null	✅ nullptr

Rationale: Be strict in what you send, lenient in what you accept.

---

Backwards Compatibility

Existing structs without field metadata continue to work with legacy behavior:

// Old style - still works
struct Person {
  std::string name;
  int32_t age;
};
FORY_STRUCT(Person, name, age);

---

Additional Context

This is the C++ equivalent of Java's @ForyField annotation. See Java issue #3000 for the original design discussion.

Protocol spec: https://fory.apache.org/docs/specification/fory_xlang_serialization_spec