iguana

A Universal Serialization Engine Based on compile-time Reflection

iguana is a modern, universal and easy-to-use serialization engine developed in C++20 and C++17.

线上讨论: 项目讨论

中文版

struct_pb

C++26 reflection/proto3 change notes

Motivation

Serialize an object to any other format data with compile-time reflection, such as json, xml, binary, table and so on. This library is designed to unify and simplify serialization in a portable cross-platform manner. This library is also easy to extend, and you can serialize any format of data with the library. This library provides a portable cross-platform way of:

• serialization of json
• serialization of xml
• serialization of yaml
• serialization of protobuf
• serialization of any customized format

compile time reflection

reflection lib introduction

With a C++26 static reflection compiler, iguana can read class members directly from the language reflection API. The CI job for this path uses the official gcc:16.1.0-trixie container with -std=gnu++26 -freflection and configures CMake with -DENABLE_CXX26_REFLECTION=ON.

Useful C++26 annotations:

struct [[= ylt::reflection::struct_name{}]] user_t {
  [[= ylt::reflection::field_name{}]]
  int user_id{};

  std::string name;

  [[= ylt::reflection::skip_field{}]]
  int local_cache{};
};

struct base_t {
  int internal_state{};
};

struct derived_t : [[= ylt::reflection::skip_base{}]] base_t {
  int value{};
};

struct xml_user_t {
  [[= ylt::reflection::field_name{}]]
  [[= iguana::xml_required{}]]
  int id{};
};

field_name changes the serialized field name for JSON/XML/YAML and generated protobuf schema. struct_name changes XML root names. skip_field excludes a member from reflection, and skip_base excludes a base class from recursive member collection. xml_required is the C++26 annotation form of the XML REQUIRED(type, fields...) macro.

Tutorial

This Tutorial is provided to give you a view of how iguana works for serialization.

Serialization of json

The first thing to do when you serialize an object is to define meta data. There is an example of defining meta data.

struct person
{
    std::string  name;
    int          age;
};
#if __cplusplus   
 
  buke 
  30 
)";
iguana::from_xml(p, xml);

Serialization of yaml

The serialization of yaml is also as simple as the above interface. Here is a complete example:

// serialization the structure to the string
person p = {"admin", 20};
iguana::string_stream ss;  // here use std::string is also ok
iguana::to_yaml(ss, p);
std::cout  b;
    int c;
    std::map d;
    std::unordered_map e;
    double f;
    std::list g;
};
YLT_REFL(composit_t, a, b, c, d, e, f, g);

Then call the simple interface:

one_t one = { 2 };
composit_t composit = { 1,{ "tom", "jack" }, 3,{ { 2,3 } },{ { 5,6 } }, 5.3,{ one } };
iguana::string_stream ss;
iguana::to_json(composit, ss);
std::cout Pro Lib
  110
  tom1.8
  
  jack2.1
  

)";
library_t lib;
iguana::from_xml(lib, str);

yaml

As always what we do, define the structure and reflect the meta data.

enum class enum_status {
  start,
  stop,
};
struct plain_type_t {
  bool isok;
  enum_status status;
  char c;
  std::optional hasprice;
  std::optional num;
  std::optional price;
};
YLT_REFL(plain_type_t, isok, status, c, hasprice, num, price);

And then, simply call the interface:

// deserialization the structure from the string
std::string str = R"(
isok: false
status: 1
c: a
hasprice: true
num:
price: 20
)";
plain_type_t p;
iguana::from_yaml(p, str);
// serialization the structure to the string
std::string ss;
iguana::to_yaml(ss, p);
std::cout  struct enum_value {
  constexpr static std::array value = {10, 11};
};
} // namespace iguana

Once this is done, we can continue writing the rest of the code as usual.

struct enum_t {
    Status a;
    Status b;
};
YLT_REFL(enum_t, a, b);

// deserialization
enum_t e;
iguana::from_json(e, str);
// serialization
enum_t e1;
e1.a = Status::START;
e1.b = Status::STOP;
std::string ss;
iguana::to_json(e1, ss);

Serialization of protobuf

Basic protobuf serialization uses the same object API:

struct person {
  int id;
  std::string name;
  int age;
  bool operator==(person const& rhs) const {
    return id == rhs.id && name == rhs.name && age == rhs.age;
  }
};

#if __cplusplus  emails;
};

YLT_REFL_PB(account, (name, 10), (age, 20), (emails, 9));

With C++26 static reflection, prefer the [[= iguana::pb_field(N)]] annotation shown below; that path reads protobuf metadata from annotations and does not depend on YLT_REFL_PB.

For advanced proto3 wire semantics on the non-C++26 path, use the descriptor helpers. The helper form keeps normal C++ field types while attaching protobuf schema metadata:

…

Helper APIs:

The explicit wrapper types iguana::pb_timestamp and iguana::pb_duration remain available when code wants the wire-shaped representation directly.

pb_field_ex can combine options. Supported options are pb_bytes, pb_zigzag, pb_fixed, pb_optional, pb_as_timestamp/as_timestamp, and pb_as_duration/as_duration.

iguana::pb_field_ex(
    "retry_count", iguana::pb_optional, iguana::pb_zigzag);

With a C++26 reflection compiler, the same metadata can be written as annotations without YLT_REFL_PB. The current C++26 test build uses GCC 16.1 with -std=gnu++26 -freflection.

…

C++26 annotation equivalents:

Supported proto3 wire metadata includes custom field numbers, bytes, sint32/sint64 zigzag encoding, fixed-width integers, explicit optional presence, oneof, google.protobuf.Timestamp, google.protobuf.Duration, and unknown field preservation. Repeated primitive fields accept packed, unpacked, and mixed input; writers use proto3 default packed output where applicable.

Field numbers must be in [1, 2^29 - 1] and cannot be in protobuf's reserved [19000, 19999] range. A message can have at most one unknown-field storage member, and it must be a std::string.

Generate a .proto view of a struct with:

std::string schema;
iguana::to_proto(schema, "demo");

The current conformance target covers the proto3 binary/protobuf-output wire-only subset. JSON mapping, text format, proto2, extensions, services, and custom options are outside this scope.

more detail and change notes

Full sources:

• More examples about json
• More examples about xml
• More examples about yaml
• More examples about struct_pb

Scripts

Automatically generate YLT_REFL macros based by struct.

To get a list of basic options and switches use:

python automatic_macro_generator.py -h

basic example:

The content of the test_macro_generator.cpp is as follows:

 struct person {
    std::string name;
    int age;
  };

  char *iguana = NULL;

  struct composit_t { int a; std::vector b; int c; std::map d; std::unordered_map e; double f;};

  char *iguana_test = NULL;

  struct composit_t2
  {
    int a;
    std::vector b;
    int iguana;
    std::map example_test;
    std::unordered_map random_name__;
    double __f__number__complex;
  };

execute script:

python automatic_macro_generator.py -i test_macro_generator.cpp

After processing by the automatic_macro_generator.py script,test_macro_generator.cpp change into:

struct person {
  std::string name;
  int age;
};
YLT_REFL(person, name, age);
char *iguana = NULL;

struct composit_t { int a; std::vector b; int c; std::map d; std::unordered_map e; double f;};
YLT_REFL(composit_t, a, b, c, d, e, f);

struct composit_t2
{
  int a;
  std::vector b;
  int iguana;
  std::map example_test;
  std::unordered_map random_name__;
  double __f__number__complex;
};
YLT_REFL(composit_t2, a, b, iguana, example_test, random_name__, __f__number__complex);

other example:

python automatic_macro_generator.py -i test_macro_generator.cpp -o have_macro.cpp

test_macro_generator.cpp will be unchanged, have_macro.cpp will be changed to source file with YLT_REFL macro.

scripts works out of the box with Python version 2.7 and 3.x on any platform.

Notes: In Python3,Will prompt DeprecationWarning: 'U' mode is deprecated.Ignore it.

F.A.Q

• Question: Why is the library called iguana?

  • Answer: I think serialization is like an iguana, because the only difference is the displaying format, however the meta data is never changed. With changeless meta data and YLT_REFL, you can serialize an object to any format, which is like how an iguana does.

• Question: Does iguana support raw pointer?

  • Answer: No. iguana doesn't support raw pointer, but it will support smart pointer in the future.

• Question: Is iguana thread-safe?

  • Answer: Not yet, but it's not a problem, you can use lock before calling from_json or to_json.

• Question: Is iguana high performance?

  • *Answer