Doris Sink Error Log in verbose mode

A note for the community

• Please vote on this issue by adding a reaction to the original issue to help the community and maintainers prioritize this request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

I am struggle with pushing data into doris. when it failed ,vector have few infomation, and I have to guess what is wrong.

the doris steam log response is mostly like :

{
    "TxnId": 2288,
    "Label": "aabd7e19-2831-45cb-aab0-67227df68e5c",
    "Comment": "",
    "TwoPhaseCommit": "false",
    "Status": "Success",
    "Message": "OK",
    "NumberTotalRows": 1,
    "NumberLoadedRows": 1,
    "NumberFilteredRows": 0,
    "NumberUnselectedRows": 0,
    "LoadBytes": 5516,
    "LoadTimeMs": 40,
    "BeginTxnTimeMs": 1,
    "StreamLoadPutTimeMs": 3,
    "ReadDataTimeMs": 0,
    "WriteDataTimeMs": 12,
    "ReceiveDataTimeMs": 7,
    "CommitAndPublishTimeMs": 22
}

but when the status is not success, it will return ErrorLog , it is very helpful for debug the data flow

{
    "TxnId": 2289,
    "Label": "628c4ac3-5a20-4a32-bc15-7c7f58a1ccbe",
    "Comment": "",
    "TwoPhaseCommit": "false",
    "Status": "Fail",
    "Message": "[DATA_QUALITY_ERROR]too many filtered rows",
    "NumberTotalRows": 1,
    "NumberLoadedRows": 0,
    "NumberFilteredRows": 1,
    "NumberUnselectedRows": 0,
    "LoadBytes": 5516,
    "LoadTimeMs": 9,
    "BeginTxnTimeMs": 0,
    "StreamLoadPutTimeMs": 3,
    "ReadDataTimeMs": 0,
    "WriteDataTimeMs": 5,
    "ReceiveDataTimeMs": 6,
    "CommitAndPublishTimeMs": 0,
    "ErrorURL": "http://xxxxxxx/api/_load_error_log?file=__shard_711/error_log_insert_stmt_c646ac01567619fe-6a399d092673181_c646ac01567619fe_6a399d092673181"
}

Use Cases

I want vector to expose the errorlog in verbose mode ,so i can debug the data remap directly.

Attempted Solutions

doris response is just receive status and message in sinks/doris/retry.rs

struct DorisStreamLoadResponse {
    #[serde(rename = "Status")]
    status: String,

    #[serde(rename = "Message")]
    message: Option<String>,

}

we can add ErrorLog and expose it in debug message

struct DorisStreamLoadResponse {
    #[serde(rename = "Status")]
    status: String,

    #[serde(rename = "Message")]
    message: Option<String>,

    #[serde(rename = "ErrorURL")]
    error_url: Option<String>,
}

Proposal

No response

References

No response

Version

No response