AEAD-2022 TCP writer reports retry buffer length after Pending, dropping appended data
The AEAD-2022 TCP writer can report more plaintext bytes written than it actually sends when a retry after Poll::Pending supplies a longer input slice. Tokio's copy and copy_bidirectional can grow that slice while waiting for the writer, so this can silently drop data during TCP forwarding.
Reproduced with shadowsocks 1.25.0, tokio 1.53.1 and Rust 1.98.1 on Linux, using AEAD2022_BLAKE3_AES_128_GCM. The same write-state logic is still present on master at 157cefa96d44de848ff218119dbec2047826c1bb; I have not run the reproducer against master.
What happens
- Call
poll_write_encryptedwithb"a". The writer encrypts it, but the underlying stream returnsPending. - Make the underlying stream writable and retry with
b"ab". - The writer sends its cached frame containing only
a, then returnsOk(2).
Expected: report 1, leaving b for the caller to submit next, or actually send both bytes before reporting 2.
The Writing state retains the encrypted buffer and its wire offset, but returns the current buf.len() after sending that buffer. It does not retain the original plaintext length. Tokio's copy implementation can append input after a pending write.
Reproducer
This uses a controlled in-memory sink. It needs no server, network, runtime, or sleeps. Run cargo run with these two files.
Cargo.toml:
[package]
name = "ss-pending-write-repro"
version = "0.1.0"
edition = "2024"
[dependencies]
shadowsocks = { version = "=1.25.0", default-features = false, features = ["aead-cipher-2022"] }
tokio = { version = "=1.53.1", features = ["io-util"] }src/main.rs:
use shadowsocks::{
config::ServerType,
context::Context as SsContext,
crypto::CipherKind,
relay::tcprelay::crypto_io::{CryptoStream, CryptoWrite, StreamType},
};
use std::{
io,
pin::Pin,
task::{Context, Poll, Waker},
};
use tokio::io::{AsyncRead, AsyncWrite, ReadBuf};
#[derive(Default)]
struct Sink {
blocked: bool,
bytes: Vec<u8>,
waker: Option<Waker>,
}
impl AsyncRead for Sink {
fn poll_read(
self: Pin<&mut Self>,
_: &mut Context<'_>,
_: &mut ReadBuf<'_>,
) -> Poll<io::Result<()>> {
Poll::Ready(Ok(()))
}
}
impl AsyncWrite for Sink {
fn poll_write(
mut self: Pin<&mut Self>,
cx: &mut Context<'_>,
data: &[u8],
) -> Poll<io::Result<usize>> {
if self.blocked {
self.waker = Some(cx.waker().clone());
return Poll::Pending;
}
self.bytes.extend_from_slice(data);
Poll::Ready(Ok(data.len()))
}
fn poll_flush(self: Pin<&mut Self>, _: &mut Context<'_>) -> Poll<io::Result<()>> {
Poll::Ready(Ok(()))
}
fn poll_shutdown(self: Pin<&mut Self>, _: &mut Context<'_>) -> Poll<io::Result<()>> {
Poll::Ready(Ok(()))
}
}
fn main() {
let context = SsContext::new_shared(ServerType::Local);
let mut stream = CryptoStream::from_stream(
&context,
Sink {
blocked: true,
..Sink::default()
},
StreamType::Client,
CipherKind::AEAD2022_BLAKE3_AES_128_GCM,
&[0; 16],
);
let mut cx = Context::from_waker(Waker::noop());
assert!(
Pin::new(&mut stream)
.poll_write_encrypted(&mut cx, b"a")
.is_pending()
);
let sink = stream.get_mut();
sink.blocked = false;
sink.waker.take().unwrap().wake();
let Poll::Ready(Ok(written)) = Pin::new(&mut stream).poll_write_encrypted(&mut cx, b"ab")
else {
panic!("write did not complete");
};
println!("Reported plaintext bytes: {written}");
println!("Ciphertext bytes: {}", stream.get_ref().bytes.len());
// The cached frame contains only 'a'. The appended 'b' has not been sent.
assert_eq!(written, 1);
}Actual output:
Reported plaintext bytes: 2
Ciphertext bytes: 60
assertion `left == right` failed
left: 2
right: 1The 60 wire bytes contain the 16-byte salt, 27-byte encrypted fixed header, and 17-byte encrypted one-byte payload. The appended byte is not in the frame.
We encountered this as an intermittent TCP payload timeout through ProxyServerStream, after the server-first greeting had succeeded. Our downstream workaround and duplex regression preserve the original write input across retries. Retaining the original plaintext length in the upstream write state appears to address the incorrect count.
Source: shadowsocks/shadowsocks-rust