DocumentSplitter emits redundant overlap-only trailing chunks for character-based split modes (token mode was fixed in #12661)

Describe the bug DocumentSplitter with split_by="token" no longer emits overlap-only trailing chunks since #12661, but the character-based modes sharing _concatenate_units (word, period, page, passage, line, sentence) still do. A trailing window consisting solely of already-covered overlap units (plus the empty artifact of a trailing delimiter) becomes its own chunk, duplicating text that is already fully contained in the previous chunk.

To Reproduce

from haystack import Document
from haystack.components.preprocessors import DocumentSplitter

splitter = DocumentSplitter(split_by="period", split_length=3, split_overlap=1)
docs = splitter.run(documents=[Document(content="s1.s2.s3.")])["documents"]
print([d.content for d in docs])
# actual:   ['s1.s2.s3.', 's3.']  <- 's3.' adds no new text
# expected: ['s1.s2.s3.']

High overlap shows it without any trailing delimiter:

splitter = DocumentSplitter(split_by="period", split_length=3, split_overlap=2)
docs = splitter.run(documents=[Document(content="s1.s2.s3.s4.")])["documents"]
print([d.content for d in docs])
# actual:   ['s1.s2.s3.', 's2.s3.s4.', 's3.s4.']  <- third chunk is fully contained in the second
# expected: ['s1.s2.s3.', 's2.s3.s4.']

Expected behavior Mirror the split_by="token" behavior from #12661: never emit a chunk that adds no new text. Partial final chunks carrying new content (e.g. word mode "t1 t2 t3 t4" with length 3 / overlap 1 -> ['t1 t2 t3 ', 't3 t4']) and below-threshold merges must keep working.

Additional context Root cause: _concatenate_units in haystack/components/preprocessors/document_splitter.py emits every trailing window yielded by more_itertools.windowed as long as its text is non-empty, without checking whether the window reaches beyond the already covered units.

Possible fixes:

• (a) skip non-first windows whose units past the covered boundary are all empty (mirrors the token-mode fix; threshold merging of genuine partial tails is preserved);
• (b) only strip the trailing empty delimiter artifact (fixes exact-fit inputs but leaves high-overlap tails like the second repro);
• (c) leave as-is (rejected: duplicate chunks pollute document stores, and I found one such phantom chunk even carrying a wrong page_number).

I have a fix along the lines of (a) with regression tests ready and can open a PR.

FAQ Check

• Have you had a look at our new FAQ page?

System:

• OS: macOS (arm64)
• Haystack version: main @ b20727a (post-#12661)