[Console] OutputFormatter is still quadratic for non-ASCII content (#63749 only fixed the ASCII path)

Symfony version(s) affected

6.4.24 and up. Reproduced on 6.4.46, 7.4.19 and the 8.2 branch

Description

OutputFormatter::formatAndWrap() is O(n × m) for non-ASCII messages, where n is the message length and m is the number of formatting tags.

The cost comes from this line, added in #61242:

// convert byte position to character position.
$pos = Helper::length(substr($message, 0, $pos));

It is evaluated once per tag, and each evaluation walks the message prefix from the very beginning: Helper::length() goes through (new UnicodeString($string))->length(), i.e. grapheme_strlen() over the whole prefix. Summed over all tags, that is quadratic.

This is the regression reported in #61845 and partially fixed by #63749. That fix added an ASCII-only fast path, so pure ASCII messages are fast again. But the check is all-or-nothing:

$isAscii = !preg_match('/[\x80-\xFF]/', $message);

A single non-ASCII byte anywhere in the message switches the whole message to the slow branch. That byte does not by itself make things slow — the magnitude is driven by n × m — but it means that no message in a non-English locale ever benefits from the fix, however large it gets.

The practical effect is a cliff rather than a gradual slowdown: an application whose output is tag-rich can be fast for months and fall off the moment one user name, one product title or one translated string contains an accented character. On v6.4.46, a 96 KB message with 800 tags formats in 0.003 s as pure ASCII and in 9.193 s once a single é is appended to it.

How to reproduce

A single file, no framework needed:

<?php
// composer require symfony/console:6.4.46
require __DIR__.'/vendor/autoload.php';

use Symfony\Component\Console\Formatter\OutputFormatter;

$formatter = new OutputFormatter(true);

function build(string $body, int $lines): string
{
    $out = '';
    for ($i = 0; $i < $lines; $i++) {
        $out .= "<info>processing</info> step $i: ".str_repeat($body.' ', 8)."\n";
    }

    return $out;
}

function bench(OutputFormatter $f, string $label, string $message): void
{
    $start = microtime(true);
    $f->format($message);
    printf("  %-24s %6.1f KB  %9.3f s\n", $label, \strlen($message) / 1024, microtime(true) - $start);
}

$ascii = build('processing', 800);

echo "-- a single non-ASCII character is enough --\n";
bench($formatter, 'pure ASCII', $ascii);
bench($formatter, 'same + one "é"', $ascii."é\n");

echo "-- quadratic growth (Portuguese) --\n";
foreach ([200, 400, 800] as $lines) {
    bench($formatter, "Portuguese, $lines lines", build('não é válido', $lines));
}

Output on v6.4.46 (affected):

-- a single non-ASCII character is enough --
  pure ASCII                 96.0 KB      0.003 s
  same + one "é"             96.0 KB      9.193 s      <-- 3000x slower
-- quadratic growth (Portuguese) --
  Portuguese, 200 lines      31.7 KB      0.480 s
  Portuguese, 400 lines      63.6 KB      1.964 s
  Portuguese, 800 lines     127.2 KB      9.583 s      <-- x4 per doubling

Output on v6.4.23 (last release before the regression):

-- a single non-ASCII character is enough --
  pure ASCII                 96.0 KB      0.003 s
  same + one "é"             96.0 KB      0.002 s
-- quadratic growth (Portuguese) --
  Portuguese, 200 lines      31.7 KB      0.000 s
  Portuguese, 400 lines      63.6 KB      0.001 s
  Portuguese, 800 lines     127.2 KB      0.002 s

Doubling the input quadruples the time, which confirms the quadratic behaviour.

The number of tags is the second factor. Keeping the payload at ~96 KB and varying only how many tags it contains:

function build(int $tags, bool $nonAscii): string
{
    $chunk = intdiv(98304, max($tags, 1));
    $out = '';
    for ($i = 0; $i < $tags; $i++) {
        $out .= '<info>x</info>'.str_repeat('a', $chunk);
    }

    return $nonAscii ? $out."é" : $out;
}

Output on v6.4.46:

tags     non-ASCII        KB       time
1        no             96.0      0.000 s
1        yes            96.0      0.042 s
10       no             96.1      0.000 s
10       yes            96.1      0.108 s
100      no             97.4      0.000 s
100      yes            97.4      1.087 s
400      no            101.2      0.001 s
400      yes           101.2      4.426 s
800      no            106.2      0.002 s
800      yes           106.3     10.562 s
2000     no            123.0      0.005 s
2000     yes           123.0     28.567 s

With the payload fixed, time grows linearly with the tag count — and stays flat for the ASCII variant of the very same message. Tag-rich output is common in practice: any command that styles per-line or per-column produces hundreds or thousands of tags in a single formatted string.

Possible Solution

The byte-to-character conversion in formatAndWrap() is not needed at all.

The original bug (#58286) was not in the outer loop — it was in applyCurrentStyle(), which cut text with substr($text, 0, $width - $currentLineLength), i.e. sliced by bytes using a character width, and counted line length with \strlen(). #61242 correctly fixed that method to use Helper::substr() / Helper::length(), and that part should stay.

But the same PR also converted the outer loop to character offsets, and that is what costs the time. It is unnecessary: the loop only ever slices the message at tag boundaries found by preg_match_all() on < and >. Those are ASCII characters, and UTF-8 continuation bytes are always in the range 0x80–0xBF, so a byte offset taken at a tag boundary can never fall inside a multibyte sequence. Slicing by bytes there is always safe.

Reverting just the outer loop to byte offsets also makes the $isAscii special case from #63749 redundant, so the method becomes simpler rather than more complex.

     public function formatAndWrap(?string $message, int $width)
     {
         if (null === $message) {
             return '';
         }
 
-        // For ASCII-only strings, byte positions equal character positions,
-        // so we can use native strlen/substr which is much faster than Helper::length/substr.
-        $isAscii = !preg_match('/[\x80-\xFF]/', $message);
-
         $offset = 0;
         $output = '';
         $openTagRegex = '[a-z](?:[^\\\\<>]*+ | \\\\.)*';
         $closeTagRegex = '[a-z][^<>]*+';
         $currentLineLength = 0;
         preg_match_all("#<(($openTagRegex) | /($closeTagRegex)?)>#ix", $message, $matches, \PREG_OFFSET_CAPTURE);
         foreach ($matches[0] as $i => $match) {
             $pos = $match[1];
             $text = $match[0];
 
             if (0 != $pos && '\\' == $message[$pos - 1]) {
                 continue;
             }
 
-            if ($isAscii) {
-                // For ASCII, byte position = character position, no conversion needed
-                $output .= $this->applyCurrentStyle(substr($message, $offset, $pos - $offset), $output, $width, $currentLineLength);
-                $offset = $pos + \strlen($text);
-            } else {
-                // convert byte position to character position.
-                $pos = Helper::length(substr($message, 0, $pos));
-                // add the text up to the next tag
-                $output .= $this->applyCurrentStyle(Helper::substr($message, $offset, $pos - $offset), $output, $width, $currentLineLength);
-                $offset = $pos + Helper::length($text);
-            }
+            // Tags are matched on "<" and ">", which are ASCII. A byte offset taken at a tag
+            // boundary can therefore never fall inside a multibyte sequence, so slicing the
+            // segment between two tags by bytes is always safe.
+            $output .= $this->applyCurrentStyle(substr($message, $offset, $pos - $offset), $output, $width, $currentLineLength);
+            $offset = $pos + \strlen($text);
 
             // opening tag?
             if ($open = '/' !== $text[1]) {
                 $tag = $matches[1][$i][0];
             } else {
                 $tag = $matches[3][$i][0] ?? '';
             }
 
             if (!$open && !$tag) {
                 // </>
                 $this->styleStack->pop();
             } elseif (null === $style = $this->createStyleFromString($tag)) {
                 $output .= $this->applyCurrentStyle($text, $output, $width, $currentLineLength);
             } elseif ($open) {
                 $this->styleStack->push($style);
             } else {
                 $this->styleStack->pop($style);
             }
         }
 
-        $output .= $this->applyCurrentStyle($isAscii ? substr($message, $offset) : Helper::substr($message, $offset), $output, $width, $currentLineLength);
+        $output .= $this->applyCurrentStyle(substr($message, $offset), $output, $width, $currentLineLength);
 
         return strtr($output, ["\0" => '\\', '\\<' => '<', '\\>' => '>']);
     }

applyCurrentStyle() is left untouched — it keeps the grapheme-aware Helper::substr() / Helper::length() calls introduced by #61242, which is where they are actually required.

The diff above is written against 6.4. It applies unchanged to 7.4 and 8.2: the body of formatAndWrap() is byte-identical across all three branches, the only difference being the : string return type on the signature line. Helper::length() is identical as well.

Verification of this patch, applied on top of v6.4.46:

1. Symfony's own test suite — Tests/Formatter: 63 tests, 173 assertions, all pass, identical to unpatched. Tests/Helper: 380 tests; the single error (DumperNativeFallbackTest::testInvoke) is present on the unpatched checkout as well and is unrelated.

2. The original bug from #58286 stays fixed — its reproducer renders without throwing and produces valid UTF-8, byte-identical to the unpatched output:

   复制

   +----------------------------------------------------+
   | Message                                            |
   +----------------------------------------------------+
   | Usuário <strong>{{user_name}}</strong> não é válid |
   | o.                                                 |
   +----------------------------------------------------+

3. Differential test — 403 generated messages (mixing tags, escaped tags, Latin diacritics, Cyrillic, CJK, emoji, combining marks, zero-width spaces, newlines and tabs) formatted with formatAndWrap() at widths 0, 1, 5, 20, 50, 120, both decorated and undecorated: 5132 cases, output byte-identical between patched and unpatched.

4. Benchmark after the patch — the cliff is gone and the quadratic growth disappears:

   复制

   -- a single non-ASCII character is enough --
     pure ASCII                 96.0 KB      0.002 s
     same + one "é"             96.0 KB      0.002 s      (was 9.193 s)
   -- quadratic growth (Portuguese) --
     Portuguese, 200 lines      31.7 KB      0.000 s
     Portuguese, 400 lines      63.6 KB      0.001 s
     Portuguese, 800 lines     127.2 KB      0.002 s      (was 9.583 s)

5. Same results on the other maintained branches. The patch was applied to v6.4.46, v7.4.19 and the 8.2 branch; in every case Tests/Formatter reports 63 tests, 173 assertions, identical with and without the patch, and the benchmark behaves the same:

   branch	96 KB + one é, before	after	Portuguese 800 lines, before	after
   6.4.46	9.193 s	0.002 s	9.583 s	0.002 s
   7.4.19	9.216 s	0.002 s	9.605 s	0.002 s
   8.2 (branch)	7.267 s	0.002 s	9.713 s	0.002 s

To re-run those checks:

git clone --branch v6.4.46 --depth 1 https://github.com/symfony/console.git
cd console
composer install && composer require --dev phpunit/phpunit:^9.6

# apply the diff above to Formatter/OutputFormatter.php, then:
php vendor/bin/phpunit --no-configuration --bootstrap vendor/autoload.php Tests/Formatter
php vendor/bin/phpunit --no-configuration --bootstrap vendor/autoload.php Tests/Helper

Additional Context

History of this code path, for context:

1. #58286 (Sep 2024) — Table with setColumnMaxWidth() threw Invalid "UTF-8" string. when wrapping text containing multibyte characters. A genuine bug in applyCurrentStyle().
2. #61242 (Jul 2025, released in v6.4.24) — fixed it, and also converted the outer loop of formatAndWrap() from byte offsets to character offsets, which introduced the O(n × m) behaviour.
3. #61845 (Sep 2025) — "Slow Laravel Migrations Caused by symfony/console". A user bisected composer.lock package by package and version by version to land on 6.4.24; trivial unit tests had started taking seconds.
4. #63749 (Mar 2026, released in v6.4.36) — added the ASCII fast path, reporting 41,965 ms → 83 ms for a 580 KB message with 20,000 tags. The PR description states explicitly: "UTF-8 content still uses the grapheme-aware functions to correctly handle multi-byte characters." That remaining half is what this issue is about.

Other scripts behave the same way — measured on comparable payloads with v6.4.46: German (größe prüfen) 8.2 s, Japanese (日本語テキスト) 39.8 s.