片状测试: EnglishRobertaTests.TestTokenizerEncoding
作者: svick创建于 2026年9月16日更新于 2026年9月19日
标签untriagedblocking-clean-ciKnown Build Error
Build Information
Build: https://dev.ZZTERM5ZZ.com/dnceng-public/cbb18261-c48f-4abb-8651-8cdcb5474649/_build/results?buildId=1599215 Build error leg or test failing: Microsoft.ML.Tokenizers.Tests.EnglishRobertaTests.TestTokenizerEncoding Pull Request: https://GitHub.com/dotnet/machinelearning/pull/7704
Error Message
DO NOT USE JSON BELOW IF THIS IS A BUILD BREAK otherwise build analysis will allow pull requests to merge that break the build worse. For a build break, do not use this issue form. Make a regular new issue. Fill the error message using step by step known issues guidance.
{
"ErrorMessage": ["EnglishRobertaTests.TestTokenizerEncoding", "ArgumentOutOfRangeException : value must be non-negative"],
"BuildRetry": false,
"ExcludeConsoleLog": false
}Microsoft.ML.Tokenizers.Tests.EnglishRobertaTests.TestTokenizerEncoding(text: "the brown fox jumped over the lazy dog!", expectedTokens: ["the", "brown", "fox", "jumped", "over", "lazy", "dog"], expectedOffsets: [Tuple(0, 3), Tuple(3, 6), Tuple(9, 4), Tuple(13, 7), Tuple(20, 5), Tuple(25, 6), Tuple(31, 7)], expectedIds: [1169, 7586, 21831, 11687, 625, 1000, 1001]),
Exception Message
System.ArgumentOutOfRangeException : value must be non-negative
Parameter name: value
Stack Trace
at System.Index.ThrowValueArgumentOutOfRange_NeedNonNegNumException()
at Microsoft.ML.Tokenizers.EnglishRobertaTokenizer.ModifyTokenListOffsets(IReadOnlyList`1 tokens, Span`1 indexMapping) in D:\a\_work\1\s\src\Microsoft.ML.Tokenizers\Model\EnglishRobertaTokenizer.cs:line 913
at Microsoft.ML.Tokenizers.EnglishRobertaTokenizer.EncodeInternal(ReadOnlySpan`1 text) in D:\a\_work\1\s\src\Microsoft.ML.Tokenizers\Model\EnglishRobertaTokenizer.cs:line 397
at Microsoft.ML.Tokenizers.EnglishRobertaTokenizer.EncodeToTokens(String text, ReadOnlySpan`1 textSpan, EncodeSettings settings) in D:\a\_work\1\s\src\Microsoft.ML.Tokenizers\Model\EnglishRobertaTokenizer.cs:line 337
at Microsoft.ML.Tokenizers.Tokenizer.EncodeToTokens(ReadOnlySpan`1 text, String& normalizedText, Boolean considerPreTokenization, Boolean considerNormalization) in D:\a\_work\1\s\src\Microsoft.ML.Tokenizers\Tokenizer.cs:line 166
at Microsoft.ML.Tokenizers.Tests.EnglishRobertaTests.TestTokenizerEncoding(String text, String[] expectedTokens, ValueTuple`2[] expectedOffsets, Int32[] expectedIds) in D:\a\_work\1\s\test\Microsoft.ML.Tokenizers.Tests\EnglishRobertaTests.cs:line 179Known issue validation
Build: :mag_right: https://dev.ZZTERM6ZZ.com/dnceng-public/public/_build/results?buildId=1599215 **Error message …
内容来源: dotnet/machinelearning