[Epic] Flaky tests roundup

Current behavior

This epic tracks flaky tests so they can be addressed one by one.

When a flaky test is added in this list, it can be set to [Ignore] on the platform or platforms where it tends to fail.

• When_TextBox_Has_ContextFlyout_Keyboard_Trigger_Shows_Flyout is flaky on iOS Skia and Wasm Skia
• TextBox_Readonly, PasswordBox_RevealInScrollViewer and FlyoutTest_Target are very flaky on iOS native
• When_ReversedMouseWheel_Then_ScrollInReversedDirection Skia X11
• When_Nested_ScrollViewers_WheelChanged is flaky on Skia WPF Islands
• TextBox_TextChanging_Capitalize is flaky on iOS
• When_Page_Navigates_Back_With_Outer_Before fails often on all targets in CI
• https://github.com/unoplatform/uno/issues/20731
• When_Page_Navigates_Back_With_Outer_Before fails often on all targets in CI
• When_Navigate and When_NavigateToString fail often on all targets in CI
• When_Year_Mode_Shown fails on iOS Native because the initial loading/switching between modes takes longer
• When_FullSizeDesired fails often on Skia Android
• NavigationViewItem fails in When_Add_Remove (Assert.AreEqual failed. Expected:<0>. Actual:<1>. ItemsRepeater/NavigationViewItemMenuItemsHost)
• When_ListViewNestedAndSwipeUpThenFlickDown_Then_ScrollThenShowIndicatorButDoNotRequest fails often on Skia Android
• When_Focus_Immediately fails on Skia targets
• Given_PipsPager is very flaky
• When_RepeatForever_ShouldLoop is flaky
• When_MultipleSelectionWithoutItemClick_Then_PointersEvents is flaky on skia targets
• When_Large_List_Scroll_To_End_Then_Back_Up_And_First_Item is flaky in WASM CI
• ValidateDynamicOverflowOrderBasic() is flaky on Android
• When_ShapeVisual_ViewBox_Shape_Combinations (https://github.com/unoplatform/uno/issues/18752)
• When_Rectangle UI test is very flaky on iOS
• BreadcrumbBar leak tests are flaky on iOS
• Given_AppWindow.WhenResize is failing in CI on Linux, passing locally
• https://github.com/unoplatform/uno/issues/18375
• #17101
• #17102
• When_Uri_Nullified fails on some Skia targets
• ValidateElementsHaveCorrectKeys is not passing (incorrectly disabled)
• VerifyKeyboardAcceleratorCanOverrideControlAccelerator, VerifyTryInvokeKeyboardAcceleratorBehaviorForLocallyScopedAccelerator, ValidateKeyboardAcceleratorEventNotInvokedWhenCollapsed and ValidateKeyboardAcceleratorBehaviorWithMultipleModifiers on Skia are unstable
• When_Image_Is_Monochromatic fails on iOS .NET 9 RC1
• https://github.com/unoplatform/uno/issues/18064
• Some Given_ImageBrush.WhenStretch fail on iOS
• GridViewGroupedResizable is too slow and fails randomly https://github.com/unoplatform/uno/pull/18926
• https://github.com/unoplatform/uno/issues/17742
• When_Hungarian_Culture_Column_Order, When_Czech_Culture_Column_Order and When_CanadaFrench_Culture_Column_Order fail on net9 wasm. It could be related to a missing/filtered ICU, or a headless browser issue, it works locally.
• MoveItemsBetweenPrimaryAndSecondaryCommands is unstable on android
• When_DownSampling is flaky and disabled in https://github.com/unoplatform/uno/pull/16991
  复制

  Running test When_DownSampling()
  GLIB UNHANDLED EXCEPTIONSystem.ArgumentNullException: Value cannot be null. (Parameter 'bitmap')
     at SkiaSharp.SKImage.FromBitmap(SKBitmap bitmap)
     at Windows.UI.Composition.CompositionSurfaceBrush.UpdatePaint(SKPaint fillPaint, SKRect bounds) in C:\a\1\s\src\Uno.UI.Composition\Composition\CompositionSurfaceBrush.skia.cs:line 91
     at Windows.UI.Composition.CompositionSpriteShape.Paint(PaintingSession& session) in C:\a\1\s\src\Uno.UI.Composition\Composition\CompositionSpriteShape.skia.cs:line 19
     at Windows.UI.Composition.BorderVisual.Paint(PaintingSession& session) in C:\a\1\s\src\Uno.UI.Composition\Composition\BorderVisual.skia.cs:line 127
     at Windows.UI.Composition.Visual.Render(PaintingSession& parentSession) in C:\a\1\s\src\Uno.UI.Composition\Composition\Visual.skia.cs:line 190
     at Windows.UI.Composition.Visual.Render(PaintingSession& parentSession) in C:\a\1\s\src\Uno.UI.Composition\Composition\Visual.skia.cs:line 199
     at Windows.UI.Composition.Visual.Render(PaintingSession& parentSession) in C:\a\1\s\src\Uno.UI.Composition\Composition\Visual.skia.cs:line 199
     at Windows.UI.Composition.Visual.Render(PaintingSession& parentSession) in C:\a\1\s\src\Uno.UI.Composition\Composition\Visual.skia.cs:line 199
     at Windows.UI.Composition.Visual.Render(PaintingSession& parentSession) in C:\a\1\s\src\Uno.UI.Composition\Composition\Visual.skia.cs:line 199
     at Windows.UI.Composition.Visual.Render(PaintingSession& parentSession) in C:\a\1\s\src\Uno.UI.Composition\Composition\Visual.skia.cs:line 199
     at Windows.UI.Composition.Visual.Render(PaintingSession& parentSession) in C:\a\1\s\src\Uno.UI.Composition\Composition\Visual.skia.cs:line 199
     at Windows.UI.Composition.Visual.Render(PaintingSession& parentSession) in C:\a\1\s\src\Uno.UI.Composition\Composition\Visual.skia.cs:line 199
     at Windows.UI.Composition.Visual.Render(PaintingSession& parentSession) in C:\a\1\s\src\Uno.UI.Composition\Composition\Visual.skia.cs:line 199
     at Windows.UI.Composition.Visual.Render(PaintingSession& parentSession) in C:\a\1\s\src\Uno.UI.Composition\Composition\Visual.skia.cs:line 199
     at Windows.UI.Composition.Visual.Render(PaintingSession& parentSession) in C:\a\1\s\src\Uno.UI.Composition\Composition\Visual.skia.cs:line 199
     at Windows.UI.Composition.Visual.Render(PaintingSession& parentSession) in C:\a\1\s\src\Uno.UI.Composition\Composition\Visual.skia.cs:line 199
     at Windows.UI.Composition.Visual.Render(PaintingSession& parentSession) in C:\a\1\s\src\Uno.UI.Composition\Composition\Visual.skia.cs:line 199
     at Windows.UI.Composition.Visual.Render(PaintingSession& parentSession) in C:\a\1\s\src\Uno.UI.Composition\Composition\Visual.skia.cs:line 199
     at Windows.UI.Composition.Visual.Render(PaintingSession& parentSession) in C:\a\1\s\src\Uno.UI.Composition\Composition\Visual.skia.cs:line 199
     at Windows.UI.Composition.Visual.Render(PaintingSession& parentSession) in C:\a\1\s\src\Uno.UI.Composition\Composition\Visual.skia.cs:line 199
     at Windows.UI.Composition.Visual.Render(PaintingSession& parentSession) in C:\a\1\s\src\Uno.UI.Composition\Composition\Visual.skia.cs:line 199
     at Windows.UI.Composition.Visual.Render(PaintingSession& parentSession) in C:\a\1\s\src\Uno.UI.Composition\Composition\Visual.skia.cs:line 199
     at Windows.UI.Composition.Visual.Render(PaintingSession& parentSession) in C:\a\1\s\src\Uno.UI.Composition\Composition\Visual.skia.cs:line 199
     at Windows.UI.Composition.Visual.Render(PaintingSession& parentSession) in C:\a\1\s\src\Uno.UI.Composition\Composition\Visual.skia.cs:line 199
     at Windows.UI.Composition.Visual.Render(PaintingSession& parentSession) in C:\a\1\s\src\Uno.UI.Composition\Composition\Visual.skia.cs:line 199
     at Windows.UI.Composition.Visual.Render(PaintingSession& parentSession) in C:\a\1\s\src\Uno.UI.Composition\Composition\Visual.skia.cs:line 199
     at Windows.UI.Composition.Visual.Render(PaintingSession& parentSession) in C:\a\1\s\src\Uno.UI.Composition\Composition\Visual.skia.cs:line 199
     at Windows.UI.Composition.Visual.Render(PaintingSession& parentSession) in C:\a\1\s\src\Uno.UI.Composition\Composition\Visual.skia.cs:line 199
     at Windows.UI.Composition.Visual.Render(PaintingSession& parentSession) in C:\a\1\s\src\Uno.UI.Composition\Composition\Visual.skia.cs:line 199
     at Windows.UI.Composition.Visual.Render(PaintingSession& parentSession) in C:\a\1\s\src\Uno.UI.Composition\Composition\Visual.skia.cs:line 199
     at Windows.UI.Composition.Visual.RenderRootVisual(SKSurface surface, Nullable`1 offsetOverride) in C:\a\1\s\src\Uno.UI.Composition\Composition\Visual.skia.cs:line 152
     at Uno.UI.Runtime.Skia.Gtk.SoftwareRenderSurface.OnDrawn(Context cr) in C:\a\1\s\src\Uno.UI.Runtime.Skia.Gtk\Rendering\SoftwareRenderSurface.cs:line 99
     at Gtk.Widget.Drawn_cb(IntPtr inst, IntPtr cr)

• #16167
• https://github.com/unoplatform/uno/issues/18105
• When_UnloadAndReload_Then_ReMaterializeItems
• When_ThemeChange on Wasm
  
• Android TestProgressRing_InitialState fails
• FlyoutTest_Unloaded
• FlyoutTest_Target is flaky on iOS
• https://github.com/unoplatform/uno/issues/13417
• PasswordBox_RevealInScrollViewer is flaky on iOS
• #15734
• TextBox_VerticalAlignment is flaky on Wasm
• Verify_MenuBarItem_Bounds_Native_Popups is flaky on Android - started being flaky after pull request #13630 (@ramezgerges)
• TextBox_Readonly is flaky on iOS
• When_Drag_Then_DoesNotScroll is not working on Android, the thumb drags when it should not
• CanChangeDisplayModeBeforeLoaded is flaky on Skia
• Holding_Tests.When_InListViewWithoutItemClick fails for missing item on android
• Holding_Tests.When_Basic fails on android
• Holding_Tests.When_InScroll fails on android
• Holding_Tests.When_InListViewWithItemClick fails on android
• TestTextBoxReleaseOutFocused is flaky on WASM
• https://github.com/unoplatform/uno/issues/15829
• When_MultipleItems is not failing sometimes (Timing related ?)
• Flyout_TemplatedParent is flaky on WASM
• https://github.com/unoplatform/uno/issues/15828
• https://github.com/unoplatform/uno/issues/15471
• TestTranslatedTap is flaky on WASM (could be related to https://github.com/unoplatform/uno/issues/13844)
• ValidateOverflowPlacement does not pass on Android
• TestTranslatedTap is failing on android
• DoesNotShowMenuIfSecondaryElementsAreCollapsed does not pass on Android
• When_NestedInSVAndOutOfViewportOnInitialLoad_Then_MaterializedEvenWhenScrollingOnMinorAxis is flaky on WASM
• #13844
• Android When_NestedIRSlowlyChangeViewport_Then_MaterializedNeededItems (Timeout even with retries) (Youssef: Can't repro)
• Android When_Verify_Canvas_With_Outer_Clip (Assert.Fail failed. ImageAssert.HasColorAt @ line 73) (Youssef: Verified - it always fails)
• UITests.Windows_UI_Xaml_Media.GradientBrushTests.GradientsPage crashes the android emulator completely (not just the app)
• Android VerifyNavigationViewItemToolTipPaneDisplayMode (Youssef: Verified - it always crashes locally)
• When_Focused_Element_Transformed is crashing the iOS/Android emulator
• Android VerifyCollectionChangeGetsRespected (Cleanup System.Exception: Event was not raised. in MUXApiTestBase.Cleanup()) (Youssef: test not disabled on master, and always passes for me)
• ListView_ObservableCollection_Unused_Space fails regularly on iOS (Expected: 132.0f But was: 88.0f)
• When_Open_Close_Root_Twice_Keep_State fails on android for missing TreeView items (Youssef: fails locally with null ref)
• When_UsingMultipleLanguages unit test fails when running with net7 or later (DateFormatter is buggy and CultureInfo itself returns wrong values)
• https://github.com/unoplatform/uno/issues/11997
• https://github.com/unoplatform/uno/issues/13257
• When_Grid_RowCollection_Changes fails with invalid child.SizePassedToArrangeOverride
• iOS 16+ tests
  • NativeCommandBar_Size fails (The first orientation change fails to be applied)
  • ListView_SelectedItems fails (Timeout ?)
  • When_Flipview_DataTemplateSelector (AssertFailedException: Assert.Fail failed. ImageAssert.HasColorAt @ line 101)
  • When_Add_Remove failure: ColorPicker
  • When_Add_Remove failure: CalendarView
• iOS When_ItemsControl.When_NestedItemsControl_RecycleTemplate (Assert.AreEqual failed. Expected a difference no greater than <1> between expected value <50> and actual value <0>. )
• iOS RepeaterTests.VerifyCorrectionsInNonScrollableDirection
• iOS/Skia VerifyFocusedItemIsRecycledOnCollectionReset (Skia fixed in https://github.com/unoplatform/uno/pull/13074)
• iOS When_WebMessageReceived (Timed out waiting for condition to be met. When_WebMessageReceived():207)
• iOS When_Flipview_Items_Modified (Timed out waiting for equality condition to be met. Expected -1 but last received value was 0.)
• iOS When_Render_Border_GetPixelsAsync (Assert.AreEqual failed. Expected:<<10, 10>>. Actual:<<10.5, 10>>. Invalid Layouted.)
• https://github.com/unoplatform/uno/issues/13095
• [All platforms] ValidateFractionalWidthDoesNotCrash (Assert.Fail failed. Timeout waiting on SpectrumRectangle.Fill to be set.) (Youssef: Test is actually skipped in WinUI. There is not much info on why it's skipped there, but I don't see anything that should trigger property changed for Fill)
• When_StretchAndAlignment on Android 28 (Youssef: Passing for me locally)
• TextBox_Readonly_ShouldNotBringUpKeyboard on Android 30 (The keyboard does not show up on emulators, breaking the test)
• When_PressOnContainerAndReleaseOnNested_Mouse on WASM (fails in CI)
• When_RepeatDuration: It was enabled in https://github.com/unoplatform/uno/pull/13060, but then failures are found and reverted in https://github.com/unoplatform/uno/pull/13089
• https://github.com/unoplatform/uno/issues/13384
• [iOS/Skia/macOS] RepeaterTests.VerifyRepeaterDoesNotLeakItemContainers (Skia started failing in https://github.com/unoplatform/uno/pull/16523)
• ExecuteScriptAsync tests are flaky on iOS (https://github.com/unoplatform/uno/issues/11997)
• WebView tests are flaky in general
• When_LargeExtent_And_Very_Large_List_Scroll_To_End_And_Back_Half_Size is flaky on Skia Android
• When_SmallExtent_And_Large_List_Scroll_To_End_Half_Size is flaky on Skia Android
• When_IsTextSelectionEnabled_ContextMenu_SelectAll is very flaky on all targets (disabled on all targets)
• When_ScrollIntoView_FreshlyAddedOffscreenItem is very flaky on all targets (disabled on all targets)
• When_ComboPopup_Rearrange_ScrollShouldNotReset is flaky on Skia WASM (disabled on Skia WASM)

In Progress

• TestTextBoxReleaseOutFocused is flaky on WASM
• TestTextBoxTap is very flaky on WASM
• Given_SymbolIcon.Validate_Size fails on all targets (Expected:<20>. Actual:<627.5>) (https://github.com/unoplatform/uno/pull/13069)
• Android VerifyDropdownItemTemplateWithNoControl (failed cleanup System.Exception: Event was not raised.) (Youssef: passing for me - https://github.com/unoplatform/uno/pull/13077)
• Android VerifyBreadcrumbDefaultAPIValues (failed cleanup System.Exception: Event was not raised.) (Youssef: passing for me - https://github.com/unoplatform/uno/pull/13077)
• ValidateOverflowButtonHidesWhenAppropriateWithNoAppBarButtons, ValidateMoreButtonCanShowWithoutSizeChanging, ValidateOverflowButtonHidesWhenAppropriateWithPrimaryAppBarButtons are failing randomly on android. (Youssef: passing for me - https://github.com/unoplatform/uno/pull/13078)
• Android/iOS When_EmbeddedResource (resource is not included when build on dotnet build) (Youssef: This appears to only be an issue for Uno.UI solution, ie, not a problem for external customers. Fixing in https://github.com/unoplatform/uno/pull/13079)
• DoubleAnimation_Tests on android. These tests fail on android when running one by one, but don't when run with others. (Youssef: Can't repro. Enabling in #13082)
  复制

  System.Exception : Error while performing WaitFor([unknown], "Failed to find [Completed] on [Text]", null, null, null)
  ----> System.TimeoutException : Failed to find [Completed] on [Text]

• When_Opacity_Paused_With_FillBehaviorHold_Then_Hold - on Android (Youssef: Can't repro. Enabling in #13082)
• When_Transforms_Paused_With_FillBehaviorStop_Then_Hold. This test tends to fail when run alone, indicating a race condition, see in the comments for screenshots of the test. (Youssef: Can't repro. Enabling in #13082)
  复制

  1) Failed : SamplesApp.UITests.Windows_UI_Xaml_Media_Animation.DoubleAnimation_Tests.When_Transforms_Paused_With_FillBehaviorStop_Then_Hold
  ImageAssert.AreEqual @ line 85
  pixelTolerance: No color tolerance
  expected: Initial (When_Transforms_Paused_With_FillBehaviorStop_Then_Hold_Initial.png {Width=1280, Height=800}) in 
  {X=615,Y=442,Width=50,Height=50}
  actual  : Final (When_Transforms_Paused_With_FillBehaviorStop_Then_Hold_Final.png {Width=1280, Height=800}) in 
  {X=615,Y=442,Width=50,Height=50}

• iOS/Android Given_ImageBrushStretch (Incorrect RenderTargetBitmap content) (youssef: Issue not related to RenderTargetBitmap. Fixing test in https://github.com/unoplatform/uno/pull/13081)

Done

• https://github.com/unoplatform/uno/issues/10164
• Android all Basics_AutomatedTransformation ( The specified child already has a parent. You must call removeView() on the child's parent first.) (https://github.com/unoplatform/uno/pull/13061)
• Android/Skia When_Render_Then_CanRenderOnCanvas (Assert.Fail failed. ImageAssert.HasColorAt @ line 134) (works locally on Skia, fails on Android and being fixed in https://github.com/unoplatform/uno/pull/13070)
• Wasm When_Reorder_To_Last_2 (https://github.com/unoplatform/uno/pull/13093)
• [All platforms] RepeaterTests.VerifyUIElementsInItemsSource (https://github.com/unoplatform/uno/pull/13074)
• [All platforms] RepeaterTests.NestedRepeaterWithDataTemplateScenario (https://github.com/unoplatform/uno/pull/13074)
• [All platforms] RepeaterTests.VerifyStoreScenarioCache (https://github.com/unoplatform/uno/pull/13074)
• https://github.com/unoplatform/uno/issues/10165 (https://github.com/unoplatform/uno/pull/13056)
• https://github.com/unoplatform/uno/pull/13071