GPU: Remove swizzle undefined matching and rework depth aliasing (#4896)

* GPU: Remove swizzle undefined matching and rework depth aliasing

@gdkchan pointed out that UI textures in TOTK seemed to be setting their texture swizzle incorrectly (texture was RGB but was sampling A, swizzle for A was wrong), so I determined that SwizzleComponentMatches was the problem and set on eliminating it. This PR combines existing work to select the most recently modified texture (now used when selecting which aliased texture to use) with some additional changes to remove the swizzle check and support aliased view creation.

The original observation (#1538) was that we wanted to match depth textures for the purposes of aliasing with color textures, but they often had different swizzle from what was sampled (as it's generally the identity swizzle once rendered). At the time, I decided to allow swizzles to match if only the defined components matched, which fixed the issue in all known cases but could easily be broken by a game _expecting_ a given swizzle, such as a 1/0 value on a component.

This error case could also occur in textures that don't even depth alias, such as R11G11B10, as the rule was created to generally apply to all cases.

The solution is now to fail this exact match test, and allow the search for an R32 texture to create a swizzled view of a D32 texture (and other such cases). This allows the creation of a view that mismatches the requested format, which wasn't present before and was the reason for the swizzle matching approach.

The exact match and view creation rules now follow the same rules over what textures to select when there are multiple options (such as a "perfect" match and an "aliased" match at the same time). It now selects the most recently modified texture, which is done with a new sequence number in the GpuContext (because we don't have enough of these).

Reportedly fixes UI having weird coloured backgrounds in TOTK. This also fixes an issue in MK8D where returning from a race resulted in the character selection cubemaps being broken. May work around issues introduced by the "short texture cache" PR due to modification ordering, though they won't be truly fixed.

Should allow (#4365) to avoid copies in more cases. Need to test that.

I tested a bunch of games #1538 originally affected and they seem to be fine. This change affects all games so it would be good to get some wide testing on it.

* Address feedback 1, fix an issue

* Workaround: Do not allow copies for format alias.

These should be removed when D32<->R32 copy dependencies become legal

src/Ryujinx.Graphics.Gpu/GpuContext.cs

@@ -98,6 +98,8 @@ namespace Ryujinx.Graphics.Gpu
         private Thread _gpuThread;
         private bool _pendingSync;
 
+        private long _modifiedSequence;
+
         /// <summary>
         /// Creates a new instance of the GPU emulation context.
         /// </summary>
@@ -200,6 +202,15 @@ namespace Ryujinx.Graphics.Gpu
             return divided * NsToTicksFractionNumerator + errorBias;
         }
 
+        /// <summary>
+        /// Gets a sequence number for resource modification ordering. This increments on each call.
+        /// </summary>
+        /// <returns>A sequence number for resource modification ordering</returns>
+        public long GetModifiedSequence()
+        {
+            return _modifiedSequence++;
+        }
+
         /// <summary>
         /// Gets the value of the GPU timer.
         /// </summary>

src/Ryujinx.Graphics.Gpu/Image/Texture.cs

@@ -1170,6 +1170,7 @@ namespace Ryujinx.Graphics.Gpu.Image
         /// <param name="caps">Host GPU capabilities</param>
         /// <param name="firstLayer">Texture view initial layer on this texture</param>
         /// <param name="firstLevel">Texture view first mipmap level on this texture</param>
+        /// <param name="flags">Texture search flags</param>
         /// <returns>The level of compatiblilty a view with the given parameters created from this texture has</returns>
         public TextureViewCompatibility IsViewCompatible(
             TextureInfo info,
@@ -1178,11 +1179,12 @@ namespace Ryujinx.Graphics.Gpu.Image
             int layerSize,
             Capabilities caps,
             out int firstLayer,
-            out int firstLevel)
+            out int firstLevel,
+            TextureSearchFlags flags = TextureSearchFlags.None)
         {
             TextureViewCompatibility result = TextureViewCompatibility.Full;
 
-            result = TextureCompatibility.PropagateViewCompatibility(result, TextureCompatibility.ViewFormatCompatible(Info, info, caps));
+            result = TextureCompatibility.PropagateViewCompatibility(result, TextureCompatibility.ViewFormatCompatible(Info, info, caps, flags));
             if (result != TextureViewCompatibility.Incompatible)
             {
                 result = TextureCompatibility.PropagateViewCompatibility(result, TextureCompatibility.ViewTargetCompatible(Info, info, ref caps));

src/Ryujinx.Graphics.Gpu/Image/TextureCache.cs

@@ -569,7 +569,7 @@ namespace Ryujinx.Graphics.Gpu.Image
 
             Texture texture = null;
 
-            TextureMatchQuality bestQuality = TextureMatchQuality.NoMatch;
+            long bestSequence = 0;
 
             for (int index = 0; index < sameAddressOverlapsCount; index++)
             {
@@ -601,17 +601,12 @@ namespace Ryujinx.Graphics.Gpu.Image
                             continue;
                         }
                     }
-                }
 
-                if (matchQuality == TextureMatchQuality.Perfect)
+                    if (texture == null || overlap.Group.ModifiedSequence - bestSequence > 0)
                     {
                         texture = overlap;
-                    break;
+                        bestSequence = overlap.Group.ModifiedSequence;
                     }
-                else if (matchQuality > bestQuality)
-                {
-                    texture = overlap;
-                    bestQuality = matchQuality;
                 }
             }
 
@@ -664,6 +659,7 @@ namespace Ryujinx.Graphics.Gpu.Image
             int fullyCompatible = 0;
 
             // Evaluate compatibility of overlaps, add temporary references
+            int preferredOverlap = -1;
 
             for (int index = 0; index < overlapsCount; index++)
             {
@@ -675,17 +671,26 @@ namespace Ryujinx.Graphics.Gpu.Image
                     sizeInfo.LayerSize,
                     _context.Capabilities,
                     out int firstLayer,
-                    out int firstLevel);
+                    out int firstLevel,
+                    flags);
 
-                if (overlapCompatibility == TextureViewCompatibility.Full)
+                if (overlapCompatibility >= TextureViewCompatibility.FormatAlias)
                 {
                     if (overlap.IsView)
                     {
-                        overlapCompatibility = TextureViewCompatibility.CopyOnly;
+                        overlapCompatibility = overlapCompatibility == TextureViewCompatibility.FormatAlias ?
+                            TextureViewCompatibility.Incompatible :
+                            TextureViewCompatibility.CopyOnly;
                     }
                     else
                     {
                         fullyCompatible++;
+
+                        if (preferredOverlap == -1 || overlap.Group.ModifiedSequence - bestSequence > 0)
+                        {
+                            preferredOverlap = index;
+                            bestSequence = overlap.Group.ModifiedSequence;
+                        }
                     }
                 }
 
@@ -695,26 +700,38 @@ namespace Ryujinx.Graphics.Gpu.Image
 
             // Search through the overlaps to find a compatible view and establish any copy dependencies.
 
-            for (int index = 0; index < overlapsCount; index++)
+            if (preferredOverlap != -1)
             {
-                Texture overlap = _textureOverlaps[index];
+                Texture overlap = _textureOverlaps[preferredOverlap];
-                OverlapInfo oInfo = _overlapInfo[index];
+                OverlapInfo oInfo = _overlapInfo[preferredOverlap];
+
+                bool aliased = oInfo.Compatibility == TextureViewCompatibility.FormatAlias;
 
-                if (oInfo.Compatibility == TextureViewCompatibility.Full)
-                {
                 if (!isSamplerTexture)
                 {
                     // If this is not a sampler texture, the size might be different from the requested size,
                     // so we need to make sure the texture information has the correct size for this base texture,
                     // before creating the view.
-                        info = info.CreateInfoForLevelView(overlap, oInfo.FirstLevel);
+
+                    info = info.CreateInfoForLevelView(overlap, oInfo.FirstLevel, aliased);
+                }
+                else if (aliased)
+                {
+                    // The format must be changed to match the parent.
+                    info = info.CreateInfoWithFormat(overlap.Info.FormatInfo);
                 }
 
                 texture = overlap.CreateView(info, sizeInfo, range.Value, oInfo.FirstLayer, oInfo.FirstLevel);
                 texture.SynchronizeMemory();
-                    break;
             }
-                else if (oInfo.Compatibility == TextureViewCompatibility.CopyOnly && fullyCompatible == 0)
+            else
+            {
+                for (int index = 0; index < overlapsCount; index++)
+                {
+                    Texture overlap = _textureOverlaps[index];
+                    OverlapInfo oInfo = _overlapInfo[index];
+
+                    if (oInfo.Compatibility == TextureViewCompatibility.CopyOnly && fullyCompatible == 0)
                     {
                         // Only copy compatible. If there's another choice for a FULLY compatible texture, choose that instead.
 
@@ -728,6 +745,7 @@ namespace Ryujinx.Graphics.Gpu.Image
                         break;
                     }
                 }
+            }
 
             if (texture != null)
             {
@@ -740,7 +758,7 @@ namespace Ryujinx.Graphics.Gpu.Image
                     Texture overlap = _textureOverlaps[index];
                     OverlapInfo oInfo = _overlapInfo[index];
 
-                    if (oInfo.Compatibility <= TextureViewCompatibility.LayoutIncompatible)
+                    if (oInfo.Compatibility <= TextureViewCompatibility.LayoutIncompatible || oInfo.Compatibility == TextureViewCompatibility.FormatAlias)
                     {
                         if (!overlap.IsView && texture.DataOverlaps(overlap, oInfo.Compatibility))
                         {

src/Ryujinx.Graphics.Gpu/Image/TextureCompatibility.cs

@@ -291,22 +291,7 @@ namespace Ryujinx.Graphics.Gpu.Image
         /// <returns>The minimum compatibility level of two provided view compatibility results</returns>
         public static TextureViewCompatibility PropagateViewCompatibility(TextureViewCompatibility first, TextureViewCompatibility second)
         {
-            if (first == TextureViewCompatibility.Incompatible || second == TextureViewCompatibility.Incompatible)
+            return (TextureViewCompatibility)Math.Min((int)first, (int)second);
-            {
-                return TextureViewCompatibility.Incompatible;
-            }
-            else if (first == TextureViewCompatibility.LayoutIncompatible || second == TextureViewCompatibility.LayoutIncompatible)
-            {
-                return TextureViewCompatibility.LayoutIncompatible;
-            }
-            else if (first == TextureViewCompatibility.CopyOnly || second == TextureViewCompatibility.CopyOnly)
-            {
-                return TextureViewCompatibility.CopyOnly;
-            }
-            else
-            {
-                return TextureViewCompatibility.Full;
-            }
         }
 
         /// <summary>
@@ -628,15 +613,21 @@ namespace Ryujinx.Graphics.Gpu.Image
         /// <param name="lhs">Texture information of the texture view</param>
         /// <param name="rhs">Texture information of the texture view</param>
         /// <param name="caps">Host GPU capabilities</param>
+        /// <param name="flags">Texture search flags</param>
         /// <returns>The view compatibility level of the texture formats</returns>
-        public static TextureViewCompatibility ViewFormatCompatible(TextureInfo lhs, TextureInfo rhs, Capabilities caps)
+        public static TextureViewCompatibility ViewFormatCompatible(TextureInfo lhs, TextureInfo rhs, Capabilities caps, TextureSearchFlags flags)
         {
             FormatInfo lhsFormat = lhs.FormatInfo;
             FormatInfo rhsFormat = rhs.FormatInfo;
 
             if (lhsFormat.Format.IsDepthOrStencil() || rhsFormat.Format.IsDepthOrStencil())
             {
-                return lhsFormat.Format == rhsFormat.Format ? TextureViewCompatibility.Full : TextureViewCompatibility.Incompatible;
+                return FormatMatches(lhs, rhs, flags.HasFlag(TextureSearchFlags.ForSampler), flags.HasFlag(TextureSearchFlags.DepthAlias)) switch
+                {
+                    TextureMatchQuality.Perfect => TextureViewCompatibility.Full,
+                    TextureMatchQuality.FormatAlias => TextureViewCompatibility.FormatAlias,
+                    _ => TextureViewCompatibility.Incompatible
+                };
             }
 
             if (IsFormatHostIncompatible(lhs, caps) || IsFormatHostIncompatible(rhs, caps))
@@ -754,49 +745,6 @@ namespace Ryujinx.Graphics.Gpu.Image
             return result ? TextureViewCompatibility.Full : TextureViewCompatibility.Incompatible;
         }
 
-        /// <summary>
-        /// Checks if a swizzle component in two textures functionally match, taking into account if the components are defined.
-        /// </summary>
-        /// <param name="lhs">Texture information to compare</param>
-        /// <param name="rhs">Texture information to compare with</param>
-        /// <param name="swizzleLhs">Swizzle component for the first texture</param>
-        /// <param name="swizzleRhs">Swizzle component for the second texture</param>
-        /// <param name="component">Component index, starting at 0 for red</param>
-        /// <returns>True if the swizzle components functionally match, false othersize</returns>
-        private static bool SwizzleComponentMatches(TextureInfo lhs, TextureInfo rhs, SwizzleComponent swizzleLhs, SwizzleComponent swizzleRhs, int component)
-        {
-            int lhsComponents = lhs.FormatInfo.Components;
-            int rhsComponents = rhs.FormatInfo.Components;
-
-            if (lhsComponents == 4 && rhsComponents == 4)
-            {
-                return swizzleLhs == swizzleRhs;
-            }
-
-            // Swizzles after the number of components a format defines are "undefined".
-            // We allow these to not be equal under certain circumstances.
-            // This can only happen when there are less than 4 components in a format.
-            // It tends to happen when float depth textures are sampled.
-
-            bool lhsDefined = (swizzleLhs - SwizzleComponent.Red) < lhsComponents;
-            bool rhsDefined = (swizzleRhs - SwizzleComponent.Red) < rhsComponents;
-
-            if (lhsDefined == rhsDefined)
-            {
-                // If both are undefined, return true. Otherwise just check if they're equal.
-                return lhsDefined ? swizzleLhs == swizzleRhs : true;
-            }
-            else
-            {
-                SwizzleComponent defined = lhsDefined ? swizzleLhs : swizzleRhs;
-                SwizzleComponent undefined = lhsDefined ? swizzleRhs : swizzleLhs;
-
-                // Undefined swizzle can be matched by a forced value (0, 1), exact equality, or expected value.
-                // For example, R___ matches R001, RGBA but not RBGA.
-                return defined == undefined || defined < SwizzleComponent.Red || defined == SwizzleComponent.Red + component;
-            }
-        }
-
         /// <summary>
         /// Checks if the texture shader sampling parameters of two texture informations match.
         /// </summary>
@@ -806,10 +754,10 @@ namespace Ryujinx.Graphics.Gpu.Image
         public static bool SamplerParamsMatches(TextureInfo lhs, TextureInfo rhs)
         {
             return lhs.DepthStencilMode == rhs.DepthStencilMode &&
-                   SwizzleComponentMatches(lhs, rhs, lhs.SwizzleR, rhs.SwizzleR, 0) &&
+                   lhs.SwizzleR == rhs.SwizzleR &&
-                   SwizzleComponentMatches(lhs, rhs, lhs.SwizzleG, rhs.SwizzleG, 1) &&
+                   lhs.SwizzleG == rhs.SwizzleG &&
-                   SwizzleComponentMatches(lhs, rhs, lhs.SwizzleB, rhs.SwizzleB, 2) &&
+                   lhs.SwizzleB == rhs.SwizzleB &&
-                   SwizzleComponentMatches(lhs, rhs, lhs.SwizzleA, rhs.SwizzleA, 3);
+                   lhs.SwizzleA == rhs.SwizzleA;
         }
 
         /// <summary>

src/Ryujinx.Graphics.Gpu/Image/TextureGroup.cs

@@ -68,6 +68,11 @@ namespace Ryujinx.Graphics.Gpu.Image
         /// </summary>
         public bool HasIncompatibleOverlaps => _incompatibleOverlaps.Count > 0;
 
+        /// <summary>
+        /// Number indicating the order this texture group was modified relative to others.
+        /// </summary>
+        public long ModifiedSequence { get; private set; }
+
         private readonly GpuContext _context;
         private readonly PhysicalMemory _physicalMemory;
 
@@ -664,6 +669,8 @@ namespace Ryujinx.Graphics.Gpu.Image
         /// <param name="texture">The texture that has been modified</param>
         public void SignalModified(Texture texture)
         {
+            ModifiedSequence = _context.GetModifiedSequence();
+
             ClearIncompatibleOverlaps(texture);
 
             EvaluateRelevantHandles(texture, (baseHandle, regionCount, split) =>
@@ -684,6 +691,8 @@ namespace Ryujinx.Graphics.Gpu.Image
         /// <param name="bound">True if this texture is being bound, false if unbound</param>
         public void SignalModifying(Texture texture, bool bound)
         {
+            ModifiedSequence = _context.GetModifiedSequence();
+
             ClearIncompatibleOverlaps(texture);
 
             EvaluateRelevantHandles(texture, (baseHandle, regionCount, split) =>

src/Ryujinx.Graphics.Gpu/Image/TextureInfo.cs

@@ -300,8 +300,9 @@ namespace Ryujinx.Graphics.Gpu.Image
         /// </summary>
         /// <param name="parent">The parent texture</param>
         /// <param name="firstLevel">The first level of the texture view</param>
+        /// <param name="parentFormat">True if the parent format should be inherited</param>
         /// <returns>The adjusted texture information with the new size</returns>
-        public TextureInfo CreateInfoForLevelView(Texture parent, int firstLevel)
+        public TextureInfo CreateInfoForLevelView(Texture parent, int firstLevel, bool parentFormat)
         {
             // When the texture is used as view of another texture, we must
             // ensure that the sizes are valid, otherwise data uploads would fail
@@ -370,7 +371,36 @@ namespace Ryujinx.Graphics.Gpu.Image
                 GobBlocksInZ,
                 GobBlocksInTileX,
                 target,
-                FormatInfo,
+                parentFormat ? parent.Info.FormatInfo : FormatInfo,
+                DepthStencilMode,
+                SwizzleR,
+                SwizzleG,
+                SwizzleB,
+                SwizzleA);
+        }
+
+        /// <summary>
+        /// Creates texture information for a given format and this information.
+        /// </summary>
+        /// <param name="formatInfo">Format for the new texture info</param>
+        /// <returns>New info with the specified format</returns>
+        public TextureInfo CreateInfoWithFormat(FormatInfo formatInfo)
+        {
+            return new TextureInfo(
+                GpuAddress,
+                Width,
+                Height,
+                DepthOrLayers,
+                Levels,
+                SamplesInX,
+                SamplesInY,
+                Stride,
+                IsLinear,
+                GobBlocksInY,
+                GobBlocksInZ,
+                GobBlocksInTileX,
+                Target,
+                formatInfo,
                 DepthStencilMode,
                 SwizzleR,
                 SwizzleG,

src/Ryujinx.Graphics.Gpu/Image/TextureViewCompatibility.cs

@@ -9,6 +9,7 @@
         Incompatible = 0,
         LayoutIncompatible,
         CopyOnly,
+        FormatAlias,
         Full
     }
 }