Compare commits

...

17 Commits

Author SHA1 Message Date
1828bc949e nuget: bump Microsoft.IO.RecyclableMemoryStream from 3.0.0 to 3.0.1 (#6936)
Bumps [Microsoft.IO.RecyclableMemoryStream](https://github.com/Microsoft/Microsoft.IO.RecyclableMemoryStream) from 3.0.0 to 3.0.1.
- [Release notes](https://github.com/Microsoft/Microsoft.IO.RecyclableMemoryStream/releases)
- [Changelog](https://github.com/microsoft/Microsoft.IO.RecyclableMemoryStream/blob/master/CHANGES.md)
- [Commits](https://github.com/Microsoft/Microsoft.IO.RecyclableMemoryStream/compare/3.0.0...v3.0.1)

---
updated-dependencies:
- dependency-name: Microsoft.IO.RecyclableMemoryStream
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-06-15 22:51:50 +02:00
c0f2491eae Vulkan separate descriptor set fixes (#6895)
* Ensure descriptor sets are only re-used when all command buffers using it have completed

* Fix some SPIR-V capabilities

* Set update after bind flag if we exceed limits

* Simpler fix for Intel

* Format whitespace

* Make struct readonly

* Add barriers for extra set arrays too
2024-06-02 22:40:28 -03:00
d7c6474729 GPU: Remove unused dynamic state and pipeline settings (#6796)
* Dynamic state for Depth Bounds should not be passed to PipelineDynamicStateCreateInfo as the command to set them is never called.

Do not pass pointer to viewport and scissor as those dynamic states should be supported on all devices.

Same as above for DepthBias values.

* Code Review Suggestion

* Pipeline derivation is not implemented and is not suggested.

* Depth Bounds are not used.
2024-06-02 22:32:10 -03:00
1ecc8fbc3b New pooled memory types (#6821)
* feat: add new types MemoryOwner and SpanOwner

* use SpanOwner instead of new array allocation

* change for loop condition to `fences.Length` instead of `count` to elide Span boundary checks on `fences`
2024-06-02 22:24:14 -03:00
888402ecaf Avoid inexact read with 'Stream.Read' (#6847) 2024-06-02 22:16:48 +02:00
971d24aef0 nuget: bump Microsoft.IdentityModel.JsonWebTokens from 7.5.2 to 7.6.0 (#6893)
Bumps [Microsoft.IdentityModel.JsonWebTokens](https://github.com/AzureAD/azure-activedirectory-identitymodel-extensions-for-dotnet) from 7.5.2 to 7.6.0.
- [Release notes](https://github.com/AzureAD/azure-activedirectory-identitymodel-extensions-for-dotnet/releases)
- [Changelog](https://github.com/AzureAD/azure-activedirectory-identitymodel-extensions-for-dotnet/blob/dev/CHANGELOG.md)
- [Commits](https://github.com/AzureAD/azure-activedirectory-identitymodel-extensions-for-dotnet/compare/7.5.2...7.6.0)

---
updated-dependencies:
- dependency-name: Microsoft.IdentityModel.JsonWebTokens
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-06-02 22:10:47 +02:00
c41fddd25e Vulkan: Extend full bindless to cover cases with phi nodes (#6853)
* Key textures using set and binding (rather than just binding)

* Extend full bindless to cover cases with phi nodes

* Log error on bindless access failure

* Shader cache version bump

* Remove constant buffer match to reduce the chances of full bindless triggering

* Re-enable it for constant buffers, paper mario does actually need it

* Format whitespace
2024-05-26 15:20:10 -03:00
2ebe929fa5 misc: Change disk shader cache compression algorithm to Brotli (RFC 7932) (#6841)
* Prefer `Brotli` compression for disk shader cache.

* Final default case for decompression switch.

* Prefer fastest compression.
2024-05-26 20:06:41 +02:00
53d096e392 Allow texture arrays to use separate descriptor sets on Vulkan (#6870)
* Report base and extra sets from the backend

* Pass texture set index everywhere

* Key textures using set and binding (rather than just binding)

* Start using extra sets for array textures

* Shader cache version bump

* Separate new commands, some PR feedback

* Introduce new manual descriptor set reservation method that prevents it from being used by something else while owned by an array

* Move bind extra sets logic to new method

* Should only use separate array is MaximumExtraSets is not zero

* Format whitespace
2024-05-26 13:30:19 -03:00
4cc00bb4b1 nuget: bump Microsoft.IdentityModel.JsonWebTokens from 7.5.1 to 7.5.2 (#6809)
Bumps [Microsoft.IdentityModel.JsonWebTokens](https://github.com/AzureAD/azure-activedirectory-identitymodel-extensions-for-dotnet) from 7.5.1 to 7.5.2.
- [Release notes](https://github.com/AzureAD/azure-activedirectory-identitymodel-extensions-for-dotnet/releases)
- [Changelog](https://github.com/AzureAD/azure-activedirectory-identitymodel-extensions-for-dotnet/blob/dev/CHANGELOG.md)
- [Commits](https://github.com/AzureAD/azure-activedirectory-identitymodel-extensions-for-dotnet/compare/7.5.1...7.5.2)

---
updated-dependencies:
- dependency-name: Microsoft.IdentityModel.JsonWebTokens
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-05-25 05:35:49 +02:00
c98b7fc702 Workaround bug on logic op with float framebuffer (#6858)
* intel workaround

built on top of the amd workaround

* forgot to update the note

* Logic Change

Enabled workaround for all vendors that aren't nvidia

* Applied Suggestions
2024-05-23 22:57:26 -03:00
e65effcb05 Workaround AMD bug on logic op with float framebuffer (#6852)
* Workaround AMD bug on logic op with float framebuffer

* Format whitespace

* Update comment
2024-05-23 01:05:32 -03:00
c1ed150949 Kernel: Wake cores from idle directly rather than through a host thread (#6837)
* Kernel: Wake cores from idle directly rather than through a host thread

Right now when a core enters an idle state, leaving that idle state requires us to first signal the core's idle thread, which then signals the correct thread that we want to run on the core. This means that in a lot of cases, we're paying double for a thread to be woken from an idle state.

This PR moves this process to happen on the thread that is waking others out of idle, instead of an idle thread that needs to be woken first.

For compatibility the process has been kept as similar as possible - the process for IdleThreadLoop has been migrated to TryLeaveIdle, and is gated by a condition variable that lets it run only once at a time for each core. A core is only considered for wake from idle if idle is both active and has been signalled - the signal is consumed and the active state is cleared when the core leaves idle.

Dummy threads (just the idle thread at the moment) have been changed to have no host thread, as the work is now done by threads entering idle and signalling out of it.

This could put a bit of extra work on threads that would have triggered `_idleInterruptEvent` before, but I'd expect less work than signalling all those reset events and the OS overhead that follows. Worst case is that other threads performing these signals at the same time will have to wait for each other, but it's still going to be a very short amount of time.

Improvements are best seen in games with heavy (or very misguided) multithreading, such as Pokemon: Legends Arceus. Improvements are expected in Scarlet/Violet and TOTK, but are harder to measure.

Testing on Linux/MacOS still to be done, definitely need to test more games as this affects all of them (obviously) and any issues might be rare to encounter.

* Remove _idleThread entirely

* Use spinwait so we don't completely blast the CPU with cmpxchg

* Didn't I already do this

* Cleanup
2024-05-22 17:47:27 -03:00
c634eb4054 Updating Concentus dependency to speed up Opus decoding (#6757)
* Implementing new features in the latest Concentus library - span-in, span-out Opus decoding (so we don't have to make temporary buffer copies), returning a more precise error code from the decoder, and automatically linking the native opus library with P/invoke if supported on the current system

* Remove stub log messages and commit package upgrade to 2.1.0

* use more correct disposal pattern

* Bump to Concentus 2.1.1

* Bump to Concentus 2.1.2

* Don't bother pulling in native opus binaries from Concentus package (using ExcludeAssets).

* Fix opus MS channel count. Explicitly disable native lib probe in OpusCodecFactory.

* Bump to package 2.2.0 which has split out the native libs, as suggested.

---------

Co-authored-by: Logan Stromberg <lostromb@microsoft.com>
2024-05-20 18:38:38 -03:00
eb1ce41b00 GPU: Migrate buffers on GPU project, pre-emptively flush device local mappings (#6794)
* GPU: Migrate buffers on GPU project, pre-emptively flush device local mappings

Essentially retreading #4540, but it's on the GPU project now instead of the backend. This allows us to have a lot more control + knowledge of where the buffer backing has been changed and allows us to pre-emptively flush pages to host memory for quicker readback. It will allow us to do other stuff in the future, but we'll get there when we get there.

Performance greatly improved in Hyrule Warriors: Age of Calamity. Performance notably improved in TOTK (average). Performance for BOTW restored to how it was before #4911, perhaps a bit better.

- Rewrites a bunch of buffer migration stuff. Might want to tighten up how dispose stuff works.
- Fixed an issue where the copy for texture pre-flush would happen _after_ the syncpoint.

TODO: remove a page from pre-flush if it isn't flushed after a certain number of copies.

* Add copy deactivation

* Fix dependent virtual buffers

* Remove logging

* Fix format issues (maybe)

* Vulkan: Remove backing swap

* Add explicit memory access types for most buffers

* Fix typo

* Add device local force expiry, change buffer inheritance behaviour

* General cleanup, OGL fix

* BufferPreFlush comments

* BufferBackingState comments

* Add an extra precaution to BufferMigration

This is very unlikely, but it's important to cover loose ends like this.

* Address some feedback

* Docs
2024-05-19 16:53:37 -03:00
2f427deb67 Fix another NullReferenceException (#6826) 2024-05-17 20:11:30 -03:00
8f51938e2b Disable keyboard controller input while swkbd is open (foreground) (second attempt) (#6808)
* Block input updates while swkbd is open in foreground mode

* Flush internal driver state before unblocking input updates

* Rename Flush to Clear and remove unnecessary attribute

* Clear the driver state only if the GamepadDriver isn't null
2024-05-17 16:58:03 -03:00
114 changed files with 3298 additions and 1079 deletions

View File

@ -11,7 +11,7 @@
<PackageVersion Include="Avalonia.Svg" Version="11.0.0.18" />
<PackageVersion Include="Avalonia.Svg.Skia" Version="11.0.0.18" />
<PackageVersion Include="CommandLineParser" Version="2.9.1" />
<PackageVersion Include="Concentus" Version="1.1.7" />
<PackageVersion Include="Concentus" Version="2.2.0" />
<PackageVersion Include="DiscordRichPresence" Version="1.2.1.24" />
<PackageVersion Include="DynamicData" Version="8.4.1" />
<PackageVersion Include="FluentAvaloniaUI" Version="2.0.5" />
@ -20,9 +20,9 @@
<PackageVersion Include="LibHac" Version="0.19.0" />
<PackageVersion Include="Microsoft.CodeAnalysis.Analyzers" Version="3.3.4" />
<PackageVersion Include="Microsoft.CodeAnalysis.CSharp" Version="4.9.2" />
<PackageVersion Include="Microsoft.IdentityModel.JsonWebTokens" Version="7.5.1" />
<PackageVersion Include="Microsoft.IdentityModel.JsonWebTokens" Version="7.6.0" />
<PackageVersion Include="Microsoft.NET.Test.Sdk" Version="17.9.0" />
<PackageVersion Include="Microsoft.IO.RecyclableMemoryStream" Version="3.0.0" />
<PackageVersion Include="Microsoft.IO.RecyclableMemoryStream" Version="3.0.1" />
<PackageVersion Include="MsgPack.Cli" Version="1.0.1" />
<PackageVersion Include="NetCoreServer" Version="8.0.7" />
<PackageVersion Include="NUnit" Version="3.13.3" />
@ -49,4 +49,4 @@
<PackageVersion Include="System.Management" Version="8.0.0" />
<PackageVersion Include="UnicornEngine.Unicorn" Version="2.0.2-rc1-fb78016" />
</ItemGroup>
</Project>
</Project>

View File

@ -237,7 +237,7 @@ namespace ARMeilleure.CodeGen.Arm64
long originalPosition = _stream.Position;
_stream.Seek(0, SeekOrigin.Begin);
_stream.Read(code, 0, code.Length);
_stream.ReadExactly(code, 0, code.Length);
_stream.Seek(originalPosition, SeekOrigin.Begin);
RelocInfo relocInfo;

View File

@ -1444,7 +1444,7 @@ namespace ARMeilleure.CodeGen.X86
Span<byte> buffer = new byte[jump.JumpPosition - _stream.Position];
_stream.Read(buffer);
_stream.ReadExactly(buffer);
_stream.Seek(ReservedBytesForJump, SeekOrigin.Current);
codeStream.Write(buffer);

View File

@ -0,0 +1,140 @@
#nullable enable
using System;
using System.Buffers;
using System.Diagnostics.CodeAnalysis;
using System.Runtime.CompilerServices;
using System.Runtime.InteropServices;
using System.Threading;
namespace Ryujinx.Common.Memory
{
/// <summary>
/// An <see cref="IMemoryOwner{T}"/> implementation with an embedded length and fast <see cref="Span{T}"/>
/// accessor, with memory allocated from <seealso cref="ArrayPool{T}.Shared"/>.
/// </summary>
/// <typeparam name="T">The type of item to store.</typeparam>
public sealed class MemoryOwner<T> : IMemoryOwner<T>
{
private readonly int _length;
private T[]? _array;
/// <summary>
/// Initializes a new instance of the <see cref="MemoryOwner{T}"/> class with the specified parameters.
/// </summary>
/// <param name="length">The length of the new memory buffer to use</param>
private MemoryOwner(int length)
{
_length = length;
_array = ArrayPool<T>.Shared.Rent(length);
}
/// <summary>
/// Creates a new <see cref="MemoryOwner{T}"/> instance with the specified length.
/// </summary>
/// <param name="length">The length of the new memory buffer to use</param>
/// <returns>A <see cref="MemoryOwner{T}"/> instance of the requested length</returns>
/// <exception cref="ArgumentOutOfRangeException">Thrown when <paramref name="length"/> is not valid</exception>
[MethodImpl(MethodImplOptions.AggressiveInlining)]
public static MemoryOwner<T> Rent(int length) => new(length);
/// <summary>
/// Creates a new <see cref="MemoryOwner{T}"/> instance with the specified length and the content cleared.
/// </summary>
/// <param name="length">The length of the new memory buffer to use</param>
/// <returns>A <see cref="MemoryOwner{T}"/> instance of the requested length and the content cleared</returns>
/// <exception cref="ArgumentOutOfRangeException">Thrown when <paramref name="length"/> is not valid</exception>
[MethodImpl(MethodImplOptions.AggressiveInlining)]
public static MemoryOwner<T> RentCleared(int length)
{
MemoryOwner<T> result = new(length);
result._array.AsSpan(0, length).Clear();
return result;
}
/// <summary>
/// Creates a new <see cref="MemoryOwner{T}"/> instance with the content copied from the specified buffer.
/// </summary>
/// <param name="buffer">The buffer to copy</param>
/// <returns>A <see cref="MemoryOwner{T}"/> instance with the same length and content as <paramref name="buffer"/></returns>
public static MemoryOwner<T> RentCopy(ReadOnlySpan<T> buffer)
{
MemoryOwner<T> result = new(buffer.Length);
buffer.CopyTo(result._array);
return result;
}
/// <summary>
/// Gets the number of items in the current instance.
/// </summary>
public int Length
{
[MethodImpl(MethodImplOptions.AggressiveInlining)]
get => _length;
}
/// <inheritdoc/>
public Memory<T> Memory
{
[MethodImpl(MethodImplOptions.AggressiveInlining)]
get
{
T[]? array = _array;
if (array is null)
{
ThrowObjectDisposedException();
}
return new(array, 0, _length);
}
}
/// <summary>
/// Gets a <see cref="Span{T}"/> wrapping the memory belonging to the current instance.
/// </summary>
/// <remarks>
/// Uses a trick made possible by the .NET 6+ runtime array layout.
/// </remarks>
public Span<T> Span
{
[MethodImpl(MethodImplOptions.AggressiveInlining)]
get
{
T[]? array = _array;
if (array is null)
{
ThrowObjectDisposedException();
}
ref T firstElementRef = ref MemoryMarshal.GetArrayDataReference(array);
return MemoryMarshal.CreateSpan(ref firstElementRef, _length);
}
}
/// <inheritdoc/>
public void Dispose()
{
T[]? array = Interlocked.Exchange(ref _array, null);
if (array is not null)
{
ArrayPool<T>.Shared.Return(array);
}
}
/// <summary>
/// Throws an <see cref="ObjectDisposedException"/> when <see cref="_array"/> is <see langword="null"/>.
/// </summary>
[DoesNotReturn]
private static void ThrowObjectDisposedException()
{
throw new ObjectDisposedException(nameof(MemoryOwner<T>), "The buffer has already been disposed.");
}
}
}

View File

@ -0,0 +1,114 @@
using System;
using System.Buffers;
using System.Runtime.CompilerServices;
using System.Runtime.InteropServices;
namespace Ryujinx.Common.Memory
{
/// <summary>
/// A stack-only type that rents a buffer of a specified length from <seealso cref="ArrayPool{T}.Shared"/>.
/// It does not implement <see cref="IDisposable"/> to avoid being boxed, but should still be disposed. This
/// is easy since C# 8, which allows use of C# `using` constructs on any type that has a public Dispose() method.
/// To keep this type simple, fast, and read-only, it does not check or guard against multiple disposals.
/// For all these reasons, all usage should be with a `using` block or statement.
/// </summary>
/// <typeparam name="T">The type of item to store.</typeparam>
public readonly ref struct SpanOwner<T>
{
private readonly int _length;
private readonly T[] _array;
/// <summary>
/// Initializes a new instance of the <see cref="SpanOwner{T}"/> struct with the specified parameters.
/// </summary>
/// <param name="length">The length of the new memory buffer to use</param>
private SpanOwner(int length)
{
_length = length;
_array = ArrayPool<T>.Shared.Rent(length);
}
/// <summary>
/// Gets an empty <see cref="SpanOwner{T}"/> instance.
/// </summary>
public static SpanOwner<T> Empty
{
[MethodImpl(MethodImplOptions.AggressiveInlining)]
get => new(0);
}
/// <summary>
/// Creates a new <see cref="SpanOwner{T}"/> instance with the specified length.
/// </summary>
/// <param name="length">The length of the new memory buffer to use</param>
/// <returns>A <see cref="SpanOwner{T}"/> instance of the requested length</returns>
/// <exception cref="ArgumentOutOfRangeException">Thrown when <paramref name="length"/> is not valid</exception>
[MethodImpl(MethodImplOptions.AggressiveInlining)]
public static SpanOwner<T> Rent(int length) => new(length);
/// <summary>
/// Creates a new <see cref="SpanOwner{T}"/> instance with the length and the content cleared.
/// </summary>
/// <param name="length">The length of the new memory buffer to use</param>
/// <returns>A <see cref="SpanOwner{T}"/> instance of the requested length and the content cleared</returns>
/// <exception cref="ArgumentOutOfRangeException">Thrown when <paramref name="length"/> is not valid</exception>
[MethodImpl(MethodImplOptions.AggressiveInlining)]
public static SpanOwner<T> RentCleared(int length)
{
SpanOwner<T> result = new(length);
result._array.AsSpan(0, length).Clear();
return result;
}
/// <summary>
/// Creates a new <see cref="SpanOwner{T}"/> instance with the content copied from the specified buffer.
/// </summary>
/// <param name="buffer">The buffer to copy</param>
/// <returns>A <see cref="SpanOwner{T}"/> instance with the same length and content as <paramref name="buffer"/></returns>
public static SpanOwner<T> RentCopy(ReadOnlySpan<T> buffer)
{
SpanOwner<T> result = new(buffer.Length);
buffer.CopyTo(result._array);
return result;
}
/// <summary>
/// Gets the number of items in the current instance
/// </summary>
public int Length
{
[MethodImpl(MethodImplOptions.AggressiveInlining)]
get => _length;
}
/// <summary>
/// Gets a <see cref="Span{T}"/> wrapping the memory belonging to the current instance.
/// </summary>
/// <remarks>
/// Uses a trick made possible by the .NET 6+ runtime array layout.
/// </remarks>
public Span<T> Span
{
[MethodImpl(MethodImplOptions.AggressiveInlining)]
get
{
ref T firstElementRef = ref MemoryMarshal.GetArrayDataReference(_array);
return MemoryMarshal.CreateSpan(ref firstElementRef, _length);
}
}
/// <summary>
/// Implements the duck-typed <see cref="IDisposable.Dispose"/> method.
/// </summary>
[MethodImpl(MethodImplOptions.AggressiveInlining)]
public void Dispose()
{
ArrayPool<T>.Shared.Return(_array);
}
}
}

View File

@ -6,8 +6,13 @@ namespace Ryujinx.Graphics.GAL
public enum BufferAccess
{
Default = 0,
FlushPersistent = 1 << 0,
Stream = 1 << 1,
SparseCompatible = 1 << 2,
HostMemory = 1,
DeviceMemory = 2,
DeviceMemoryMapped = 3,
MemoryTypeMask = 0xf,
Stream = 1 << 4,
SparseCompatible = 1 << 5,
}
}

View File

@ -6,6 +6,7 @@ namespace Ryujinx.Graphics.GAL
{
public readonly TargetApi Api;
public readonly string VendorName;
public readonly SystemMemoryType MemoryType;
public readonly bool HasFrontFacingBug;
public readonly bool HasVectorIndexingBug;
@ -50,6 +51,13 @@ namespace Ryujinx.Graphics.GAL
public readonly bool SupportsIndirectParameters;
public readonly bool SupportsDepthClipControl;
public readonly int UniformBufferSetIndex;
public readonly int StorageBufferSetIndex;
public readonly int TextureSetIndex;
public readonly int ImageSetIndex;
public readonly int ExtraSetBaseIndex;
public readonly int MaximumExtraSets;
public readonly uint MaximumUniformBuffersPerStage;
public readonly uint MaximumStorageBuffersPerStage;
public readonly uint MaximumTexturesPerStage;
@ -66,6 +74,7 @@ namespace Ryujinx.Graphics.GAL
public Capabilities(
TargetApi api,
string vendorName,
SystemMemoryType memoryType,
bool hasFrontFacingBug,
bool hasVectorIndexingBug,
bool needsFragmentOutputSpecialization,
@ -107,6 +116,12 @@ namespace Ryujinx.Graphics.GAL
bool supportsViewportSwizzle,
bool supportsIndirectParameters,
bool supportsDepthClipControl,
int uniformBufferSetIndex,
int storageBufferSetIndex,
int textureSetIndex,
int imageSetIndex,
int extraSetBaseIndex,
int maximumExtraSets,
uint maximumUniformBuffersPerStage,
uint maximumStorageBuffersPerStage,
uint maximumTexturesPerStage,
@ -120,6 +135,7 @@ namespace Ryujinx.Graphics.GAL
{
Api = api;
VendorName = vendorName;
MemoryType = memoryType;
HasFrontFacingBug = hasFrontFacingBug;
HasVectorIndexingBug = hasVectorIndexingBug;
NeedsFragmentOutputSpecialization = needsFragmentOutputSpecialization;
@ -161,6 +177,12 @@ namespace Ryujinx.Graphics.GAL
SupportsViewportSwizzle = supportsViewportSwizzle;
SupportsIndirectParameters = supportsIndirectParameters;
SupportsDepthClipControl = supportsDepthClipControl;
UniformBufferSetIndex = uniformBufferSetIndex;
StorageBufferSetIndex = storageBufferSetIndex;
TextureSetIndex = textureSetIndex;
ImageSetIndex = imageSetIndex;
ExtraSetBaseIndex = extraSetBaseIndex;
MaximumExtraSets = maximumExtraSets;
MaximumUniformBuffersPerStage = maximumUniformBuffersPerStage;
MaximumStorageBuffersPerStage = maximumStorageBuffersPerStage;
MaximumTexturesPerStage = maximumTexturesPerStage;

View File

@ -711,5 +711,36 @@ namespace Ryujinx.Graphics.GAL
{
return format.IsUint() || format.IsSint();
}
/// <summary>
/// Checks if the texture format is a float or sRGB color format.
/// </summary>
/// <remarks>
/// Does not include normalized, compressed or depth formats.
/// Float and sRGB formats do not participate in logical operations.
/// </remarks>
/// <param name="format">Texture format</param>
/// <returns>True if the format is a float or sRGB color format, false otherwise</returns>
public static bool IsFloatOrSrgb(this Format format)
{
switch (format)
{
case Format.R8G8B8A8Srgb:
case Format.B8G8R8A8Srgb:
case Format.R16Float:
case Format.R16G16Float:
case Format.R16G16B16Float:
case Format.R16G16B16A16Float:
case Format.R32Float:
case Format.R32G32Float:
case Format.R32G32B32Float:
case Format.R32G32B32A32Float:
case Format.R11G11B10Float:
case Format.R9G9B9E5Float:
return true;
}
return false;
}
}
}

View File

@ -1,6 +1,8 @@
using System;
namespace Ryujinx.Graphics.GAL
{
public interface IImageArray
public interface IImageArray : IDisposable
{
void SetFormats(int index, Format[] imageFormats);
void SetImages(int index, ITexture[] images);

View File

@ -60,6 +60,7 @@ namespace Ryujinx.Graphics.GAL
void SetImage(ShaderStage stage, int binding, ITexture texture, Format imageFormat);
void SetImageArray(ShaderStage stage, int binding, IImageArray array);
void SetImageArraySeparate(ShaderStage stage, int setIndex, IImageArray array);
void SetLineParameters(float width, bool smooth);
@ -91,6 +92,7 @@ namespace Ryujinx.Graphics.GAL
void SetTextureAndSampler(ShaderStage stage, int binding, ITexture texture, ISampler sampler);
void SetTextureArray(ShaderStage stage, int binding, ITextureArray array);
void SetTextureArraySeparate(ShaderStage stage, int setIndex, ITextureArray array);
void SetTransformFeedbackBuffers(ReadOnlySpan<BufferRange> buffers);
void SetUniformBuffers(ReadOnlySpan<BufferAssignment> buffers);

View File

@ -17,7 +17,6 @@ namespace Ryujinx.Graphics.GAL
void BackgroundContextAction(Action action, bool alwaysBackground = false);
BufferHandle CreateBuffer(int size, BufferAccess access = BufferAccess.Default);
BufferHandle CreateBuffer(int size, BufferAccess access, BufferHandle storageHint);
BufferHandle CreateBuffer(nint pointer, int size);
BufferHandle CreateBufferSparse(ReadOnlySpan<BufferRange> storageBuffers);

View File

@ -1,6 +1,8 @@
using System;
namespace Ryujinx.Graphics.GAL
{
public interface ITextureArray
public interface ITextureArray : IDisposable
{
void SetSamplers(int index, ISampler[] samplers);
void SetTextures(int index, ITexture[] textures);

View File

@ -44,7 +44,6 @@ namespace Ryujinx.Graphics.GAL.Multithreading
}
Register<ActionCommand>(CommandType.Action);
Register<CreateBufferCommand>(CommandType.CreateBuffer);
Register<CreateBufferAccessCommand>(CommandType.CreateBufferAccess);
Register<CreateBufferSparseCommand>(CommandType.CreateBufferSparse);
Register<CreateHostBufferCommand>(CommandType.CreateHostBuffer);
@ -67,6 +66,7 @@ namespace Ryujinx.Graphics.GAL.Multithreading
Register<CounterEventDisposeCommand>(CommandType.CounterEventDispose);
Register<CounterEventFlushCommand>(CommandType.CounterEventFlush);
Register<ImageArrayDisposeCommand>(CommandType.ImageArrayDispose);
Register<ImageArraySetFormatsCommand>(CommandType.ImageArraySetFormats);
Register<ImageArraySetImagesCommand>(CommandType.ImageArraySetImages);
@ -89,6 +89,7 @@ namespace Ryujinx.Graphics.GAL.Multithreading
Register<TextureSetDataSliceRegionCommand>(CommandType.TextureSetDataSliceRegion);
Register<TextureSetStorageCommand>(CommandType.TextureSetStorage);
Register<TextureArrayDisposeCommand>(CommandType.TextureArrayDispose);
Register<TextureArraySetSamplersCommand>(CommandType.TextureArraySetSamplers);
Register<TextureArraySetTexturesCommand>(CommandType.TextureArraySetTextures);
@ -125,6 +126,7 @@ namespace Ryujinx.Graphics.GAL.Multithreading
Register<SetUniformBuffersCommand>(CommandType.SetUniformBuffers);
Register<SetImageCommand>(CommandType.SetImage);
Register<SetImageArrayCommand>(CommandType.SetImageArray);
Register<SetImageArraySeparateCommand>(CommandType.SetImageArraySeparate);
Register<SetIndexBufferCommand>(CommandType.SetIndexBuffer);
Register<SetLineParametersCommand>(CommandType.SetLineParameters);
Register<SetLogicOpStateCommand>(CommandType.SetLogicOpState);
@ -142,6 +144,7 @@ namespace Ryujinx.Graphics.GAL.Multithreading
Register<SetStencilTestCommand>(CommandType.SetStencilTest);
Register<SetTextureAndSamplerCommand>(CommandType.SetTextureAndSampler);
Register<SetTextureArrayCommand>(CommandType.SetTextureArray);
Register<SetTextureArraySeparateCommand>(CommandType.SetTextureArraySeparate);
Register<SetUserClipDistanceCommand>(CommandType.SetUserClipDistance);
Register<SetVertexAttribsCommand>(CommandType.SetVertexAttribs);
Register<SetVertexBuffersCommand>(CommandType.SetVertexBuffers);

View File

@ -3,7 +3,6 @@ namespace Ryujinx.Graphics.GAL.Multithreading
enum CommandType : byte
{
Action,
CreateBuffer,
CreateBufferAccess,
CreateBufferSparse,
CreateHostBuffer,
@ -27,6 +26,7 @@ namespace Ryujinx.Graphics.GAL.Multithreading
CounterEventDispose,
CounterEventFlush,
ImageArrayDispose,
ImageArraySetFormats,
ImageArraySetImages,
@ -49,6 +49,7 @@ namespace Ryujinx.Graphics.GAL.Multithreading
TextureSetDataSliceRegion,
TextureSetStorage,
TextureArrayDispose,
TextureArraySetSamplers,
TextureArraySetTextures,
@ -85,6 +86,7 @@ namespace Ryujinx.Graphics.GAL.Multithreading
SetUniformBuffers,
SetImage,
SetImageArray,
SetImageArraySeparate,
SetIndexBuffer,
SetLineParameters,
SetLogicOpState,
@ -102,6 +104,7 @@ namespace Ryujinx.Graphics.GAL.Multithreading
SetStencilTest,
SetTextureAndSampler,
SetTextureArray,
SetTextureArraySeparate,
SetUserClipDistance,
SetVertexAttribs,
SetVertexBuffers,

View File

@ -0,0 +1,21 @@
using Ryujinx.Graphics.GAL.Multithreading.Model;
using Ryujinx.Graphics.GAL.Multithreading.Resources;
namespace Ryujinx.Graphics.GAL.Multithreading.Commands.ImageArray
{
struct ImageArrayDisposeCommand : IGALCommand, IGALCommand<ImageArrayDisposeCommand>
{
public readonly CommandType CommandType => CommandType.ImageArrayDispose;
private TableRef<ThreadedImageArray> _imageArray;
public void Set(TableRef<ThreadedImageArray> imageArray)
{
_imageArray = imageArray;
}
public static void Run(ref ImageArrayDisposeCommand command, ThreadedRenderer threaded, IRenderer renderer)
{
command._imageArray.Get(threaded).Base.Dispose();
}
}
}

View File

@ -1,31 +0,0 @@
namespace Ryujinx.Graphics.GAL.Multithreading.Commands.Renderer
{
struct CreateBufferCommand : IGALCommand, IGALCommand<CreateBufferCommand>
{
public readonly CommandType CommandType => CommandType.CreateBuffer;
private BufferHandle _threadedHandle;
private int _size;
private BufferAccess _access;
private BufferHandle _storageHint;
public void Set(BufferHandle threadedHandle, int size, BufferAccess access, BufferHandle storageHint)
{
_threadedHandle = threadedHandle;
_size = size;
_access = access;
_storageHint = storageHint;
}
public static void Run(ref CreateBufferCommand command, ThreadedRenderer threaded, IRenderer renderer)
{
BufferHandle hint = BufferHandle.Null;
if (command._storageHint != BufferHandle.Null)
{
hint = threaded.Buffers.MapBuffer(command._storageHint);
}
threaded.Buffers.AssignBuffer(command._threadedHandle, renderer.CreateBuffer(command._size, command._access, hint));
}
}
}

View File

@ -0,0 +1,26 @@
using Ryujinx.Graphics.GAL.Multithreading.Model;
using Ryujinx.Graphics.GAL.Multithreading.Resources;
using Ryujinx.Graphics.Shader;
namespace Ryujinx.Graphics.GAL.Multithreading.Commands
{
struct SetImageArraySeparateCommand : IGALCommand, IGALCommand<SetImageArraySeparateCommand>
{
public readonly CommandType CommandType => CommandType.SetImageArraySeparate;
private ShaderStage _stage;
private int _setIndex;
private TableRef<IImageArray> _array;
public void Set(ShaderStage stage, int setIndex, TableRef<IImageArray> array)
{
_stage = stage;
_setIndex = setIndex;
_array = array;
}
public static void Run(ref SetImageArraySeparateCommand command, ThreadedRenderer threaded, IRenderer renderer)
{
renderer.Pipeline.SetImageArraySeparate(command._stage, command._setIndex, command._array.GetAs<ThreadedImageArray>(threaded)?.Base);
}
}
}

View File

@ -0,0 +1,26 @@
using Ryujinx.Graphics.GAL.Multithreading.Model;
using Ryujinx.Graphics.GAL.Multithreading.Resources;
using Ryujinx.Graphics.Shader;
namespace Ryujinx.Graphics.GAL.Multithreading.Commands
{
struct SetTextureArraySeparateCommand : IGALCommand, IGALCommand<SetTextureArraySeparateCommand>
{
public readonly CommandType CommandType => CommandType.SetTextureArraySeparate;
private ShaderStage _stage;
private int _setIndex;
private TableRef<ITextureArray> _array;
public void Set(ShaderStage stage, int setIndex, TableRef<ITextureArray> array)
{
_stage = stage;
_setIndex = setIndex;
_array = array;
}
public static void Run(ref SetTextureArraySeparateCommand command, ThreadedRenderer threaded, IRenderer renderer)
{
renderer.Pipeline.SetTextureArraySeparate(command._stage, command._setIndex, command._array.GetAs<ThreadedTextureArray>(threaded)?.Base);
}
}
}

View File

@ -0,0 +1,21 @@
using Ryujinx.Graphics.GAL.Multithreading.Model;
using Ryujinx.Graphics.GAL.Multithreading.Resources;
namespace Ryujinx.Graphics.GAL.Multithreading.Commands.TextureArray
{
struct TextureArrayDisposeCommand : IGALCommand, IGALCommand<TextureArrayDisposeCommand>
{
public readonly CommandType CommandType => CommandType.TextureArrayDispose;
private TableRef<ThreadedTextureArray> _textureArray;
public void Set(TableRef<ThreadedTextureArray> textureArray)
{
_textureArray = textureArray;
}
public static void Run(ref TextureArrayDisposeCommand command, ThreadedRenderer threaded, IRenderer renderer)
{
command._textureArray.Get(threaded).Base.Dispose();
}
}
}

View File

@ -21,6 +21,12 @@ namespace Ryujinx.Graphics.GAL.Multithreading.Resources
return new TableRef<T>(_renderer, reference);
}
public void Dispose()
{
_renderer.New<ImageArrayDisposeCommand>().Set(Ref(this));
_renderer.QueueCommand();
}
public void SetFormats(int index, Format[] imageFormats)
{
_renderer.New<ImageArraySetFormatsCommand>().Set(Ref(this), index, Ref(imageFormats));

View File

@ -22,6 +22,12 @@ namespace Ryujinx.Graphics.GAL.Multithreading.Resources
return new TableRef<T>(_renderer, reference);
}
public void Dispose()
{
_renderer.New<TextureArrayDisposeCommand>().Set(Ref(this));
_renderer.QueueCommand();
}
public void SetSamplers(int index, ISampler[] samplers)
{
_renderer.New<TextureArraySetSamplersCommand>().Set(Ref(this), index, Ref(samplers.ToArray()));

View File

@ -189,6 +189,12 @@ namespace Ryujinx.Graphics.GAL.Multithreading
_renderer.QueueCommand();
}
public void SetImageArraySeparate(ShaderStage stage, int setIndex, IImageArray array)
{
_renderer.New<SetImageArraySeparateCommand>().Set(stage, setIndex, Ref(array));
_renderer.QueueCommand();
}
public void SetIndexBuffer(BufferRange buffer, IndexType type)
{
_renderer.New<SetIndexBufferCommand>().Set(buffer, type);
@ -297,6 +303,12 @@ namespace Ryujinx.Graphics.GAL.Multithreading
_renderer.QueueCommand();
}
public void SetTextureArraySeparate(ShaderStage stage, int setIndex, ITextureArray array)
{
_renderer.New<SetTextureArraySeparateCommand>().Set(stage, setIndex, Ref(array));
_renderer.QueueCommand();
}
public void SetTransformFeedbackBuffers(ReadOnlySpan<BufferRange> buffers)
{
_renderer.New<SetTransformFeedbackBuffersCommand>().Set(_renderer.CopySpan(buffers));

View File

@ -272,15 +272,6 @@ namespace Ryujinx.Graphics.GAL.Multithreading
return handle;
}
public BufferHandle CreateBuffer(int size, BufferAccess access, BufferHandle storageHint)
{
BufferHandle handle = Buffers.CreateBufferHandle();
New<CreateBufferCommand>().Set(handle, size, access, storageHint);
QueueCommand();
return handle;
}
public BufferHandle CreateBuffer(nint pointer, int size)
{
BufferHandle handle = Buffers.CreateBufferHandle();

View File

@ -0,0 +1,29 @@
namespace Ryujinx.Graphics.GAL
{
public enum SystemMemoryType
{
/// <summary>
/// The backend manages the ownership of memory. This mode never supports host imported memory.
/// </summary>
BackendManaged,
/// <summary>
/// Device memory has similar performance to host memory, usually because it's shared between CPU/GPU.
/// Use host memory whenever possible.
/// </summary>
UnifiedMemory,
/// <summary>
/// GPU storage to host memory goes though a slow interconnect, but it would still be preferable to use it if the data is flushed back often.
/// Assumes constant buffer access to host memory is rather fast.
/// </summary>
DedicatedMemory,
/// <summary>
/// GPU storage to host memory goes though a slow interconnect, that is very slow when doing access from storage.
/// When frequently accessed, copy buffers to host memory using DMA.
/// Assumes constant buffer access to host memory is rather fast.
/// </summary>
DedicatedMemorySlowStorage
}
}

View File

@ -5,6 +5,7 @@ using Ryujinx.Graphics.GAL;
using Ryujinx.Graphics.Gpu.Engine.GPFifo;
using Ryujinx.Graphics.Gpu.Engine.Threed;
using Ryujinx.Graphics.Gpu.Engine.Types;
using Ryujinx.Graphics.Gpu.Memory;
using Ryujinx.Memory.Range;
using System;
using System.Collections.Generic;
@ -495,8 +496,8 @@ namespace Ryujinx.Graphics.Gpu.Engine.MME
ulong indirectBufferSize = (ulong)maxDrawCount * (ulong)stride;
MultiRange indirectBufferRange = bufferCache.TranslateAndCreateMultiBuffers(_processor.MemoryManager, indirectBufferGpuVa, indirectBufferSize);
MultiRange parameterBufferRange = bufferCache.TranslateAndCreateMultiBuffers(_processor.MemoryManager, parameterBufferGpuVa, 4);
MultiRange indirectBufferRange = bufferCache.TranslateAndCreateMultiBuffers(_processor.MemoryManager, indirectBufferGpuVa, indirectBufferSize, BufferStage.Indirect);
MultiRange parameterBufferRange = bufferCache.TranslateAndCreateMultiBuffers(_processor.MemoryManager, parameterBufferGpuVa, 4, BufferStage.Indirect);
_processor.ThreedClass.DrawIndirect(
topology,

View File

@ -438,7 +438,7 @@ namespace Ryujinx.Graphics.Gpu.Engine.Threed.ComputeDraw
ReadOnlySpan<byte> dataBytes = MemoryMarshal.Cast<int, byte>(data);
BufferHandle buffer = _context.Renderer.CreateBuffer(dataBytes.Length);
BufferHandle buffer = _context.Renderer.CreateBuffer(dataBytes.Length, BufferAccess.DeviceMemory);
_context.Renderer.SetBufferData(buffer, 0, dataBytes);
return new IndexBuffer(buffer, count, dataBytes.Length);
@ -529,7 +529,7 @@ namespace Ryujinx.Graphics.Gpu.Engine.Threed.ComputeDraw
{
if (_dummyBuffer == BufferHandle.Null)
{
_dummyBuffer = _context.Renderer.CreateBuffer(DummyBufferSize);
_dummyBuffer = _context.Renderer.CreateBuffer(DummyBufferSize, BufferAccess.DeviceMemory);
_context.Renderer.Pipeline.ClearBuffer(_dummyBuffer, 0, DummyBufferSize, 0);
}
@ -550,7 +550,7 @@ namespace Ryujinx.Graphics.Gpu.Engine.Threed.ComputeDraw
_context.Renderer.DeleteBuffer(_sequentialIndexBuffer);
}
_sequentialIndexBuffer = _context.Renderer.CreateBuffer(count * sizeof(uint));
_sequentialIndexBuffer = _context.Renderer.CreateBuffer(count * sizeof(uint), BufferAccess.DeviceMemory);
_sequentialIndexBufferCount = count;
Span<int> data = new int[count];
@ -583,7 +583,7 @@ namespace Ryujinx.Graphics.Gpu.Engine.Threed.ComputeDraw
_context.Renderer.DeleteBuffer(buffer.Handle);
}
buffer.Handle = _context.Renderer.CreateBuffer(newSize);
buffer.Handle = _context.Renderer.CreateBuffer(newSize, BufferAccess.DeviceMemory);
buffer.Size = newSize;
}

View File

@ -3,6 +3,7 @@ using Ryujinx.Common.Logging;
using Ryujinx.Graphics.GAL;
using Ryujinx.Graphics.Gpu.Engine.Types;
using Ryujinx.Graphics.Gpu.Image;
using Ryujinx.Graphics.Gpu.Memory;
using Ryujinx.Graphics.Gpu.Shader;
using Ryujinx.Graphics.Shader;
using Ryujinx.Graphics.Shader.Translation;
@ -370,7 +371,7 @@ namespace Ryujinx.Graphics.Gpu.Engine.Threed.ComputeDraw
{
var memoryManager = _channel.MemoryManager;
BufferRange range = memoryManager.Physical.BufferCache.GetBufferRange(memoryManager.GetPhysicalRegions(address, size));
BufferRange range = memoryManager.Physical.BufferCache.GetBufferRange(memoryManager.GetPhysicalRegions(address, size), BufferStage.VertexBuffer);
ITexture bufferTexture = _vacContext.EnsureBufferTexture(index + 2, format);
bufferTexture.SetStorage(range);
@ -412,7 +413,9 @@ namespace Ryujinx.Graphics.Gpu.Engine.Threed.ComputeDraw
var memoryManager = _channel.MemoryManager;
ulong misalign = address & ((ulong)_context.Capabilities.TextureBufferOffsetAlignment - 1);
BufferRange range = memoryManager.Physical.BufferCache.GetBufferRange(memoryManager.GetPhysicalRegions(address + indexOffset - misalign, size + misalign));
BufferRange range = memoryManager.Physical.BufferCache.GetBufferRange(
memoryManager.GetPhysicalRegions(address + indexOffset - misalign, size + misalign),
BufferStage.IndexBuffer);
misalignedOffset = (int)misalign >> shift;
SetIndexBufferTexture(reservations, range, format);

View File

@ -684,8 +684,8 @@ namespace Ryujinx.Graphics.Gpu.Engine.Threed
if (hasCount)
{
var indirectBuffer = memory.BufferCache.GetBufferRange(indirectBufferRange);
var parameterBuffer = memory.BufferCache.GetBufferRange(parameterBufferRange);
var indirectBuffer = memory.BufferCache.GetBufferRange(indirectBufferRange, BufferStage.Indirect);
var parameterBuffer = memory.BufferCache.GetBufferRange(parameterBufferRange, BufferStage.Indirect);
if (indexed)
{
@ -698,7 +698,7 @@ namespace Ryujinx.Graphics.Gpu.Engine.Threed
}
else
{
var indirectBuffer = memory.BufferCache.GetBufferRange(indirectBufferRange);
var indirectBuffer = memory.BufferCache.GetBufferRange(indirectBufferRange, BufferStage.Indirect);
if (indexed)
{

View File

@ -393,17 +393,18 @@ namespace Ryujinx.Graphics.Gpu
if (force || _pendingSync || (syncpoint && SyncpointActions.Count > 0))
{
Renderer.CreateSync(SyncNumber, strict);
foreach (var action in SyncActions)
{
action.SyncPreAction(syncpoint);
}
foreach (var action in SyncpointActions)
{
action.SyncPreAction(syncpoint);
}
Renderer.CreateSync(SyncNumber, strict);
SyncNumber++;
SyncActions.RemoveAll(action => action.SyncAction(syncpoint));

View File

@ -19,6 +19,11 @@ namespace Ryujinx.Graphics.Gpu.Image
/// </summary>
public Format Format { get; }
/// <summary>
/// Shader texture host set index.
/// </summary>
public int Set { get; }
/// <summary>
/// Shader texture host binding point.
/// </summary>
@ -54,15 +59,17 @@ namespace Ryujinx.Graphics.Gpu.Image
/// </summary>
/// <param name="target">The shader sampler target type</param>
/// <param name="format">Format of the image as declared on the shader</param>
/// <param name="set">Shader texture host set index</param>
/// <param name="binding">The shader texture binding point</param>
/// <param name="arrayLength">For array of textures, this indicates the length of the array. A value of one indicates it is not an array</param>
/// <param name="cbufSlot">Constant buffer slot where the texture handle is located</param>
/// <param name="handle">The shader texture handle (read index into the texture constant buffer)</param>
/// <param name="flags">The texture's usage flags, indicating how it is used in the shader</param>
public TextureBindingInfo(Target target, Format format, int binding, int arrayLength, int cbufSlot, int handle, TextureUsageFlags flags)
public TextureBindingInfo(Target target, Format format, int set, int binding, int arrayLength, int cbufSlot, int handle, TextureUsageFlags flags)
{
Target = target;
Format = format;
Set = set;
Binding = binding;
ArrayLength = arrayLength;
CbufSlot = cbufSlot;
@ -74,6 +81,7 @@ namespace Ryujinx.Graphics.Gpu.Image
/// Constructs the texture binding information structure.
/// </summary>
/// <param name="target">The shader sampler target type</param>
/// <param name="set">Shader texture host set index</param>
/// <param name="binding">The shader texture binding point</param>
/// <param name="arrayLength">For array of textures, this indicates the length of the array. A value of one indicates it is not an array</param>
/// <param name="cbufSlot">Constant buffer slot where the texture handle is located</param>
@ -82,12 +90,13 @@ namespace Ryujinx.Graphics.Gpu.Image
/// <param name="isSamplerOnly">Indicates that the binding is for a sampler</param>
public TextureBindingInfo(
Target target,
int set,
int binding,
int arrayLength,
int cbufSlot,
int handle,
TextureUsageFlags flags,
bool isSamplerOnly) : this(target, 0, binding, arrayLength, cbufSlot, handle, flags)
bool isSamplerOnly) : this(target, 0, set, binding, arrayLength, cbufSlot, handle, flags)
{
IsSamplerOnly = isSamplerOnly;
}

View File

@ -566,7 +566,7 @@ namespace Ryujinx.Graphics.Gpu.Image
int stageIndex,
int textureBufferIndex,
SamplerIndex samplerIndex,
TextureBindingInfo bindingInfo)
in TextureBindingInfo bindingInfo)
{
Update(texturePool, samplerPool, stage, stageIndex, textureBufferIndex, isImage: false, samplerIndex, bindingInfo);
}
@ -579,7 +579,7 @@ namespace Ryujinx.Graphics.Gpu.Image
/// <param name="stageIndex">Shader stage index where the array is used</param>
/// <param name="textureBufferIndex">Texture constant buffer index</param>
/// <param name="bindingInfo">Array binding information</param>
public void UpdateImageArray(TexturePool texturePool, ShaderStage stage, int stageIndex, int textureBufferIndex, TextureBindingInfo bindingInfo)
public void UpdateImageArray(TexturePool texturePool, ShaderStage stage, int stageIndex, int textureBufferIndex, in TextureBindingInfo bindingInfo)
{
Update(texturePool, null, stage, stageIndex, textureBufferIndex, isImage: true, SamplerIndex.ViaHeaderIndex, bindingInfo);
}
@ -603,7 +603,7 @@ namespace Ryujinx.Graphics.Gpu.Image
int textureBufferIndex,
bool isImage,
SamplerIndex samplerIndex,
TextureBindingInfo bindingInfo)
in TextureBindingInfo bindingInfo)
{
if (IsDirectHandleType(bindingInfo.Handle))
{
@ -623,7 +623,7 @@ namespace Ryujinx.Graphics.Gpu.Image
/// <param name="stage">Shader stage where the array is used</param>
/// <param name="isImage">Whether the array is a image or texture array</param>
/// <param name="bindingInfo">Array binding information</param>
private void UpdateFromPool(TexturePool texturePool, SamplerPool samplerPool, ShaderStage stage, bool isImage, TextureBindingInfo bindingInfo)
private void UpdateFromPool(TexturePool texturePool, SamplerPool samplerPool, ShaderStage stage, bool isImage, in TextureBindingInfo bindingInfo)
{
CacheEntry entry = GetOrAddEntry(texturePool, samplerPool, bindingInfo, isImage, out bool isNewEntry);
@ -638,11 +638,11 @@ namespace Ryujinx.Graphics.Gpu.Image
if (isImage)
{
_context.Renderer.Pipeline.SetImageArray(stage, bindingInfo.Binding, entry.ImageArray);
SetImageArray(stage, bindingInfo, entry.ImageArray);
}
else
{
_context.Renderer.Pipeline.SetTextureArray(stage, bindingInfo.Binding, entry.TextureArray);
SetTextureArray(stage, bindingInfo, entry.TextureArray);
}
return;
@ -708,11 +708,11 @@ namespace Ryujinx.Graphics.Gpu.Image
format = texture.Format;
}
_channel.BufferManager.SetBufferTextureStorage(entry.ImageArray, hostTexture, texture.Range, bindingInfo, index, format);
_channel.BufferManager.SetBufferTextureStorage(stage, entry.ImageArray, hostTexture, texture.Range, bindingInfo, index, format);
}
else
{
_channel.BufferManager.SetBufferTextureStorage(entry.TextureArray, hostTexture, texture.Range, bindingInfo, index, format);
_channel.BufferManager.SetBufferTextureStorage(stage, entry.TextureArray, hostTexture, texture.Range, bindingInfo, index, format);
}
}
else if (isImage)
@ -737,14 +737,14 @@ namespace Ryujinx.Graphics.Gpu.Image
entry.ImageArray.SetFormats(0, formats);
entry.ImageArray.SetImages(0, textures);
_context.Renderer.Pipeline.SetImageArray(stage, bindingInfo.Binding, entry.ImageArray);
SetImageArray(stage, bindingInfo, entry.ImageArray);
}
else
{
entry.TextureArray.SetSamplers(0, samplers);
entry.TextureArray.SetTextures(0, textures);
_context.Renderer.Pipeline.SetTextureArray(stage, bindingInfo.Binding, entry.TextureArray);
SetTextureArray(stage, bindingInfo, entry.TextureArray);
}
}
@ -767,7 +767,7 @@ namespace Ryujinx.Graphics.Gpu.Image
int textureBufferIndex,
bool isImage,
SamplerIndex samplerIndex,
TextureBindingInfo bindingInfo)
in TextureBindingInfo bindingInfo)
{
(textureBufferIndex, int samplerBufferIndex) = TextureHandle.UnpackSlots(bindingInfo.CbufSlot, textureBufferIndex);
@ -800,11 +800,11 @@ namespace Ryujinx.Graphics.Gpu.Image
if (isImage)
{
_context.Renderer.Pipeline.SetImageArray(stage, bindingInfo.Binding, entry.ImageArray);
SetImageArray(stage, bindingInfo, entry.ImageArray);
}
else
{
_context.Renderer.Pipeline.SetTextureArray(stage, bindingInfo.Binding, entry.TextureArray);
SetTextureArray(stage, bindingInfo, entry.TextureArray);
}
return;
@ -829,11 +829,11 @@ namespace Ryujinx.Graphics.Gpu.Image
if (isImage)
{
_context.Renderer.Pipeline.SetImageArray(stage, bindingInfo.Binding, entry.ImageArray);
SetImageArray(stage, bindingInfo, entry.ImageArray);
}
else
{
_context.Renderer.Pipeline.SetTextureArray(stage, bindingInfo.Binding, entry.TextureArray);
SetTextureArray(stage, bindingInfo, entry.TextureArray);
}
return;
@ -921,11 +921,11 @@ namespace Ryujinx.Graphics.Gpu.Image
format = texture.Format;
}
_channel.BufferManager.SetBufferTextureStorage(entry.ImageArray, hostTexture, texture.Range, bindingInfo, index, format);
_channel.BufferManager.SetBufferTextureStorage(stage, entry.ImageArray, hostTexture, texture.Range, bindingInfo, index, format);
}
else
{
_channel.BufferManager.SetBufferTextureStorage(entry.TextureArray, hostTexture, texture.Range, bindingInfo, index, format);
_channel.BufferManager.SetBufferTextureStorage(stage, entry.TextureArray, hostTexture, texture.Range, bindingInfo, index, format);
}
}
else if (isImage)
@ -950,14 +950,50 @@ namespace Ryujinx.Graphics.Gpu.Image
entry.ImageArray.SetFormats(0, formats);
entry.ImageArray.SetImages(0, textures);
_context.Renderer.Pipeline.SetImageArray(stage, bindingInfo.Binding, entry.ImageArray);
SetImageArray(stage, bindingInfo, entry.ImageArray);
}
else
{
entry.TextureArray.SetSamplers(0, samplers);
entry.TextureArray.SetTextures(0, textures);
_context.Renderer.Pipeline.SetTextureArray(stage, bindingInfo.Binding, entry.TextureArray);
SetTextureArray(stage, bindingInfo, entry.TextureArray);
}
}
/// <summary>
/// Updates a texture array binding on the host.
/// </summary>
/// <param name="stage">Shader stage where the array is used</param>
/// <param name="bindingInfo">Array binding information</param>
/// <param name="array">Texture array</param>
private void SetTextureArray(ShaderStage stage, in TextureBindingInfo bindingInfo, ITextureArray array)
{
if (bindingInfo.Set >= _context.Capabilities.ExtraSetBaseIndex && _context.Capabilities.MaximumExtraSets != 0)
{
_context.Renderer.Pipeline.SetTextureArraySeparate(stage, bindingInfo.Set, array);
}
else
{
_context.Renderer.Pipeline.SetTextureArray(stage, bindingInfo.Binding, array);
}
}
/// <summary>
/// Updates a image array binding on the host.
/// </summary>
/// <param name="stage">Shader stage where the array is used</param>
/// <param name="bindingInfo">Array binding information</param>
/// <param name="array">Image array</param>
private void SetImageArray(ShaderStage stage, in TextureBindingInfo bindingInfo, IImageArray array)
{
if (bindingInfo.Set >= _context.Capabilities.ExtraSetBaseIndex && _context.Capabilities.MaximumExtraSets != 0)
{
_context.Renderer.Pipeline.SetImageArraySeparate(stage, bindingInfo.Set, array);
}
else
{
_context.Renderer.Pipeline.SetImageArray(stage, bindingInfo.Binding, array);
}
}
@ -973,7 +1009,7 @@ namespace Ryujinx.Graphics.Gpu.Image
private CacheEntry GetOrAddEntry(
TexturePool texturePool,
SamplerPool samplerPool,
TextureBindingInfo bindingInfo,
in TextureBindingInfo bindingInfo,
bool isImage,
out bool isNew)
{
@ -1015,7 +1051,7 @@ namespace Ryujinx.Graphics.Gpu.Image
private CacheEntryFromBuffer GetOrAddEntry(
TexturePool texturePool,
SamplerPool samplerPool,
TextureBindingInfo bindingInfo,
in TextureBindingInfo bindingInfo,
bool isImage,
ref BufferBounds textureBufferBounds,
out bool isNew)
@ -1077,6 +1113,15 @@ namespace Ryujinx.Graphics.Gpu.Image
nextNode = nextNode.Next;
_cacheFromBuffer.Remove(toRemove.Value.Key);
_lruCache.Remove(toRemove);
if (toRemove.Value.Key.IsImage)
{
toRemove.Value.ImageArray.Dispose();
}
else
{
toRemove.Value.TextureArray.Dispose();
}
}
}
@ -1088,11 +1133,20 @@ namespace Ryujinx.Graphics.Gpu.Image
{
List<CacheEntryFromPoolKey> keysToRemove = null;
foreach (CacheEntryFromPoolKey key in _cacheFromPool.Keys)
foreach ((CacheEntryFromPoolKey key, CacheEntry entry) in _cacheFromPool)
{
if (key.MatchesPool(pool))
{
(keysToRemove ??= new()).Add(key);
if (key.IsImage)
{
entry.ImageArray.Dispose();
}
else
{
entry.TextureArray.Dispose();
}
}
}

View File

@ -645,7 +645,7 @@ namespace Ryujinx.Graphics.Gpu.Image
}
else
{
_flushBuffer = _context.Renderer.CreateBuffer((int)Storage.Size, BufferAccess.FlushPersistent);
_flushBuffer = _context.Renderer.CreateBuffer((int)Storage.Size, BufferAccess.HostMemory);
_flushBufferImported = false;
}

View File

@ -10,6 +10,8 @@ using System.Threading;
namespace Ryujinx.Graphics.Gpu.Memory
{
delegate void BufferFlushAction(ulong address, ulong size, ulong syncNumber);
/// <summary>
/// Buffer, used to store vertex and index data, uniform and storage buffers, and others.
/// </summary>
@ -23,7 +25,7 @@ namespace Ryujinx.Graphics.Gpu.Memory
/// <summary>
/// Host buffer handle.
/// </summary>
public BufferHandle Handle { get; }
public BufferHandle Handle { get; private set; }
/// <summary>
/// Start address of the buffer in guest memory.
@ -60,6 +62,17 @@ namespace Ryujinx.Graphics.Gpu.Memory
/// </remarks>
private BufferModifiedRangeList _modifiedRanges = null;
/// <summary>
/// A structure that is used to flush buffer data back to a host mapped buffer for cached readback.
/// Only used if the buffer data is explicitly owned by device local memory.
/// </summary>
private BufferPreFlush _preFlush = null;
/// <summary>
/// Usage tracking state that determines what type of backing the buffer should use.
/// </summary>
public BufferBackingState BackingState;
private readonly MultiRegionHandle _memoryTrackingGranular;
private readonly RegionHandle _memoryTracking;
@ -87,6 +100,7 @@ namespace Ryujinx.Graphics.Gpu.Memory
/// <param name="physicalMemory">Physical memory where the buffer is mapped</param>
/// <param name="address">Start address of the buffer</param>
/// <param name="size">Size of the buffer in bytes</param>
/// <param name="stage">The type of usage that created the buffer</param>
/// <param name="sparseCompatible">Indicates if the buffer can be used in a sparse buffer mapping</param>
/// <param name="baseBuffers">Buffers which this buffer contains, and will inherit tracking handles from</param>
public Buffer(
@ -94,6 +108,7 @@ namespace Ryujinx.Graphics.Gpu.Memory
PhysicalMemory physicalMemory,
ulong address,
ulong size,
BufferStage stage,
bool sparseCompatible,
IEnumerable<Buffer> baseBuffers = null)
{
@ -103,9 +118,11 @@ namespace Ryujinx.Graphics.Gpu.Memory
Size = size;
SparseCompatible = sparseCompatible;
BufferAccess access = sparseCompatible ? BufferAccess.SparseCompatible : BufferAccess.Default;
BackingState = new BufferBackingState(_context, this, stage, baseBuffers);
Handle = context.Renderer.CreateBuffer((int)size, access, baseBuffers?.MaxBy(x => x.Size).Handle ?? BufferHandle.Null);
BufferAccess access = BackingState.SwitchAccess(this);
Handle = context.Renderer.CreateBuffer((int)size, access);
_useGranular = size > GranularBufferThreshold;
@ -161,6 +178,29 @@ namespace Ryujinx.Graphics.Gpu.Memory
_virtualDependenciesLock = new ReaderWriterLockSlim();
}
/// <summary>
/// Recreates the backing buffer based on the desired access type
/// reported by the backing state struct.
/// </summary>
private void ChangeBacking()
{
BufferAccess access = BackingState.SwitchAccess(this);
BufferHandle newHandle = _context.Renderer.CreateBuffer((int)Size, access);
_context.Renderer.Pipeline.CopyBuffer(Handle, newHandle, 0, 0, (int)Size);
_modifiedRanges?.SelfMigration();
// If swtiching from device local to host mapped, pre-flushing data no longer makes sense.
// This is set to null and disposed when the migration fully completes.
_preFlush = null;
Handle = newHandle;
_physicalMemory.BufferCache.BufferBackingChanged(this);
}
/// <summary>
/// Gets a sub-range from the buffer, from a start address til a page boundary after the given size.
/// </summary>
@ -246,6 +286,7 @@ namespace Ryujinx.Graphics.Gpu.Memory
}
else
{
BackingState.RecordSet();
_context.Renderer.SetBufferData(Handle, 0, _physicalMemory.GetSpan(Address, (int)Size));
CopyToDependantVirtualBuffers();
}
@ -283,15 +324,35 @@ namespace Ryujinx.Graphics.Gpu.Memory
_modifiedRanges ??= new BufferModifiedRangeList(_context, this, Flush);
}
/// <summary>
/// Checks if a backing change is deemed necessary from the given usage.
/// If it is, queues a backing change to happen on the next sync action.
/// </summary>
/// <param name="stage">Buffer stage that can change backing type</param>
private void TryQueueBackingChange(BufferStage stage)
{
if (BackingState.ShouldChangeBacking(stage))
{
if (!_syncActionRegistered)
{
_context.RegisterSyncAction(this);
_syncActionRegistered = true;
}
}
}
/// <summary>
/// Signal that the given region of the buffer has been modified.
/// </summary>
/// <param name="address">The start address of the modified region</param>
/// <param name="size">The size of the modified region</param>
public void SignalModified(ulong address, ulong size)
/// <param name="stage">Buffer stage that triggered the modification</param>
public void SignalModified(ulong address, ulong size, BufferStage stage)
{
EnsureRangeList();
TryQueueBackingChange(stage);
_modifiedRanges.SignalModified(address, size);
if (!_syncActionRegistered)
@ -311,6 +372,37 @@ namespace Ryujinx.Graphics.Gpu.Memory
_modifiedRanges?.Clear(address, size);
}
/// <summary>
/// Action to be performed immediately before sync is created.
/// This will copy any buffer ranges designated for pre-flushing.
/// </summary>
/// <param name="syncpoint">True if the action is a guest syncpoint</param>
public void SyncPreAction(bool syncpoint)
{
if (_referenceCount == 0)
{
return;
}
if (BackingState.ShouldChangeBacking())
{
ChangeBacking();
}
if (BackingState.IsDeviceLocal)
{
_preFlush ??= new BufferPreFlush(_context, this, FlushImpl);
if (_preFlush.ShouldCopy)
{
_modifiedRanges?.GetRangesAtSync(Address, Size, _context.SyncNumber, (address, size) =>
{
_preFlush.CopyModified(address, size);
});
}
}
}
/// <summary>
/// Action to be performed when a syncpoint is reached after modification.
/// This will register read/write tracking to flush the buffer from GPU when its memory is used.
@ -466,6 +558,8 @@ namespace Ryujinx.Graphics.Gpu.Memory
/// <param name="mSize">Size of the modified region</param>
private void LoadRegion(ulong mAddress, ulong mSize)
{
BackingState.RecordSet();
int offset = (int)(mAddress - Address);
_context.Renderer.SetBufferData(Handle, offset, _physicalMemory.GetSpan(mAddress, (int)mSize));
@ -539,18 +633,84 @@ namespace Ryujinx.Graphics.Gpu.Memory
/// Flushes a range of the buffer.
/// This writes the range data back into guest memory.
/// </summary>
/// <param name="handle">Buffer handle to flush data from</param>
/// <param name="address">Start address of the range</param>
/// <param name="size">Size in bytes of the range</param>
public void Flush(ulong address, ulong size)
private void FlushImpl(BufferHandle handle, ulong address, ulong size)
{
int offset = (int)(address - Address);
using PinnedSpan<byte> data = _context.Renderer.GetBufferData(Handle, offset, (int)size);
using PinnedSpan<byte> data = _context.Renderer.GetBufferData(handle, offset, (int)size);
// TODO: When write tracking shaders, they will need to be aware of changes in overlapping buffers.
_physicalMemory.WriteUntracked(address, CopyFromDependantVirtualBuffers(data.Get(), address, size));
}
/// <summary>
/// Flushes a range of the buffer.
/// This writes the range data back into guest memory.
/// </summary>
/// <param name="address">Start address of the range</param>
/// <param name="size">Size in bytes of the range</param>
private void FlushImpl(ulong address, ulong size)
{
FlushImpl(Handle, address, size);
}
/// <summary>
/// Flushes a range of the buffer from the most optimal source.
/// This writes the range data back into guest memory.
/// </summary>
/// <param name="address">Start address of the range</param>
/// <param name="size">Size in bytes of the range</param>
/// <param name="syncNumber">Sync number waited for before flushing the data</param>
public void Flush(ulong address, ulong size, ulong syncNumber)
{
BackingState.RecordFlush();
BufferPreFlush preFlush = _preFlush;
if (preFlush != null)
{
preFlush.FlushWithAction(address, size, syncNumber);
}
else
{
FlushImpl(address, size);
}
}
/// <summary>
/// Gets an action that disposes the backing buffer using its current handle.
/// Useful for deleting an old copy of the buffer after the handle changes.
/// </summary>
/// <returns>An action that flushes data from the specified range, using the buffer handle at the time the method is generated</returns>
public Action GetSnapshotDisposeAction()
{
BufferHandle handle = Handle;
BufferPreFlush preFlush = _preFlush;
return () =>
{
_context.Renderer.DeleteBuffer(handle);
preFlush?.Dispose();
};
}
/// <summary>
/// Gets an action that flushes a range of the buffer using its current handle.
/// Useful for flushing data from old copies of the buffer after the handle changes.
/// </summary>
/// <returns>An action that flushes data from the specified range, using the buffer handle at the time the method is generated</returns>
public BufferFlushAction GetSnapshotFlushAction()
{
BufferHandle handle = Handle;
return (ulong address, ulong size, ulong _) =>
{
FlushImpl(handle, address, size);
};
}
/// <summary>
/// Align a given address and size region to page boundaries.
/// </summary>
@ -857,6 +1017,8 @@ namespace Ryujinx.Graphics.Gpu.Memory
_modifiedRanges?.Clear();
_context.Renderer.DeleteBuffer(Handle);
_preFlush?.Dispose();
_preFlush = null;
UnmappedSequence++;
}

View File

@ -0,0 +1,294 @@
using Ryujinx.Graphics.GAL;
using System;
using System.Collections.Generic;
namespace Ryujinx.Graphics.Gpu.Memory
{
/// <summary>
/// Type of backing memory.
/// In ascending order of priority when merging multiple buffer backing states.
/// </summary>
internal enum BufferBackingType
{
HostMemory,
DeviceMemory,
DeviceMemoryWithFlush
}
/// <summary>
/// Keeps track of buffer usage to decide what memory heap that buffer memory is placed on.
/// Dedicated GPUs prefer certain types of resources to be device local,
/// and if we need data to be read back, we might prefer that they're in host memory.
///
/// The measurements recorded here compare to a set of heruristics (thresholds and conditions)
/// that appear to produce good performance in most software.
/// </summary>
internal struct BufferBackingState
{
private const int DeviceLocalSizeThreshold = 256 * 1024; // 256kb
private const int SetCountThreshold = 100;
private const int WriteCountThreshold = 50;
private const int FlushCountThreshold = 5;
private const int DeviceLocalForceExpiry = 100;
public readonly bool IsDeviceLocal => _activeType != BufferBackingType.HostMemory;
private readonly SystemMemoryType _systemMemoryType;
private BufferBackingType _activeType;
private BufferBackingType _desiredType;
private bool _canSwap;
private int _setCount;
private int _writeCount;
private int _flushCount;
private int _flushTemp;
private int _lastFlushWrite;
private int _deviceLocalForceCount;
private readonly int _size;
/// <summary>
/// Initialize the buffer backing state for a given parent buffer.
/// </summary>
/// <param name="context">GPU context</param>
/// <param name="parent">Parent buffer</param>
/// <param name="stage">Initial buffer stage</param>
/// <param name="baseBuffers">Buffers to inherit state from</param>
public BufferBackingState(GpuContext context, Buffer parent, BufferStage stage, IEnumerable<Buffer> baseBuffers = null)
{
_size = (int)parent.Size;
_systemMemoryType = context.Capabilities.MemoryType;
// Backend managed is always auto, unified memory is always host.
_desiredType = BufferBackingType.HostMemory;
_canSwap = _systemMemoryType != SystemMemoryType.BackendManaged && _systemMemoryType != SystemMemoryType.UnifiedMemory;
if (_canSwap)
{
// Might want to start certain buffers as being device local,
// and the usage might also lock those buffers into being device local.
BufferStage storageFlags = stage & BufferStage.StorageMask;
if (parent.Size > DeviceLocalSizeThreshold && baseBuffers == null)
{
_desiredType = BufferBackingType.DeviceMemory;
}
if (storageFlags != 0)
{
// Storage buffer bindings may require special treatment.
var rawStage = stage & BufferStage.StageMask;
if (rawStage == BufferStage.Fragment)
{
// Fragment read should start device local.
_desiredType = BufferBackingType.DeviceMemory;
if (storageFlags != BufferStage.StorageRead)
{
// Fragment write should stay device local until the use doesn't happen anymore.
_deviceLocalForceCount = DeviceLocalForceExpiry;
}
}
// TODO: Might be nice to force atomic access to be device local for any stage.
}
if (baseBuffers != null)
{
foreach (Buffer buffer in baseBuffers)
{
CombineState(buffer.BackingState);
}
}
}
}
/// <summary>
/// Combine buffer backing types, selecting the one with highest priority.
/// </summary>
/// <param name="left">First buffer backing type</param>
/// <param name="right">Second buffer backing type</param>
/// <returns>Combined buffer backing type</returns>
private static BufferBackingType CombineTypes(BufferBackingType left, BufferBackingType right)
{
return (BufferBackingType)Math.Max((int)left, (int)right);
}
/// <summary>
/// Combine the state from the given buffer backing state with this one,
/// so that the state isn't lost when migrating buffers.
/// </summary>
/// <param name="oldState">Buffer state to combine into this state</param>
private void CombineState(BufferBackingState oldState)
{
_setCount += oldState._setCount;
_writeCount += oldState._writeCount;
_flushCount += oldState._flushCount;
_flushTemp += oldState._flushTemp;
_lastFlushWrite = -1;
_deviceLocalForceCount = Math.Max(_deviceLocalForceCount, oldState._deviceLocalForceCount);
_canSwap &= oldState._canSwap;
_desiredType = CombineTypes(_desiredType, oldState._desiredType);
}
/// <summary>
/// Get the buffer access for the desired backing type, and record that type as now being active.
/// </summary>
/// <param name="parent">Parent buffer</param>
/// <returns>Buffer access</returns>
public BufferAccess SwitchAccess(Buffer parent)
{
BufferAccess access = parent.SparseCompatible ? BufferAccess.SparseCompatible : BufferAccess.Default;
bool isBackendManaged = _systemMemoryType == SystemMemoryType.BackendManaged;
if (!isBackendManaged)
{
switch (_desiredType)
{
case BufferBackingType.HostMemory:
access |= BufferAccess.HostMemory;
break;
case BufferBackingType.DeviceMemory:
access |= BufferAccess.DeviceMemory;
break;
case BufferBackingType.DeviceMemoryWithFlush:
access |= BufferAccess.DeviceMemoryMapped;
break;
}
}
_activeType = _desiredType;
return access;
}
/// <summary>
/// Record when data has been uploaded to the buffer.
/// </summary>
public void RecordSet()
{
_setCount++;
ConsiderUseCounts();
}
/// <summary>
/// Record when data has been flushed from the buffer.
/// </summary>
public void RecordFlush()
{
if (_lastFlushWrite != _writeCount)
{
// If it's on the same page as the last flush, ignore it.
_lastFlushWrite = _writeCount;
_flushCount++;
}
}
/// <summary>
/// Determine if the buffer backing should be changed.
/// </summary>
/// <returns>True if the desired backing type is different from the current type</returns>
public readonly bool ShouldChangeBacking()
{
return _desiredType != _activeType;
}
/// <summary>
/// Determine if the buffer backing should be changed, considering a new use with the given buffer stage.
/// </summary>
/// <param name="stage">Buffer stage for the use</param>
/// <returns>True if the desired backing type is different from the current type</returns>
public bool ShouldChangeBacking(BufferStage stage)
{
if (!_canSwap)
{
return false;
}
BufferStage storageFlags = stage & BufferStage.StorageMask;
if (storageFlags != 0)
{
if (storageFlags != BufferStage.StorageRead)
{
// Storage write.
_writeCount++;
var rawStage = stage & BufferStage.StageMask;
if (rawStage == BufferStage.Fragment)
{
// Switch to device memory, swap back only if this use disappears.
_desiredType = CombineTypes(_desiredType, BufferBackingType.DeviceMemory);
_deviceLocalForceCount = DeviceLocalForceExpiry;
// TODO: Might be nice to force atomic access to be device local for any stage.
}
}
ConsiderUseCounts();
}
return _desiredType != _activeType;
}
/// <summary>
/// Evaluate the current counts to determine what the buffer's desired backing type is.
/// This method depends on heuristics devised by testing a variety of software.
/// </summary>
private void ConsiderUseCounts()
{
if (_canSwap)
{
if (_writeCount >= WriteCountThreshold || _setCount >= SetCountThreshold || _flushCount >= FlushCountThreshold)
{
if (_deviceLocalForceCount > 0 && --_deviceLocalForceCount != 0)
{
// Some buffer usage demanded that the buffer stay device local.
// The desired type was selected when this counter was set.
}
else if (_flushCount > 0 || _flushTemp-- > 0)
{
// Buffers that flush should ideally be mapped in host address space for easy copies.
// If the buffer is large it will do better on GPU memory, as there will be more writes than data flushes (typically individual pages).
// If it is small, then it's likely most of the buffer will be flushed so we want it on host memory, as access is cached.
_desiredType = _size > DeviceLocalSizeThreshold ? BufferBackingType.DeviceMemoryWithFlush : BufferBackingType.HostMemory;
}
else if (_writeCount >= WriteCountThreshold)
{
// Buffers that are written often should ideally be in the device local heap. (Storage buffers)
_desiredType = BufferBackingType.DeviceMemory;
}
else if (_setCount > SetCountThreshold)
{
// Buffers that have their data set often should ideally be host mapped. (Constant buffers)
_desiredType = BufferBackingType.HostMemory;
}
// It's harder for a buffer that is flushed to revert to another type of mapping.
if (_flushCount > 0)
{
_flushTemp = 1000;
}
_lastFlushWrite = -1;
_flushCount = 0;
_writeCount = 0;
_setCount = 0;
}
}
}
}
}

View File

@ -107,8 +107,9 @@ namespace Ryujinx.Graphics.Gpu.Memory
/// <param name="memoryManager">GPU memory manager where the buffer is mapped</param>
/// <param name="gpuVa">Start GPU virtual address of the buffer</param>
/// <param name="size">Size in bytes of the buffer</param>
/// <param name="stage">The type of usage that created the buffer</param>
/// <returns>Contiguous physical range of the buffer, after address translation</returns>
public MultiRange TranslateAndCreateBuffer(MemoryManager memoryManager, ulong gpuVa, ulong size)
public MultiRange TranslateAndCreateBuffer(MemoryManager memoryManager, ulong gpuVa, ulong size, BufferStage stage)
{
if (gpuVa == 0)
{
@ -119,7 +120,7 @@ namespace Ryujinx.Graphics.Gpu.Memory
if (address != MemoryManager.PteUnmapped)
{
CreateBuffer(address, size);
CreateBuffer(address, size, stage);
}
return new MultiRange(address, size);
@ -132,8 +133,9 @@ namespace Ryujinx.Graphics.Gpu.Memory
/// <param name="memoryManager">GPU memory manager where the buffer is mapped</param>
/// <param name="gpuVa">Start GPU virtual address of the buffer</param>
/// <param name="size">Size in bytes of the buffer</param>
/// <param name="stage">The type of usage that created the buffer</param>
/// <returns>Physical ranges of the buffer, after address translation</returns>
public MultiRange TranslateAndCreateMultiBuffers(MemoryManager memoryManager, ulong gpuVa, ulong size)
public MultiRange TranslateAndCreateMultiBuffers(MemoryManager memoryManager, ulong gpuVa, ulong size, BufferStage stage)
{
if (gpuVa == 0)
{
@ -149,7 +151,7 @@ namespace Ryujinx.Graphics.Gpu.Memory
return range;
}
CreateBuffer(range);
CreateBuffer(range, stage);
return range;
}
@ -161,8 +163,9 @@ namespace Ryujinx.Graphics.Gpu.Memory
/// <param name="memoryManager">GPU memory manager where the buffer is mapped</param>
/// <param name="gpuVa">Start GPU virtual address of the buffer</param>
/// <param name="size">Size in bytes of the buffer</param>
/// <param name="stage">The type of usage that created the buffer</param>
/// <returns>Physical ranges of the buffer, after address translation</returns>
public MultiRange TranslateAndCreateMultiBuffersPhysicalOnly(MemoryManager memoryManager, ulong gpuVa, ulong size)
public MultiRange TranslateAndCreateMultiBuffersPhysicalOnly(MemoryManager memoryManager, ulong gpuVa, ulong size, BufferStage stage)
{
if (gpuVa == 0)
{
@ -186,11 +189,11 @@ namespace Ryujinx.Graphics.Gpu.Memory
{
if (range.Count > 1)
{
CreateBuffer(subRange.Address, subRange.Size, SparseBufferAlignmentSize);
CreateBuffer(subRange.Address, subRange.Size, stage, SparseBufferAlignmentSize);
}
else
{
CreateBuffer(subRange.Address, subRange.Size);
CreateBuffer(subRange.Address, subRange.Size, stage);
}
}
}
@ -203,11 +206,12 @@ namespace Ryujinx.Graphics.Gpu.Memory
/// This can be used to ensure the existance of a buffer.
/// </summary>
/// <param name="range">Physical ranges of memory where the buffer data is located</param>
public void CreateBuffer(MultiRange range)
/// <param name="stage">The type of usage that created the buffer</param>
public void CreateBuffer(MultiRange range, BufferStage stage)
{
if (range.Count > 1)
{
CreateMultiRangeBuffer(range);
CreateMultiRangeBuffer(range, stage);
}
else
{
@ -215,7 +219,7 @@ namespace Ryujinx.Graphics.Gpu.Memory
if (subRange.Address != MemoryManager.PteUnmapped)
{
CreateBuffer(subRange.Address, subRange.Size);
CreateBuffer(subRange.Address, subRange.Size, stage);
}
}
}
@ -226,7 +230,8 @@ namespace Ryujinx.Graphics.Gpu.Memory
/// </summary>
/// <param name="address">Address of the buffer in memory</param>
/// <param name="size">Size of the buffer in bytes</param>
public void CreateBuffer(ulong address, ulong size)
/// <param name="stage">The type of usage that created the buffer</param>
public void CreateBuffer(ulong address, ulong size, BufferStage stage)
{
ulong endAddress = address + size;
@ -239,7 +244,7 @@ namespace Ryujinx.Graphics.Gpu.Memory
alignedEndAddress += BufferAlignmentSize;
}
CreateBufferAligned(alignedAddress, alignedEndAddress - alignedAddress);
CreateBufferAligned(alignedAddress, alignedEndAddress - alignedAddress, stage);
}
/// <summary>
@ -248,8 +253,9 @@ namespace Ryujinx.Graphics.Gpu.Memory
/// </summary>
/// <param name="address">Address of the buffer in memory</param>
/// <param name="size">Size of the buffer in bytes</param>
/// <param name="stage">The type of usage that created the buffer</param>
/// <param name="alignment">Alignment of the start address of the buffer in bytes</param>
public void CreateBuffer(ulong address, ulong size, ulong alignment)
public void CreateBuffer(ulong address, ulong size, BufferStage stage, ulong alignment)
{
ulong alignmentMask = alignment - 1;
ulong pageAlignmentMask = BufferAlignmentMask;
@ -264,7 +270,7 @@ namespace Ryujinx.Graphics.Gpu.Memory
alignedEndAddress += pageAlignmentMask;
}
CreateBufferAligned(alignedAddress, alignedEndAddress - alignedAddress, alignment);
CreateBufferAligned(alignedAddress, alignedEndAddress - alignedAddress, stage, alignment);
}
/// <summary>
@ -272,7 +278,8 @@ namespace Ryujinx.Graphics.Gpu.Memory
/// if it does not exist yet.
/// </summary>
/// <param name="range">Physical ranges of memory</param>
private void CreateMultiRangeBuffer(MultiRange range)
/// <param name="stage">The type of usage that created the buffer</param>
private void CreateMultiRangeBuffer(MultiRange range, BufferStage stage)
{
// Ensure all non-contiguous buffer we might use are sparse aligned.
for (int i = 0; i < range.Count; i++)
@ -281,7 +288,7 @@ namespace Ryujinx.Graphics.Gpu.Memory
if (subRange.Address != MemoryManager.PteUnmapped)
{
CreateBuffer(subRange.Address, subRange.Size, SparseBufferAlignmentSize);
CreateBuffer(subRange.Address, subRange.Size, stage, SparseBufferAlignmentSize);
}
}
@ -431,9 +438,9 @@ namespace Ryujinx.Graphics.Gpu.Memory
result.EndGpuAddress < gpuVa + size ||
result.UnmappedSequence != result.Buffer.UnmappedSequence)
{
MultiRange range = TranslateAndCreateBuffer(memoryManager, gpuVa, size);
MultiRange range = TranslateAndCreateBuffer(memoryManager, gpuVa, size, BufferStage.Internal);
ulong address = range.GetSubRange(0).Address;
result = new BufferCacheEntry(address, gpuVa, GetBuffer(address, size));
result = new BufferCacheEntry(address, gpuVa, GetBuffer(address, size, BufferStage.Internal));
_dirtyCache[gpuVa] = result;
}
@ -466,9 +473,9 @@ namespace Ryujinx.Graphics.Gpu.Memory
result.EndGpuAddress < alignedEndGpuVa ||
result.UnmappedSequence != result.Buffer.UnmappedSequence)
{
MultiRange range = TranslateAndCreateBuffer(memoryManager, alignedGpuVa, size);
MultiRange range = TranslateAndCreateBuffer(memoryManager, alignedGpuVa, size, BufferStage.None);
ulong address = range.GetSubRange(0).Address;
result = new BufferCacheEntry(address, alignedGpuVa, GetBuffer(address, size));
result = new BufferCacheEntry(address, alignedGpuVa, GetBuffer(address, size, BufferStage.None));
_modifiedCache[alignedGpuVa] = result;
}
@ -485,7 +492,8 @@ namespace Ryujinx.Graphics.Gpu.Memory
/// </summary>
/// <param name="address">Address of the buffer in guest memory</param>
/// <param name="size">Size in bytes of the buffer</param>
private void CreateBufferAligned(ulong address, ulong size)
/// <param name="stage">The type of usage that created the buffer</param>
private void CreateBufferAligned(ulong address, ulong size, BufferStage stage)
{
Buffer[] overlaps = _bufferOverlaps;
int overlapsCount = _buffers.FindOverlapsNonOverlapping(address, size, ref overlaps);
@ -546,13 +554,13 @@ namespace Ryujinx.Graphics.Gpu.Memory
ulong newSize = endAddress - address;
CreateBufferAligned(address, newSize, anySparseCompatible, overlaps, overlapsCount);
CreateBufferAligned(address, newSize, stage, anySparseCompatible, overlaps, overlapsCount);
}
}
else
{
// No overlap, just create a new buffer.
Buffer buffer = new(_context, _physicalMemory, address, size, sparseCompatible: false);
Buffer buffer = new(_context, _physicalMemory, address, size, stage, sparseCompatible: false);
lock (_buffers)
{
@ -570,8 +578,9 @@ namespace Ryujinx.Graphics.Gpu.Memory
/// </summary>
/// <param name="address">Address of the buffer in guest memory</param>
/// <param name="size">Size in bytes of the buffer</param>
/// <param name="stage">The type of usage that created the buffer</param>
/// <param name="alignment">Alignment of the start address of the buffer</param>
private void CreateBufferAligned(ulong address, ulong size, ulong alignment)
private void CreateBufferAligned(ulong address, ulong size, BufferStage stage, ulong alignment)
{
Buffer[] overlaps = _bufferOverlaps;
int overlapsCount = _buffers.FindOverlapsNonOverlapping(address, size, ref overlaps);
@ -624,13 +633,13 @@ namespace Ryujinx.Graphics.Gpu.Memory
ulong newSize = endAddress - address;
CreateBufferAligned(address, newSize, sparseAligned, overlaps, overlapsCount);
CreateBufferAligned(address, newSize, stage, sparseAligned, overlaps, overlapsCount);
}
}
else
{
// No overlap, just create a new buffer.
Buffer buffer = new(_context, _physicalMemory, address, size, sparseAligned);
Buffer buffer = new(_context, _physicalMemory, address, size, stage, sparseAligned);
lock (_buffers)
{
@ -648,12 +657,13 @@ namespace Ryujinx.Graphics.Gpu.Memory
/// </summary>
/// <param name="address">Address of the buffer in guest memory</param>
/// <param name="size">Size in bytes of the buffer</param>
/// <param name="stage">The type of usage that created the buffer</param>
/// <param name="sparseCompatible">Indicates if the buffer can be used in a sparse buffer mapping</param>
/// <param name="overlaps">Buffers overlapping the range</param>
/// <param name="overlapsCount">Total of overlaps</param>
private void CreateBufferAligned(ulong address, ulong size, bool sparseCompatible, Buffer[] overlaps, int overlapsCount)
private void CreateBufferAligned(ulong address, ulong size, BufferStage stage, bool sparseCompatible, Buffer[] overlaps, int overlapsCount)
{
Buffer newBuffer = new Buffer(_context, _physicalMemory, address, size, sparseCompatible, overlaps.Take(overlapsCount));
Buffer newBuffer = new Buffer(_context, _physicalMemory, address, size, stage, sparseCompatible, overlaps.Take(overlapsCount));
lock (_buffers)
{
@ -704,7 +714,7 @@ namespace Ryujinx.Graphics.Gpu.Memory
for (int index = 0; index < overlapCount; index++)
{
CreateMultiRangeBuffer(overlaps[index].Range);
CreateMultiRangeBuffer(overlaps[index].Range, BufferStage.None);
}
}
@ -731,8 +741,8 @@ namespace Ryujinx.Graphics.Gpu.Memory
/// <param name="size">Size in bytes of the copy</param>
public void CopyBuffer(MemoryManager memoryManager, ulong srcVa, ulong dstVa, ulong size)
{
MultiRange srcRange = TranslateAndCreateMultiBuffersPhysicalOnly(memoryManager, srcVa, size);
MultiRange dstRange = TranslateAndCreateMultiBuffersPhysicalOnly(memoryManager, dstVa, size);
MultiRange srcRange = TranslateAndCreateMultiBuffersPhysicalOnly(memoryManager, srcVa, size, BufferStage.Copy);
MultiRange dstRange = TranslateAndCreateMultiBuffersPhysicalOnly(memoryManager, dstVa, size, BufferStage.Copy);
if (srcRange.Count == 1 && dstRange.Count == 1)
{
@ -788,8 +798,8 @@ namespace Ryujinx.Graphics.Gpu.Memory
/// <param name="size">Size in bytes of the copy</param>
private void CopyBufferSingleRange(MemoryManager memoryManager, ulong srcAddress, ulong dstAddress, ulong size)
{
Buffer srcBuffer = GetBuffer(srcAddress, size);
Buffer dstBuffer = GetBuffer(dstAddress, size);
Buffer srcBuffer = GetBuffer(srcAddress, size, BufferStage.Copy);
Buffer dstBuffer = GetBuffer(dstAddress, size, BufferStage.Copy);
int srcOffset = (int)(srcAddress - srcBuffer.Address);
int dstOffset = (int)(dstAddress - dstBuffer.Address);
@ -803,7 +813,7 @@ namespace Ryujinx.Graphics.Gpu.Memory
if (srcBuffer.IsModified(srcAddress, size))
{
dstBuffer.SignalModified(dstAddress, size);
dstBuffer.SignalModified(dstAddress, size, BufferStage.Copy);
}
else
{
@ -828,12 +838,12 @@ namespace Ryujinx.Graphics.Gpu.Memory
/// <param name="value">Value to be written into the buffer</param>
public void ClearBuffer(MemoryManager memoryManager, ulong gpuVa, ulong size, uint value)
{
MultiRange range = TranslateAndCreateMultiBuffersPhysicalOnly(memoryManager, gpuVa, size);
MultiRange range = TranslateAndCreateMultiBuffersPhysicalOnly(memoryManager, gpuVa, size, BufferStage.Copy);
for (int index = 0; index < range.Count; index++)
{
MemoryRange subRange = range.GetSubRange(index);
Buffer buffer = GetBuffer(subRange.Address, subRange.Size);
Buffer buffer = GetBuffer(subRange.Address, subRange.Size, BufferStage.Copy);
int offset = (int)(subRange.Address - buffer.Address);
@ -849,18 +859,19 @@ namespace Ryujinx.Graphics.Gpu.Memory
/// Gets a buffer sub-range starting at a given memory address, aligned to the next page boundary.
/// </summary>
/// <param name="range">Physical regions of memory where the buffer is mapped</param>
/// <param name="stage">Buffer stage that triggered the access</param>
/// <param name="write">Whether the buffer will be written to by this use</param>
/// <returns>The buffer sub-range starting at the given memory address</returns>
public BufferRange GetBufferRangeAligned(MultiRange range, bool write = false)
public BufferRange GetBufferRangeAligned(MultiRange range, BufferStage stage, bool write = false)
{
if (range.Count > 1)
{
return GetBuffer(range, write).GetRange(range);
return GetBuffer(range, stage, write).GetRange(range);
}
else
{
MemoryRange subRange = range.GetSubRange(0);
return GetBuffer(subRange.Address, subRange.Size, write).GetRangeAligned(subRange.Address, subRange.Size, write);
return GetBuffer(subRange.Address, subRange.Size, stage, write).GetRangeAligned(subRange.Address, subRange.Size, write);
}
}
@ -868,18 +879,19 @@ namespace Ryujinx.Graphics.Gpu.Memory
/// Gets a buffer sub-range for a given memory range.
/// </summary>
/// <param name="range">Physical regions of memory where the buffer is mapped</param>
/// <param name="stage">Buffer stage that triggered the access</param>
/// <param name="write">Whether the buffer will be written to by this use</param>
/// <returns>The buffer sub-range for the given range</returns>
public BufferRange GetBufferRange(MultiRange range, bool write = false)
public BufferRange GetBufferRange(MultiRange range, BufferStage stage, bool write = false)
{
if (range.Count > 1)
{
return GetBuffer(range, write).GetRange(range);
return GetBuffer(range, stage, write).GetRange(range);
}
else
{
MemoryRange subRange = range.GetSubRange(0);
return GetBuffer(subRange.Address, subRange.Size, write).GetRange(subRange.Address, subRange.Size, write);
return GetBuffer(subRange.Address, subRange.Size, stage, write).GetRange(subRange.Address, subRange.Size, write);
}
}
@ -888,9 +900,10 @@ namespace Ryujinx.Graphics.Gpu.Memory
/// A buffer overlapping with the specified range is assumed to already exist on the cache.
/// </summary>
/// <param name="range">Physical regions of memory where the buffer is mapped</param>
/// <param name="stage">Buffer stage that triggered the access</param>
/// <param name="write">Whether the buffer will be written to by this use</param>
/// <returns>The buffer where the range is fully contained</returns>
private MultiRangeBuffer GetBuffer(MultiRange range, bool write = false)
private MultiRangeBuffer GetBuffer(MultiRange range, BufferStage stage, bool write = false)
{
for (int i = 0; i < range.Count; i++)
{
@ -902,7 +915,7 @@ namespace Ryujinx.Graphics.Gpu.Memory
if (write)
{
subBuffer.SignalModified(subRange.Address, subRange.Size);
subBuffer.SignalModified(subRange.Address, subRange.Size, stage);
}
}
@ -935,9 +948,10 @@ namespace Ryujinx.Graphics.Gpu.Memory
/// </summary>
/// <param name="address">Start address of the memory range</param>
/// <param name="size">Size in bytes of the memory range</param>
/// <param name="stage">Buffer stage that triggered the access</param>
/// <param name="write">Whether the buffer will be written to by this use</param>
/// <returns>The buffer where the range is fully contained</returns>
private Buffer GetBuffer(ulong address, ulong size, bool write = false)
private Buffer GetBuffer(ulong address, ulong size, BufferStage stage, bool write = false)
{
Buffer buffer;
@ -950,7 +964,7 @@ namespace Ryujinx.Graphics.Gpu.Memory
if (write)
{
buffer.SignalModified(address, size);
buffer.SignalModified(address, size, stage);
}
}
else
@ -1004,6 +1018,18 @@ namespace Ryujinx.Graphics.Gpu.Memory
}
}
/// <summary>
/// Signal that the given buffer's handle has changed,
/// forcing rebind and any overlapping multi-range buffers to be recreated.
/// </summary>
/// <param name="buffer">The buffer that has changed handle</param>
public void BufferBackingChanged(Buffer buffer)
{
NotifyBuffersModified?.Invoke();
RecreateMultiRangeBuffers(buffer.Address, buffer.Size);
}
/// <summary>
/// Prune any invalid entries from a quick access dictionary.
/// </summary>

View File

@ -156,7 +156,7 @@ namespace Ryujinx.Graphics.Gpu.Memory
/// <param name="type">Type of each index buffer element</param>
public void SetIndexBuffer(ulong gpuVa, ulong size, IndexType type)
{
MultiRange range = _channel.MemoryManager.Physical.BufferCache.TranslateAndCreateBuffer(_channel.MemoryManager, gpuVa, size);
MultiRange range = _channel.MemoryManager.Physical.BufferCache.TranslateAndCreateBuffer(_channel.MemoryManager, gpuVa, size, BufferStage.IndexBuffer);
_indexBuffer.Range = range;
_indexBuffer.Type = type;
@ -186,7 +186,7 @@ namespace Ryujinx.Graphics.Gpu.Memory
/// <param name="divisor">Vertex divisor of the buffer, for instanced draws</param>
public void SetVertexBuffer(int index, ulong gpuVa, ulong size, int stride, int divisor)
{
MultiRange range = _channel.MemoryManager.Physical.BufferCache.TranslateAndCreateBuffer(_channel.MemoryManager, gpuVa, size);
MultiRange range = _channel.MemoryManager.Physical.BufferCache.TranslateAndCreateBuffer(_channel.MemoryManager, gpuVa, size, BufferStage.VertexBuffer);
_vertexBuffers[index].Range = range;
_vertexBuffers[index].Stride = stride;
@ -213,7 +213,7 @@ namespace Ryujinx.Graphics.Gpu.Memory
/// <param name="size">Size in bytes of the transform feedback buffer</param>
public void SetTransformFeedbackBuffer(int index, ulong gpuVa, ulong size)
{
MultiRange range = _channel.MemoryManager.Physical.BufferCache.TranslateAndCreateMultiBuffers(_channel.MemoryManager, gpuVa, size);
MultiRange range = _channel.MemoryManager.Physical.BufferCache.TranslateAndCreateMultiBuffers(_channel.MemoryManager, gpuVa, size, BufferStage.TransformFeedback);
_transformFeedbackBuffers[index] = new BufferBounds(range);
_transformFeedbackBuffersDirty = true;
@ -260,7 +260,7 @@ namespace Ryujinx.Graphics.Gpu.Memory
gpuVa = BitUtils.AlignDown<ulong>(gpuVa, (ulong)_context.Capabilities.StorageBufferOffsetAlignment);
MultiRange range = _channel.MemoryManager.Physical.BufferCache.TranslateAndCreateMultiBuffers(_channel.MemoryManager, gpuVa, size);
MultiRange range = _channel.MemoryManager.Physical.BufferCache.TranslateAndCreateMultiBuffers(_channel.MemoryManager, gpuVa, size, BufferStageUtils.ComputeStorage(flags));
_cpStorageBuffers.SetBounds(index, range, flags);
}
@ -284,7 +284,7 @@ namespace Ryujinx.Graphics.Gpu.Memory
gpuVa = BitUtils.AlignDown<ulong>(gpuVa, (ulong)_context.Capabilities.StorageBufferOffsetAlignment);
MultiRange range = _channel.MemoryManager.Physical.BufferCache.TranslateAndCreateMultiBuffers(_channel.MemoryManager, gpuVa, size);
MultiRange range = _channel.MemoryManager.Physical.BufferCache.TranslateAndCreateMultiBuffers(_channel.MemoryManager, gpuVa, size, BufferStageUtils.GraphicsStorage(stage, flags));
if (!buffers.Buffers[index].Range.Equals(range))
{
@ -303,7 +303,7 @@ namespace Ryujinx.Graphics.Gpu.Memory
/// <param name="size">Size in bytes of the storage buffer</param>
public void SetComputeUniformBuffer(int index, ulong gpuVa, ulong size)
{
MultiRange range = _channel.MemoryManager.Physical.BufferCache.TranslateAndCreateBuffer(_channel.MemoryManager, gpuVa, size);
MultiRange range = _channel.MemoryManager.Physical.BufferCache.TranslateAndCreateBuffer(_channel.MemoryManager, gpuVa, size, BufferStage.Compute);
_cpUniformBuffers.SetBounds(index, range);
}
@ -318,7 +318,7 @@ namespace Ryujinx.Graphics.Gpu.Memory
/// <param name="size">Size in bytes of the storage buffer</param>
public void SetGraphicsUniformBuffer(int stage, int index, ulong gpuVa, ulong size)
{
MultiRange range = _channel.MemoryManager.Physical.BufferCache.TranslateAndCreateBuffer(_channel.MemoryManager, gpuVa, size);
MultiRange range = _channel.MemoryManager.Physical.BufferCache.TranslateAndCreateBuffer(_channel.MemoryManager, gpuVa, size, BufferStageUtils.FromShaderStage(stage));
_gpUniformBuffers[stage].SetBounds(index, range);
_gpUniformBuffersDirty = true;
@ -502,7 +502,7 @@ namespace Ryujinx.Graphics.Gpu.Memory
foreach (var binding in _bufferTextures)
{
var isStore = binding.BindingInfo.Flags.HasFlag(TextureUsageFlags.ImageStore);
var range = bufferCache.GetBufferRange(binding.Range, isStore);
var range = bufferCache.GetBufferRange(binding.Range, BufferStageUtils.TextureBuffer(binding.Stage, binding.BindingInfo.Flags), isStore);
binding.Texture.SetStorage(range);
// The texture must be rebound to use the new storage if it was updated.
@ -526,7 +526,7 @@ namespace Ryujinx.Graphics.Gpu.Memory
foreach (var binding in _bufferTextureArrays)
{
var range = bufferCache.GetBufferRange(binding.Range);
var range = bufferCache.GetBufferRange(binding.Range, BufferStage.None);
binding.Texture.SetStorage(range);
textureArray[0] = binding.Texture;
@ -536,7 +536,7 @@ namespace Ryujinx.Graphics.Gpu.Memory
foreach (var binding in _bufferImageArrays)
{
var isStore = binding.BindingInfo.Flags.HasFlag(TextureUsageFlags.ImageStore);
var range = bufferCache.GetBufferRange(binding.Range, isStore);
var range = bufferCache.GetBufferRange(binding.Range, BufferStage.None, isStore);
binding.Texture.SetStorage(range);
textureArray[0] = binding.Texture;
@ -565,7 +565,7 @@ namespace Ryujinx.Graphics.Gpu.Memory
if (!_indexBuffer.Range.IsUnmapped)
{
BufferRange buffer = bufferCache.GetBufferRange(_indexBuffer.Range);
BufferRange buffer = bufferCache.GetBufferRange(_indexBuffer.Range, BufferStage.IndexBuffer);
_context.Renderer.Pipeline.SetIndexBuffer(buffer, _indexBuffer.Type);
}
@ -597,7 +597,7 @@ namespace Ryujinx.Graphics.Gpu.Memory
continue;
}
BufferRange buffer = bufferCache.GetBufferRange(vb.Range);
BufferRange buffer = bufferCache.GetBufferRange(vb.Range, BufferStage.VertexBuffer);
vertexBuffers[index] = new VertexBufferDescriptor(buffer, vb.Stride, vb.Divisor);
}
@ -637,7 +637,7 @@ namespace Ryujinx.Graphics.Gpu.Memory
continue;
}
tfbs[index] = bufferCache.GetBufferRange(tfb.Range, write: true);
tfbs[index] = bufferCache.GetBufferRange(tfb.Range, BufferStage.TransformFeedback, write: true);
}
_context.Renderer.Pipeline.SetTransformFeedbackBuffers(tfbs);
@ -684,7 +684,7 @@ namespace Ryujinx.Graphics.Gpu.Memory
_context.SupportBufferUpdater.SetTfeOffset(index, tfeOffset);
buffers[index] = new BufferAssignment(index, bufferCache.GetBufferRange(range, write: true));
buffers[index] = new BufferAssignment(index, bufferCache.GetBufferRange(range, BufferStage.TransformFeedback, write: true));
}
}
@ -751,6 +751,7 @@ namespace Ryujinx.Graphics.Gpu.Memory
for (ShaderStage stage = ShaderStage.Vertex; stage <= ShaderStage.Fragment; stage++)
{
ref var buffers = ref bindings[(int)stage - 1];
BufferStage bufferStage = BufferStageUtils.FromShaderStage(stage);
for (int index = 0; index < buffers.Count; index++)
{
@ -762,8 +763,8 @@ namespace Ryujinx.Graphics.Gpu.Memory
{
var isWrite = bounds.Flags.HasFlag(BufferUsageFlags.Write);
var range = isStorage
? bufferCache.GetBufferRangeAligned(bounds.Range, isWrite)
: bufferCache.GetBufferRange(bounds.Range);
? bufferCache.GetBufferRangeAligned(bounds.Range, bufferStage | BufferStageUtils.FromUsage(bounds.Flags), isWrite)
: bufferCache.GetBufferRange(bounds.Range, bufferStage);
ranges[rangesCount++] = new BufferAssignment(bindingInfo.Binding, range);
}
@ -799,8 +800,8 @@ namespace Ryujinx.Graphics.Gpu.Memory
{
var isWrite = bounds.Flags.HasFlag(BufferUsageFlags.Write);
var range = isStorage
? bufferCache.GetBufferRangeAligned(bounds.Range, isWrite)
: bufferCache.GetBufferRange(bounds.Range);
? bufferCache.GetBufferRangeAligned(bounds.Range, BufferStageUtils.ComputeStorage(bounds.Flags), isWrite)
: bufferCache.GetBufferRange(bounds.Range, BufferStage.Compute);
ranges[rangesCount++] = new BufferAssignment(bindingInfo.Binding, range);
}
@ -875,7 +876,7 @@ namespace Ryujinx.Graphics.Gpu.Memory
Format format,
bool isImage)
{
_channel.MemoryManager.Physical.BufferCache.CreateBuffer(range);
_channel.MemoryManager.Physical.BufferCache.CreateBuffer(range, BufferStageUtils.TextureBuffer(stage, bindingInfo.Flags));
_bufferTextures.Add(new BufferTextureBinding(stage, texture, range, bindingInfo, format, isImage));
}
@ -883,6 +884,7 @@ namespace Ryujinx.Graphics.Gpu.Memory
/// <summary>
/// Sets the buffer storage of a buffer texture array element. This will be bound when the buffer manager commits bindings.
/// </summary>
/// <param name="stage">Shader stage accessing the texture</param>
/// <param name="array">Texture array where the element will be inserted</param>
/// <param name="texture">Buffer texture</param>
/// <param name="range">Physical ranges of memory where the buffer texture data is located</param>
@ -890,6 +892,7 @@ namespace Ryujinx.Graphics.Gpu.Memory
/// <param name="index">Index of the binding on the array</param>
/// <param name="format">Format of the buffer texture</param>
public void SetBufferTextureStorage(
ShaderStage stage,
ITextureArray array,
ITexture texture,
MultiRange range,
@ -897,7 +900,7 @@ namespace Ryujinx.Graphics.Gpu.Memory
int index,
Format format)
{
_channel.MemoryManager.Physical.BufferCache.CreateBuffer(range);
_channel.MemoryManager.Physical.BufferCache.CreateBuffer(range, BufferStageUtils.TextureBuffer(stage, bindingInfo.Flags));
_bufferTextureArrays.Add(new BufferTextureArrayBinding<ITextureArray>(array, texture, range, bindingInfo, index, format));
}
@ -905,6 +908,7 @@ namespace Ryujinx.Graphics.Gpu.Memory
/// <summary>
/// Sets the buffer storage of a buffer image array element. This will be bound when the buffer manager commits bindings.
/// </summary>
/// <param name="stage">Shader stage accessing the texture</param>
/// <param name="array">Image array where the element will be inserted</param>
/// <param name="texture">Buffer texture</param>
/// <param name="range">Physical ranges of memory where the buffer texture data is located</param>
@ -912,6 +916,7 @@ namespace Ryujinx.Graphics.Gpu.Memory
/// <param name="index">Index of the binding on the array</param>
/// <param name="format">Format of the buffer texture</param>
public void SetBufferTextureStorage(
ShaderStage stage,
IImageArray array,
ITexture texture,
MultiRange range,
@ -919,7 +924,7 @@ namespace Ryujinx.Graphics.Gpu.Memory
int index,
Format format)
{
_channel.MemoryManager.Physical.BufferCache.CreateBuffer(range);
_channel.MemoryManager.Physical.BufferCache.CreateBuffer(range, BufferStageUtils.TextureBuffer(stage, bindingInfo.Flags));
_bufferImageArrays.Add(new BufferTextureArrayBinding<IImageArray>(array, texture, range, bindingInfo, index, format));
}

View File

@ -1,37 +1,21 @@
using System;
using System.Threading;
namespace Ryujinx.Graphics.Gpu.Memory
{
/// <summary>
/// A record of when buffer data was copied from one buffer to another, along with the SyncNumber when the migration will be complete.
/// Keeps the source buffer alive for data flushes until the migration is complete.
/// A record of when buffer data was copied from multiple buffers to one migration target,
/// along with the SyncNumber when the migration will be complete.
/// Keeps the source buffers alive for data flushes until the migration is complete.
/// All spans cover the full range of the "destination" buffer.
/// </summary>
internal class BufferMigration : IDisposable
{
/// <summary>
/// The offset for the migrated region.
/// Ranges from source buffers that were copied as part of this migration.
/// Ordered by increasing base address.
/// </summary>
private readonly ulong _offset;
/// <summary>
/// The size for the migrated region.
/// </summary>
private readonly ulong _size;
/// <summary>
/// The buffer that was migrated from.
/// </summary>
private readonly Buffer _buffer;
/// <summary>
/// The source range action, to be called on overlap with an unreached sync number.
/// </summary>
private readonly Action<ulong, ulong> _sourceRangeAction;
/// <summary>
/// The source range list.
/// </summary>
private readonly BufferModifiedRangeList _source;
public BufferMigrationSpan[] Spans { get; private set; }
/// <summary>
/// The destination range list. This range list must be updated when flushing the source.
@ -43,55 +27,193 @@ namespace Ryujinx.Graphics.Gpu.Memory
/// </summary>
public readonly ulong SyncNumber;
/// <summary>
/// Number of active users there are traversing this migration's spans.
/// </summary>
private int _refCount;
/// <summary>
/// Create a new buffer migration.
/// </summary>
/// <param name="spans">Source spans for the migration</param>
/// <param name="destination">Destination buffer range list</param>
/// <param name="syncNumber">Sync number where this migration will be complete</param>
public BufferMigration(BufferMigrationSpan[] spans, BufferModifiedRangeList destination, ulong syncNumber)
{
Spans = spans;
Destination = destination;
SyncNumber = syncNumber;
}
/// <summary>
/// Add a span to the migration. Allocates a new array with the target size, and replaces it.
/// </summary>
/// <remarks>
/// The base address for the span is assumed to be higher than all other spans in the migration,
/// to keep the span array ordered.
/// </remarks>
public void AddSpanToEnd(BufferMigrationSpan span)
{
BufferMigrationSpan[] oldSpans = Spans;
BufferMigrationSpan[] newSpans = new BufferMigrationSpan[oldSpans.Length + 1];
oldSpans.CopyTo(newSpans, 0);
newSpans[oldSpans.Length] = span;
Spans = newSpans;
}
/// <summary>
/// Performs the given range action, or one from a migration that overlaps and has not synced yet.
/// </summary>
/// <param name="offset">The offset to pass to the action</param>
/// <param name="size">The size to pass to the action</param>
/// <param name="syncNumber">The sync number that has been reached</param>
/// <param name="rangeAction">The action to perform</param>
public void RangeActionWithMigration(ulong offset, ulong size, ulong syncNumber, BufferFlushAction rangeAction)
{
long syncDiff = (long)(syncNumber - SyncNumber);
if (syncDiff >= 0)
{
// The migration has completed. Run the parent action.
rangeAction(offset, size, syncNumber);
}
else
{
Interlocked.Increment(ref _refCount);
ulong prevAddress = offset;
ulong endAddress = offset + size;
foreach (BufferMigrationSpan span in Spans)
{
if (!span.Overlaps(offset, size))
{
continue;
}
if (span.Address > prevAddress)
{
// There's a gap between this span and the last (or the start address). Flush the range using the parent action.
rangeAction(prevAddress, span.Address - prevAddress, syncNumber);
}
span.RangeActionWithMigration(offset, size, syncNumber);
prevAddress = span.Address + span.Size;
}
if (endAddress > prevAddress)
{
// There's a gap at the end of the range with no migration. Flush the range using the parent action.
rangeAction(prevAddress, endAddress - prevAddress, syncNumber);
}
Interlocked.Decrement(ref _refCount);
}
}
/// <summary>
/// Dispose the buffer migration. This removes the reference from the destination range list,
/// and runs all the dispose buffers for the migration spans. (typically disposes the source buffer)
/// </summary>
public void Dispose()
{
while (Volatile.Read(ref _refCount) > 0)
{
// Coming into this method, the sync for the migration will be met, so nothing can increment the ref count.
// However, an existing traversal of the spans for data flush could still be in progress.
// Spin if this is ever the case, so they don't get disposed before the operation is complete.
}
Destination.RemoveMigration(this);
foreach (BufferMigrationSpan span in Spans)
{
span.Dispose();
}
}
}
/// <summary>
/// A record of when buffer data was copied from one buffer to another, for a specific range in a source buffer.
/// Keeps the source buffer alive for data flushes until the migration is complete.
/// </summary>
internal readonly struct BufferMigrationSpan : IDisposable
{
/// <summary>
/// The offset for the migrated region.
/// </summary>
public readonly ulong Address;
/// <summary>
/// The size for the migrated region.
/// </summary>
public readonly ulong Size;
/// <summary>
/// The action to perform when the migration isn't needed anymore.
/// </summary>
private readonly Action _disposeAction;
/// <summary>
/// The source range action, to be called on overlap with an unreached sync number.
/// </summary>
private readonly BufferFlushAction _sourceRangeAction;
/// <summary>
/// Optional migration for the source data. Can chain together if many migrations happen in a short time.
/// If this is null, then _sourceRangeAction will always provide up to date data.
/// </summary>
private readonly BufferMigration _source;
/// <summary>
/// Creates a record for a buffer migration.
/// </summary>
/// <param name="buffer">The source buffer for this migration</param>
/// <param name="disposeAction">The action to perform when the migration isn't needed anymore</param>
/// <param name="sourceRangeAction">The flush action for the source buffer</param>
/// <param name="source">The modified range list for the source buffer</param>
/// <param name="dest">The modified range list for the destination buffer</param>
/// <param name="syncNumber">The sync number for when the migration is complete</param>
public BufferMigration(
/// <param name="source">Pending migration for the source buffer</param>
public BufferMigrationSpan(
Buffer buffer,
Action<ulong, ulong> sourceRangeAction,
BufferModifiedRangeList source,
BufferModifiedRangeList dest,
ulong syncNumber)
Action disposeAction,
BufferFlushAction sourceRangeAction,
BufferMigration source)
{
_offset = buffer.Address;
_size = buffer.Size;
_buffer = buffer;
Address = buffer.Address;
Size = buffer.Size;
_disposeAction = disposeAction;
_sourceRangeAction = sourceRangeAction;
_source = source;
Destination = dest;
SyncNumber = syncNumber;
}
/// <summary>
/// Creates a record for a buffer migration, using the default buffer dispose action.
/// </summary>
/// <param name="buffer">The source buffer for this migration</param>
/// <param name="sourceRangeAction">The flush action for the source buffer</param>
/// <param name="source">Pending migration for the source buffer</param>
public BufferMigrationSpan(
Buffer buffer,
BufferFlushAction sourceRangeAction,
BufferMigration source) : this(buffer, buffer.DecrementReferenceCount, sourceRangeAction, source) { }
/// <summary>
/// Determine if the given range overlaps this migration, and has not been completed yet.
/// </summary>
/// <param name="offset">Start offset</param>
/// <param name="size">Range size</param>
/// <param name="syncNumber">The sync number that was waited on</param>
/// <returns>True if overlapping and in progress, false otherwise</returns>
public bool Overlaps(ulong offset, ulong size, ulong syncNumber)
public bool Overlaps(ulong offset, ulong size)
{
ulong end = offset + size;
ulong destEnd = _offset + _size;
long syncDiff = (long)(syncNumber - SyncNumber); // syncNumber is less if the copy has not completed.
ulong destEnd = Address + Size;
return !(end <= _offset || offset >= destEnd) && syncDiff < 0;
}
/// <summary>
/// Determine if the given range matches this migration.
/// </summary>
/// <param name="offset">Start offset</param>
/// <param name="size">Range size</param>
/// <returns>True if the range exactly matches, false otherwise</returns>
public bool FullyMatches(ulong offset, ulong size)
{
return _offset == offset && _size == size;
return !(end <= Address || offset >= destEnd);
}
/// <summary>
@ -100,26 +222,30 @@ namespace Ryujinx.Graphics.Gpu.Memory
/// <param name="offset">Start offset</param>
/// <param name="size">Range size</param>
/// <param name="syncNumber">Current sync number</param>
/// <param name="parent">The modified range list that originally owned this range</param>
public void RangeActionWithMigration(ulong offset, ulong size, ulong syncNumber, BufferModifiedRangeList parent)
public void RangeActionWithMigration(ulong offset, ulong size, ulong syncNumber)
{
ulong end = offset + size;
end = Math.Min(_offset + _size, end);
offset = Math.Max(_offset, offset);
end = Math.Min(Address + Size, end);
offset = Math.Max(Address, offset);
size = end - offset;
_source.RangeActionWithMigration(offset, size, syncNumber, parent, _sourceRangeAction);
if (_source != null)
{
_source.RangeActionWithMigration(offset, size, syncNumber, _sourceRangeAction);
}
else
{
_sourceRangeAction(offset, size, syncNumber);
}
}
/// <summary>
/// Removes this reference to the range list, potentially allowing for the source buffer to be disposed.
/// Removes this migration span, potentially allowing for the source buffer to be disposed.
/// </summary>
public void Dispose()
{
Destination.RemoveMigration(this);
_buffer.DecrementReferenceCount();
_disposeAction();
}
}
}

View File

@ -1,7 +1,6 @@
using Ryujinx.Common.Pools;
using Ryujinx.Memory.Range;
using System;
using System.Collections.Generic;
using System.Linq;
namespace Ryujinx.Graphics.Gpu.Memory
@ -72,10 +71,10 @@ namespace Ryujinx.Graphics.Gpu.Memory
private readonly GpuContext _context;
private readonly Buffer _parent;
private readonly Action<ulong, ulong> _flushAction;
private readonly BufferFlushAction _flushAction;
private List<BufferMigration> _sources;
private BufferMigration _migrationTarget;
private BufferMigration _source;
private BufferModifiedRangeList _migrationTarget;
private readonly object _lock = new();
@ -99,7 +98,7 @@ namespace Ryujinx.Graphics.Gpu.Memory
/// <param name="context">GPU context that the buffer range list belongs to</param>
/// <param name="parent">The parent buffer that owns this range list</param>
/// <param name="flushAction">The flush action for the parent buffer</param>
public BufferModifiedRangeList(GpuContext context, Buffer parent, Action<ulong, ulong> flushAction) : base(BackingInitialSize)
public BufferModifiedRangeList(GpuContext context, Buffer parent, BufferFlushAction flushAction) : base(BackingInitialSize)
{
_context = context;
_parent = parent;
@ -199,6 +198,36 @@ namespace Ryujinx.Graphics.Gpu.Memory
}
}
/// <summary>
/// Gets modified ranges within the specified region, and then fires the given action for each range individually.
/// </summary>
/// <param name="address">Start address to query</param>
/// <param name="size">Size to query</param>
/// <param name="syncNumber">Sync number required for a range to be signalled</param>
/// <param name="rangeAction">The action to call for each modified range</param>
public void GetRangesAtSync(ulong address, ulong size, ulong syncNumber, Action<ulong, ulong> rangeAction)
{
int count = 0;
ref var overlaps = ref ThreadStaticArray<BufferModifiedRange>.Get();
// Range list must be consistent for this operation.
lock (_lock)
{
count = FindOverlapsNonOverlapping(address, size, ref overlaps);
}
for (int i = 0; i < count; i++)
{
BufferModifiedRange overlap = overlaps[i];
if (overlap.SyncNumber == syncNumber)
{
rangeAction(overlap.Address, overlap.Size);
}
}
}
/// <summary>
/// Gets modified ranges within the specified region, and then fires the given action for each range individually.
/// </summary>
@ -245,41 +274,16 @@ namespace Ryujinx.Graphics.Gpu.Memory
/// <param name="offset">The offset to pass to the action</param>
/// <param name="size">The size to pass to the action</param>
/// <param name="syncNumber">The sync number that has been reached</param>
/// <param name="parent">The modified range list that originally owned this range</param>
/// <param name="rangeAction">The action to perform</param>
public void RangeActionWithMigration(ulong offset, ulong size, ulong syncNumber, BufferModifiedRangeList parent, Action<ulong, ulong> rangeAction)
public void RangeActionWithMigration(ulong offset, ulong size, ulong syncNumber, BufferFlushAction rangeAction)
{
bool firstSource = true;
if (parent != this)
if (_source != null)
{
lock (_lock)
{
if (_sources != null)
{
foreach (BufferMigration source in _sources)
{
if (source.Overlaps(offset, size, syncNumber))
{
if (firstSource && !source.FullyMatches(offset, size))
{
// Perform this buffer's action first. The migrations will run after.
rangeAction(offset, size);
}
source.RangeActionWithMigration(offset, size, syncNumber, parent);
firstSource = false;
}
}
}
}
_source.RangeActionWithMigration(offset, size, syncNumber, rangeAction);
}
if (firstSource)
else
{
// No overlapping migrations, or they are not meant for this range, flush the data using the given action.
rangeAction(offset, size);
rangeAction(offset, size, syncNumber);
}
}
@ -319,7 +323,7 @@ namespace Ryujinx.Graphics.Gpu.Memory
ClearPart(overlap, clampAddress, clampEnd);
RangeActionWithMigration(clampAddress, clampEnd - clampAddress, waitSync, overlap.Parent, _flushAction);
RangeActionWithMigration(clampAddress, clampEnd - clampAddress, waitSync, _flushAction);
}
}
@ -329,7 +333,7 @@ namespace Ryujinx.Graphics.Gpu.Memory
// There is a migration target to call instead. This can't be changed after set so accessing it outside the lock is fine.
_migrationTarget.Destination.RemoveRangesAndFlush(overlaps, rangeCount, highestDiff, currentSync, address, endAddress);
_migrationTarget.RemoveRangesAndFlush(overlaps, rangeCount, highestDiff, currentSync, address, endAddress);
}
/// <summary>
@ -367,7 +371,7 @@ namespace Ryujinx.Graphics.Gpu.Memory
if (rangeCount == -1)
{
_migrationTarget.Destination.WaitForAndFlushRanges(address, size);
_migrationTarget.WaitForAndFlushRanges(address, size);
return;
}
@ -407,6 +411,9 @@ namespace Ryujinx.Graphics.Gpu.Memory
/// <summary>
/// Inherit ranges from another modified range list.
/// </summary>
/// <remarks>
/// Assumes that ranges will be inherited in address ascending order.
/// </remarks>
/// <param name="ranges">The range list to inherit from</param>
/// <param name="registerRangeAction">The action to call for each modified range</param>
public void InheritRanges(BufferModifiedRangeList ranges, Action<ulong, ulong> registerRangeAction)
@ -415,18 +422,31 @@ namespace Ryujinx.Graphics.Gpu.Memory
lock (ranges._lock)
{
BufferMigration migration = new(ranges._parent, ranges._flushAction, ranges, this, _context.SyncNumber);
ranges._parent.IncrementReferenceCount();
ranges._migrationTarget = migration;
_context.RegisterBufferMigration(migration);
inheritRanges = ranges.ToArray();
lock (_lock)
{
(_sources ??= new List<BufferMigration>()).Add(migration);
// Copy over the migration from the previous range list
BufferMigration oldMigration = ranges._source;
BufferMigrationSpan span = new BufferMigrationSpan(ranges._parent, ranges._flushAction, oldMigration);
ranges._parent.IncrementReferenceCount();
if (_source == null)
{
// Create a new migration.
_source = new BufferMigration(new BufferMigrationSpan[] { span }, this, _context.SyncNumber);
_context.RegisterBufferMigration(_source);
}
else
{
// Extend the migration
_source.AddSpanToEnd(span);
}
ranges._migrationTarget = this;
foreach (BufferModifiedRange range in inheritRanges)
{
@ -445,6 +465,27 @@ namespace Ryujinx.Graphics.Gpu.Memory
}
}
/// <summary>
/// Register a migration from previous buffer storage. This migration is from a snapshot of the buffer's
/// current handle to its handle in the future, and is assumed to be complete when the sync action completes.
/// When the migration completes, the handle is disposed.
/// </summary>
public void SelfMigration()
{
lock (_lock)
{
BufferMigrationSpan span = new(_parent, _parent.GetSnapshotDisposeAction(), _parent.GetSnapshotFlushAction(), _source);
BufferMigration migration = new(new BufferMigrationSpan[] { span }, this, _context.SyncNumber);
// Migration target is used to redirect flush actions to the latest range list,
// so we don't need to set it here. (this range list is still the latest)
_context.RegisterBufferMigration(migration);
_source = migration;
}
}
/// <summary>
/// Removes a source buffer migration, indicating its copy has completed.
/// </summary>
@ -453,7 +494,10 @@ namespace Ryujinx.Graphics.Gpu.Memory
{
lock (_lock)
{
_sources.Remove(migration);
if (_source == migration)
{
_source = null;
}
}
}

View File

@ -0,0 +1,295 @@
using Ryujinx.Common;
using Ryujinx.Graphics.GAL;
using System;
namespace Ryujinx.Graphics.Gpu.Memory
{
/// <summary>
/// Manages flushing ranges from buffers in advance for easy access, if they are flushed often.
/// Typically, from device local memory to a host mapped target for cached access.
/// </summary>
internal class BufferPreFlush : IDisposable
{
private const ulong PageSize = MemoryManager.PageSize;
/// <summary>
/// Threshold for the number of copies without a flush required to disable preflush on a page.
/// </summary>
private const int DeactivateCopyThreshold = 200;
/// <summary>
/// Value that indicates whether a page has been flushed or copied before.
/// </summary>
private enum PreFlushState
{
None,
HasFlushed,
HasCopied
}
/// <summary>
/// Flush state for each page of the buffer.
/// Controls whether data should be copied to the flush buffer, what sync is expected
/// and unflushed copy counting for stopping copies that are no longer needed.
/// </summary>
private struct PreFlushPage
{
public PreFlushState State;
public ulong FirstActivatedSync;
public ulong LastCopiedSync;
public int CopyCount;
}
/// <summary>
/// True if there are ranges that should copy to the flush buffer, false otherwise.
/// </summary>
public bool ShouldCopy { get; private set; }
private readonly GpuContext _context;
private readonly Buffer _buffer;
private readonly PreFlushPage[] _pages;
private readonly ulong _address;
private readonly ulong _size;
private readonly ulong _misalignment;
private readonly Action<BufferHandle, ulong, ulong> _flushAction;
private BufferHandle _flushBuffer;
public BufferPreFlush(GpuContext context, Buffer parent, Action<BufferHandle, ulong, ulong> flushAction)
{
_context = context;
_buffer = parent;
_address = parent.Address;
_size = parent.Size;
_pages = new PreFlushPage[BitUtils.DivRoundUp(_size, PageSize)];
_misalignment = _address & (PageSize - 1);
_flushAction = flushAction;
}
/// <summary>
/// Ensure that the flush buffer exists.
/// </summary>
private void EnsureFlushBuffer()
{
if (_flushBuffer == BufferHandle.Null)
{
_flushBuffer = _context.Renderer.CreateBuffer((int)_size, BufferAccess.HostMemory);
}
}
/// <summary>
/// Gets a page range from an address and size byte range.
/// </summary>
/// <param name="address">Range address</param>
/// <param name="size">Range size</param>
/// <returns>A page index and count</returns>
private (int index, int count) GetPageRange(ulong address, ulong size)
{
ulong offset = address - _address;
ulong endOffset = offset + size;
int basePage = (int)(offset / PageSize);
int endPage = (int)((endOffset - 1) / PageSize);
return (basePage, 1 + endPage - basePage);
}
/// <summary>
/// Gets an offset and size range in the parent buffer from a page index and count.
/// </summary>
/// <param name="startPage">Range start page</param>
/// <param name="count">Range page count</param>
/// <returns>Offset and size range</returns>
private (int offset, int size) GetOffset(int startPage, int count)
{
int offset = (int)((ulong)startPage * PageSize - _misalignment);
int endOffset = (int)((ulong)(startPage + count) * PageSize - _misalignment);
offset = Math.Max(0, offset);
endOffset = Math.Min((int)_size, endOffset);
return (offset, endOffset - offset);
}
/// <summary>
/// Copy a range of pages from the parent buffer into the flush buffer.
/// </summary>
/// <param name="startPage">Range start page</param>
/// <param name="count">Range page count</param>
private void CopyPageRange(int startPage, int count)
{
(int offset, int size) = GetOffset(startPage, count);
EnsureFlushBuffer();
_context.Renderer.Pipeline.CopyBuffer(_buffer.Handle, _flushBuffer, offset, offset, size);
}
/// <summary>
/// Copy a modified range into the flush buffer if it's marked as flushed.
/// Any pages the range overlaps are copied, and copies aren't repeated in the same sync number.
/// </summary>
/// <param name="address">Range address</param>
/// <param name="size">Range size</param>
public void CopyModified(ulong address, ulong size)
{
(int baseIndex, int count) = GetPageRange(address, size);
ulong syncNumber = _context.SyncNumber;
int startPage = -1;
for (int i = 0; i < count; i++)
{
int pageIndex = baseIndex + i;
ref PreFlushPage page = ref _pages[pageIndex];
if (page.State > PreFlushState.None)
{
// Perform the copy, and update the state of each page.
if (startPage == -1)
{
startPage = pageIndex;
}
if (page.State != PreFlushState.HasCopied)
{
page.FirstActivatedSync = syncNumber;
page.State = PreFlushState.HasCopied;
}
else if (page.CopyCount++ >= DeactivateCopyThreshold)
{
page.CopyCount = 0;
page.State = PreFlushState.None;
}
if (page.LastCopiedSync != syncNumber)
{
page.LastCopiedSync = syncNumber;
}
}
else if (startPage != -1)
{
CopyPageRange(startPage, pageIndex - startPage);
startPage = -1;
}
}
if (startPage != -1)
{
CopyPageRange(startPage, (baseIndex + count) - startPage);
}
}
/// <summary>
/// Flush the given page range back into guest memory, optionally using data from the flush buffer.
/// The actual flushed range is an intersection of the page range and the address range.
/// </summary>
/// <param name="address">Address range start</param>
/// <param name="size">Address range size</param>
/// <param name="startPage">Page range start</param>
/// <param name="count">Page range count</param>
/// <param name="preFlush">True if the data should come from the flush buffer</param>
private void FlushPageRange(ulong address, ulong size, int startPage, int count, bool preFlush)
{
(int pageOffset, int pageSize) = GetOffset(startPage, count);
int offset = (int)(address - _address);
int end = offset + (int)size;
offset = Math.Max(offset, pageOffset);
end = Math.Min(end, pageOffset + pageSize);
if (end >= offset)
{
BufferHandle handle = preFlush ? _flushBuffer : _buffer.Handle;
_flushAction(handle, _address + (ulong)offset, (ulong)(end - offset));
}
}
/// <summary>
/// Flush the given address range back into guest memory, optionally using data from the flush buffer.
/// When a copy has been performed on or before the waited sync number, the data can come from the flush buffer.
/// Otherwise, it flushes the parent buffer directly.
/// </summary>
/// <param name="address">Range address</param>
/// <param name="size">Range size</param>
/// <param name="syncNumber">Sync number that has been waited for</param>
public void FlushWithAction(ulong address, ulong size, ulong syncNumber)
{
// Copy the parts of the range that have pre-flush copies that have been completed.
// Run the flush action for ranges that don't have pre-flush copies.
// If a range doesn't have a pre-flush copy, consider adding one.
(int baseIndex, int count) = GetPageRange(address, size);
bool rangePreFlushed = false;
int startPage = -1;
for (int i = 0; i < count; i++)
{
int pageIndex = baseIndex + i;
ref PreFlushPage page = ref _pages[pageIndex];
bool flushPage = false;
page.CopyCount = 0;
if (page.State == PreFlushState.HasCopied)
{
if (syncNumber >= page.FirstActivatedSync)
{
// After the range is first activated, its data will always be copied to the preflush buffer on each sync.
flushPage = true;
}
}
else if (page.State == PreFlushState.None)
{
page.State = PreFlushState.HasFlushed;
ShouldCopy = true;
}
if (flushPage)
{
if (!rangePreFlushed || startPage == -1)
{
if (startPage != -1)
{
FlushPageRange(address, size, startPage, pageIndex - startPage, false);
}
rangePreFlushed = true;
startPage = pageIndex;
}
}
else if (rangePreFlushed || startPage == -1)
{
if (startPage != -1)
{
FlushPageRange(address, size, startPage, pageIndex - startPage, true);
}
rangePreFlushed = false;
startPage = pageIndex;
}
}
if (startPage != -1)
{
FlushPageRange(address, size, startPage, (baseIndex + count) - startPage, rangePreFlushed);
}
}
/// <summary>
/// Dispose the flush buffer, if present.
/// </summary>
public void Dispose()
{
if (_flushBuffer != BufferHandle.Null)
{
_context.Renderer.DeleteBuffer(_flushBuffer);
}
}
}
}

View File

@ -0,0 +1,99 @@
using Ryujinx.Graphics.Shader;
using System.Runtime.CompilerServices;
namespace Ryujinx.Graphics.Gpu.Memory
{
/// <summary>
/// Pipeline stages that can modify buffer data, as well as flags indicating storage usage.
/// Must match ShaderStage for the shader stages, though anything after that can be in any order.
/// </summary>
internal enum BufferStage : byte
{
Compute,
Vertex,
TessellationControl,
TessellationEvaluation,
Geometry,
Fragment,
Indirect,
VertexBuffer,
IndexBuffer,
Copy,
TransformFeedback,
Internal,
None,
StageMask = 0x3f,
StorageMask = 0xc0,
StorageRead = 0x40,
StorageWrite = 0x80,
#pragma warning disable CA1069 // Enums values should not be duplicated
StorageAtomic = 0xc0
#pragma warning restore CA1069 // Enums values should not be duplicated
}
/// <summary>
/// Utility methods to convert shader stages and binding flags into buffer stages.
/// </summary>
internal static class BufferStageUtils
{
[MethodImpl(MethodImplOptions.AggressiveInlining)]
public static BufferStage FromShaderStage(ShaderStage stage)
{
return (BufferStage)stage;
}
[MethodImpl(MethodImplOptions.AggressiveInlining)]
public static BufferStage FromShaderStage(int stageIndex)
{
return (BufferStage)(stageIndex + 1);
}
[MethodImpl(MethodImplOptions.AggressiveInlining)]
public static BufferStage FromUsage(BufferUsageFlags flags)
{
if (flags.HasFlag(BufferUsageFlags.Write))
{
return BufferStage.StorageWrite;
}
else
{
return BufferStage.StorageRead;
}
}
[MethodImpl(MethodImplOptions.AggressiveInlining)]
public static BufferStage FromUsage(TextureUsageFlags flags)
{
if (flags.HasFlag(TextureUsageFlags.ImageStore))
{
return BufferStage.StorageWrite;
}
else
{
return BufferStage.StorageRead;
}
}
[MethodImpl(MethodImplOptions.AggressiveInlining)]
public static BufferStage TextureBuffer(ShaderStage shaderStage, TextureUsageFlags flags)
{
return FromShaderStage(shaderStage) | FromUsage(flags);
}
[MethodImpl(MethodImplOptions.AggressiveInlining)]
public static BufferStage GraphicsStorage(int stageIndex, BufferUsageFlags flags)
{
return FromShaderStage(stageIndex) | FromUsage(flags);
}
[MethodImpl(MethodImplOptions.AggressiveInlining)]
public static BufferStage ComputeStorage(BufferUsageFlags flags)
{
return BufferStage.Compute | FromUsage(flags);
}
}
}

View File

@ -62,6 +62,7 @@ namespace Ryujinx.Graphics.Gpu.Shader
var result = new TextureBindingInfo(
target,
descriptor.Set,
descriptor.Binding,
descriptor.ArrayLength,
descriptor.CbufSlot,
@ -90,6 +91,7 @@ namespace Ryujinx.Graphics.Gpu.Shader
var result = new TextureBindingInfo(
target,
format,
descriptor.Set,
descriptor.Binding,
descriptor.ArrayLength,
descriptor.CbufSlot,

View File

@ -125,9 +125,18 @@ namespace Ryujinx.Graphics.Gpu.Shader.DiskCache
CompressionAlgorithm algorithm = CompressionAlgorithm.None;
Read(ref algorithm);
if (algorithm == CompressionAlgorithm.Deflate)
switch (algorithm)
{
_activeStream = new DeflateStream(_stream, CompressionMode.Decompress, true);
case CompressionAlgorithm.None:
break;
case CompressionAlgorithm.Deflate:
_activeStream = new DeflateStream(_stream, CompressionMode.Decompress, true);
break;
case CompressionAlgorithm.Brotli:
_activeStream = new BrotliStream(_stream, CompressionMode.Decompress, true);
break;
default:
throw new ArgumentException($"Invalid compression algorithm \"{algorithm}\"");
}
}
@ -139,9 +148,18 @@ namespace Ryujinx.Graphics.Gpu.Shader.DiskCache
{
Write(ref algorithm);
if (algorithm == CompressionAlgorithm.Deflate)
switch (algorithm)
{
_activeStream = new DeflateStream(_stream, CompressionLevel.Fastest, true);
case CompressionAlgorithm.None:
break;
case CompressionAlgorithm.Deflate:
_activeStream = new DeflateStream(_stream, CompressionLevel.Fastest, true);
break;
case CompressionAlgorithm.Brotli:
_activeStream = new BrotliStream(_stream, CompressionLevel.Fastest, true);
break;
default:
throw new ArgumentException($"Invalid compression algorithm \"{algorithm}\"");
}
}
@ -177,7 +195,7 @@ namespace Ryujinx.Graphics.Gpu.Shader.DiskCache
switch (algorithm)
{
case CompressionAlgorithm.None:
stream.Read(data);
stream.ReadExactly(data);
break;
case CompressionAlgorithm.Deflate:
stream = new DeflateStream(stream, CompressionMode.Decompress, true);
@ -187,6 +205,14 @@ namespace Ryujinx.Graphics.Gpu.Shader.DiskCache
}
stream.Dispose();
break;
case CompressionAlgorithm.Brotli:
stream = new BrotliStream(stream, CompressionMode.Decompress, true);
for (int offset = 0; offset < data.Length;)
{
offset += stream.Read(data[offset..]);
}
stream.Dispose();
break;
}
}
@ -210,6 +236,11 @@ namespace Ryujinx.Graphics.Gpu.Shader.DiskCache
stream.Write(data);
stream.Dispose();
break;
case CompressionAlgorithm.Brotli:
stream = new BrotliStream(stream, CompressionLevel.Fastest, true);
stream.Write(data);
stream.Dispose();
break;
}
}
}

View File

@ -14,5 +14,10 @@ namespace Ryujinx.Graphics.Gpu.Shader.DiskCache
/// Deflate compression (RFC 1951).
/// </summary>
Deflate,
/// <summary>
/// Brotli compression (RFC 7932).
/// </summary>
Brotli,
}
}

View File

@ -51,7 +51,7 @@ namespace Ryujinx.Graphics.Gpu.Shader.DiskCache
/// <returns>Compression algorithm</returns>
public static CompressionAlgorithm GetCompressionAlgorithm()
{
return CompressionAlgorithm.Deflate;
return CompressionAlgorithm.Brotli;
}
}
}

View File

@ -220,7 +220,7 @@ namespace Ryujinx.Graphics.Gpu.Shader.DiskCache
}
dataFileStream.Seek((long)entry.Offset, SeekOrigin.Begin);
dataFileStream.Read(cb1Data);
dataFileStream.ReadExactly(cb1Data);
BinarySerializer.ReadCompressed(dataFileStream, guestCode);
_cache[index] = (guestCode, cb1Data);
@ -279,7 +279,7 @@ namespace Ryujinx.Graphics.Gpu.Shader.DiskCache
dataFileStream.Seek((long)entry.Offset, SeekOrigin.Begin);
byte[] cachedCode = new byte[entry.CodeSize];
byte[] cachedCb1Data = new byte[entry.Cb1DataSize];
dataFileStream.Read(cachedCb1Data);
dataFileStream.ReadExactly(cachedCb1Data);
BinarySerializer.ReadCompressed(dataFileStream, cachedCode);
if (data.SequenceEqual(cachedCode) && cb1Data.SequenceEqual(cachedCb1Data))

View File

@ -22,7 +22,7 @@ namespace Ryujinx.Graphics.Gpu.Shader.DiskCache
private const ushort FileFormatVersionMajor = 1;
private const ushort FileFormatVersionMinor = 2;
private const uint FileFormatVersionPacked = ((uint)FileFormatVersionMajor << 16) | FileFormatVersionMinor;
private const uint CodeGenVersion = 5936;
private const uint CodeGenVersion = 6852;
private const string SharedTocFileName = "shared.toc";
private const string SharedDataFileName = "shared.data";

View File

@ -51,7 +51,7 @@ namespace Ryujinx.Graphics.Gpu.Shader
_reservedImages = rrc.ReservedImages;
}
public int CreateConstantBufferBinding(int index)
public SetBindingPair CreateConstantBufferBinding(int index)
{
int binding;
@ -64,10 +64,10 @@ namespace Ryujinx.Graphics.Gpu.Shader
binding = _resourceCounts.UniformBuffersCount++;
}
return binding + _reservedConstantBuffers;
return new SetBindingPair(_context.Capabilities.UniformBufferSetIndex, binding + _reservedConstantBuffers);
}
public int CreateImageBinding(int count, bool isBuffer)
public SetBindingPair CreateImageBinding(int count, bool isBuffer)
{
int binding;
@ -96,10 +96,10 @@ namespace Ryujinx.Graphics.Gpu.Shader
_resourceCounts.ImagesCount += count;
}
return binding + _reservedImages;
return new SetBindingPair(_context.Capabilities.ImageSetIndex, binding + _reservedImages);
}
public int CreateStorageBufferBinding(int index)
public SetBindingPair CreateStorageBufferBinding(int index)
{
int binding;
@ -112,10 +112,10 @@ namespace Ryujinx.Graphics.Gpu.Shader
binding = _resourceCounts.StorageBuffersCount++;
}
return binding + _reservedStorageBuffers;
return new SetBindingPair(_context.Capabilities.StorageBufferSetIndex, binding + _reservedStorageBuffers);
}
public int CreateTextureBinding(int count, bool isBuffer)
public SetBindingPair CreateTextureBinding(int count, bool isBuffer)
{
int binding;
@ -144,7 +144,7 @@ namespace Ryujinx.Graphics.Gpu.Shader
_resourceCounts.TexturesCount += count;
}
return binding + _reservedTextures;
return new SetBindingPair(_context.Capabilities.TextureSetIndex, binding + _reservedTextures);
}
private int GetBindingFromIndex(int index, uint maxPerStage, string resourceName)
@ -183,6 +183,16 @@ namespace Ryujinx.Graphics.Gpu.Shader
return maxPerStage * Constants.ShaderStages;
}
public int CreateExtraSet()
{
if (_resourceCounts.SetsCount >= _context.Capabilities.MaximumExtraSets)
{
return -1;
}
return _context.Capabilities.ExtraSetBaseIndex + _resourceCounts.SetsCount++;
}
public int QueryHostGatherBiasPrecision() => _context.Capabilities.GatherBiasPrecision;
public bool QueryHostReducedPrecision() => _context.Capabilities.ReduceShaderPrecision;

View File

@ -24,5 +24,10 @@ namespace Ryujinx.Graphics.Gpu.Shader
/// Total of images used by the shaders.
/// </summary>
public int ImagesCount;
/// <summary>
/// Total of extra sets used by the shaders.
/// </summary>
public int SetsCount;
}
}

View File

@ -1,5 +1,6 @@
using Ryujinx.Graphics.GAL;
using Ryujinx.Graphics.Shader;
using System;
using System.Collections.Generic;
namespace Ryujinx.Graphics.Gpu.Shader
@ -9,13 +10,6 @@ namespace Ryujinx.Graphics.Gpu.Shader
/// </summary>
class ShaderInfoBuilder
{
private const int TotalSets = 4;
private const int UniformSetIndex = 0;
private const int StorageSetIndex = 1;
private const int TextureSetIndex = 2;
private const int ImageSetIndex = 3;
private const ResourceStages SupportBufferStages =
ResourceStages.Compute |
ResourceStages.Vertex |
@ -36,8 +30,8 @@ namespace Ryujinx.Graphics.Gpu.Shader
private readonly int _reservedTextures;
private readonly int _reservedImages;
private readonly List<ResourceDescriptor>[] _resourceDescriptors;
private readonly List<ResourceUsage>[] _resourceUsages;
private List<ResourceDescriptor>[] _resourceDescriptors;
private List<ResourceUsage>[] _resourceUsages;
/// <summary>
/// Creates a new shader info builder.
@ -51,17 +45,27 @@ namespace Ryujinx.Graphics.Gpu.Shader
_fragmentOutputMap = -1;
_resourceDescriptors = new List<ResourceDescriptor>[TotalSets];
_resourceUsages = new List<ResourceUsage>[TotalSets];
int uniformSetIndex = context.Capabilities.UniformBufferSetIndex;
int storageSetIndex = context.Capabilities.StorageBufferSetIndex;
int textureSetIndex = context.Capabilities.TextureSetIndex;
int imageSetIndex = context.Capabilities.ImageSetIndex;
for (int index = 0; index < TotalSets; index++)
int totalSets = Math.Max(uniformSetIndex, storageSetIndex);
totalSets = Math.Max(totalSets, textureSetIndex);
totalSets = Math.Max(totalSets, imageSetIndex);
totalSets++;
_resourceDescriptors = new List<ResourceDescriptor>[totalSets];
_resourceUsages = new List<ResourceUsage>[totalSets];
for (int index = 0; index < totalSets; index++)
{
_resourceDescriptors[index] = new();
_resourceUsages[index] = new();
}
AddDescriptor(SupportBufferStages, ResourceType.UniformBuffer, UniformSetIndex, 0, 1);
AddUsage(SupportBufferStages, ResourceType.UniformBuffer, UniformSetIndex, 0, 1);
AddDescriptor(SupportBufferStages, ResourceType.UniformBuffer, uniformSetIndex, 0, 1);
AddUsage(SupportBufferStages, ResourceType.UniformBuffer, uniformSetIndex, 0, 1);
ResourceReservationCounts rrc = new(!context.Capabilities.SupportsTransformFeedback && tfEnabled, vertexAsCompute);
@ -73,12 +77,20 @@ namespace Ryujinx.Graphics.Gpu.Shader
// TODO: Handle that better? Maybe we should only set the binding that are really needed on each shader.
ResourceStages stages = vertexAsCompute ? ResourceStages.Compute | ResourceStages.Vertex : VtgStages;
PopulateDescriptorAndUsages(stages, ResourceType.UniformBuffer, UniformSetIndex, 1, rrc.ReservedConstantBuffers - 1);
PopulateDescriptorAndUsages(stages, ResourceType.StorageBuffer, StorageSetIndex, 0, rrc.ReservedStorageBuffers);
PopulateDescriptorAndUsages(stages, ResourceType.BufferTexture, TextureSetIndex, 0, rrc.ReservedTextures);
PopulateDescriptorAndUsages(stages, ResourceType.BufferImage, ImageSetIndex, 0, rrc.ReservedImages);
PopulateDescriptorAndUsages(stages, ResourceType.UniformBuffer, uniformSetIndex, 1, rrc.ReservedConstantBuffers - 1);
PopulateDescriptorAndUsages(stages, ResourceType.StorageBuffer, storageSetIndex, 0, rrc.ReservedStorageBuffers);
PopulateDescriptorAndUsages(stages, ResourceType.BufferTexture, textureSetIndex, 0, rrc.ReservedTextures);
PopulateDescriptorAndUsages(stages, ResourceType.BufferImage, imageSetIndex, 0, rrc.ReservedImages);
}
/// <summary>
/// Populates descriptors and usages for vertex as compute and transform feedback emulation reserved resources.
/// </summary>
/// <param name="stages">Shader stages where the resources are used</param>
/// <param name="type">Resource type</param>
/// <param name="setIndex">Resource set index where the resources are used</param>
/// <param name="start">First binding number</param>
/// <param name="count">Amount of bindings</param>
private void PopulateDescriptorAndUsages(ResourceStages stages, ResourceType type, int setIndex, int start, int count)
{
AddDescriptor(stages, type, setIndex, start, count);
@ -127,18 +139,23 @@ namespace Ryujinx.Graphics.Gpu.Shader
int textureBinding = _reservedTextures + stageIndex * texturesPerStage * 2;
int imageBinding = _reservedImages + stageIndex * imagesPerStage * 2;
AddDescriptor(stages, ResourceType.UniformBuffer, UniformSetIndex, uniformBinding, uniformsPerStage);
AddDescriptor(stages, ResourceType.StorageBuffer, StorageSetIndex, storageBinding, storagesPerStage);
AddDualDescriptor(stages, ResourceType.TextureAndSampler, ResourceType.BufferTexture, TextureSetIndex, textureBinding, texturesPerStage);
AddDualDescriptor(stages, ResourceType.Image, ResourceType.BufferImage, ImageSetIndex, imageBinding, imagesPerStage);
int uniformSetIndex = _context.Capabilities.UniformBufferSetIndex;
int storageSetIndex = _context.Capabilities.StorageBufferSetIndex;
int textureSetIndex = _context.Capabilities.TextureSetIndex;
int imageSetIndex = _context.Capabilities.ImageSetIndex;
AddArrayDescriptors(info.Textures, stages, TextureSetIndex, isImage: false);
AddArrayDescriptors(info.Images, stages, TextureSetIndex, isImage: true);
AddDescriptor(stages, ResourceType.UniformBuffer, uniformSetIndex, uniformBinding, uniformsPerStage);
AddDescriptor(stages, ResourceType.StorageBuffer, storageSetIndex, storageBinding, storagesPerStage);
AddDualDescriptor(stages, ResourceType.TextureAndSampler, ResourceType.BufferTexture, textureSetIndex, textureBinding, texturesPerStage);
AddDualDescriptor(stages, ResourceType.Image, ResourceType.BufferImage, imageSetIndex, imageBinding, imagesPerStage);
AddUsage(info.CBuffers, stages, UniformSetIndex, isStorage: false);
AddUsage(info.SBuffers, stages, StorageSetIndex, isStorage: true);
AddUsage(info.Textures, stages, TextureSetIndex, isImage: false);
AddUsage(info.Images, stages, ImageSetIndex, isImage: true);
AddArrayDescriptors(info.Textures, stages, isImage: false);
AddArrayDescriptors(info.Images, stages, isImage: true);
AddUsage(info.CBuffers, stages, isStorage: false);
AddUsage(info.SBuffers, stages, isStorage: true);
AddUsage(info.Textures, stages, isImage: false);
AddUsage(info.Images, stages, isImage: true);
}
/// <summary>
@ -177,9 +194,8 @@ namespace Ryujinx.Graphics.Gpu.Shader
/// </summary>
/// <param name="textures">Textures to be added</param>
/// <param name="stages">Stages where the textures are used</param>
/// <param name="setIndex">Descriptor set index where the textures will be bound</param>
/// <param name="isImage">True for images, false for textures</param>
private void AddArrayDescriptors(IEnumerable<TextureDescriptor> textures, ResourceStages stages, int setIndex, bool isImage)
private void AddArrayDescriptors(IEnumerable<TextureDescriptor> textures, ResourceStages stages, bool isImage)
{
foreach (TextureDescriptor texture in textures)
{
@ -187,7 +203,7 @@ namespace Ryujinx.Graphics.Gpu.Shader
{
ResourceType type = GetTextureResourceType(texture, isImage);
_resourceDescriptors[setIndex].Add(new ResourceDescriptor(texture.Binding, texture.ArrayLength, type, stages));
GetDescriptors(texture.Set).Add(new ResourceDescriptor(texture.Binding, texture.ArrayLength, type, stages));
}
}
}
@ -213,13 +229,12 @@ namespace Ryujinx.Graphics.Gpu.Shader
/// </summary>
/// <param name="buffers">Buffers to be added</param>
/// <param name="stages">Stages where the buffers are used</param>
/// <param name="setIndex">Descriptor set index where the buffers will be bound</param>
/// <param name="isStorage">True for storage buffers, false for uniform buffers</param>
private void AddUsage(IEnumerable<BufferDescriptor> buffers, ResourceStages stages, int setIndex, bool isStorage)
private void AddUsage(IEnumerable<BufferDescriptor> buffers, ResourceStages stages, bool isStorage)
{
foreach (BufferDescriptor buffer in buffers)
{
_resourceUsages[setIndex].Add(new ResourceUsage(
GetUsages(buffer.Set).Add(new ResourceUsage(
buffer.Binding,
1,
isStorage ? ResourceType.StorageBuffer : ResourceType.UniformBuffer,
@ -232,18 +247,65 @@ namespace Ryujinx.Graphics.Gpu.Shader
/// </summary>
/// <param name="textures">Textures to be added</param>
/// <param name="stages">Stages where the textures are used</param>
/// <param name="setIndex">Descriptor set index where the textures will be bound</param>
/// <param name="isImage">True for images, false for textures</param>
private void AddUsage(IEnumerable<TextureDescriptor> textures, ResourceStages stages, int setIndex, bool isImage)
private void AddUsage(IEnumerable<TextureDescriptor> textures, ResourceStages stages, bool isImage)
{
foreach (TextureDescriptor texture in textures)
{
ResourceType type = GetTextureResourceType(texture, isImage);
_resourceUsages[setIndex].Add(new ResourceUsage(texture.Binding, texture.ArrayLength, type, stages));
GetUsages(texture.Set).Add(new ResourceUsage(texture.Binding, texture.ArrayLength, type, stages));
}
}
/// <summary>
/// Gets the list of resource descriptors for a given set index. A new list will be created if needed.
/// </summary>
/// <param name="setIndex">Resource set index</param>
/// <returns>List of resource descriptors</returns>
private List<ResourceDescriptor> GetDescriptors(int setIndex)
{
if (_resourceDescriptors.Length <= setIndex)
{
int oldLength = _resourceDescriptors.Length;
Array.Resize(ref _resourceDescriptors, setIndex + 1);
for (int index = oldLength; index <= setIndex; index++)
{
_resourceDescriptors[index] = new();
}
}
return _resourceDescriptors[setIndex];
}
/// <summary>
/// Gets the list of resource usages for a given set index. A new list will be created if needed.
/// </summary>
/// <param name="setIndex">Resource set index</param>
/// <returns>List of resource usages</returns>
private List<ResourceUsage> GetUsages(int setIndex)
{
if (_resourceUsages.Length <= setIndex)
{
int oldLength = _resourceUsages.Length;
Array.Resize(ref _resourceUsages, setIndex + 1);
for (int index = oldLength; index <= setIndex; index++)
{
_resourceUsages[index] = new();
}
}
return _resourceUsages[setIndex];
}
/// <summary>
/// Gets a resource type from a texture descriptor.
/// </summary>
/// <param name="texture">Texture descriptor</param>
/// <param name="isImage">Whether the texture is a image texture (writable) or not (sampled)</param>
/// <returns>Resource type</returns>
private static ResourceType GetTextureResourceType(TextureDescriptor texture, bool isImage)
{
bool isBuffer = (texture.Type & SamplerType.Mask) == SamplerType.TextureBuffer;
@ -278,10 +340,12 @@ namespace Ryujinx.Graphics.Gpu.Shader
/// <returns>Shader information</returns>
public ShaderInfo Build(ProgramPipelineState? pipeline, bool fromCache = false)
{
var descriptors = new ResourceDescriptorCollection[TotalSets];
var usages = new ResourceUsageCollection[TotalSets];
int totalSets = _resourceDescriptors.Length;
for (int index = 0; index < TotalSets; index++)
var descriptors = new ResourceDescriptorCollection[totalSets];
var usages = new ResourceUsageCollection[totalSets];
for (int index = 0; index < totalSets; index++)
{
descriptors[index] = new ResourceDescriptorCollection(_resourceDescriptors[index].ToArray().AsReadOnly());
usages[index] = new ResourceUsageCollection(_resourceUsages[index].ToArray().AsReadOnly());

View File

@ -63,5 +63,9 @@ namespace Ryujinx.Graphics.OpenGL.Image
}
}
}
public void Dispose()
{
}
}
}

View File

@ -48,5 +48,9 @@ namespace Ryujinx.Graphics.OpenGL.Image
}
}
}
public void Dispose()
{
}
}
}

View File

@ -61,7 +61,9 @@ namespace Ryujinx.Graphics.OpenGL
{
BufferCount++;
if (access.HasFlag(GAL.BufferAccess.FlushPersistent))
var memType = access & GAL.BufferAccess.MemoryTypeMask;
if (memType == GAL.BufferAccess.HostMemory)
{
BufferHandle handle = Buffer.CreatePersistent(size);
@ -75,11 +77,6 @@ namespace Ryujinx.Graphics.OpenGL
}
}
public BufferHandle CreateBuffer(int size, GAL.BufferAccess access, BufferHandle storageHint)
{
return CreateBuffer(size, access);
}
public BufferHandle CreateBuffer(nint pointer, int size)
{
throw new NotSupportedException();
@ -148,6 +145,7 @@ namespace Ryujinx.Graphics.OpenGL
return new Capabilities(
api: TargetApi.OpenGL,
vendorName: GpuVendor,
memoryType: SystemMemoryType.BackendManaged,
hasFrontFacingBug: intelWindows,
hasVectorIndexingBug: amdWindows,
needsFragmentOutputSpecialization: false,
@ -189,6 +187,12 @@ namespace Ryujinx.Graphics.OpenGL
supportsViewportSwizzle: HwCapabilities.SupportsViewportSwizzle,
supportsIndirectParameters: HwCapabilities.SupportsIndirectParameters,
supportsDepthClipControl: true,
uniformBufferSetIndex: 0,
storageBufferSetIndex: 1,
textureSetIndex: 2,
imageSetIndex: 3,
extraSetBaseIndex: 0,
maximumExtraSets: 0,
maximumUniformBuffersPerStage: 13, // TODO: Avoid hardcoding those limits here and get from driver?
maximumStorageBuffersPerStage: 16,
maximumTexturesPerStage: 32,

View File

@ -963,6 +963,11 @@ namespace Ryujinx.Graphics.OpenGL
(array as ImageArray).Bind(binding);
}
public void SetImageArraySeparate(ShaderStage stage, int setIndex, IImageArray array)
{
throw new NotSupportedException("OpenGL does not support descriptor sets.");
}
public void SetIndexBuffer(BufferRange buffer, IndexType type)
{
_elementsType = type.Convert();
@ -1312,6 +1317,11 @@ namespace Ryujinx.Graphics.OpenGL
(array as TextureArray).Bind(binding);
}
public void SetTextureArraySeparate(ShaderStage stage, int setIndex, ITextureArray array)
{
throw new NotSupportedException("OpenGL does not support descriptor sets.");
}
public void SetTransformFeedbackBuffers(ReadOnlySpan<BufferRange> buffers)
{
if (_tfEnabled)

View File

@ -4,14 +4,16 @@ namespace Ryujinx.Graphics.Shader
{
// New fields should be added to the end of the struct to keep disk shader cache compatibility.
public readonly int Set;
public readonly int Binding;
public readonly byte Slot;
public readonly byte SbCbSlot;
public readonly ushort SbCbOffset;
public readonly BufferUsageFlags Flags;
public BufferDescriptor(int binding, int slot)
public BufferDescriptor(int set, int binding, int slot)
{
Set = set;
Binding = binding;
Slot = (byte)slot;
SbCbSlot = 0;
@ -19,8 +21,9 @@ namespace Ryujinx.Graphics.Shader
Flags = BufferUsageFlags.None;
}
public BufferDescriptor(int binding, int slot, int sbCbSlot, int sbCbOffset, BufferUsageFlags flags)
public BufferDescriptor(int set, int binding, int slot, int sbCbSlot, int sbCbOffset, BufferUsageFlags flags)
{
Set = set;
Binding = binding;
Slot = (byte)slot;
SbCbSlot = (byte)sbCbSlot;

View File

@ -462,7 +462,7 @@ namespace Ryujinx.Graphics.Shader.CodeGen.Glsl.Instructions
}
else
{
context.Properties.Textures.TryGetValue(texOp.Binding, out TextureDefinition definition);
context.Properties.Textures.TryGetValue(texOp.GetTextureSetAndBinding(), out TextureDefinition definition);
bool hasLod = !definition.Type.HasFlag(SamplerType.Multisample) && (definition.Type & SamplerType.Mask) != SamplerType.TextureBuffer;
string texCall;
@ -639,7 +639,7 @@ namespace Ryujinx.Graphics.Shader.CodeGen.Glsl.Instructions
private static string GetSamplerName(CodeGenContext context, AstTextureOperation texOp, ref int srcIndex)
{
TextureDefinition textureDefinition = context.Properties.Textures[texOp.Binding];
TextureDefinition textureDefinition = context.Properties.Textures[texOp.GetTextureSetAndBinding()];
string name = textureDefinition.Name;
if (textureDefinition.ArrayLength != 1)
@ -649,7 +649,7 @@ namespace Ryujinx.Graphics.Shader.CodeGen.Glsl.Instructions
if (texOp.IsSeparate)
{
TextureDefinition samplerDefinition = context.Properties.Textures[texOp.SamplerBinding];
TextureDefinition samplerDefinition = context.Properties.Textures[texOp.GetSamplerSetAndBinding()];
string samplerName = samplerDefinition.Name;
if (samplerDefinition.ArrayLength != 1)
@ -665,7 +665,7 @@ namespace Ryujinx.Graphics.Shader.CodeGen.Glsl.Instructions
private static string GetImageName(CodeGenContext context, AstTextureOperation texOp, ref int srcIndex)
{
TextureDefinition definition = context.Properties.Images[texOp.Binding];
TextureDefinition definition = context.Properties.Images[texOp.GetTextureSetAndBinding()];
string name = definition.Name;
if (definition.ArrayLength != 1)

View File

@ -33,9 +33,9 @@ namespace Ryujinx.Graphics.Shader.CodeGen.Spirv
public Dictionary<int, Instruction> LocalMemories { get; } = new();
public Dictionary<int, Instruction> SharedMemories { get; } = new();
public Dictionary<int, SamplerType> SamplersTypes { get; } = new();
public Dictionary<int, SamplerDeclaration> Samplers { get; } = new();
public Dictionary<int, ImageDeclaration> Images { get; } = new();
public Dictionary<SetBindingPair, SamplerType> SamplersTypes { get; } = new();
public Dictionary<SetBindingPair, SamplerDeclaration> Samplers { get; } = new();
public Dictionary<SetBindingPair, ImageDeclaration> Images { get; } = new();
public Dictionary<IoDefinition, Instruction> Inputs { get; } = new();
public Dictionary<IoDefinition, Instruction> Outputs { get; } = new();
@ -98,11 +98,6 @@ namespace Ryujinx.Graphics.Shader.CodeGen.Spirv
Logger = parameters.Logger;
TargetApi = parameters.TargetApi;
AddCapability(Capability.Shader);
AddCapability(Capability.Float64);
SetMemoryModel(AddressingModel.Logical, MemoryModel.GLSL450);
Delegates = new SpirvDelegates(this);
}

View File

@ -208,13 +208,13 @@ namespace Ryujinx.Graphics.Shader.CodeGen.Spirv
var sampledImageVariable = context.Variable(sampledImageArrayPointerType, StorageClass.UniformConstant);
context.Samplers.Add(sampler.Binding, new SamplerDeclaration(
context.Samplers.Add(new(sampler.Set, sampler.Binding), new SamplerDeclaration(
imageType,
sampledImageType,
sampledImagePointerType,
sampledImageVariable,
sampler.ArrayLength != 1));
context.SamplersTypes.Add(sampler.Binding, sampler.Type);
context.SamplersTypes.Add(new(sampler.Set, sampler.Binding), sampler.Type);
context.Name(sampledImageVariable, sampler.Name);
context.Decorate(sampledImageVariable, Decoration.DescriptorSet, (LiteralInteger)setIndex);
@ -256,7 +256,7 @@ namespace Ryujinx.Graphics.Shader.CodeGen.Spirv
var imageVariable = context.Variable(imageArrayPointerType, StorageClass.UniformConstant);
context.Images.Add(image.Binding, new ImageDeclaration(imageType, imagePointerType, imageVariable, image.ArrayLength != 1));
context.Images.Add(new(image.Set, image.Binding), new ImageDeclaration(imageType, imagePointerType, imageVariable, image.ArrayLength != 1));
context.Name(imageVariable, image.Name);
context.Decorate(imageVariable, Decoration.DescriptorSet, (LiteralInteger)setIndex);

View File

@ -602,7 +602,7 @@ namespace Ryujinx.Graphics.Shader.CodeGen.Spirv
return context.Get(type, texOp.GetSource(srcIndex++));
}
ImageDeclaration declaration = context.Images[texOp.Binding];
ImageDeclaration declaration = context.Images[texOp.GetTextureSetAndBinding()];
SpvInstruction image = declaration.Image;
SpvInstruction resultType = context.GetType(componentType);
@ -681,7 +681,7 @@ namespace Ryujinx.Graphics.Shader.CodeGen.Spirv
return context.Get(type, texOp.GetSource(srcIndex++));
}
ImageDeclaration declaration = context.Images[texOp.Binding];
ImageDeclaration declaration = context.Images[texOp.GetTextureSetAndBinding()];
SpvInstruction image = declaration.Image;
if (declaration.IsIndexed)
@ -738,7 +738,7 @@ namespace Ryujinx.Graphics.Shader.CodeGen.Spirv
return context.Get(type, texOp.GetSource(srcIndex++));
}
ImageDeclaration declaration = context.Images[texOp.Binding];
ImageDeclaration declaration = context.Images[texOp.GetTextureSetAndBinding()];
SpvInstruction image = declaration.Image;
if (declaration.IsIndexed)
@ -837,7 +837,7 @@ namespace Ryujinx.Graphics.Shader.CodeGen.Spirv
return context.Get(type, texOp.GetSource(srcIndex++));
}
SamplerDeclaration declaration = context.Samplers[texOp.Binding];
SamplerDeclaration declaration = context.Samplers[texOp.GetTextureSetAndBinding()];
SpvInstruction image = GenerateSampledImageLoad(context, texOp, declaration, ref srcIndex);
int pCount = texOp.Type.GetDimensions();
@ -1161,7 +1161,7 @@ namespace Ryujinx.Graphics.Shader.CodeGen.Spirv
return context.Get(type, texOp.GetSource(srcIndex++));
}
SamplerDeclaration declaration = context.Samplers[texOp.Binding];
SamplerDeclaration declaration = context.Samplers[texOp.GetTextureSetAndBinding()];
SpvInstruction image = GenerateSampledImageLoad(context, texOp, declaration, ref srcIndex);
int coordsCount = texOp.Type.GetDimensions();
@ -1433,7 +1433,7 @@ namespace Ryujinx.Graphics.Shader.CodeGen.Spirv
int srcIndex = 0;
SamplerDeclaration declaration = context.Samplers[texOp.Binding];
SamplerDeclaration declaration = context.Samplers[texOp.GetTextureSetAndBinding()];
SpvInstruction image = GenerateSampledImageLoad(context, texOp, declaration, ref srcIndex);
image = context.Image(declaration.ImageType, image);
@ -1449,7 +1449,7 @@ namespace Ryujinx.Graphics.Shader.CodeGen.Spirv
int srcIndex = 0;
SamplerDeclaration declaration = context.Samplers[texOp.Binding];
SamplerDeclaration declaration = context.Samplers[texOp.GetTextureSetAndBinding()];
SpvInstruction image = GenerateSampledImageLoad(context, texOp, declaration, ref srcIndex);
image = context.Image(declaration.ImageType, image);
@ -1460,7 +1460,7 @@ namespace Ryujinx.Graphics.Shader.CodeGen.Spirv
}
else
{
var type = context.SamplersTypes[texOp.Binding];
var type = context.SamplersTypes[texOp.GetTextureSetAndBinding()];
bool hasLod = !type.HasFlag(SamplerType.Multisample) && type != SamplerType.TextureBuffer;
int dimensions = (type & SamplerType.Mask) == SamplerType.TextureCube ? 2 : type.GetDimensions();
@ -1889,7 +1889,7 @@ namespace Ryujinx.Graphics.Shader.CodeGen.Spirv
{
image = context.Load(declaration.ImageType, image);
SamplerDeclaration samplerDeclaration = context.Samplers[texOp.SamplerBinding];
SamplerDeclaration samplerDeclaration = context.Samplers[texOp.GetSamplerSetAndBinding()];
SpvInstruction sampler = samplerDeclaration.Image;

View File

@ -43,6 +43,10 @@ namespace Ryujinx.Graphics.Shader.CodeGen.Spirv
CodeGenContext context = new(info, parameters, instPool, integerPool);
context.AddCapability(Capability.Shader);
context.SetMemoryModel(AddressingModel.Logical, MemoryModel.GLSL450);
context.AddCapability(Capability.GroupNonUniformBallot);
context.AddCapability(Capability.GroupNonUniformShuffle);
context.AddCapability(Capability.GroupNonUniformVote);
@ -51,6 +55,11 @@ namespace Ryujinx.Graphics.Shader.CodeGen.Spirv
context.AddCapability(Capability.ImageQuery);
context.AddCapability(Capability.SampledBuffer);
if (parameters.HostCapabilities.SupportsShaderFloat64)
{
context.AddCapability(Capability.Float64);
}
if (parameters.Definitions.TransformFeedbackEnabled && parameters.Definitions.LastInVertexPipeline)
{
context.AddCapability(Capability.TransformFeedback);
@ -58,7 +67,8 @@ namespace Ryujinx.Graphics.Shader.CodeGen.Spirv
if (parameters.Definitions.Stage == ShaderStage.Fragment)
{
if (context.Info.IoDefinitions.Contains(new IoDefinition(StorageKind.Input, IoVariable.Layer)))
if (context.Info.IoDefinitions.Contains(new IoDefinition(StorageKind.Input, IoVariable.Layer)) ||
context.Info.IoDefinitions.Contains(new IoDefinition(StorageKind.Input, IoVariable.PrimitiveId)))
{
context.AddCapability(Capability.Geometry);
}

View File

@ -27,34 +27,43 @@ namespace Ryujinx.Graphics.Shader
ReadOnlySpan<ulong> GetCode(ulong address, int minimumSize);
/// <summary>
/// Queries the binding number of a constant buffer.
/// Gets the binding number of a constant buffer.
/// </summary>
/// <param name="index">Constant buffer index</param>
/// <returns>Binding number</returns>
int CreateConstantBufferBinding(int index);
SetBindingPair CreateConstantBufferBinding(int index);
/// <summary>
/// Queries the binding number of an image.
/// Gets the binding number of an image.
/// </summary>
/// <param name="count">For array of images, the number of elements of the array, otherwise it should be 1</param>
/// <param name="isBuffer">Indicates if the image is a buffer image</param>
/// <returns>Binding number</returns>
int CreateImageBinding(int count, bool isBuffer);
SetBindingPair CreateImageBinding(int count, bool isBuffer);
/// <summary>
/// Queries the binding number of a storage buffer.
/// Gets the binding number of a storage buffer.
/// </summary>
/// <param name="index">Storage buffer index</param>
/// <returns>Binding number</returns>
int CreateStorageBufferBinding(int index);
SetBindingPair CreateStorageBufferBinding(int index);
/// <summary>
/// Queries the binding number of a texture.
/// Gets the binding number of a texture.
/// </summary>
/// <param name="count">For array of textures, the number of elements of the array, otherwise it should be 1</param>
/// <param name="isBuffer">Indicates if the texture is a buffer texture</param>
/// <returns>Binding number</returns>
int CreateTextureBinding(int count, bool isBuffer);
SetBindingPair CreateTextureBinding(int count, bool isBuffer);
/// <summary>
/// Gets the set index for an additional set, or -1 if there's no extra set available.
/// </summary>
/// <returns>Extra set index, or -1 if not available</returns>
int CreateExtraSet()
{
return -1;
}
/// <summary>
/// Queries Local Size X for compute shaders.

View File

@ -278,7 +278,7 @@ namespace Ryujinx.Graphics.Shader.Instructions
flags |= TextureFlags.Bindless;
}
int binding = isBindless ? 0 : context.ResourceManager.GetTextureOrImageBinding(
SetBindingPair setAndBinding = isBindless ? default : context.ResourceManager.GetTextureOrImageBinding(
Instruction.ImageAtomic,
type,
format,
@ -286,7 +286,7 @@ namespace Ryujinx.Graphics.Shader.Instructions
TextureOperation.DefaultCbufSlot,
imm);
Operand res = context.ImageAtomic(type, format, flags, binding, sources);
Operand res = context.ImageAtomic(type, format, flags, setAndBinding, sources);
context.Copy(d, res);
}
@ -389,7 +389,7 @@ namespace Ryujinx.Graphics.Shader.Instructions
TextureFormat format = isBindless ? TextureFormat.Unknown : ShaderProperties.GetTextureFormat(context.TranslatorContext.GpuAccessor, handle);
int binding = isBindless ? 0 : context.ResourceManager.GetTextureOrImageBinding(
SetBindingPair setAndBinding = isBindless ? default : context.ResourceManager.GetTextureOrImageBinding(
Instruction.ImageLoad,
type,
format,
@ -397,7 +397,7 @@ namespace Ryujinx.Graphics.Shader.Instructions
TextureOperation.DefaultCbufSlot,
handle);
context.ImageLoad(type, format, flags, binding, (int)componentMask, dests, sources);
context.ImageLoad(type, format, flags, setAndBinding, (int)componentMask, dests, sources);
}
else
{
@ -432,7 +432,7 @@ namespace Ryujinx.Graphics.Shader.Instructions
TextureFormat format = GetTextureFormat(size);
int binding = isBindless ? 0 : context.ResourceManager.GetTextureOrImageBinding(
SetBindingPair setAndBinding = isBindless ? default : context.ResourceManager.GetTextureOrImageBinding(
Instruction.ImageLoad,
type,
format,
@ -440,7 +440,7 @@ namespace Ryujinx.Graphics.Shader.Instructions
TextureOperation.DefaultCbufSlot,
handle);
context.ImageLoad(type, format, flags, binding, compMask, dests, sources);
context.ImageLoad(type, format, flags, setAndBinding, compMask, dests, sources);
switch (size)
{
@ -552,7 +552,7 @@ namespace Ryujinx.Graphics.Shader.Instructions
flags |= TextureFlags.Bindless;
}
int binding = isBindless ? 0 : context.ResourceManager.GetTextureOrImageBinding(
SetBindingPair setAndBinding = isBindless ? default : context.ResourceManager.GetTextureOrImageBinding(
Instruction.ImageAtomic,
type,
format,
@ -560,7 +560,7 @@ namespace Ryujinx.Graphics.Shader.Instructions
TextureOperation.DefaultCbufSlot,
imm);
context.ImageAtomic(type, format, flags, binding, sources);
context.ImageAtomic(type, format, flags, setAndBinding, sources);
}
private static void EmitSust(
@ -679,7 +679,7 @@ namespace Ryujinx.Graphics.Shader.Instructions
flags |= TextureFlags.Coherent;
}
int binding = isBindless ? 0 : context.ResourceManager.GetTextureOrImageBinding(
SetBindingPair setAndBinding = isBindless ? default : context.ResourceManager.GetTextureOrImageBinding(
Instruction.ImageStore,
type,
format,
@ -687,7 +687,7 @@ namespace Ryujinx.Graphics.Shader.Instructions
TextureOperation.DefaultCbufSlot,
handle);
context.ImageStore(type, format, flags, binding, sources);
context.ImageStore(type, format, flags, setAndBinding, sources);
}
private static int GetComponentSizeInBytesLog2(SuatomSize size)

View File

@ -885,7 +885,7 @@ namespace Ryujinx.Graphics.Shader.Instructions
return Register(dest++, RegisterType.Gpr);
}
int binding = isBindless ? 0 : context.ResourceManager.GetTextureOrImageBinding(
SetBindingPair setAndBinding = isBindless ? default : context.ResourceManager.GetTextureOrImageBinding(
Instruction.Lod,
type,
TextureFormat.Unknown,
@ -913,7 +913,7 @@ namespace Ryujinx.Graphics.Shader.Instructions
else
{
// The instruction component order is the inverse of GLSL's.
Operand res = context.Lod(type, flags, binding, compIndex ^ 1, sources);
Operand res = context.Lod(type, flags, setAndBinding, compIndex ^ 1, sources);
res = context.FPMultiply(res, ConstF(256.0f));
@ -1116,12 +1116,12 @@ namespace Ryujinx.Graphics.Shader.Instructions
}
TextureFlags flags = isBindless ? TextureFlags.Bindless : TextureFlags.None;
int binding;
SetBindingPair setAndBinding;
switch (query)
{
case TexQuery.TexHeaderDimension:
binding = isBindless ? 0 : context.ResourceManager.GetTextureOrImageBinding(
setAndBinding = isBindless ? default : context.ResourceManager.GetTextureOrImageBinding(
Instruction.TextureQuerySize,
type,
TextureFormat.Unknown,
@ -1140,13 +1140,13 @@ namespace Ryujinx.Graphics.Shader.Instructions
break;
}
context.Copy(d, context.TextureQuerySize(type, flags, binding, compIndex, sources));
context.Copy(d, context.TextureQuerySize(type, flags, setAndBinding, compIndex, sources));
}
}
break;
case TexQuery.TexHeaderTextureType:
binding = isBindless ? 0 : context.ResourceManager.GetTextureOrImageBinding(
setAndBinding = isBindless ? default : context.ResourceManager.GetTextureOrImageBinding(
Instruction.TextureQuerySamples,
type,
TextureFormat.Unknown,
@ -1171,7 +1171,7 @@ namespace Ryujinx.Graphics.Shader.Instructions
if (d != null)
{
context.Copy(d, context.TextureQuerySamples(type, flags, binding, sources));
context.Copy(d, context.TextureQuerySamples(type, flags, setAndBinding, sources));
}
}
break;
@ -1191,7 +1191,7 @@ namespace Ryujinx.Graphics.Shader.Instructions
Operand[] dests,
Operand[] sources)
{
int binding = flags.HasFlag(TextureFlags.Bindless) ? 0 : context.ResourceManager.GetTextureOrImageBinding(
SetBindingPair setAndBinding = flags.HasFlag(TextureFlags.Bindless) ? default : context.ResourceManager.GetTextureOrImageBinding(
Instruction.TextureSample,
type,
TextureFormat.Unknown,
@ -1199,7 +1199,7 @@ namespace Ryujinx.Graphics.Shader.Instructions
TextureOperation.DefaultCbufSlot,
handle);
context.TextureSample(type, flags, binding, componentMask, dests, sources);
context.TextureSample(type, flags, setAndBinding, componentMask, dests, sources);
}
private static SamplerType ConvertSamplerType(TexDim dimensions)

View File

@ -8,7 +8,9 @@ namespace Ryujinx.Graphics.Shader.IntermediateRepresentation
public TextureFormat Format { get; set; }
public TextureFlags Flags { get; private set; }
public int Set { get; private set; }
public int Binding { get; private set; }
public int SamplerSet { get; private set; }
public int SamplerBinding { get; private set; }
public TextureOperation(
@ -16,6 +18,7 @@ namespace Ryujinx.Graphics.Shader.IntermediateRepresentation
SamplerType type,
TextureFormat format,
TextureFlags flags,
int set,
int binding,
int compIndex,
Operand[] dests,
@ -24,24 +27,28 @@ namespace Ryujinx.Graphics.Shader.IntermediateRepresentation
Type = type;
Format = format;
Flags = flags;
Set = set;
Binding = binding;
SamplerSet = -1;
SamplerBinding = -1;
}
public void TurnIntoArray(int binding)
public void TurnIntoArray(SetBindingPair setAndBinding)
{
Flags &= ~TextureFlags.Bindless;
Binding = binding;
Set = setAndBinding.SetIndex;
Binding = setAndBinding.Binding;
}
public void TurnIntoArray(int textureBinding, int samplerBinding)
public void TurnIntoArray(SetBindingPair textureSetAndBinding, SetBindingPair samplerSetAndBinding)
{
TurnIntoArray(textureBinding);
TurnIntoArray(textureSetAndBinding);
SamplerBinding = samplerBinding;
SamplerSet = samplerSetAndBinding.SetIndex;
SamplerBinding = samplerSetAndBinding.Binding;
}
public void SetBinding(int binding)
public void SetBinding(SetBindingPair setAndBinding)
{
if ((Flags & TextureFlags.Bindless) != 0)
{
@ -50,7 +57,8 @@ namespace Ryujinx.Graphics.Shader.IntermediateRepresentation
RemoveSource(0);
}
Binding = binding;
Set = setAndBinding.SetIndex;
Binding = setAndBinding.Binding;
}
public void SetLodLevelFlag()

View File

@ -0,0 +1,4 @@
namespace Ryujinx.Graphics.Shader
{
public readonly record struct SetBindingPair(int SetIndex, int Binding);
}

View File

@ -8,7 +8,9 @@ namespace Ryujinx.Graphics.Shader.StructuredIr
public TextureFormat Format { get; }
public TextureFlags Flags { get; }
public int Set { get; }
public int Binding { get; }
public int SamplerSet { get; }
public int SamplerBinding { get; }
public bool IsSeparate => SamplerBinding >= 0;
@ -18,7 +20,9 @@ namespace Ryujinx.Graphics.Shader.StructuredIr
SamplerType type,
TextureFormat format,
TextureFlags flags,
int set,
int binding,
int samplerSet,
int samplerBinding,
int index,
params IAstNode[] sources) : base(inst, StorageKind.None, false, index, sources, sources.Length)
@ -26,8 +30,20 @@ namespace Ryujinx.Graphics.Shader.StructuredIr
Type = type;
Format = format;
Flags = flags;
Set = set;
Binding = binding;
SamplerSet = samplerSet;
SamplerBinding = samplerBinding;
}
public SetBindingPair GetTextureSetAndBinding()
{
return new SetBindingPair(Set, Binding);
}
public SetBindingPair GetSamplerSetAndBinding()
{
return new SetBindingPair(SamplerSet, SamplerBinding);
}
}
}

View File

@ -6,15 +6,15 @@ namespace Ryujinx.Graphics.Shader.StructuredIr
{
private readonly Dictionary<int, BufferDefinition> _constantBuffers;
private readonly Dictionary<int, BufferDefinition> _storageBuffers;
private readonly Dictionary<int, TextureDefinition> _textures;
private readonly Dictionary<int, TextureDefinition> _images;
private readonly Dictionary<SetBindingPair, TextureDefinition> _textures;
private readonly Dictionary<SetBindingPair, TextureDefinition> _images;
private readonly Dictionary<int, MemoryDefinition> _localMemories;
private readonly Dictionary<int, MemoryDefinition> _sharedMemories;
public IReadOnlyDictionary<int, BufferDefinition> ConstantBuffers => _constantBuffers;
public IReadOnlyDictionary<int, BufferDefinition> StorageBuffers => _storageBuffers;
public IReadOnlyDictionary<int, TextureDefinition> Textures => _textures;
public IReadOnlyDictionary<int, TextureDefinition> Images => _images;
public IReadOnlyDictionary<SetBindingPair, TextureDefinition> Textures => _textures;
public IReadOnlyDictionary<SetBindingPair, TextureDefinition> Images => _images;
public IReadOnlyDictionary<int, MemoryDefinition> LocalMemories => _localMemories;
public IReadOnlyDictionary<int, MemoryDefinition> SharedMemories => _sharedMemories;
@ -22,8 +22,8 @@ namespace Ryujinx.Graphics.Shader.StructuredIr
{
_constantBuffers = new Dictionary<int, BufferDefinition>();
_storageBuffers = new Dictionary<int, BufferDefinition>();
_textures = new Dictionary<int, TextureDefinition>();
_images = new Dictionary<int, TextureDefinition>();
_textures = new Dictionary<SetBindingPair, TextureDefinition>();
_images = new Dictionary<SetBindingPair, TextureDefinition>();
_localMemories = new Dictionary<int, MemoryDefinition>();
_sharedMemories = new Dictionary<int, MemoryDefinition>();
}
@ -40,12 +40,12 @@ namespace Ryujinx.Graphics.Shader.StructuredIr
public void AddOrUpdateTexture(TextureDefinition definition)
{
_textures[definition.Binding] = definition;
_textures[new(definition.Set, definition.Binding)] = definition;
}
public void AddOrUpdateImage(TextureDefinition definition)
{
_images[definition.Binding] = definition;
_images[new(definition.Set, definition.Binding)] = definition;
}
public int AddLocalMemory(MemoryDefinition definition)

View File

@ -169,7 +169,17 @@ namespace Ryujinx.Graphics.Shader.StructuredIr
AstTextureOperation GetAstTextureOperation(TextureOperation texOp)
{
return new AstTextureOperation(inst, texOp.Type, texOp.Format, texOp.Flags, texOp.Binding, texOp.SamplerBinding, texOp.Index, sources);
return new AstTextureOperation(
inst,
texOp.Type,
texOp.Format,
texOp.Flags,
texOp.Set,
texOp.Binding,
texOp.SamplerSet,
texOp.SamplerBinding,
texOp.Index,
sources);
}
int componentsCount = BitOperations.PopCount((uint)operation.Index);

View File

@ -4,6 +4,7 @@ namespace Ryujinx.Graphics.Shader
{
// New fields should be added to the end of the struct to keep disk shader cache compatibility.
public readonly int Set;
public readonly int Binding;
public readonly SamplerType Type;
@ -18,6 +19,7 @@ namespace Ryujinx.Graphics.Shader
public readonly TextureUsageFlags Flags;
public TextureDescriptor(
int set,
int binding,
SamplerType type,
TextureFormat format,
@ -27,6 +29,7 @@ namespace Ryujinx.Graphics.Shader
bool separate,
TextureUsageFlags flags)
{
Set = set;
Binding = binding;
Type = type;
Format = format;

View File

@ -124,7 +124,7 @@ namespace Ryujinx.Graphics.Shader.Translation
this.TextureSample(
SamplerType.TextureBuffer,
TextureFlags.IntCoords,
ResourceManager.Reservations.IndexBufferTextureBinding,
ResourceManager.Reservations.GetIndexBufferTextureSetAndBinding(),
1,
new[] { vertexIndexVr },
new[] { this.IAdd(ibBaseOffset, outputVertexOffset) });
@ -145,7 +145,7 @@ namespace Ryujinx.Graphics.Shader.Translation
this.TextureSample(
SamplerType.TextureBuffer,
TextureFlags.IntCoords,
ResourceManager.Reservations.TopologyRemapBufferTextureBinding,
ResourceManager.Reservations.GetTopologyRemapBufferTextureSetAndBinding(),
1,
new[] { vertexIndex },
new[] { this.IAdd(baseVertex, Const(index)) });

View File

@ -618,12 +618,21 @@ namespace Ryujinx.Graphics.Shader.Translation
SamplerType type,
TextureFormat format,
TextureFlags flags,
int binding,
SetBindingPair setAndBinding,
Operand[] sources)
{
Operand dest = Local();
context.Add(new TextureOperation(Instruction.ImageAtomic, type, format, flags, binding, 0, new[] { dest }, sources));
context.Add(new TextureOperation(
Instruction.ImageAtomic,
type,
format,
flags,
setAndBinding.SetIndex,
setAndBinding.Binding,
0,
new[] { dest },
sources));
return dest;
}
@ -633,12 +642,21 @@ namespace Ryujinx.Graphics.Shader.Translation
SamplerType type,
TextureFormat format,
TextureFlags flags,
int binding,
SetBindingPair setAndBinding,
int compMask,
Operand[] dests,
Operand[] sources)
{
context.Add(new TextureOperation(Instruction.ImageLoad, type, format, flags, binding, compMask, dests, sources));
context.Add(new TextureOperation(
Instruction.ImageLoad,
type,
format,
flags,
setAndBinding.SetIndex,
setAndBinding.Binding,
compMask,
dests,
sources));
}
public static void ImageStore(
@ -646,10 +664,19 @@ namespace Ryujinx.Graphics.Shader.Translation
SamplerType type,
TextureFormat format,
TextureFlags flags,
int binding,
SetBindingPair setAndBinding,
Operand[] sources)
{
context.Add(new TextureOperation(Instruction.ImageStore, type, format, flags, binding, 0, null, sources));
context.Add(new TextureOperation(
Instruction.ImageStore,
type,
format,
flags,
setAndBinding.SetIndex,
setAndBinding.Binding,
0,
null,
sources));
}
public static Operand IsNan(this EmitterContext context, Operand a, Instruction fpType = Instruction.FP32)
@ -718,13 +745,22 @@ namespace Ryujinx.Graphics.Shader.Translation
this EmitterContext context,
SamplerType type,
TextureFlags flags,
int binding,
SetBindingPair setAndBinding,
int compIndex,
Operand[] sources)
{
Operand dest = Local();
context.Add(new TextureOperation(Instruction.Lod, type, TextureFormat.Unknown, flags, binding, compIndex, new[] { dest }, sources));
context.Add(new TextureOperation(
Instruction.Lod,
type,
TextureFormat.Unknown,
flags,
setAndBinding.SetIndex,
setAndBinding.Binding,
compIndex,
new[] { dest },
sources));
return dest;
}
@ -889,24 +925,42 @@ namespace Ryujinx.Graphics.Shader.Translation
this EmitterContext context,
SamplerType type,
TextureFlags flags,
int binding,
SetBindingPair setAndBinding,
int compMask,
Operand[] dests,
Operand[] sources)
{
context.Add(new TextureOperation(Instruction.TextureSample, type, TextureFormat.Unknown, flags, binding, compMask, dests, sources));
context.Add(new TextureOperation(
Instruction.TextureSample,
type,
TextureFormat.Unknown,
flags,
setAndBinding.SetIndex,
setAndBinding.Binding,
compMask,
dests,
sources));
}
public static Operand TextureQuerySamples(
this EmitterContext context,
SamplerType type,
TextureFlags flags,
int binding,
SetBindingPair setAndBinding,
Operand[] sources)
{
Operand dest = Local();
context.Add(new TextureOperation(Instruction.TextureQuerySamples, type, TextureFormat.Unknown, flags, binding, 0, new[] { dest }, sources));
context.Add(new TextureOperation(
Instruction.TextureQuerySamples,
type,
TextureFormat.Unknown,
flags,
setAndBinding.SetIndex,
setAndBinding.Binding,
0,
new[] { dest },
sources));
return dest;
}
@ -915,13 +969,22 @@ namespace Ryujinx.Graphics.Shader.Translation
this EmitterContext context,
SamplerType type,
TextureFlags flags,
int binding,
SetBindingPair setAndBinding,
int compIndex,
Operand[] sources)
{
Operand dest = Local();
context.Add(new TextureOperation(Instruction.TextureQuerySize, type, TextureFormat.Unknown, flags, binding, compIndex, new[] { dest }, sources));
context.Add(new TextureOperation(
Instruction.TextureQuerySize,
type,
TextureFormat.Unknown,
flags,
setAndBinding.SetIndex,
setAndBinding.Binding,
compIndex,
new[] { dest },
sources));
return dest;
}

View File

@ -8,6 +8,7 @@ namespace Ryujinx.Graphics.Shader.Translation
public readonly bool SupportsGeometryShaderPassthrough;
public readonly bool SupportsShaderBallot;
public readonly bool SupportsShaderBarrierDivergence;
public readonly bool SupportsShaderFloat64;
public readonly bool SupportsTextureShadowLod;
public readonly bool SupportsViewportMask;
@ -18,6 +19,7 @@ namespace Ryujinx.Graphics.Shader.Translation
bool supportsGeometryShaderPassthrough,
bool supportsShaderBallot,
bool supportsShaderBarrierDivergence,
bool supportsShaderFloat64,
bool supportsTextureShadowLod,
bool supportsViewportMask)
{
@ -27,6 +29,7 @@ namespace Ryujinx.Graphics.Shader.Translation
SupportsGeometryShaderPassthrough = supportsGeometryShaderPassthrough;
SupportsShaderBallot = supportsShaderBallot;
SupportsShaderBarrierDivergence = supportsShaderBarrierDivergence;
SupportsShaderFloat64 = supportsShaderFloat64;
SupportsTextureShadowLod = supportsTextureShadowLod;
SupportsViewportMask = supportsViewportMask;
}

View File

@ -38,6 +38,12 @@ namespace Ryujinx.Graphics.Shader.Translation.Optimizations
// If we can't do bindless elimination, remove the texture operation.
// Set any destination variables to zero.
string typeName = texOp.Inst.IsImage()
? texOp.Type.ToGlslImageType(texOp.Format.GetComponentType())
: texOp.Type.ToGlslTextureType();
gpuAccessor.Log($"Failed to find handle source for bindless access of type \"{typeName}\".");
for (int destIndex = 0; destIndex < texOp.DestsCount; destIndex++)
{
block.Operations.AddBefore(node, new Operation(Instruction.Copy, texOp.GetDest(destIndex), OperandHelper.Const(0)));
@ -62,17 +68,22 @@ namespace Ryujinx.Graphics.Shader.Translation.Optimizations
return false;
}
Operand nvHandle = texOp.GetSource(0);
Operand bindlessHandle = texOp.GetSource(0);
if (nvHandle.AsgOp is not Operation handleOp ||
handleOp.Inst != Instruction.Load ||
(handleOp.StorageKind != StorageKind.Input && handleOp.StorageKind != StorageKind.StorageBuffer))
if (bindlessHandle.AsgOp is PhiNode phi)
{
// Right now, we only allow bindless access when the handle comes from a shader input or storage buffer.
// This is an artificial limitation to prevent it from being used in cases where it
// would have a large performance impact of loading all textures in the pool.
// It might be removed in the future, if we can mitigate the performance impact.
for (int srcIndex = 0; srcIndex < phi.SourcesCount; srcIndex++)
{
Operand phiSource = phi.GetSource(srcIndex);
if (phiSource.AsgOp is not PhiNode && !IsBindlessAccessAllowed(phiSource))
{
return false;
}
}
}
else if (!IsBindlessAccessAllowed(bindlessHandle))
{
return false;
}
@ -80,8 +91,8 @@ namespace Ryujinx.Graphics.Shader.Translation.Optimizations
Operand samplerHandle = OperandHelper.Local();
Operand textureIndex = OperandHelper.Local();
block.Operations.AddBefore(node, new Operation(Instruction.BitwiseAnd, textureHandle, nvHandle, OperandHelper.Const(0xfffff)));
block.Operations.AddBefore(node, new Operation(Instruction.ShiftRightU32, samplerHandle, nvHandle, OperandHelper.Const(20)));
block.Operations.AddBefore(node, new Operation(Instruction.BitwiseAnd, textureHandle, bindlessHandle, OperandHelper.Const(0xfffff)));
block.Operations.AddBefore(node, new Operation(Instruction.ShiftRightU32, samplerHandle, bindlessHandle, OperandHelper.Const(20)));
int texturePoolLength = Math.Max(BindlessToArray.MinimumArrayLength, gpuAccessor.QueryTextureArrayLengthFromPool());
@ -91,7 +102,7 @@ namespace Ryujinx.Graphics.Shader.Translation.Optimizations
bool hasSampler = !texOp.Inst.IsImage();
int textureBinding = resourceManager.GetTextureOrImageBinding(
SetBindingPair textureSetAndBinding = resourceManager.GetTextureOrImageBinding(
texOp.Inst,
texOp.Type,
texOp.Format,
@ -111,7 +122,7 @@ namespace Ryujinx.Graphics.Shader.Translation.Optimizations
texOp.InsertSource(1, samplerIndex);
int samplerBinding = resourceManager.GetTextureOrImageBinding(
SetBindingPair samplerSetAndBinding = resourceManager.GetTextureOrImageBinding(
texOp.Inst,
SamplerType.None,
texOp.Format,
@ -120,11 +131,35 @@ namespace Ryujinx.Graphics.Shader.Translation.Optimizations
TextureHandle.PackOffsets(0, 0, TextureHandleType.Direct),
samplerPoolLength);
texOp.TurnIntoArray(textureBinding, samplerBinding);
texOp.TurnIntoArray(textureSetAndBinding, samplerSetAndBinding);
}
else
{
texOp.TurnIntoArray(textureBinding);
texOp.TurnIntoArray(textureSetAndBinding);
}
return true;
}
private static bool IsBindlessAccessAllowed(Operand nvHandle)
{
if (nvHandle.Type == OperandType.ConstantBuffer)
{
// Bindless access with handles from constant buffer is allowed.
return true;
}
if (nvHandle.AsgOp is not Operation handleOp ||
handleOp.Inst != Instruction.Load ||
(handleOp.StorageKind != StorageKind.Input && handleOp.StorageKind != StorageKind.StorageBuffer))
{
// Right now, we only allow bindless access when the handle comes from a shader input or storage buffer.
// This is an artificial limitation to prevent it from being used in cases where it
// would have a large performance impact of loading all textures in the pool.
// It might be removed in the future, if we can mitigate the performance impact.
return false;
}
return true;
@ -445,7 +480,7 @@ namespace Ryujinx.Graphics.Shader.Translation.Optimizations
}
}
int binding = resourceManager.GetTextureOrImageBinding(
SetBindingPair setAndBinding = resourceManager.GetTextureOrImageBinding(
texOp.Inst,
texOp.Type,
texOp.Format,
@ -453,7 +488,7 @@ namespace Ryujinx.Graphics.Shader.Translation.Optimizations
cbufSlot,
cbufOffset);
texOp.SetBinding(binding);
texOp.SetBinding(setAndBinding);
}
}
}

View File

@ -221,7 +221,7 @@ namespace Ryujinx.Graphics.Shader.Translation.Optimizations
private static void TurnIntoArray(ResourceManager resourceManager, TextureOperation texOp, int cbufSlot, int handleIndex, int length)
{
int binding = resourceManager.GetTextureOrImageBinding(
SetBindingPair setAndBinding = resourceManager.GetTextureOrImageBinding(
texOp.Inst,
texOp.Type,
texOp.Format,
@ -230,7 +230,7 @@ namespace Ryujinx.Graphics.Shader.Translation.Optimizations
handleIndex,
length);
texOp.TurnIntoArray(binding);
texOp.TurnIntoArray(setAndBinding);
}
}
}

View File

@ -20,8 +20,8 @@ namespace Ryujinx.Graphics.Shader.Translation
private readonly ShaderStage _stage;
private readonly string _stagePrefix;
private readonly int[] _cbSlotToBindingMap;
private readonly int[] _sbSlotToBindingMap;
private readonly SetBindingPair[] _cbSlotToBindingMap;
private readonly SetBindingPair[] _sbSlotToBindingMap;
private uint _sbSlotWritten;
private readonly Dictionary<int, int> _sbSlots;
@ -33,6 +33,7 @@ namespace Ryujinx.Graphics.Shader.Translation
private struct TextureMeta
{
public int Set;
public int Binding;
public bool AccurateType;
public SamplerType Type;
@ -64,10 +65,10 @@ namespace Ryujinx.Graphics.Shader.Translation
_stage = stage;
_stagePrefix = GetShaderStagePrefix(stage);
_cbSlotToBindingMap = new int[18];
_sbSlotToBindingMap = new int[16];
_cbSlotToBindingMap.AsSpan().Fill(-1);
_sbSlotToBindingMap.AsSpan().Fill(-1);
_cbSlotToBindingMap = new SetBindingPair[18];
_sbSlotToBindingMap = new SetBindingPair[16];
_cbSlotToBindingMap.AsSpan().Fill(new(-1, -1));
_sbSlotToBindingMap.AsSpan().Fill(new(-1, -1));
_sbSlots = new();
_sbSlotsReverse = new();
@ -146,16 +147,16 @@ namespace Ryujinx.Graphics.Shader.Translation
public int GetConstantBufferBinding(int slot)
{
int binding = _cbSlotToBindingMap[slot];
if (binding < 0)
SetBindingPair setAndBinding = _cbSlotToBindingMap[slot];
if (setAndBinding.Binding < 0)
{
binding = _gpuAccessor.CreateConstantBufferBinding(slot);
_cbSlotToBindingMap[slot] = binding;
setAndBinding = _gpuAccessor.CreateConstantBufferBinding(slot);
_cbSlotToBindingMap[slot] = setAndBinding;
string slotNumber = slot.ToString(CultureInfo.InvariantCulture);
AddNewConstantBuffer(binding, $"{_stagePrefix}_c{slotNumber}");
AddNewConstantBuffer(setAndBinding.SetIndex, setAndBinding.Binding, $"{_stagePrefix}_c{slotNumber}");
}
return binding;
return setAndBinding.Binding;
}
public bool TryGetStorageBufferBinding(int sbCbSlot, int sbCbOffset, bool write, out int binding)
@ -166,14 +167,14 @@ namespace Ryujinx.Graphics.Shader.Translation
return false;
}
binding = _sbSlotToBindingMap[slot];
SetBindingPair setAndBinding = _sbSlotToBindingMap[slot];
if (binding < 0)
if (setAndBinding.Binding < 0)
{
binding = _gpuAccessor.CreateStorageBufferBinding(slot);
_sbSlotToBindingMap[slot] = binding;
setAndBinding = _gpuAccessor.CreateStorageBufferBinding(slot);
_sbSlotToBindingMap[slot] = setAndBinding;
string slotNumber = slot.ToString(CultureInfo.InvariantCulture);
AddNewStorageBuffer(binding, $"{_stagePrefix}_s{slotNumber}");
AddNewStorageBuffer(setAndBinding.SetIndex, setAndBinding.Binding, $"{_stagePrefix}_s{slotNumber}");
}
if (write)
@ -181,6 +182,7 @@ namespace Ryujinx.Graphics.Shader.Translation
_sbSlotWritten |= 1u << slot;
}
binding = setAndBinding.Binding;
return true;
}
@ -208,7 +210,7 @@ namespace Ryujinx.Graphics.Shader.Translation
{
for (slot = 0; slot < _cbSlotToBindingMap.Length; slot++)
{
if (_cbSlotToBindingMap[slot] == binding)
if (_cbSlotToBindingMap[slot].Binding == binding)
{
return true;
}
@ -218,7 +220,7 @@ namespace Ryujinx.Graphics.Shader.Translation
return false;
}
public int GetTextureOrImageBinding(
public SetBindingPair GetTextureOrImageBinding(
Instruction inst,
SamplerType type,
TextureFormat format,
@ -240,7 +242,7 @@ namespace Ryujinx.Graphics.Shader.Translation
format = TextureFormat.Unknown;
}
int binding = GetTextureOrImageBinding(
SetBindingPair setAndBinding = GetTextureOrImageBinding(
cbufSlot,
handle,
arrayLength,
@ -255,10 +257,10 @@ namespace Ryujinx.Graphics.Shader.Translation
_gpuAccessor.RegisterTexture(handle, cbufSlot);
return binding;
return setAndBinding;
}
private int GetTextureOrImageBinding(
private SetBindingPair GetTextureOrImageBinding(
int cbufSlot,
int handle,
int arrayLength,
@ -311,21 +313,38 @@ namespace Ryujinx.Graphics.Shader.Translation
UsageFlags = usageFlags,
};
int setIndex;
int binding;
if (dict.TryGetValue(info, out var existingMeta))
{
dict[info] = MergeTextureMeta(meta, existingMeta);
setIndex = existingMeta.Set;
binding = existingMeta.Binding;
}
else
{
bool isBuffer = (type & SamplerType.Mask) == SamplerType.TextureBuffer;
if (arrayLength > 1 && (setIndex = _gpuAccessor.CreateExtraSet()) >= 0)
{
// We reserved an "extra set" for the array.
// In this case the binding is always the first one (0).
// Using separate sets for array is better as we need to do less descriptor set updates.
binding = isImage
? _gpuAccessor.CreateImageBinding(arrayLength, isBuffer)
: _gpuAccessor.CreateTextureBinding(arrayLength, isBuffer);
binding = 0;
}
else
{
bool isBuffer = (type & SamplerType.Mask) == SamplerType.TextureBuffer;
SetBindingPair setAndBinding = isImage
? _gpuAccessor.CreateImageBinding(arrayLength, isBuffer)
: _gpuAccessor.CreateTextureBinding(arrayLength, isBuffer);
setIndex = setAndBinding.SetIndex;
binding = setAndBinding.Binding;
}
meta.Set = setIndex;
meta.Binding = binding;
dict.Add(info, meta);
@ -355,7 +374,7 @@ namespace Ryujinx.Graphics.Shader.Translation
}
var definition = new TextureDefinition(
isImage ? 3 : 2,
setIndex,
binding,
arrayLength,
separate,
@ -373,11 +392,12 @@ namespace Ryujinx.Graphics.Shader.Translation
Properties.AddOrUpdateTexture(definition);
}
return binding;
return new SetBindingPair(setIndex, binding);
}
private static TextureMeta MergeTextureMeta(TextureMeta meta, TextureMeta existingMeta)
{
meta.Set = existingMeta.Set;
meta.Binding = existingMeta.Binding;
meta.UsageFlags |= existingMeta.UsageFlags;
@ -440,11 +460,11 @@ namespace Ryujinx.Graphics.Shader.Translation
for (int slot = 0; slot < _cbSlotToBindingMap.Length; slot++)
{
int binding = _cbSlotToBindingMap[slot];
SetBindingPair setAndBinding = _cbSlotToBindingMap[slot];
if (binding >= 0 && _usedConstantBufferBindings.Contains(binding))
if (setAndBinding.Binding >= 0 && _usedConstantBufferBindings.Contains(setAndBinding.Binding))
{
descriptors[descriptorIndex++] = new BufferDescriptor(binding, slot);
descriptors[descriptorIndex++] = new BufferDescriptor(setAndBinding.SetIndex, setAndBinding.Binding, slot);
}
}
@ -464,13 +484,13 @@ namespace Ryujinx.Graphics.Shader.Translation
foreach ((int key, int slot) in _sbSlots)
{
int binding = _sbSlotToBindingMap[slot];
SetBindingPair setAndBinding = _sbSlotToBindingMap[slot];
if (binding >= 0)
if (setAndBinding.Binding >= 0)
{
(int sbCbSlot, int sbCbOffset) = UnpackSbCbInfo(key);
BufferUsageFlags flags = (_sbSlotWritten & (1u << slot)) != 0 ? BufferUsageFlags.Write : BufferUsageFlags.None;
descriptors[descriptorIndex++] = new BufferDescriptor(binding, slot, sbCbSlot, sbCbOffset, flags);
descriptors[descriptorIndex++] = new BufferDescriptor(setAndBinding.SetIndex, setAndBinding.Binding, slot, sbCbSlot, sbCbOffset, flags);
}
}
@ -507,6 +527,7 @@ namespace Ryujinx.Graphics.Shader.Translation
}
descriptors.Add(new TextureDescriptor(
meta.Set,
meta.Binding,
meta.Type,
info.Format,
@ -527,6 +548,7 @@ namespace Ryujinx.Graphics.Shader.Translation
}
descriptors.Add(new TextureDescriptor(
meta.Set,
meta.Binding,
meta.Type,
info.Format,
@ -587,24 +609,24 @@ namespace Ryujinx.Graphics.Shader.Translation
return false;
}
private void AddNewConstantBuffer(int binding, string name)
private void AddNewConstantBuffer(int setIndex, int binding, string name)
{
StructureType type = new(new[]
{
new StructureField(AggregateType.Array | AggregateType.Vector4 | AggregateType.FP32, "data", Constants.ConstantBufferSize / 16),
});
Properties.AddOrUpdateConstantBuffer(new(BufferLayout.Std140, 0, binding, name, type));
Properties.AddOrUpdateConstantBuffer(new(BufferLayout.Std140, setIndex, binding, name, type));
}
private void AddNewStorageBuffer(int binding, string name)
private void AddNewStorageBuffer(int setIndex, int binding, string name)
{
StructureType type = new(new[]
{
new StructureField(AggregateType.Array | AggregateType.U32, "data", 0),
});
Properties.AddOrUpdateStorageBuffer(new(BufferLayout.Std430, 1, binding, name, type));
Properties.AddOrUpdateStorageBuffer(new(BufferLayout.Std430, setIndex, binding, name, type));
}
public static string GetShaderStagePrefix(ShaderStage stage)

View File

@ -11,6 +11,8 @@ namespace Ryujinx.Graphics.Shader.Translation
public const int MaxVertexBufferTextures = 32;
private const int TextureSetIndex = 2; // TODO: Get from GPU accessor.
public int VertexInfoConstantBufferBinding { get; }
public int VertexOutputStorageBufferBinding { get; }
public int GeometryVertexOutputStorageBufferBinding { get; }
@ -163,6 +165,21 @@ namespace Ryujinx.Graphics.Shader.Translation
return _vertexBufferTextureBaseBinding + vaLocation;
}
public SetBindingPair GetVertexBufferTextureSetAndBinding(int vaLocation)
{
return new SetBindingPair(TextureSetIndex, GetVertexBufferTextureBinding(vaLocation));
}
public SetBindingPair GetIndexBufferTextureSetAndBinding()
{
return new SetBindingPair(TextureSetIndex, IndexBufferTextureBinding);
}
public SetBindingPair GetTopologyRemapBufferTextureSetAndBinding()
{
return new SetBindingPair(TextureSetIndex, TopologyRemapBufferTextureBinding);
}
internal bool TryGetOffset(StorageKind storageKind, int location, int component, out int offset)
{
return _offsets.TryGetValue(new IoDefinition(storageKind, IoVariable.UserDefined, location, component), out offset);

View File

@ -182,6 +182,7 @@ namespace Ryujinx.Graphics.Shader.Translation.Transforms
texOp.Type,
texOp.Format,
texOp.Flags,
texOp.Set,
texOp.Binding,
index,
new[] { coordSize },
@ -251,6 +252,7 @@ namespace Ryujinx.Graphics.Shader.Translation.Transforms
texOp.Type,
texOp.Format,
texOp.Flags,
texOp.Set,
texOp.Binding,
index,
new[] { coordSize },
@ -471,6 +473,7 @@ namespace Ryujinx.Graphics.Shader.Translation.Transforms
texOp.Type,
texOp.Format,
texOp.Flags & ~(TextureFlags.Offset | TextureFlags.Offsets),
texOp.Set,
texOp.Binding,
1 << 3, // W component: i=0, j=0
new[] { dests[destIndex++] },
@ -527,6 +530,7 @@ namespace Ryujinx.Graphics.Shader.Translation.Transforms
texOp.Type,
texOp.Format,
texOp.Flags & ~(TextureFlags.Offset | TextureFlags.Offsets),
texOp.Set,
texOp.Binding,
componentIndex,
dests,
@ -573,6 +577,7 @@ namespace Ryujinx.Graphics.Shader.Translation.Transforms
texOp.Type,
texOp.Format,
texOp.Flags,
texOp.Set,
texOp.Binding,
index,
new[] { texSizes[index] },
@ -603,6 +608,7 @@ namespace Ryujinx.Graphics.Shader.Translation.Transforms
texOp.Type,
texOp.Format,
texOp.Flags,
texOp.Set,
texOp.Binding,
0,
new[] { lod },
@ -633,6 +639,7 @@ namespace Ryujinx.Graphics.Shader.Translation.Transforms
texOp.Type,
texOp.Format,
texOp.Flags,
texOp.Set,
texOp.Binding,
index,
new[] { texSizes[index] },

View File

@ -54,6 +54,7 @@ namespace Ryujinx.Graphics.Shader.Translation.Transforms
{
bool needsSextNorm = context.Definitions.IsAttributePackedRgb10A2Signed(location);
SetBindingPair setAndBinding = context.ResourceManager.Reservations.GetVertexBufferTextureSetAndBinding(location);
Operand temp = needsSextNorm ? Local() : dest;
Operand vertexElemOffset = GenerateVertexOffset(context.ResourceManager, node, location, 0);
@ -62,7 +63,8 @@ namespace Ryujinx.Graphics.Shader.Translation.Transforms
SamplerType.TextureBuffer,
TextureFormat.Unknown,
TextureFlags.IntCoords,
context.ResourceManager.Reservations.GetVertexBufferTextureBinding(location),
setAndBinding.SetIndex,
setAndBinding.Binding,
1 << component,
new[] { temp },
new[] { vertexElemOffset }));
@ -75,6 +77,7 @@ namespace Ryujinx.Graphics.Shader.Translation.Transforms
}
else
{
SetBindingPair setAndBinding = context.ResourceManager.Reservations.GetVertexBufferTextureSetAndBinding(location);
Operand temp = component > 0 ? Local() : dest;
Operand vertexElemOffset = GenerateVertexOffset(context.ResourceManager, node, location, component);
@ -83,7 +86,8 @@ namespace Ryujinx.Graphics.Shader.Translation.Transforms
SamplerType.TextureBuffer,
TextureFormat.Unknown,
TextureFlags.IntCoords,
context.ResourceManager.Reservations.GetVertexBufferTextureBinding(location),
setAndBinding.SetIndex,
setAndBinding.Binding,
1,
new[] { temp },
new[] { vertexElemOffset }));

View File

@ -363,6 +363,7 @@ namespace Ryujinx.Graphics.Shader.Translation
GpuAccessor.QueryHostSupportsGeometryShaderPassthrough(),
GpuAccessor.QueryHostSupportsShaderBallot(),
GpuAccessor.QueryHostSupportsShaderBarrierDivergence(),
GpuAccessor.QueryHostSupportsShaderFloat64(),
GpuAccessor.QueryHostSupportsTextureShadowLod(),
GpuAccessor.QueryHostSupportsViewportMask());
@ -412,8 +413,8 @@ namespace Ryujinx.Graphics.Shader.Translation
if (Stage == ShaderStage.Vertex)
{
int ibBinding = resourceManager.Reservations.IndexBufferTextureBinding;
TextureDefinition indexBuffer = new(2, ibBinding, "ib_data", SamplerType.TextureBuffer);
SetBindingPair ibSetAndBinding = resourceManager.Reservations.GetIndexBufferTextureSetAndBinding();
TextureDefinition indexBuffer = new(ibSetAndBinding.SetIndex, ibSetAndBinding.Binding, "ib_data", SamplerType.TextureBuffer);
resourceManager.Properties.AddOrUpdateTexture(indexBuffer);
int inputMap = _program.AttributeUsage.UsedInputAttributes;
@ -421,8 +422,8 @@ namespace Ryujinx.Graphics.Shader.Translation
while (inputMap != 0)
{
int location = BitOperations.TrailingZeroCount(inputMap);
int binding = resourceManager.Reservations.GetVertexBufferTextureBinding(location);
TextureDefinition vaBuffer = new(2, binding, $"vb_data{location}", SamplerType.TextureBuffer);
SetBindingPair setAndBinding = resourceManager.Reservations.GetVertexBufferTextureSetAndBinding(location);
TextureDefinition vaBuffer = new(setAndBinding.SetIndex, setAndBinding.Binding, $"vb_data{location}", SamplerType.TextureBuffer);
resourceManager.Properties.AddOrUpdateTexture(vaBuffer);
inputMap &= ~(1 << location);
@ -430,8 +431,8 @@ namespace Ryujinx.Graphics.Shader.Translation
}
else if (Stage == ShaderStage.Geometry)
{
int trbBinding = resourceManager.Reservations.TopologyRemapBufferTextureBinding;
TextureDefinition remapBuffer = new(2, trbBinding, "trb_data", SamplerType.TextureBuffer);
SetBindingPair trbSetAndBinding = resourceManager.Reservations.GetTopologyRemapBufferTextureSetAndBinding();
TextureDefinition remapBuffer = new(trbSetAndBinding.SetIndex, trbSetAndBinding.Binding, "trb_data", SamplerType.TextureBuffer);
resourceManager.Properties.AddOrUpdateTexture(remapBuffer);
int geometryVbOutputSbBinding = resourceManager.Reservations.GeometryVertexOutputStorageBufferBinding;

View File

@ -1,4 +1,3 @@
using Ryujinx.Common.Logging;
using Ryujinx.Graphics.GAL;
using Silk.NET.Vulkan;
using System;
@ -31,40 +30,29 @@ namespace Ryujinx.Graphics.Vulkan
private readonly VulkanRenderer _gd;
private readonly Device _device;
private MemoryAllocation _allocation;
private Auto<DisposableBuffer> _buffer;
private Auto<MemoryAllocation> _allocationAuto;
private readonly MemoryAllocation _allocation;
private readonly Auto<DisposableBuffer> _buffer;
private readonly Auto<MemoryAllocation> _allocationAuto;
private readonly bool _allocationImported;
private ulong _bufferHandle;
private readonly ulong _bufferHandle;
private CacheByRange<BufferHolder> _cachedConvertedBuffers;
public int Size { get; }
private IntPtr _map;
private readonly IntPtr _map;
private MultiFenceHolder _waitable;
private readonly MultiFenceHolder _waitable;
private bool _lastAccessIsWrite;
private BufferAllocationType _baseType;
private BufferAllocationType _currentType;
private bool _swapQueued;
public BufferAllocationType DesiredType { get; private set; }
private int _setCount;
private int _writeCount;
private int _flushCount;
private int _flushTemp;
private int _lastFlushWrite = -1;
private readonly BufferAllocationType _baseType;
private readonly BufferAllocationType _activeType;
private readonly ReaderWriterLockSlim _flushLock;
private FenceHolder _flushFence;
private int _flushWaiting;
private List<Action> _swapActions;
private byte[] _pendingData;
private BufferMirrorRangeList _pendingDataRanges;
private Dictionary<ulong, StagingBufferReserved> _mirrors;
@ -83,8 +71,7 @@ namespace Ryujinx.Graphics.Vulkan
_map = allocation.HostPointer;
_baseType = type;
_currentType = currentType;
DesiredType = currentType;
_activeType = currentType;
_flushLock = new ReaderWriterLockSlim();
_useMirrors = gd.IsTBDR;
@ -104,8 +91,7 @@ namespace Ryujinx.Graphics.Vulkan
_map = _allocation.HostPointer + offset;
_baseType = type;
_currentType = currentType;
DesiredType = currentType;
_activeType = currentType;
_flushLock = new ReaderWriterLockSlim();
}
@ -120,164 +106,11 @@ namespace Ryujinx.Graphics.Vulkan
Size = size;
_baseType = BufferAllocationType.Sparse;
_currentType = BufferAllocationType.Sparse;
DesiredType = BufferAllocationType.Sparse;
_activeType = BufferAllocationType.Sparse;
_flushLock = new ReaderWriterLockSlim();
}
public bool TryBackingSwap(ref CommandBufferScoped? cbs)
{
if (_swapQueued && DesiredType != _currentType)
{
// Only swap if the buffer is not used in any queued command buffer.
bool isRented = _buffer.HasRentedCommandBufferDependency(_gd.CommandBufferPool);
if (!isRented && _gd.CommandBufferPool.OwnedByCurrentThread && !_flushLock.IsReadLockHeld && (_pendingData == null || cbs != null))
{
var currentAllocation = _allocationAuto;
var currentBuffer = _buffer;
IntPtr currentMap = _map;
(VkBuffer buffer, MemoryAllocation allocation, BufferAllocationType resultType) = _gd.BufferManager.CreateBacking(_gd, Size, DesiredType, false, false, _currentType);
if (buffer.Handle != 0)
{
if (cbs != null)
{
ClearMirrors(cbs.Value, 0, Size);
}
_flushLock.EnterWriteLock();
ClearFlushFence();
_waitable = new MultiFenceHolder(Size);
_allocation = allocation;
_allocationAuto = new Auto<MemoryAllocation>(allocation);
_buffer = new Auto<DisposableBuffer>(new DisposableBuffer(_gd.Api, _device, buffer), this, _waitable, _allocationAuto);
_bufferHandle = buffer.Handle;
_map = allocation.HostPointer;
if (_map != IntPtr.Zero && currentMap != IntPtr.Zero)
{
// Copy data directly. Readbacks don't have to wait if this is done.
unsafe
{
new Span<byte>((void*)currentMap, Size).CopyTo(new Span<byte>((void*)_map, Size));
}
}
else
{
cbs ??= _gd.CommandBufferPool.Rent();
CommandBufferScoped cbsV = cbs.Value;
Copy(_gd, cbsV, currentBuffer, _buffer, 0, 0, Size);
// Need to wait for the data to reach the new buffer before data can be flushed.
_flushFence = _gd.CommandBufferPool.GetFence(cbsV.CommandBufferIndex);
_flushFence.Get();
}
Logger.Debug?.PrintMsg(LogClass.Gpu, $"Converted {Size} buffer {_currentType} to {resultType}");
_currentType = resultType;
if (_swapActions != null)
{
foreach (var action in _swapActions)
{
action();
}
_swapActions.Clear();
}
currentBuffer.Dispose();
currentAllocation.Dispose();
_gd.PipelineInternal.SwapBuffer(currentBuffer, _buffer);
_flushLock.ExitWriteLock();
}
_swapQueued = false;
return true;
}
return false;
}
_swapQueued = false;
return true;
}
private void ConsiderBackingSwap()
{
if (_baseType == BufferAllocationType.Auto)
{
// When flushed, wait for a bit more info to make a decision.
bool wasFlushed = _flushTemp > 0;
int multiplier = wasFlushed ? 2 : 0;
if (_writeCount >= (WriteCountThreshold << multiplier) || _setCount >= (SetCountThreshold << multiplier) || _flushCount >= (FlushCountThreshold << multiplier))
{
if (_flushCount > 0 || _flushTemp-- > 0)
{
// Buffers that flush should ideally be mapped in host address space for easy copies.
// If the buffer is large it will do better on GPU memory, as there will be more writes than data flushes (typically individual pages).
// If it is small, then it's likely most of the buffer will be flushed so we want it on host memory, as access is cached.
bool hostMappingSensitive = _gd.Vendor == Vendor.Nvidia;
bool deviceLocalMapped = Size > DeviceLocalSizeThreshold || (wasFlushed && _writeCount > _flushCount * 10 && hostMappingSensitive) || _currentType == BufferAllocationType.DeviceLocalMapped;
DesiredType = deviceLocalMapped ? BufferAllocationType.DeviceLocalMapped : BufferAllocationType.HostMapped;
// It's harder for a buffer that is flushed to revert to another type of mapping.
if (_flushCount > 0)
{
_flushTemp = 1000;
}
}
else if (_writeCount >= (WriteCountThreshold << multiplier))
{
// Buffers that are written often should ideally be in the device local heap. (Storage buffers)
DesiredType = BufferAllocationType.DeviceLocal;
}
else if (_setCount > (SetCountThreshold << multiplier))
{
// Buffers that have their data set often should ideally be host mapped. (Constant buffers)
DesiredType = BufferAllocationType.HostMapped;
}
_lastFlushWrite = -1;
_flushCount = 0;
_writeCount = 0;
_setCount = 0;
}
if (!_swapQueued && DesiredType != _currentType)
{
_swapQueued = true;
_gd.PipelineInternal.AddBackingSwap(this);
}
}
}
public void Pin()
{
if (_baseType == BufferAllocationType.Auto)
{
_baseType = _currentType;
}
}
public unsafe Auto<DisposableBufferView> CreateView(VkFormat format, int offset, int size, Action invalidateView)
{
var bufferViewCreateInfo = new BufferViewCreateInfo
@ -291,19 +124,9 @@ namespace Ryujinx.Graphics.Vulkan
_gd.Api.CreateBufferView(_device, bufferViewCreateInfo, null, out var bufferView).ThrowOnError();
(_swapActions ??= new List<Action>()).Add(invalidateView);
return new Auto<DisposableBufferView>(new DisposableBufferView(_gd.Api, _device, bufferView), this, _waitable, _buffer);
}
public void InheritMetrics(BufferHolder other)
{
_setCount = other._setCount;
_writeCount = other._writeCount;
_flushCount = other._flushCount;
_flushTemp = other._flushTemp;
}
public unsafe void InsertBarrier(CommandBuffer commandBuffer, bool isWrite)
{
// If the last access is write, we always need a barrier to be sure we will read or modify
@ -423,18 +246,8 @@ namespace Ryujinx.Graphics.Vulkan
{
if (isWrite)
{
_writeCount++;
SignalWrite(0, Size);
}
else if (isSSBO)
{
// Always consider SSBO access for swapping to device local memory.
_writeCount++;
ConsiderBackingSwap();
}
return _buffer;
}
@ -443,8 +256,6 @@ namespace Ryujinx.Graphics.Vulkan
{
if (isWrite)
{
_writeCount++;
SignalWrite(offset, size);
}
@ -543,8 +354,6 @@ namespace Ryujinx.Graphics.Vulkan
public void SignalWrite(int offset, int size)
{
ConsiderBackingSwap();
if (offset == 0 && size == Size)
{
_cachedConvertedBuffers.Clear();
@ -624,13 +433,6 @@ namespace Ryujinx.Graphics.Vulkan
WaitForFlushFence();
if (_lastFlushWrite != _writeCount)
{
// If it's on the same page as the last flush, ignore it.
_lastFlushWrite = _writeCount;
_flushCount++;
}
Span<byte> result;
if (_map != IntPtr.Zero)
@ -711,8 +513,7 @@ namespace Ryujinx.Graphics.Vulkan
return;
}
_setCount++;
bool allowMirror = _useMirrors && allowCbsWait && cbs != null && _currentType <= BufferAllocationType.HostMapped;
bool allowMirror = _useMirrors && allowCbsWait && cbs != null && _activeType <= BufferAllocationType.HostMapped;
if (_map != IntPtr.Zero)
{
@ -863,8 +664,6 @@ namespace Ryujinx.Graphics.Vulkan
var dstBuffer = GetBuffer(cbs.CommandBuffer, dstOffset, data.Length, true).Get(cbs, dstOffset, data.Length, true).Value;
_writeCount--;
InsertBufferBarrier(
_gd,
cbs.CommandBuffer,
@ -1100,8 +899,6 @@ namespace Ryujinx.Graphics.Vulkan
public void Dispose()
{
_swapQueued = false;
_gd.PipelineInternal?.FlushCommandsIfWeightExceeding(_buffer, (ulong)Size);
_buffer.Dispose();

View File

@ -165,10 +165,6 @@ namespace Ryujinx.Graphics.Vulkan
if (TryGetBuffer(range.Handle, out var existingHolder))
{
// Since this buffer now also owns the memory from the referenced buffer,
// we pin it to ensure the memory location will not change.
existingHolder.Pin();
(var memory, var offset) = existingHolder.GetDeviceMemoryAndOffset();
memoryBinds[index] = new SparseMemoryBind()
@ -235,10 +231,9 @@ namespace Ryujinx.Graphics.Vulkan
int size,
bool sparseCompatible = false,
BufferAllocationType baseType = BufferAllocationType.HostMapped,
BufferHandle storageHint = default,
bool forceMirrors = false)
{
return CreateWithHandle(gd, size, out _, sparseCompatible, baseType, storageHint, forceMirrors);
return CreateWithHandle(gd, size, out _, sparseCompatible, baseType, forceMirrors);
}
public BufferHandle CreateWithHandle(
@ -247,10 +242,9 @@ namespace Ryujinx.Graphics.Vulkan
out BufferHolder holder,
bool sparseCompatible = false,
BufferAllocationType baseType = BufferAllocationType.HostMapped,
BufferHandle storageHint = default,
bool forceMirrors = false)
{
holder = Create(gd, size, forConditionalRendering: false, sparseCompatible, baseType, storageHint);
holder = Create(gd, size, forConditionalRendering: false, sparseCompatible, baseType);
if (holder == null)
{
return BufferHandle.Null;
@ -387,31 +381,13 @@ namespace Ryujinx.Graphics.Vulkan
int size,
bool forConditionalRendering = false,
bool sparseCompatible = false,
BufferAllocationType baseType = BufferAllocationType.HostMapped,
BufferHandle storageHint = default)
BufferAllocationType baseType = BufferAllocationType.HostMapped)
{
BufferAllocationType type = baseType;
BufferHolder storageHintHolder = null;
if (baseType == BufferAllocationType.Auto)
{
if (gd.IsSharedMemory)
{
baseType = BufferAllocationType.HostMapped;
type = baseType;
}
else
{
type = size >= BufferHolder.DeviceLocalSizeThreshold ? BufferAllocationType.DeviceLocal : BufferAllocationType.HostMapped;
}
if (storageHint != BufferHandle.Null)
{
if (TryGetBuffer(storageHint, out storageHintHolder))
{
type = storageHintHolder.DesiredType;
}
}
type = BufferAllocationType.HostMapped;
}
(VkBuffer buffer, MemoryAllocation allocation, BufferAllocationType resultType) =
@ -421,11 +397,6 @@ namespace Ryujinx.Graphics.Vulkan
{
var holder = new BufferHolder(gd, _device, buffer, allocation, size, baseType, resultType);
if (storageHintHolder != null)
{
holder.InheritMetrics(storageHintHolder);
}
return holder;
}

View File

@ -69,17 +69,7 @@ namespace Ryujinx.Graphics.Vulkan
}
}
private record struct ArrayRef<T>
{
public ShaderStage Stage;
public T Array;
public ArrayRef(ShaderStage stage, T array)
{
Stage = stage;
Array = array;
}
}
private readonly record struct ArrayRef<T>(ShaderStage Stage, T Array);
private readonly VulkanRenderer _gd;
private readonly Device _device;
@ -97,6 +87,9 @@ namespace Ryujinx.Graphics.Vulkan
private ArrayRef<TextureArray>[] _textureArrayRefs;
private ArrayRef<ImageArray>[] _imageArrayRefs;
private ArrayRef<TextureArray>[] _textureArrayExtraRefs;
private ArrayRef<ImageArray>[] _imageArrayExtraRefs;
private readonly DescriptorBufferInfo[] _uniformBuffers;
private readonly DescriptorBufferInfo[] _storageBuffers;
private readonly DescriptorImageInfo[] _textures;
@ -152,6 +145,9 @@ namespace Ryujinx.Graphics.Vulkan
_textureArrayRefs = Array.Empty<ArrayRef<TextureArray>>();
_imageArrayRefs = Array.Empty<ArrayRef<ImageArray>>();
_textureArrayExtraRefs = Array.Empty<ArrayRef<TextureArray>>();
_imageArrayExtraRefs = Array.Empty<ArrayRef<ImageArray>>();
_uniformBuffers = new DescriptorBufferInfo[Constants.MaxUniformBufferBindings];
_storageBuffers = new DescriptorBufferInfo[Constants.MaxStorageBufferBindings];
_textures = new DescriptorImageInfo[Constants.MaxTexturesPerStage];
@ -295,8 +291,9 @@ namespace Ryujinx.Graphics.Vulkan
}
else
{
PipelineStageFlags stageFlags = _textureArrayRefs[segment.Binding].Stage.ConvertToPipelineStageFlags();
_textureArrayRefs[segment.Binding].Array?.QueueWriteToReadBarriers(cbs, stageFlags);
ref var arrayRef = ref _textureArrayRefs[segment.Binding];
PipelineStageFlags stageFlags = arrayRef.Stage.ConvertToPipelineStageFlags();
arrayRef.Array?.QueueWriteToReadBarriers(cbs, stageFlags);
}
}
}
@ -315,8 +312,40 @@ namespace Ryujinx.Graphics.Vulkan
}
else
{
PipelineStageFlags stageFlags = _imageArrayRefs[segment.Binding].Stage.ConvertToPipelineStageFlags();
_imageArrayRefs[segment.Binding].Array?.QueueWriteToReadBarriers(cbs, stageFlags);
ref var arrayRef = ref _imageArrayRefs[segment.Binding];
PipelineStageFlags stageFlags = arrayRef.Stage.ConvertToPipelineStageFlags();
arrayRef.Array?.QueueWriteToReadBarriers(cbs, stageFlags);
}
}
}
for (int setIndex = PipelineBase.DescriptorSetLayouts; setIndex < _program.BindingSegments.Length; setIndex++)
{
var bindingSegments = _program.BindingSegments[setIndex];
if (bindingSegments.Length == 0)
{
continue;
}
ResourceBindingSegment segment = bindingSegments[0];
if (segment.IsArray)
{
if (segment.Type == ResourceType.Texture ||
segment.Type == ResourceType.Sampler ||
segment.Type == ResourceType.TextureAndSampler ||
segment.Type == ResourceType.BufferTexture)
{
ref var arrayRef = ref _textureArrayExtraRefs[setIndex - PipelineBase.DescriptorSetLayouts];
PipelineStageFlags stageFlags = arrayRef.Stage.ConvertToPipelineStageFlags();
arrayRef.Array?.QueueWriteToReadBarriers(cbs, stageFlags);
}
else if (segment.Type == ResourceType.Image || segment.Type == ResourceType.BufferImage)
{
ref var arrayRef = ref _imageArrayExtraRefs[setIndex - PipelineBase.DescriptorSetLayouts];
PipelineStageFlags stageFlags = arrayRef.Stage.ConvertToPipelineStageFlags();
arrayRef.Array?.QueueWriteToReadBarriers(cbs, stageFlags);
}
}
}
@ -495,25 +524,39 @@ namespace Ryujinx.Graphics.Vulkan
public void SetTextureArray(CommandBufferScoped cbs, ShaderStage stage, int binding, ITextureArray array)
{
if (_textureArrayRefs.Length <= binding)
{
Array.Resize(ref _textureArrayRefs, binding + ArrayGrowthSize);
}
ref ArrayRef<TextureArray> arrayRef = ref GetArrayRef(ref _textureArrayRefs, binding, ArrayGrowthSize);
if (_textureArrayRefs[binding].Stage != stage || _textureArrayRefs[binding].Array != array)
if (arrayRef.Stage != stage || arrayRef.Array != array)
{
if (_textureArrayRefs[binding].Array != null)
{
_textureArrayRefs[binding].Array.Bound = false;
}
arrayRef.Array?.DecrementBindCount();
if (array is TextureArray textureArray)
{
textureArray.Bound = true;
textureArray.IncrementBindCount();
textureArray.QueueWriteToReadBarriers(cbs, stage.ConvertToPipelineStageFlags());
}
_textureArrayRefs[binding] = new ArrayRef<TextureArray>(stage, array as TextureArray);
arrayRef = new ArrayRef<TextureArray>(stage, array as TextureArray);
SignalDirty(DirtyFlags.Texture);
}
}
public void SetTextureArraySeparate(CommandBufferScoped cbs, ShaderStage stage, int setIndex, ITextureArray array)
{
ref ArrayRef<TextureArray> arrayRef = ref GetArrayRef(ref _textureArrayExtraRefs, setIndex - PipelineBase.DescriptorSetLayouts);
if (arrayRef.Stage != stage || arrayRef.Array != array)
{
arrayRef.Array?.DecrementBindCount();
if (array is TextureArray textureArray)
{
textureArray.IncrementBindCount();
textureArray.QueueWriteToReadBarriers(cbs, stage.ConvertToPipelineStageFlags());
}
arrayRef = new ArrayRef<TextureArray>(stage, array as TextureArray);
SignalDirty(DirtyFlags.Texture);
}
@ -521,30 +564,56 @@ namespace Ryujinx.Graphics.Vulkan
public void SetImageArray(CommandBufferScoped cbs, ShaderStage stage, int binding, IImageArray array)
{
if (_imageArrayRefs.Length <= binding)
{
Array.Resize(ref _imageArrayRefs, binding + ArrayGrowthSize);
}
ref ArrayRef<ImageArray> arrayRef = ref GetArrayRef(ref _imageArrayRefs, binding, ArrayGrowthSize);
if (_imageArrayRefs[binding].Stage != stage || _imageArrayRefs[binding].Array != array)
if (arrayRef.Stage != stage || arrayRef.Array != array)
{
if (_imageArrayRefs[binding].Array != null)
{
_imageArrayRefs[binding].Array.Bound = false;
}
arrayRef.Array?.DecrementBindCount();
if (array is ImageArray imageArray)
{
imageArray.Bound = true;
imageArray.IncrementBindCount();
imageArray.QueueWriteToReadBarriers(cbs, stage.ConvertToPipelineStageFlags());
}
_imageArrayRefs[binding] = new ArrayRef<ImageArray>(stage, array as ImageArray);
arrayRef = new ArrayRef<ImageArray>(stage, array as ImageArray);
SignalDirty(DirtyFlags.Image);
}
}
public void SetImageArraySeparate(CommandBufferScoped cbs, ShaderStage stage, int setIndex, IImageArray array)
{
ref ArrayRef<ImageArray> arrayRef = ref GetArrayRef(ref _imageArrayExtraRefs, setIndex - PipelineBase.DescriptorSetLayouts);
if (arrayRef.Stage != stage || arrayRef.Array != array)
{
arrayRef.Array?.DecrementBindCount();
if (array is ImageArray imageArray)
{
imageArray.IncrementBindCount();
imageArray.QueueWriteToReadBarriers(cbs, stage.ConvertToPipelineStageFlags());
}
arrayRef = new ArrayRef<ImageArray>(stage, array as ImageArray);
SignalDirty(DirtyFlags.Image);
}
}
private static ref ArrayRef<T> GetArrayRef<T>(ref ArrayRef<T>[] array, int index, int growthSize = 1)
{
ArgumentOutOfRangeException.ThrowIfNegative(index);
if (array.Length <= index)
{
Array.Resize(ref array, index + growthSize);
}
return ref array[index];
}
public void SetUniformBuffers(CommandBuffer commandBuffer, ReadOnlySpan<BufferAssignment> buffers)
{
for (int i = 0; i < buffers.Length; i++)
@ -594,31 +663,40 @@ namespace Ryujinx.Graphics.Vulkan
return;
}
var program = _program;
if (_dirty.HasFlag(DirtyFlags.Uniform))
{
if (_program.UsePushDescriptors)
if (program.UsePushDescriptors)
{
UpdateAndBindUniformBufferPd(cbs, pbp);
UpdateAndBindUniformBufferPd(cbs);
}
else
{
UpdateAndBind(cbs, PipelineBase.UniformSetIndex, pbp);
UpdateAndBind(cbs, program, PipelineBase.UniformSetIndex, pbp);
}
}
if (_dirty.HasFlag(DirtyFlags.Storage))
{
UpdateAndBind(cbs, PipelineBase.StorageSetIndex, pbp);
UpdateAndBind(cbs, program, PipelineBase.StorageSetIndex, pbp);
}
if (_dirty.HasFlag(DirtyFlags.Texture))
{
UpdateAndBind(cbs, PipelineBase.TextureSetIndex, pbp);
UpdateAndBind(cbs, program, PipelineBase.TextureSetIndex, pbp);
}
if (_dirty.HasFlag(DirtyFlags.Image))
{
UpdateAndBind(cbs, PipelineBase.ImageSetIndex, pbp);
UpdateAndBind(cbs, program, PipelineBase.ImageSetIndex, pbp);
}
if (program.BindingSegments.Length > PipelineBase.DescriptorSetLayouts)
{
// Program is using extra sets, we need to bind those too.
BindExtraSets(cbs, program, pbp);
}
_dirty = DirtyFlags.None;
@ -658,9 +736,8 @@ namespace Ryujinx.Graphics.Vulkan
}
[MethodImpl(MethodImplOptions.AggressiveInlining)]
private void UpdateAndBind(CommandBufferScoped cbs, int setIndex, PipelineBindPoint pbp)
private void UpdateAndBind(CommandBufferScoped cbs, ShaderCollection program, int setIndex, PipelineBindPoint pbp)
{
var program = _program;
var bindingSegments = program.BindingSegments[setIndex];
if (bindingSegments.Length == 0)
@ -869,7 +946,7 @@ namespace Ryujinx.Graphics.Vulkan
}
[MethodImpl(MethodImplOptions.AggressiveInlining)]
private void UpdateAndBindUniformBufferPd(CommandBufferScoped cbs, PipelineBindPoint pbp)
private void UpdateAndBindUniformBufferPd(CommandBufferScoped cbs)
{
int sequence = _pdSequence;
var bindingSegments = _program.BindingSegments[PipelineBase.UniformSetIndex];
@ -933,6 +1010,56 @@ namespace Ryujinx.Graphics.Vulkan
}
}
private void BindExtraSets(CommandBufferScoped cbs, ShaderCollection program, PipelineBindPoint pbp)
{
for (int setIndex = PipelineBase.DescriptorSetLayouts; setIndex < program.BindingSegments.Length; setIndex++)
{
var bindingSegments = program.BindingSegments[setIndex];
if (bindingSegments.Length == 0)
{
continue;
}
ResourceBindingSegment segment = bindingSegments[0];
if (segment.IsArray)
{
DescriptorSet[] sets = null;
if (segment.Type == ResourceType.Texture ||
segment.Type == ResourceType.Sampler ||
segment.Type == ResourceType.TextureAndSampler ||
segment.Type == ResourceType.BufferTexture)
{
sets = _textureArrayExtraRefs[setIndex - PipelineBase.DescriptorSetLayouts].Array.GetDescriptorSets(
_device,
cbs,
_templateUpdater,
program,
setIndex,
_dummyTexture,
_dummySampler);
}
else if (segment.Type == ResourceType.Image || segment.Type == ResourceType.BufferImage)
{
sets = _imageArrayExtraRefs[setIndex - PipelineBase.DescriptorSetLayouts].Array.GetDescriptorSets(
_device,
cbs,
_templateUpdater,
program,
setIndex,
_dummyTexture);
}
if (sets != null)
{
_gd.Api.CmdBindDescriptorSets(cbs.CommandBuffer, pbp, _program.PipelineLayout, (uint)setIndex, 1, sets, 0, ReadOnlySpan<uint>.Empty);
}
}
}
}
public void SignalCommandBufferChange()
{
_updateDescriptorCacheCbIndex = true;

View File

@ -424,10 +424,20 @@ namespace Ryujinx.Graphics.Vulkan
public static BufferAllocationType Convert(this BufferAccess access)
{
if (access.HasFlag(BufferAccess.FlushPersistent) || access.HasFlag(BufferAccess.Stream))
BufferAccess memType = access & BufferAccess.MemoryTypeMask;
if (memType == BufferAccess.HostMemory || access.HasFlag(BufferAccess.Stream))
{
return BufferAllocationType.HostMapped;
}
else if (memType == BufferAccess.DeviceMemory)
{
return BufferAllocationType.DeviceLocal;
}
else if (memType == BufferAccess.DeviceMemoryMapped)
{
return BufferAllocationType.DeviceLocalMapped;
}
return BufferAllocationType.Auto;
}

View File

@ -24,6 +24,7 @@ namespace Ryujinx.Graphics.Vulkan
public VkFormat[] AttachmentFormats { get; }
public int[] AttachmentIndices { get; }
public uint AttachmentIntegerFormatMask { get; }
public bool LogicOpsAllowed { get; }
public int AttachmentsCount { get; }
public int MaxColorAttachmentIndex => AttachmentIndices.Length > 0 ? AttachmentIndices[^1] : -1;
@ -32,7 +33,9 @@ namespace Ryujinx.Graphics.Vulkan
public FramebufferParams(Device device, TextureView view, uint width, uint height)
{
bool isDepthStencil = view.Info.Format.IsDepthOrStencil();
var format = view.Info.Format;
bool isDepthStencil = format.IsDepthOrStencil();
_device = device;
_attachments = new[] { view.GetImageViewForAttachment() };
@ -56,6 +59,8 @@ namespace Ryujinx.Graphics.Vulkan
AttachmentSamples = new[] { (uint)view.Info.Samples };
AttachmentFormats = new[] { view.VkFormat };
AttachmentIndices = isDepthStencil ? Array.Empty<int>() : new[] { 0 };
AttachmentIntegerFormatMask = format.IsInteger() ? 1u : 0u;
LogicOpsAllowed = !format.IsFloatOrSrgb();
AttachmentsCount = 1;
@ -85,6 +90,7 @@ namespace Ryujinx.Graphics.Vulkan
int index = 0;
int bindIndex = 0;
uint attachmentIntegerFormatMask = 0;
bool allFormatsFloatOrSrgb = colorsCount != 0;
foreach (ITexture color in colors)
{
@ -101,11 +107,15 @@ namespace Ryujinx.Graphics.Vulkan
AttachmentFormats[index] = texture.VkFormat;
AttachmentIndices[index] = bindIndex;
if (texture.Info.Format.IsInteger())
var format = texture.Info.Format;
if (format.IsInteger())
{
attachmentIntegerFormatMask |= 1u << bindIndex;
}
allFormatsFloatOrSrgb &= format.IsFloatOrSrgb();
width = Math.Min(width, (uint)texture.Width);
height = Math.Min(height, (uint)texture.Height);
layers = Math.Min(layers, (uint)texture.Layers);
@ -120,6 +130,7 @@ namespace Ryujinx.Graphics.Vulkan
}
AttachmentIntegerFormatMask = attachmentIntegerFormatMask;
LogicOpsAllowed = !allFormatsFloatOrSrgb;
if (depthStencil is TextureView dsTexture && dsTexture.Valid)
{

View File

@ -5,7 +5,7 @@ using System.Collections.Generic;
namespace Ryujinx.Graphics.Vulkan
{
class ImageArray : IImageArray
class ImageArray : ResourceArray, IImageArray
{
private readonly VulkanRenderer _gd;
@ -29,8 +29,6 @@ namespace Ryujinx.Graphics.Vulkan
private readonly bool _isBuffer;
public bool Bound;
public ImageArray(VulkanRenderer gd, int size, bool isBuffer)
{
_gd = gd;
@ -97,8 +95,7 @@ namespace Ryujinx.Graphics.Vulkan
{
_cachedCommandBufferIndex = -1;
_storages = null;
_gd.PipelineInternal.ForceImageDirty();
SetDirty(_gd);
}
public void QueueWriteToReadBarriers(CommandBufferScoped cbs, PipelineStageFlags stageFlags)
@ -175,5 +172,47 @@ namespace Ryujinx.Graphics.Vulkan
return bufferTextures;
}
public DescriptorSet[] GetDescriptorSets(
Device device,
CommandBufferScoped cbs,
DescriptorSetTemplateUpdater templateUpdater,
ShaderCollection program,
int setIndex,
TextureView dummyTexture)
{
if (TryGetCachedDescriptorSets(cbs, program, setIndex, out DescriptorSet[] sets))
{
// We still need to ensure the current command buffer holds a reference to all used textures.
if (!_isBuffer)
{
GetImageInfos(_gd, cbs, dummyTexture);
}
else
{
GetBufferViews(cbs);
}
return sets;
}
DescriptorSetTemplate template = program.Templates[setIndex];
DescriptorSetTemplateWriter tu = templateUpdater.Begin(template);
if (!_isBuffer)
{
tu.Push(GetImageInfos(_gd, cbs, dummyTexture));
}
else
{
tu.Push(GetBufferViews(cbs));
}
templateUpdater.Commit(_gd, device, sets[0]);
return sets;
}
}
}

View File

@ -1,3 +1,4 @@
using Ryujinx.Common.Memory;
using Silk.NET.Vulkan;
using System;
@ -165,14 +166,15 @@ namespace Ryujinx.Graphics.Vulkan
/// <returns>True if all fences were signaled before the timeout expired, false otherwise</returns>
private bool WaitForFencesImpl(Vk api, Device device, int offset, int size, bool hasTimeout, ulong timeout)
{
Span<FenceHolder> fenceHolders = new FenceHolder[CommandBufferPool.MaxCommandBuffers];
using SpanOwner<FenceHolder> fenceHoldersOwner = SpanOwner<FenceHolder>.Rent(CommandBufferPool.MaxCommandBuffers);
Span<FenceHolder> fenceHolders = fenceHoldersOwner.Span;
int count = size != 0 ? GetOverlappingFences(fenceHolders, offset, size) : GetFences(fenceHolders);
Span<Fence> fences = stackalloc Fence[count];
int fenceCount = 0;
for (int i = 0; i < count; i++)
for (int i = 0; i < fences.Length; i++)
{
if (fenceHolders[i].TryGet(out Fence fence))
{

View File

@ -751,14 +751,12 @@ namespace Ryujinx.Graphics.Vulkan
_vertexBufferUpdater.Commit(Cbs);
}
#pragma warning disable CA1822 // Mark member as static
public void SetAlphaTest(bool enable, float reference, CompareOp op)
{
// This is currently handled using shader specialization, as Vulkan does not support alpha test.
// In the future, we may want to use this to write the reference value into the support buffer,
// to avoid creating one version of the shader per reference value used.
}
#pragma warning restore CA1822
public void SetBlendState(AdvancedBlendDescriptor blend)
{
@ -903,6 +901,11 @@ namespace Ryujinx.Graphics.Vulkan
_descriptorSetUpdater.SetImageArray(Cbs, stage, binding, array);
}
public void SetImageArraySeparate(ShaderStage stage, int setIndex, IImageArray array)
{
_descriptorSetUpdater.SetImageArraySeparate(Cbs, stage, setIndex, array);
}
public void SetIndexBuffer(BufferRange buffer, IndexType type)
{
if (buffer.Handle != BufferHandle.Null)
@ -945,7 +948,6 @@ namespace Ryujinx.Graphics.Vulkan
// TODO: Default levels (likely needs emulation on shaders?)
}
#pragma warning disable CA1822 // Mark member as static
public void SetPointParameters(float size, bool isProgramPointSize, bool enablePointSprite, Origin origin)
{
// TODO.
@ -955,7 +957,6 @@ namespace Ryujinx.Graphics.Vulkan
{
// TODO.
}
#pragma warning restore CA1822
public void SetPrimitiveRestart(bool enable, int index)
{
@ -1156,6 +1157,11 @@ namespace Ryujinx.Graphics.Vulkan
_descriptorSetUpdater.SetTextureArray(Cbs, stage, binding, array);
}
public void SetTextureArraySeparate(ShaderStage stage, int setIndex, ITextureArray array)
{
_descriptorSetUpdater.SetTextureArraySeparate(Cbs, stage, setIndex, array);
}
public void SetTransformFeedbackBuffers(ReadOnlySpan<BufferRange> buffers)
{
PauseTransformFeedbackInternal();
@ -1186,12 +1192,10 @@ namespace Ryujinx.Graphics.Vulkan
_descriptorSetUpdater.SetUniformBuffers(CommandBuffer, buffers);
}
#pragma warning disable CA1822 // Mark member as static
public void SetUserClipDistance(int index, bool enableClip)
{
// TODO.
}
#pragma warning restore CA1822
public void SetVertexAttribs(ReadOnlySpan<VertexAttribDescriptor> vertexAttribs)
{
@ -1498,6 +1502,7 @@ namespace Ryujinx.Graphics.Vulkan
var dstAttachmentFormats = _newState.Internal.AttachmentFormats.AsSpan();
FramebufferParams.AttachmentFormats.CopyTo(dstAttachmentFormats);
_newState.Internal.AttachmentIntegerFormatMask = FramebufferParams.AttachmentIntegerFormatMask;
_newState.Internal.LogicOpsAllowed = FramebufferParams.LogicOpsAllowed;
for (int i = FramebufferParams.AttachmentFormats.Length; i < dstAttachmentFormats.Length; i++)
{

View File

@ -180,9 +180,6 @@ namespace Ryujinx.Graphics.Vulkan
pipeline.LogicOpEnable = state.LogicOpEnable;
pipeline.LogicOp = state.LogicOp.Convert();
pipeline.MinDepthBounds = 0f; // Not implemented.
pipeline.MaxDepthBounds = 0f; // Not implemented.
pipeline.PatchControlPoints = state.PatchControlPoints;
pipeline.PolygonMode = PolygonMode.Fill; // Not implemented.
pipeline.PrimitiveRestartEnable = state.PrimitiveRestartEnable;
@ -208,17 +205,11 @@ namespace Ryujinx.Graphics.Vulkan
pipeline.StencilFrontPassOp = state.StencilTest.FrontDpPass.Convert();
pipeline.StencilFrontDepthFailOp = state.StencilTest.FrontDpFail.Convert();
pipeline.StencilFrontCompareOp = state.StencilTest.FrontFunc.Convert();
pipeline.StencilFrontCompareMask = 0;
pipeline.StencilFrontWriteMask = 0;
pipeline.StencilFrontReference = 0;
pipeline.StencilBackFailOp = state.StencilTest.BackSFail.Convert();
pipeline.StencilBackPassOp = state.StencilTest.BackDpPass.Convert();
pipeline.StencilBackDepthFailOp = state.StencilTest.BackDpFail.Convert();
pipeline.StencilBackCompareOp = state.StencilTest.BackFunc.Convert();
pipeline.StencilBackCompareMask = 0;
pipeline.StencilBackWriteMask = 0;
pipeline.StencilBackReference = 0;
pipeline.StencilTestEnable = state.StencilTest.TestEnable;
@ -302,6 +293,7 @@ namespace Ryujinx.Graphics.Vulkan
int attachmentCount = 0;
int maxColorAttachmentIndex = -1;
uint attachmentIntegerFormatMask = 0;
bool allFormatsFloatOrSrgb = true;
for (int i = 0; i < Constants.MaxRenderTargets; i++)
{
@ -314,6 +306,8 @@ namespace Ryujinx.Graphics.Vulkan
{
attachmentIntegerFormatMask |= 1u << i;
}
allFormatsFloatOrSrgb &= state.AttachmentFormats[i].IsFloatOrSrgb();
}
}
@ -325,6 +319,7 @@ namespace Ryujinx.Graphics.Vulkan
pipeline.ColorBlendAttachmentStateCount = (uint)(maxColorAttachmentIndex + 1);
pipeline.VertexAttributeDescriptionsCount = (uint)Math.Min(Constants.MaxVertexAttributes, state.VertexAttribCount);
pipeline.Internal.AttachmentIntegerFormatMask = attachmentIntegerFormatMask;
pipeline.Internal.LogicOpsAllowed = attachmentCount == 0 || !allFormatsFloatOrSrgb;
return pipeline;
}

View File

@ -222,20 +222,6 @@ namespace Ryujinx.Graphics.Vulkan
}
}
private void TryBackingSwaps()
{
CommandBufferScoped? cbs = null;
_backingSwaps.RemoveAll(holder => holder.TryBackingSwap(ref cbs));
cbs?.Dispose();
}
public void AddBackingSwap(BufferHolder holder)
{
_backingSwaps.Add(holder);
}
public void Restore()
{
if (Pipeline != null)
@ -291,8 +277,6 @@ namespace Ryujinx.Graphics.Vulkan
Gd.ResetCounterPool();
TryBackingSwaps();
Restore();
}

View File

@ -3,6 +3,8 @@ using Silk.NET.Vulkan;
using System;
using System.Collections.Generic;
using System.Collections.ObjectModel;
using System.Diagnostics;
using System.Runtime.InteropServices;
namespace Ryujinx.Graphics.Vulkan
{
@ -14,6 +16,7 @@ namespace Ryujinx.Graphics.Vulkan
private readonly Device _device;
public DescriptorSetLayout[] DescriptorSetLayouts { get; }
public bool[] DescriptorSetLayoutsUpdateAfterBind { get; }
public PipelineLayout PipelineLayout { get; }
private readonly int[] _consumedDescriptorsPerSet;
@ -27,6 +30,41 @@ namespace Ryujinx.Graphics.Vulkan
private int _dsLastCbIndex;
private int _dsLastSubmissionCount;
private struct ManualDescriptorSetEntry
{
public Auto<DescriptorSetCollection> DescriptorSet;
public uint CbRefMask;
public bool InUse;
public ManualDescriptorSetEntry(Auto<DescriptorSetCollection> descriptorSet, int cbIndex)
{
DescriptorSet = descriptorSet;
CbRefMask = 1u << cbIndex;
InUse = true;
}
}
private readonly struct PendingManualDsConsumption
{
public FenceHolder Fence { get; }
public int CommandBufferIndex { get; }
public int SetIndex { get; }
public int CacheIndex { get; }
public PendingManualDsConsumption(FenceHolder fence, int commandBufferIndex, int setIndex, int cacheIndex)
{
Fence = fence;
CommandBufferIndex = commandBufferIndex;
SetIndex = setIndex;
CacheIndex = cacheIndex;
fence.Get();
}
}
private readonly List<ManualDescriptorSetEntry>[] _manualDsCache;
private readonly Queue<PendingManualDsConsumption> _pendingManualDsConsumptions;
private readonly Queue<int>[] _freeManualDsCacheEntries;
private readonly Dictionary<long, DescriptorSetTemplate> _pdTemplates;
private readonly ResourceDescriptorCollection _pdDescriptors;
private long _lastPdUsage;
@ -50,6 +88,9 @@ namespace Ryujinx.Graphics.Vulkan
}
_dsCacheCursor = new int[setsCount];
_manualDsCache = new List<ManualDescriptorSetEntry>[setsCount];
_pendingManualDsConsumptions = new Queue<PendingManualDsConsumption>();
_freeManualDsCacheEntries = new Queue<int>[setsCount];
}
public PipelineLayoutCacheEntry(
@ -58,7 +99,11 @@ namespace Ryujinx.Graphics.Vulkan
ReadOnlyCollection<ResourceDescriptorCollection> setDescriptors,
bool usePushDescriptors) : this(gd, device, setDescriptors.Count)
{
(DescriptorSetLayouts, PipelineLayout) = PipelineLayoutFactory.Create(gd, device, setDescriptors, usePushDescriptors);
ResourceLayouts layouts = PipelineLayoutFactory.Create(gd, device, setDescriptors, usePushDescriptors);
DescriptorSetLayouts = layouts.DescriptorSetLayouts;
DescriptorSetLayoutsUpdateAfterBind = layouts.DescriptorSetLayoutsUpdateAfterBind;
PipelineLayout = layouts.PipelineLayout;
_consumedDescriptorsPerSet = new int[setDescriptors.Count];
_poolSizes = new DescriptorPoolSize[setDescriptors.Count][];
@ -113,7 +158,7 @@ namespace Ryujinx.Graphics.Vulkan
_poolSizes[setIndex],
setIndex,
_consumedDescriptorsPerSet[setIndex],
false);
DescriptorSetLayoutsUpdateAfterBind[setIndex]);
list.Add(dsc);
isNew = true;
@ -124,6 +169,101 @@ namespace Ryujinx.Graphics.Vulkan
return list[index];
}
public Auto<DescriptorSetCollection> GetNewManualDescriptorSetCollection(CommandBufferScoped cbs, int setIndex, out int cacheIndex)
{
FreeCompletedManualDescriptorSets();
var list = _manualDsCache[setIndex] ??= new();
var span = CollectionsMarshal.AsSpan(list);
Queue<int> freeQueue = _freeManualDsCacheEntries[setIndex];
// Do we have at least one freed descriptor set? If so, just use that.
if (freeQueue != null && freeQueue.TryDequeue(out int freeIndex))
{
ref ManualDescriptorSetEntry entry = ref span[freeIndex];
Debug.Assert(!entry.InUse && entry.CbRefMask == 0);
entry.InUse = true;
entry.CbRefMask = 1u << cbs.CommandBufferIndex;
cacheIndex = freeIndex;
_pendingManualDsConsumptions.Enqueue(new PendingManualDsConsumption(cbs.GetFence(), cbs.CommandBufferIndex, setIndex, freeIndex));
return entry.DescriptorSet;
}
// Otherwise create a new descriptor set, and add to our pending queue for command buffer consumption tracking.
var dsc = _descriptorSetManager.AllocateDescriptorSet(
_gd.Api,
DescriptorSetLayouts[setIndex],
_poolSizes[setIndex],
setIndex,
_consumedDescriptorsPerSet[setIndex],
DescriptorSetLayoutsUpdateAfterBind[setIndex]);
cacheIndex = list.Count;
list.Add(new ManualDescriptorSetEntry(dsc, cbs.CommandBufferIndex));
_pendingManualDsConsumptions.Enqueue(new PendingManualDsConsumption(cbs.GetFence(), cbs.CommandBufferIndex, setIndex, cacheIndex));
return dsc;
}
public void UpdateManualDescriptorSetCollectionOwnership(CommandBufferScoped cbs, int setIndex, int cacheIndex)
{
FreeCompletedManualDescriptorSets();
var list = _manualDsCache[setIndex];
var span = CollectionsMarshal.AsSpan(list);
ref var entry = ref span[cacheIndex];
uint cbMask = 1u << cbs.CommandBufferIndex;
if ((entry.CbRefMask & cbMask) == 0)
{
entry.CbRefMask |= cbMask;
_pendingManualDsConsumptions.Enqueue(new PendingManualDsConsumption(cbs.GetFence(), cbs.CommandBufferIndex, setIndex, cacheIndex));
}
}
private void FreeCompletedManualDescriptorSets()
{
FenceHolder signalledFence = null;
while (_pendingManualDsConsumptions.TryPeek(out var pds) && (pds.Fence == signalledFence || pds.Fence.IsSignaled()))
{
signalledFence = pds.Fence; // Already checked - don't need to do it again.
var dequeued = _pendingManualDsConsumptions.Dequeue();
Debug.Assert(dequeued.Fence == pds.Fence);
pds.Fence.Put();
var span = CollectionsMarshal.AsSpan(_manualDsCache[dequeued.SetIndex]);
ref var entry = ref span[dequeued.CacheIndex];
entry.CbRefMask &= ~(1u << dequeued.CommandBufferIndex);
if (!entry.InUse && entry.CbRefMask == 0)
{
// If not in use by any array, and not bound to any command buffer, the descriptor set can be re-used immediately.
(_freeManualDsCacheEntries[dequeued.SetIndex] ??= new()).Enqueue(dequeued.CacheIndex);
}
}
}
public void ReleaseManualDescriptorSetCollection(int setIndex, int cacheIndex)
{
var list = _manualDsCache[setIndex];
var span = CollectionsMarshal.AsSpan(list);
span[cacheIndex].InUse = false;
if (span[cacheIndex].CbRefMask == 0)
{
// This is no longer in use by any array, so if not bound to any command buffer, the descriptor set can be re-used immediately.
(_freeManualDsCacheEntries[setIndex] ??= new()).Enqueue(cacheIndex);
}
}
private static Span<DescriptorPoolSize> GetDescriptorPoolSizes(Span<DescriptorPoolSize> output, ResourceDescriptorCollection setDescriptor, uint multiplier)
{
int count = 0;
@ -204,6 +344,21 @@ namespace Ryujinx.Graphics.Vulkan
}
}
for (int i = 0; i < _manualDsCache.Length; i++)
{
if (_manualDsCache[i] == null)
{
continue;
}
for (int j = 0; j < _manualDsCache[i].Count; j++)
{
_manualDsCache[i][j].DescriptorSet.Dispose();
}
_manualDsCache[i].Clear();
}
_gd.Api.DestroyPipelineLayout(_device, PipelineLayout, null);
for (int i = 0; i < DescriptorSetLayouts.Length; i++)
@ -211,6 +366,11 @@ namespace Ryujinx.Graphics.Vulkan
_gd.Api.DestroyDescriptorSetLayout(_device, DescriptorSetLayouts[i], null);
}
while (_pendingManualDsConsumptions.TryDequeue(out var pds))
{
pds.Fence.Put();
}
_descriptorSetManager.Dispose();
}
}

View File

@ -1,18 +1,23 @@
using Ryujinx.Common.Memory;
using Ryujinx.Graphics.GAL;
using Silk.NET.Vulkan;
using System;
using System.Collections.ObjectModel;
namespace Ryujinx.Graphics.Vulkan
{
record struct ResourceLayouts(DescriptorSetLayout[] DescriptorSetLayouts, bool[] DescriptorSetLayoutsUpdateAfterBind, PipelineLayout PipelineLayout);
static class PipelineLayoutFactory
{
public static unsafe (DescriptorSetLayout[], PipelineLayout) Create(
public static unsafe ResourceLayouts Create(
VulkanRenderer gd,
Device device,
ReadOnlyCollection<ResourceDescriptorCollection> setDescriptors,
bool usePushDescriptors)
{
DescriptorSetLayout[] layouts = new DescriptorSetLayout[setDescriptors.Count];
bool[] updateAfterBindFlags = new bool[setDescriptors.Count];
bool isMoltenVk = gd.IsMoltenVk;
@ -32,10 +37,11 @@ namespace Ryujinx.Graphics.Vulkan
DescriptorSetLayoutBinding[] layoutBindings = new DescriptorSetLayoutBinding[rdc.Descriptors.Count];
bool hasArray = false;
for (int descIndex = 0; descIndex < rdc.Descriptors.Count; descIndex++)
{
ResourceDescriptor descriptor = rdc.Descriptors[descIndex];
ResourceStages stages = descriptor.Stages;
if (descriptor.Type == ResourceType.StorageBuffer && isMoltenVk)
@ -52,16 +58,37 @@ namespace Ryujinx.Graphics.Vulkan
DescriptorCount = (uint)descriptor.Count,
StageFlags = stages.Convert(),
};
if (descriptor.Count > 1)
{
hasArray = true;
}
}
fixed (DescriptorSetLayoutBinding* pLayoutBindings = layoutBindings)
{
DescriptorSetLayoutCreateFlags flags = DescriptorSetLayoutCreateFlags.None;
if (usePushDescriptors && setIndex == 0)
{
flags = DescriptorSetLayoutCreateFlags.PushDescriptorBitKhr;
}
if (gd.Vendor == Vendor.Intel && hasArray)
{
// Some vendors (like Intel) have low per-stage limits.
// We must set the flag if we exceed those limits.
flags |= DescriptorSetLayoutCreateFlags.UpdateAfterBindPoolBit;
updateAfterBindFlags[setIndex] = true;
}
var descriptorSetLayoutCreateInfo = new DescriptorSetLayoutCreateInfo
{
SType = StructureType.DescriptorSetLayoutCreateInfo,
PBindings = pLayoutBindings,
BindingCount = (uint)layoutBindings.Length,
Flags = usePushDescriptors && setIndex == 0 ? DescriptorSetLayoutCreateFlags.PushDescriptorBitKhr : DescriptorSetLayoutCreateFlags.None,
Flags = flags,
};
gd.Api.CreateDescriptorSetLayout(device, descriptorSetLayoutCreateInfo, null, out layouts[setIndex]).ThrowOnError();
@ -82,7 +109,7 @@ namespace Ryujinx.Graphics.Vulkan
gd.Api.CreatePipelineLayout(device, &pipelineLayoutCreateInfo, null, out layout).ThrowOnError();
}
return (layouts, layout);
return new ResourceLayouts(layouts, updateAfterBindFlags, layout);
}
}
}

View File

@ -71,244 +71,232 @@ namespace Ryujinx.Graphics.Vulkan
set => Internal.Id4 = (Internal.Id4 & 0xFFFFFFFF) | ((ulong)value << 32);
}
public float MinDepthBounds
{
readonly get => BitConverter.Int32BitsToSingle((int)((Internal.Id5 >> 0) & 0xFFFFFFFF));
set => Internal.Id5 = (Internal.Id5 & 0xFFFFFFFF00000000) | ((ulong)(uint)BitConverter.SingleToInt32Bits(value) << 0);
}
public float MaxDepthBounds
{
readonly get => BitConverter.Int32BitsToSingle((int)((Internal.Id5 >> 32) & 0xFFFFFFFF));
set => Internal.Id5 = (Internal.Id5 & 0xFFFFFFFF) | ((ulong)(uint)BitConverter.SingleToInt32Bits(value) << 32);
}
public PolygonMode PolygonMode
{
readonly get => (PolygonMode)((Internal.Id6 >> 0) & 0x3FFFFFFF);
set => Internal.Id6 = (Internal.Id6 & 0xFFFFFFFFC0000000) | ((ulong)value << 0);
readonly get => (PolygonMode)((Internal.Id5 >> 0) & 0x3FFFFFFF);
set => Internal.Id5 = (Internal.Id5 & 0xFFFFFFFFC0000000) | ((ulong)value << 0);
}
public uint StagesCount
{
readonly get => (byte)((Internal.Id6 >> 30) & 0xFF);
set => Internal.Id6 = (Internal.Id6 & 0xFFFFFFC03FFFFFFF) | ((ulong)value << 30);
readonly get => (byte)((Internal.Id5 >> 30) & 0xFF);
set => Internal.Id5 = (Internal.Id5 & 0xFFFFFFC03FFFFFFF) | ((ulong)value << 30);
}
public uint VertexAttributeDescriptionsCount
{
readonly get => (byte)((Internal.Id6 >> 38) & 0xFF);
set => Internal.Id6 = (Internal.Id6 & 0xFFFFC03FFFFFFFFF) | ((ulong)value << 38);
readonly get => (byte)((Internal.Id5 >> 38) & 0xFF);
set => Internal.Id5 = (Internal.Id5 & 0xFFFFC03FFFFFFFFF) | ((ulong)value << 38);
}
public uint VertexBindingDescriptionsCount
{
readonly get => (byte)((Internal.Id6 >> 46) & 0xFF);
set => Internal.Id6 = (Internal.Id6 & 0xFFC03FFFFFFFFFFF) | ((ulong)value << 46);
readonly get => (byte)((Internal.Id5 >> 46) & 0xFF);
set => Internal.Id5 = (Internal.Id5 & 0xFFC03FFFFFFFFFFF) | ((ulong)value << 46);
}
public uint ViewportsCount
{
readonly get => (byte)((Internal.Id6 >> 54) & 0xFF);
set => Internal.Id6 = (Internal.Id6 & 0xC03FFFFFFFFFFFFF) | ((ulong)value << 54);
readonly get => (byte)((Internal.Id5 >> 54) & 0xFF);
set => Internal.Id5 = (Internal.Id5 & 0xC03FFFFFFFFFFFFF) | ((ulong)value << 54);
}
public uint ScissorsCount
{
readonly get => (byte)((Internal.Id7 >> 0) & 0xFF);
set => Internal.Id7 = (Internal.Id7 & 0xFFFFFFFFFFFFFF00) | ((ulong)value << 0);
readonly get => (byte)((Internal.Id6 >> 0) & 0xFF);
set => Internal.Id6 = (Internal.Id6 & 0xFFFFFFFFFFFFFF00) | ((ulong)value << 0);
}
public uint ColorBlendAttachmentStateCount
{
readonly get => (byte)((Internal.Id7 >> 8) & 0xFF);
set => Internal.Id7 = (Internal.Id7 & 0xFFFFFFFFFFFF00FF) | ((ulong)value << 8);
readonly get => (byte)((Internal.Id6 >> 8) & 0xFF);
set => Internal.Id6 = (Internal.Id6 & 0xFFFFFFFFFFFF00FF) | ((ulong)value << 8);
}
public PrimitiveTopology Topology
{
readonly get => (PrimitiveTopology)((Internal.Id7 >> 16) & 0xF);
set => Internal.Id7 = (Internal.Id7 & 0xFFFFFFFFFFF0FFFF) | ((ulong)value << 16);
readonly get => (PrimitiveTopology)((Internal.Id6 >> 16) & 0xF);
set => Internal.Id6 = (Internal.Id6 & 0xFFFFFFFFFFF0FFFF) | ((ulong)value << 16);
}
public LogicOp LogicOp
{
readonly get => (LogicOp)((Internal.Id7 >> 20) & 0xF);
set => Internal.Id7 = (Internal.Id7 & 0xFFFFFFFFFF0FFFFF) | ((ulong)value << 20);
readonly get => (LogicOp)((Internal.Id6 >> 20) & 0xF);
set => Internal.Id6 = (Internal.Id6 & 0xFFFFFFFFFF0FFFFF) | ((ulong)value << 20);
}
public CompareOp DepthCompareOp
{
readonly get => (CompareOp)((Internal.Id7 >> 24) & 0x7);
set => Internal.Id7 = (Internal.Id7 & 0xFFFFFFFFF8FFFFFF) | ((ulong)value << 24);
readonly get => (CompareOp)((Internal.Id6 >> 24) & 0x7);
set => Internal.Id6 = (Internal.Id6 & 0xFFFFFFFFF8FFFFFF) | ((ulong)value << 24);
}
public StencilOp StencilFrontFailOp
{
readonly get => (StencilOp)((Internal.Id7 >> 27) & 0x7);
set => Internal.Id7 = (Internal.Id7 & 0xFFFFFFFFC7FFFFFF) | ((ulong)value << 27);
readonly get => (StencilOp)((Internal.Id6 >> 27) & 0x7);
set => Internal.Id6 = (Internal.Id6 & 0xFFFFFFFFC7FFFFFF) | ((ulong)value << 27);
}
public StencilOp StencilFrontPassOp
{
readonly get => (StencilOp)((Internal.Id7 >> 30) & 0x7);
set => Internal.Id7 = (Internal.Id7 & 0xFFFFFFFE3FFFFFFF) | ((ulong)value << 30);
readonly get => (StencilOp)((Internal.Id6 >> 30) & 0x7);
set => Internal.Id6 = (Internal.Id6 & 0xFFFFFFFE3FFFFFFF) | ((ulong)value << 30);
}
public StencilOp StencilFrontDepthFailOp
{
readonly get => (StencilOp)((Internal.Id7 >> 33) & 0x7);
set => Internal.Id7 = (Internal.Id7 & 0xFFFFFFF1FFFFFFFF) | ((ulong)value << 33);
readonly get => (StencilOp)((Internal.Id6 >> 33) & 0x7);
set => Internal.Id6 = (Internal.Id6 & 0xFFFFFFF1FFFFFFFF) | ((ulong)value << 33);
}
public CompareOp StencilFrontCompareOp
{
readonly get => (CompareOp)((Internal.Id7 >> 36) & 0x7);
set => Internal.Id7 = (Internal.Id7 & 0xFFFFFF8FFFFFFFFF) | ((ulong)value << 36);
readonly get => (CompareOp)((Internal.Id6 >> 36) & 0x7);
set => Internal.Id6 = (Internal.Id6 & 0xFFFFFF8FFFFFFFFF) | ((ulong)value << 36);
}
public StencilOp StencilBackFailOp
{
readonly get => (StencilOp)((Internal.Id7 >> 39) & 0x7);
set => Internal.Id7 = (Internal.Id7 & 0xFFFFFC7FFFFFFFFF) | ((ulong)value << 39);
readonly get => (StencilOp)((Internal.Id6 >> 39) & 0x7);
set => Internal.Id6 = (Internal.Id6 & 0xFFFFFC7FFFFFFFFF) | ((ulong)value << 39);
}
public StencilOp StencilBackPassOp
{
readonly get => (StencilOp)((Internal.Id7 >> 42) & 0x7);
set => Internal.Id7 = (Internal.Id7 & 0xFFFFE3FFFFFFFFFF) | ((ulong)value << 42);
readonly get => (StencilOp)((Internal.Id6 >> 42) & 0x7);
set => Internal.Id6 = (Internal.Id6 & 0xFFFFE3FFFFFFFFFF) | ((ulong)value << 42);
}
public StencilOp StencilBackDepthFailOp
{
readonly get => (StencilOp)((Internal.Id7 >> 45) & 0x7);
set => Internal.Id7 = (Internal.Id7 & 0xFFFF1FFFFFFFFFFF) | ((ulong)value << 45);
readonly get => (StencilOp)((Internal.Id6 >> 45) & 0x7);
set => Internal.Id6 = (Internal.Id6 & 0xFFFF1FFFFFFFFFFF) | ((ulong)value << 45);
}
public CompareOp StencilBackCompareOp
{
readonly get => (CompareOp)((Internal.Id7 >> 48) & 0x7);
set => Internal.Id7 = (Internal.Id7 & 0xFFF8FFFFFFFFFFFF) | ((ulong)value << 48);
readonly get => (CompareOp)((Internal.Id6 >> 48) & 0x7);
set => Internal.Id6 = (Internal.Id6 & 0xFFF8FFFFFFFFFFFF) | ((ulong)value << 48);
}
public CullModeFlags CullMode
{
readonly get => (CullModeFlags)((Internal.Id7 >> 51) & 0x3);
set => Internal.Id7 = (Internal.Id7 & 0xFFE7FFFFFFFFFFFF) | ((ulong)value << 51);
readonly get => (CullModeFlags)((Internal.Id6 >> 51) & 0x3);
set => Internal.Id6 = (Internal.Id6 & 0xFFE7FFFFFFFFFFFF) | ((ulong)value << 51);
}
public bool PrimitiveRestartEnable
{
readonly get => ((Internal.Id7 >> 53) & 0x1) != 0UL;
set => Internal.Id7 = (Internal.Id7 & 0xFFDFFFFFFFFFFFFF) | ((value ? 1UL : 0UL) << 53);
readonly get => ((Internal.Id6 >> 53) & 0x1) != 0UL;
set => Internal.Id6 = (Internal.Id6 & 0xFFDFFFFFFFFFFFFF) | ((value ? 1UL : 0UL) << 53);
}
public bool DepthClampEnable
{
readonly get => ((Internal.Id7 >> 54) & 0x1) != 0UL;
set => Internal.Id7 = (Internal.Id7 & 0xFFBFFFFFFFFFFFFF) | ((value ? 1UL : 0UL) << 54);
readonly get => ((Internal.Id6 >> 54) & 0x1) != 0UL;
set => Internal.Id6 = (Internal.Id6 & 0xFFBFFFFFFFFFFFFF) | ((value ? 1UL : 0UL) << 54);
}
public bool RasterizerDiscardEnable
{
readonly get => ((Internal.Id7 >> 55) & 0x1) != 0UL;
set => Internal.Id7 = (Internal.Id7 & 0xFF7FFFFFFFFFFFFF) | ((value ? 1UL : 0UL) << 55);
readonly get => ((Internal.Id6 >> 55) & 0x1) != 0UL;
set => Internal.Id6 = (Internal.Id6 & 0xFF7FFFFFFFFFFFFF) | ((value ? 1UL : 0UL) << 55);
}
public FrontFace FrontFace
{
readonly get => (FrontFace)((Internal.Id7 >> 56) & 0x1);
set => Internal.Id7 = (Internal.Id7 & 0xFEFFFFFFFFFFFFFF) | ((ulong)value << 56);
readonly get => (FrontFace)((Internal.Id6 >> 56) & 0x1);
set => Internal.Id6 = (Internal.Id6 & 0xFEFFFFFFFFFFFFFF) | ((ulong)value << 56);
}
public bool DepthBiasEnable
{
readonly get => ((Internal.Id7 >> 57) & 0x1) != 0UL;
set => Internal.Id7 = (Internal.Id7 & 0xFDFFFFFFFFFFFFFF) | ((value ? 1UL : 0UL) << 57);
readonly get => ((Internal.Id6 >> 57) & 0x1) != 0UL;
set => Internal.Id6 = (Internal.Id6 & 0xFDFFFFFFFFFFFFFF) | ((value ? 1UL : 0UL) << 57);
}
public bool DepthTestEnable
{
readonly get => ((Internal.Id7 >> 58) & 0x1) != 0UL;
set => Internal.Id7 = (Internal.Id7 & 0xFBFFFFFFFFFFFFFF) | ((value ? 1UL : 0UL) << 58);
readonly get => ((Internal.Id6 >> 58) & 0x1) != 0UL;
set => Internal.Id6 = (Internal.Id6 & 0xFBFFFFFFFFFFFFFF) | ((value ? 1UL : 0UL) << 58);
}
public bool DepthWriteEnable
{
readonly get => ((Internal.Id7 >> 59) & 0x1) != 0UL;
set => Internal.Id7 = (Internal.Id7 & 0xF7FFFFFFFFFFFFFF) | ((value ? 1UL : 0UL) << 59);
readonly get => ((Internal.Id6 >> 59) & 0x1) != 0UL;
set => Internal.Id6 = (Internal.Id6 & 0xF7FFFFFFFFFFFFFF) | ((value ? 1UL : 0UL) << 59);
}
public bool DepthBoundsTestEnable
{
readonly get => ((Internal.Id7 >> 60) & 0x1) != 0UL;
set => Internal.Id7 = (Internal.Id7 & 0xEFFFFFFFFFFFFFFF) | ((value ? 1UL : 0UL) << 60);
readonly get => ((Internal.Id6 >> 60) & 0x1) != 0UL;
set => Internal.Id6 = (Internal.Id6 & 0xEFFFFFFFFFFFFFFF) | ((value ? 1UL : 0UL) << 60);
}
public bool StencilTestEnable
{
readonly get => ((Internal.Id7 >> 61) & 0x1) != 0UL;
set => Internal.Id7 = (Internal.Id7 & 0xDFFFFFFFFFFFFFFF) | ((value ? 1UL : 0UL) << 61);
readonly get => ((Internal.Id6 >> 61) & 0x1) != 0UL;
set => Internal.Id6 = (Internal.Id6 & 0xDFFFFFFFFFFFFFFF) | ((value ? 1UL : 0UL) << 61);
}
public bool LogicOpEnable
{
readonly get => ((Internal.Id7 >> 62) & 0x1) != 0UL;
set => Internal.Id7 = (Internal.Id7 & 0xBFFFFFFFFFFFFFFF) | ((value ? 1UL : 0UL) << 62);
readonly get => ((Internal.Id6 >> 62) & 0x1) != 0UL;
set => Internal.Id6 = (Internal.Id6 & 0xBFFFFFFFFFFFFFFF) | ((value ? 1UL : 0UL) << 62);
}
public bool HasDepthStencil
{
readonly get => ((Internal.Id7 >> 63) & 0x1) != 0UL;
set => Internal.Id7 = (Internal.Id7 & 0x7FFFFFFFFFFFFFFF) | ((value ? 1UL : 0UL) << 63);
readonly get => ((Internal.Id6 >> 63) & 0x1) != 0UL;
set => Internal.Id6 = (Internal.Id6 & 0x7FFFFFFFFFFFFFFF) | ((value ? 1UL : 0UL) << 63);
}
public uint PatchControlPoints
{
readonly get => (uint)((Internal.Id8 >> 0) & 0xFFFFFFFF);
set => Internal.Id8 = (Internal.Id8 & 0xFFFFFFFF00000000) | ((ulong)value << 0);
readonly get => (uint)((Internal.Id7 >> 0) & 0xFFFFFFFF);
set => Internal.Id7 = (Internal.Id7 & 0xFFFFFFFF00000000) | ((ulong)value << 0);
}
public uint SamplesCount
{
readonly get => (uint)((Internal.Id8 >> 32) & 0xFFFFFFFF);
set => Internal.Id8 = (Internal.Id8 & 0xFFFFFFFF) | ((ulong)value << 32);
readonly get => (uint)((Internal.Id7 >> 32) & 0xFFFFFFFF);
set => Internal.Id7 = (Internal.Id7 & 0xFFFFFFFF) | ((ulong)value << 32);
}
public bool AlphaToCoverageEnable
{
readonly get => ((Internal.Id9 >> 0) & 0x1) != 0UL;
set => Internal.Id9 = (Internal.Id9 & 0xFFFFFFFFFFFFFFFE) | ((value ? 1UL : 0UL) << 0);
readonly get => ((Internal.Id8 >> 0) & 0x1) != 0UL;
set => Internal.Id8 = (Internal.Id8 & 0xFFFFFFFFFFFFFFFE) | ((value ? 1UL : 0UL) << 0);
}
public bool AlphaToOneEnable
{
readonly get => ((Internal.Id9 >> 1) & 0x1) != 0UL;
set => Internal.Id9 = (Internal.Id9 & 0xFFFFFFFFFFFFFFFD) | ((value ? 1UL : 0UL) << 1);
readonly get => ((Internal.Id8 >> 1) & 0x1) != 0UL;
set => Internal.Id8 = (Internal.Id8 & 0xFFFFFFFFFFFFFFFD) | ((value ? 1UL : 0UL) << 1);
}
public bool AdvancedBlendSrcPreMultiplied
{
readonly get => ((Internal.Id9 >> 2) & 0x1) != 0UL;
set => Internal.Id9 = (Internal.Id9 & 0xFFFFFFFFFFFFFFFB) | ((value ? 1UL : 0UL) << 2);
readonly get => ((Internal.Id8 >> 2) & 0x1) != 0UL;
set => Internal.Id8 = (Internal.Id8 & 0xFFFFFFFFFFFFFFFB) | ((value ? 1UL : 0UL) << 2);
}
public bool AdvancedBlendDstPreMultiplied
{
readonly get => ((Internal.Id9 >> 3) & 0x1) != 0UL;
set => Internal.Id9 = (Internal.Id9 & 0xFFFFFFFFFFFFFFF7) | ((value ? 1UL : 0UL) << 3);
readonly get => ((Internal.Id8 >> 3) & 0x1) != 0UL;
set => Internal.Id8 = (Internal.Id8 & 0xFFFFFFFFFFFFFFF7) | ((value ? 1UL : 0UL) << 3);
}
public BlendOverlapEXT AdvancedBlendOverlap
{
readonly get => (BlendOverlapEXT)((Internal.Id9 >> 4) & 0x3);
set => Internal.Id9 = (Internal.Id9 & 0xFFFFFFFFFFFFFFCF) | ((ulong)value << 4);
readonly get => (BlendOverlapEXT)((Internal.Id8 >> 4) & 0x3);
set => Internal.Id8 = (Internal.Id8 & 0xFFFFFFFFFFFFFFCF) | ((ulong)value << 4);
}
public bool DepthMode
{
readonly get => ((Internal.Id9 >> 6) & 0x1) != 0UL;
set => Internal.Id9 = (Internal.Id9 & 0xFFFFFFFFFFFFFFBF) | ((value ? 1UL : 0UL) << 6);
readonly get => ((Internal.Id8 >> 6) & 0x1) != 0UL;
set => Internal.Id8 = (Internal.Id8 & 0xFFFFFFFFFFFFFFBF) | ((value ? 1UL : 0UL) << 6);
}
public bool HasTessellationControlShader;
@ -408,8 +396,6 @@ namespace Ryujinx.Graphics.Vulkan
fixed (VertexInputAttributeDescription* pVertexAttributeDescriptions = &Internal.VertexAttributeDescriptions[0])
fixed (VertexInputAttributeDescription* pVertexAttributeDescriptions2 = &_vertexAttributeDescriptions2[0])
fixed (VertexInputBindingDescription* pVertexBindingDescriptions = &Internal.VertexBindingDescriptions[0])
fixed (Viewport* pViewports = &Internal.Viewports[0])
fixed (Rect2D* pScissors = &Internal.Scissors[0])
fixed (PipelineColorBlendAttachmentState* pColorBlendAttachmentState = &Internal.ColorBlendAttachmentState[0])
{
var vertexInputState = new PipelineVertexInputStateCreateInfo
@ -472,18 +458,13 @@ namespace Ryujinx.Graphics.Vulkan
CullMode = CullMode,
FrontFace = FrontFace,
DepthBiasEnable = DepthBiasEnable,
DepthBiasClamp = DepthBiasClamp,
DepthBiasConstantFactor = DepthBiasConstantFactor,
DepthBiasSlopeFactor = DepthBiasSlopeFactor,
};
var viewportState = new PipelineViewportStateCreateInfo
{
SType = StructureType.PipelineViewportStateCreateInfo,
ViewportCount = ViewportsCount,
PViewports = pViewports,
ScissorCount = ScissorsCount,
PScissors = pScissors,
};
if (gd.Capabilities.SupportsDepthClipControl)
@ -511,19 +492,13 @@ namespace Ryujinx.Graphics.Vulkan
StencilFrontFailOp,
StencilFrontPassOp,
StencilFrontDepthFailOp,
StencilFrontCompareOp,
StencilFrontCompareMask,
StencilFrontWriteMask,
StencilFrontReference);
StencilFrontCompareOp);
var stencilBack = new StencilOpState(
StencilBackFailOp,
StencilBackPassOp,
StencilBackDepthFailOp,
StencilBackCompareOp,
StencilBackCompareMask,
StencilBackWriteMask,
StencilBackReference);
StencilBackCompareOp);
var depthStencilState = new PipelineDepthStencilStateCreateInfo
{
@ -531,12 +506,10 @@ namespace Ryujinx.Graphics.Vulkan
DepthTestEnable = DepthTestEnable,
DepthWriteEnable = DepthWriteEnable,
DepthCompareOp = DepthCompareOp,
DepthBoundsTestEnable = DepthBoundsTestEnable,
DepthBoundsTestEnable = false,
StencilTestEnable = StencilTestEnable,
Front = stencilFront,
Back = stencilBack,
MinDepthBounds = MinDepthBounds,
MaxDepthBounds = MaxDepthBounds,
};
uint blendEnables = 0;
@ -560,10 +533,14 @@ namespace Ryujinx.Graphics.Vulkan
}
}
// Vendors other than NVIDIA have a bug where it enables logical operations even for float formats,
// so we need to force disable them here.
bool logicOpEnable = LogicOpEnable && (gd.Vendor == Vendor.Nvidia || Internal.LogicOpsAllowed);
var colorBlendState = new PipelineColorBlendStateCreateInfo
{
SType = StructureType.PipelineColorBlendStateCreateInfo,
LogicOpEnable = LogicOpEnable,
LogicOpEnable = logicOpEnable,
LogicOp = LogicOp,
AttachmentCount = ColorBlendAttachmentStateCount,
PAttachments = pColorBlendAttachmentState,
@ -587,22 +564,21 @@ namespace Ryujinx.Graphics.Vulkan
}
bool supportsExtDynamicState = gd.Capabilities.SupportsExtendedDynamicState;
int dynamicStatesCount = supportsExtDynamicState ? 9 : 8;
int dynamicStatesCount = supportsExtDynamicState ? 8 : 7;
DynamicState* dynamicStates = stackalloc DynamicState[dynamicStatesCount];
dynamicStates[0] = DynamicState.Viewport;
dynamicStates[1] = DynamicState.Scissor;
dynamicStates[2] = DynamicState.DepthBias;
dynamicStates[3] = DynamicState.DepthBounds;
dynamicStates[4] = DynamicState.StencilCompareMask;
dynamicStates[5] = DynamicState.StencilWriteMask;
dynamicStates[6] = DynamicState.StencilReference;
dynamicStates[7] = DynamicState.BlendConstants;
dynamicStates[3] = DynamicState.StencilCompareMask;
dynamicStates[4] = DynamicState.StencilWriteMask;
dynamicStates[5] = DynamicState.StencilReference;
dynamicStates[6] = DynamicState.BlendConstants;
if (supportsExtDynamicState)
{
dynamicStates[8] = DynamicState.VertexInputBindingStrideExt;
dynamicStates[7] = DynamicState.VertexInputBindingStrideExt;
}
var pipelineDynamicStateCreateInfo = new PipelineDynamicStateCreateInfo
@ -628,7 +604,6 @@ namespace Ryujinx.Graphics.Vulkan
PDynamicState = &pipelineDynamicStateCreateInfo,
Layout = PipelineLayout,
RenderPass = renderPass,
BasePipelineIndex = -1,
};
Result result = gd.Api.CreateGraphicsPipelines(device, cache, 1, &pipelineCreateInfo, null, &pipelineHandle);

View File

@ -17,23 +17,21 @@ namespace Ryujinx.Graphics.Vulkan
public ulong Id4;
public ulong Id5;
public ulong Id6;
public ulong Id7;
public ulong Id8;
public ulong Id9;
private readonly uint VertexAttributeDescriptionsCount => (byte)((Id6 >> 38) & 0xFF);
private readonly uint VertexBindingDescriptionsCount => (byte)((Id6 >> 46) & 0xFF);
private readonly uint ColorBlendAttachmentStateCount => (byte)((Id7 >> 8) & 0xFF);
private readonly bool HasDepthStencil => ((Id7 >> 63) & 0x1) != 0UL;
private readonly uint VertexAttributeDescriptionsCount => (byte)((Id5 >> 38) & 0xFF);
private readonly uint VertexBindingDescriptionsCount => (byte)((Id5 >> 46) & 0xFF);
private readonly uint ColorBlendAttachmentStateCount => (byte)((Id6 >> 8) & 0xFF);
private readonly bool HasDepthStencil => ((Id6 >> 63) & 0x1) != 0UL;
public Array32<VertexInputAttributeDescription> VertexAttributeDescriptions;
public Array33<VertexInputBindingDescription> VertexBindingDescriptions;
public Array16<Viewport> Viewports;
public Array16<Rect2D> Scissors;
public Array8<PipelineColorBlendAttachmentState> ColorBlendAttachmentState;
public Array9<Format> AttachmentFormats;
public uint AttachmentIntegerFormatMask;
public bool LogicOpsAllowed;
public readonly override bool Equals(object obj)
{
@ -44,7 +42,7 @@ namespace Ryujinx.Graphics.Vulkan
{
if (!Unsafe.As<ulong, Vector256<byte>>(ref Id0).Equals(Unsafe.As<ulong, Vector256<byte>>(ref other.Id0)) ||
!Unsafe.As<ulong, Vector256<byte>>(ref Id4).Equals(Unsafe.As<ulong, Vector256<byte>>(ref other.Id4)) ||
!Unsafe.As<ulong, Vector128<byte>>(ref Id8).Equals(Unsafe.As<ulong, Vector128<byte>>(ref other.Id8)))
!Unsafe.As<ulong, Vector128<byte>>(ref Id7).Equals(Unsafe.As<ulong, Vector128<byte>>(ref other.Id7)))
{
return false;
}
@ -87,8 +85,7 @@ namespace Ryujinx.Graphics.Vulkan
Id5 * 23 ^
Id6 * 23 ^
Id7 * 23 ^
Id8 * 23 ^
Id9 * 23;
Id8 * 23;
for (int i = 0; i < (int)VertexAttributeDescriptionsCount; i++)
{

View File

@ -0,0 +1,74 @@
using Silk.NET.Vulkan;
using System;
using System.Diagnostics;
namespace Ryujinx.Graphics.Vulkan
{
class ResourceArray : IDisposable
{
private DescriptorSet[] _cachedDescriptorSets;
private ShaderCollection _cachedDscProgram;
private int _cachedDscSetIndex;
private int _cachedDscIndex;
private int _bindCount;
protected void SetDirty(VulkanRenderer gd)
{
ReleaseDescriptorSet();
if (_bindCount != 0)
{
gd.PipelineInternal.ForceTextureDirty();
}
}
public bool TryGetCachedDescriptorSets(CommandBufferScoped cbs, ShaderCollection program, int setIndex, out DescriptorSet[] sets)
{
if (_cachedDescriptorSets != null)
{
_cachedDscProgram.UpdateManualDescriptorSetCollectionOwnership(cbs, _cachedDscSetIndex, _cachedDscIndex);
sets = _cachedDescriptorSets;
return true;
}
var dsc = program.GetNewManualDescriptorSetCollection(cbs, setIndex, out _cachedDscIndex).Get(cbs);
sets = dsc.GetSets();
_cachedDescriptorSets = sets;
_cachedDscProgram = program;
_cachedDscSetIndex = setIndex;
return false;
}
public void IncrementBindCount()
{
_bindCount++;
}
public void DecrementBindCount()
{
int newBindCount = --_bindCount;
Debug.Assert(newBindCount >= 0);
}
private void ReleaseDescriptorSet()
{
if (_cachedDescriptorSets != null)
{
_cachedDscProgram.ReleaseManualDescriptorSetCollection(_cachedDscSetIndex, _cachedDscIndex);
_cachedDescriptorSets = null;
}
}
public void Dispose()
{
ReleaseDescriptorSet();
}
}
}

View File

@ -604,6 +604,21 @@ namespace Ryujinx.Graphics.Vulkan
return _plce.GetNewDescriptorSetCollection(setIndex, out isNew);
}
public Auto<DescriptorSetCollection> GetNewManualDescriptorSetCollection(CommandBufferScoped cbs, int setIndex, out int cacheIndex)
{
return _plce.GetNewManualDescriptorSetCollection(cbs, setIndex, out cacheIndex);
}
public void UpdateManualDescriptorSetCollectionOwnership(CommandBufferScoped cbs, int setIndex, int cacheIndex)
{
_plce.UpdateManualDescriptorSetCollectionOwnership(cbs, setIndex, cacheIndex);
}
public void ReleaseManualDescriptorSetCollection(int setIndex, int cacheIndex)
{
_plce.ReleaseManualDescriptorSetCollection(setIndex, cacheIndex);
}
public bool HasSameLayout(ShaderCollection other)
{
return other != null && _plce == other._plce;

View File

@ -5,7 +5,7 @@ using System.Collections.Generic;
namespace Ryujinx.Graphics.Vulkan
{
class TextureArray : ITextureArray
class TextureArray : ResourceArray, ITextureArray
{
private readonly VulkanRenderer _gd;
@ -29,8 +29,6 @@ namespace Ryujinx.Graphics.Vulkan
private readonly bool _isBuffer;
public bool Bound;
public TextureArray(VulkanRenderer gd, int size, bool isBuffer)
{
_gd = gd;
@ -106,8 +104,7 @@ namespace Ryujinx.Graphics.Vulkan
{
_cachedCommandBufferIndex = -1;
_storages = null;
_gd.PipelineInternal.ForceTextureDirty();
SetDirty(_gd);
}
public void QueueWriteToReadBarriers(CommandBufferScoped cbs, PipelineStageFlags stageFlags)
@ -190,5 +187,48 @@ namespace Ryujinx.Graphics.Vulkan
return bufferTextures;
}
public DescriptorSet[] GetDescriptorSets(
Device device,
CommandBufferScoped cbs,
DescriptorSetTemplateUpdater templateUpdater,
ShaderCollection program,
int setIndex,
TextureView dummyTexture,
SamplerHolder dummySampler)
{
if (TryGetCachedDescriptorSets(cbs, program, setIndex, out DescriptorSet[] sets))
{
// We still need to ensure the current command buffer holds a reference to all used textures.
if (!_isBuffer)
{
GetImageInfos(_gd, cbs, dummyTexture, dummySampler);
}
else
{
GetBufferViews(cbs);
}
return sets;
}
DescriptorSetTemplate template = program.Templates[setIndex];
DescriptorSetTemplateWriter tu = templateUpdater.Begin(template);
if (!_isBuffer)
{
tu.Push(GetImageInfos(_gd, cbs, dummyTexture, dummySampler));
}
else
{
tu.Push(GetBufferViews(cbs));
}
templateUpdater.Commit(_gd, device, sets[0]);
return sets;
}
}
}

View File

@ -486,12 +486,7 @@ namespace Ryujinx.Graphics.Vulkan
public BufferHandle CreateBuffer(int size, BufferAccess access)
{
return BufferManager.CreateWithHandle(this, size, access.HasFlag(BufferAccess.SparseCompatible), access.Convert(), default, access == BufferAccess.Stream);
}
public BufferHandle CreateBuffer(int size, BufferAccess access, BufferHandle storageHint)
{
return BufferManager.CreateWithHandle(this, size, access.HasFlag(BufferAccess.SparseCompatible), access.Convert(), storageHint);
return BufferManager.CreateWithHandle(this, size, access.HasFlag(BufferAccess.SparseCompatible), access.Convert(), access.HasFlag(BufferAccess.Stream));
}
public BufferHandle CreateBuffer(nint pointer, int size)
@ -675,9 +670,23 @@ namespace Ryujinx.Graphics.Vulkan
var limits = _physicalDevice.PhysicalDeviceProperties.Limits;
var mainQueueProperties = _physicalDevice.QueueFamilyProperties[QueueFamilyIndex];
SystemMemoryType memoryType;
if (IsSharedMemory)
{
memoryType = SystemMemoryType.UnifiedMemory;
}
else
{
memoryType = Vendor == Vendor.Nvidia ?
SystemMemoryType.DedicatedMemorySlowStorage :
SystemMemoryType.DedicatedMemory;
}
return new Capabilities(
api: TargetApi.Vulkan,
GpuVendor,
memoryType: memoryType,
hasFrontFacingBug: IsIntelWindows,
hasVectorIndexingBug: Vendor == Vendor.Qualcomm,
needsFragmentOutputSpecialization: IsMoltenVk,
@ -719,6 +728,12 @@ namespace Ryujinx.Graphics.Vulkan
supportsViewportSwizzle: false,
supportsIndirectParameters: true,
supportsDepthClipControl: Capabilities.SupportsDepthClipControl,
uniformBufferSetIndex: PipelineBase.UniformSetIndex,
storageBufferSetIndex: PipelineBase.StorageSetIndex,
textureSetIndex: PipelineBase.TextureSetIndex,
imageSetIndex: PipelineBase.ImageSetIndex,
extraSetBaseIndex: PipelineBase.DescriptorSetLayouts,
maximumExtraSets: Math.Max(0, (int)limits.MaxBoundDescriptorSets - PipelineBase.DescriptorSetLayouts),
maximumUniformBuffersPerStage: Constants.MaxUniformBuffersPerStage,
maximumStorageBuffersPerStage: Constants.MaxStorageBuffersPerStage,
maximumTexturesPerStage: Constants.MaxTexturesPerStage,

View File

@ -81,6 +81,11 @@ namespace Ryujinx.Input.GTK3
return _pressedKeys.Contains(nativeKey);
}
public void Clear()
{
_pressedKeys.Clear();
}
public IGamepad GetGamepad(string id)
{
if (!_keyboardIdentifers[0].Equals(id))

View File

@ -107,6 +107,8 @@ namespace Ryujinx.UI.Applet
swkbdDialog.SetInputLengthValidation(args.StringLengthMin, args.StringLengthMax);
swkbdDialog.SetInputValidation(args.KeyboardMode);
((MainWindow)_parent).RendererWidget.NpadManager.BlockInputUpdates();
if (swkbdDialog.Run() == (int)ResponseType.Ok)
{
inputText = swkbdDialog.InputEntry.Text;
@ -128,6 +130,7 @@ namespace Ryujinx.UI.Applet
});
dialogCloseEvent.WaitOne();
((MainWindow)_parent).RendererWidget.NpadManager.UnblockInputUpdates();
userText = error ? null : inputText;

Some files were not shown because too many files have changed in this diff Show More