Commit graph

20 commits

Author SHA1 Message Date
riperiperi
a539303e71
Remove unused physical region tracking (#2085)
* Remove unused physical region tracking

* Update comments
2021-03-06 20:21:53 -03:00
riperiperi
8d36681bf1
Improve handling for unmapped GPU resources (#2083)
* Improve handling for unmapped GPU resources

- Fixed a memory tracking bug that would set protection on empty PTEs
- When a texture's memory is (partially) unmapped, all pool references are forcibly removed and the texture must be rediscovered to draw with it. This will also force the texture discovery to always compare the texture's range for a match.
- RegionHandles now know if they are unmapped, and automatically unset their dirty flag when unmapped.
- Partial texture sync now loads only the region of texture that has been modified. Unmapped memory tracking handles cause dirty flags for a texture group handle to be ignored.

This greatly improves the emulator's stability for newer UE4 games.

* Address feedback, fix MultiRange slice

Fixed an issue where the size of the multi-range slice would be miscalculated.

* Update Ryujinx.Memory/Range/MultiRange.cs (feedback)

Co-authored-by: Mary <thog@protonmail.com>

Co-authored-by: Mary <thog@protonmail.com>
2021-03-06 11:43:55 -03:00
riperiperi
b530f0e110
Texture Cache: "Texture Groups" and "Texture Dependencies" (#2001)
* Initial implementation (3d tex mips broken)

This works rather well for most games, just need to fix 3d texture mips.

* Cleanup

* Address feedback

* Copy Dependencies and various other fixes

* Fix layer/level offset for copy from view<->view.

* Remove dirty flag from dependency

The dirty flag behaviour is not needed - DeferredCopy is all we need.

* Fix tracking mip slices.

* Propagate granularity (fix astral chain)

* Address Feedback pt 1

* Save slice sizes as part of SizeInfo

* Fix nits

* Fix disposing multiple dependencies causing a crash

This list is obviously modified when removing dependencies, so create a copy of it.
2021-03-02 19:30:54 -03:00
Caian Benedicto
f7b2daf5ec
Fix virtual address overflow near ulong limit (#2044)
* Fix virtual address overflow near ulong limit

* Fix comments

* Improve overflow checking for large size values

* Add overflow checking to AddressSpaceManager class

* Add overflow protection to read and write functions
2021-02-28 11:48:04 +01:00
LDj3SNuD
dc0adb533d
PPTC & Pool Enhancements. (#1968)
* PPTC & Pool Enhancements.

* Avoid buffer allocations in CodeGenContext.GetCode(). Avoid stream allocations in PTC.PtcInfo.

Refactoring/nits.

* Use XXHash128, for Ptc.Load & Ptc.Save, x10 faster than Md5.

* Why not a nice Span.

* Added a simple PtcFormatter library for deserialization/serialization, which does not require reflection, in use at PtcJumpTable and PtcProfiler; improves maintainability and simplicity/readability of affected code.

* Nits.

* Revert #1987.

* Revert "Revert #1987."

This reverts commit 998be765cf.
2021-02-22 03:23:48 +01:00
gdkchan
734747ae58
Reduce temporary copy/fill buffer size (#1926) 2021-01-19 13:31:15 +11:00
riperiperi
a1f77a5b6a
Implement lazy flush-on-read for Buffers (SSBO/Copy) (#1790)
* Initial implementation of buffer flush (VERY WIP)

* Host shaders need to be rebuilt for the SSBO write flag.

* New approach with reserved regions and gl sync

* Fix a ton of buffer issues.

* Remove unused buffer unmapped behaviour

* Revert "Remove unused buffer unmapped behaviour"

This reverts commit f1700e52fb8760180ac5e0987a07d409d1e70ece.

* Delete modified ranges on unmap

Fixes potential crashes in Super Smash Bros, where a previously modified range could lie on either side of an unmap.

* Cache some more delegates.

* Dispose Sync on Close

* Also create host sync for GPFifo syncpoint increment.

* Copy buffer optimization, add docs

* Fix race condition with OpenGL Sync

* Enable read tracking on CommandBuffer, insert syncpoint on WaitForIdle

* Performance: Only flush individual pages of SSBO at a time

This avoids flushing large amounts of data when only a small amount is actually used.

* Signal Modified rather than flushing after clear

* Fix some docs and code style.

* Introduce a new test for tracking memory protection.

Sucessfully demonstrates that the bug causing write protection to be cleared by a read action has been fixed. (these tests fail on master)

* Address Comments

* Add host sync for SetReference

This ensures that any indirect draws will correctly flush any related buffer data written before them. Fixes some flashing and misplaced world geometry in MH rise.

* Make PageAlign static

* Re-enable read tracking, for reads.
2021-01-17 17:08:06 -03:00
gdkchan
c4f56c5704
Support for resources on non-contiguous GPU memory regions (#1905)
* Support for resources on non-contiguous GPU memory regions

* Implement MultiRange physical addresses, only used with a single range for now

* Actually use non-contiguous ranges

* GetPhysicalRegions fixes

* Documentation and remove Address property from TextureInfo

* Finish implementing GetWritableRegion

* Fix typo
2021-01-17 19:44:34 +01:00
LDj3SNuD
68f6b79fd3
Add a simple Pools Limiter. (#1830)
* Added support for offline invalidation, via PPTC, of low cq translations replaced by high cq translations; both on a single run and between runs.

Added invalidation of .cache files in the event of reuse on a different user operating system.

Added .info and .cache files invalidation in case of a failed stream decompression.

Nits.

* InternalVersion = 1712;

* Nits.

* Address comment.

* Get rid of BinaryFormatter.

Nits.

* Move Ptc.LoadTranslations().

Nits.

* Nits.

* Fixed corner cases (in case backup copies have to be used). Added save logs.

* Not core fixes.

* Complement to the previous commit. Added load logs. Removed BinaryFormatter leftovers.

* Add LoadTranslations log.

* Nits.

* Removed the search and management of LowCq overlapping functions.

* Final increment of .info and .cache flags.

* Nit.

* Free up memory allocated by Pools during any PPTC translations at boot time.

* Nit due to rebase.

* Add a simple Pools Limiter.

* Nits.

* Fix missing JumpTable.RegisterFunction() due to rebase.

Clear MemoryStreams as soon as possible, when they are no longer needed.

* Code cleaning.

* Nit for retrigger Checks.

* Update Ptc.cs

* Contextual refactoring of Translator. Ignore resetting of pools for DirectCallStubs.

* Nit for retrigger Checks.
2021-01-12 19:04:02 +01:00
EmulationFanatic
d05db79a3e
Update InvalidMemoryRegionException.cs to abide by English rules. (#1834) 2020-12-25 04:09:02 +01:00
gdkchan
19d18662ea
Correct type of executable sizes (#1802) 2020-12-13 08:30:27 +01:00
riperiperi
4594c3b310
Signal memory tracking before/after mapping into another process (#1785)
* Signal memory tracking before/after mapping into another process

* Wording.

* Add missing method.
2020-12-07 21:42:17 -03:00
gdkchan
cf6cd71488
IPC refactor part 2: Use ReplyAndReceive on HLE services and remove special handling from kernel (#1458)
* IPC refactor part 2: Use ReplyAndReceive on HLE services and remove special handling from kernel

* Fix for applet transfer memory + some nits

* Keep handles if possible to avoid server handle table exhaustion

* Fix IPC ZeroFill bug

* am: Correctly implement CreateManagedDisplayLayer and implement CreateManagedDisplaySeparableLayer

CreateManagedDisplaySeparableLayer is requires since 10.x+ when appletResourceUserId != 0

* Make it exit properly

* Make ServiceNotImplementedException show the full message again

* Allow yielding execution to avoid starving other threads

* Only wait if active

* Merge IVirtualMemoryManager and IAddressSpaceManager

* Fix Ro loading data from the wrong process

Co-authored-by: Thog <me@thog.eu>
2020-12-02 00:23:43 +01:00
Mary
aa129fdbdf
infra: Migrate to .NET 5 (#1694)
* infra: Migrate to .NET 5

This migrate projects and CI to .NET 5

* Remove language version restrictions (now on 9.0 by default)

* infra: pin .NET 5 to avoid later issues

* infra: Cleanup csproj files

* infra: update dependencies

* infra: Add temporary workaround for a bug in Vector128.Create

see https://github.com/dotnet/runtime/issues/44704 for more informations
2020-11-15 19:27:15 +01:00
gdkchan
3c60d4b0ea
Do not report unmapped pages as dirty (#1672)
* Do not report unmapped pages as dirty

* Make tests pass again

* PR feedback
2020-11-10 22:07:52 -03:00
riperiperi
b4d8d893a4
Memory Read/Write Tracking using Region Handles (#1272)
* WIP Range Tracking

- Texture invalidation seems to have large problems
- Buffer/Pool invalidation may have problems
- Mirror memory tracking puts an additional `add` in compiled code, we likely just want to make HLE access slower if this is the final solution.
- Native project is in the messiest possible location.
- [HACK] JIT memory access always uses native "fast" path
- [HACK] Trying some things with texture invalidation and views.

It works :)

Still a few hacks, messy things, slow things

More work in progress stuff (also move to memory project)

Quite a bit faster now.
- Unmapping GPU VA and CPU VA will now correctly update write tracking regions, and invalidate textures for the former.
- The Virtual range list is now non-overlapping like the physical one.
- Fixed some bugs where regions could leak.
- Introduced a weird bug that I still need to track down (consistent invalid buffer in MK8 ribbon road)

Move some stuff.

I think we'll eventually just put the dll and so for this in a nuget package.

Fix rebase.

[WIP] MultiRegionHandle variable size ranges

- Avoid reprotecting regions that change often (needs some tweaking)
- There's still a bug in buffers, somehow.
- Might want different api for minimum granularity

Fix rebase issue

Commit everything needed for software only tracking.

Remove native components.

Remove more native stuff.

Cleanup

Use a separate window for the background context, update opentk. (fixes linux)

Some experimental changes

Should get things working up to scratch - still need to try some things with flush/modification and res scale.

Include address with the region action.

Initial work to make range tracking work

Still a ton of bugs

Fix some issues with the new stuff.

* Fix texture flush instability

There's still some weird behaviour, but it's much improved without this. (textures with cpu modified data were flushing over it)

* Find the destination texture for Buffer->Texture full copy

Greatly improves performance for nvdec videos (with range tracking)

* Further improve texture tracking

* Disable Memory Tracking for view parents

This is a temporary approach to better match behaviour on master (where invalidations would be soaked up by views, rather than trigger twice)

The assumption is that when views are created to a texture, they will cover all of its data anyways. Of course, this can easily be improved in future.

* Introduce some tracking tests.

WIP

* Complete base tests.

* Add more tests for multiregion, fix existing test.

* Cleanup Part 1

* Remove unnecessary code from memory tracking

* Fix some inconsistencies with 3D texture rule.

* Add dispose tests.

* Use a background thread for the background context.

Rather than setting and unsetting a context as current, doing the work on a dedicated thread with signals seems to be a bit faster.

Also nerf the multithreading test a bit.

* Copy to texture with matching alignment

This extends the copy to work for some videos with unusual size, such as tutorial videos in SMO. It will only occur if the destination texture already exists at XCount size.

* Track reads for buffer copies. Synchronize new buffers before copying overlaps.

* Remove old texture flushing mechanisms.

Range tracking all the way, baby.

* Wake the background thread when disposing.

Avoids a deadlock when games are closed.

* Address Feedback 1

* Separate TextureCopy instance for background thread

Also `BackgroundContextWorker.InBackground` for a more sensible idenfifier for if we're in a background thread.

* Add missing XML docs.

* Address Feedback

* Maybe I should start drinking coffee.

* Some more feedback.

* Remove flush warning, Refocus window after making background context
2020-10-16 17:18:35 -03:00
gdkchan
57bb0abda3
Print guest stack trace on invalid memory access (#1407)
* Print guest stack trace on invalid memory access

* Improve XML docs
2020-07-30 23:16:41 +10:00
gdkchan
9f6b24edfd
Improve kernel IPC related syscalls (#1379)
* Implement session count decrement when the handle is closed

* Remove unused field

* Implement SendSyncRequestWithUserBuffer, SendAsyncRequestWithUserBuffer and ReplyAndReceiveWithUserBuffer syscalls

* Nits

* Fix swapped copy dst/src

* Add missing pointer buffer descriptor write on reply

* Fix IPC unaligned buffer copy and restoring client attributes on reply

* Oops

* Fix SetIpcMappingPermission

* Fix unaligned copy bugs

* Free memory used for temporary IPC buffers
2020-07-17 14:19:07 +10:00
gdkchan
4d02a2d2c0
New NVDEC and VIC implementation (#1384)
* Initial NVDEC and VIC implementation

* Update FFmpeg.AutoGen to 4.3.0

* Add nvdec dependencies for Windows

* Unify some VP9 structures

* Rename VP9 structure fields

* Improvements to Video API

* XML docs for Common.Memory

* Remove now unused or redundant overloads from MemoryAccessor

* NVDEC UV surface read/write scalar paths

* Add FIXME comments about hacky things/stuff that will need to be fixed in the future

* Cleaned up VP9 memory allocation

* Remove some debug logs

* Rename some VP9 structs

* Remove unused struct

* No need to compile Ryujinx.Graphics.Host1x with unsafe anymore

* Name AsyncWorkQueue threads to make debugging easier

* Make Vp9PictureInfo a ref struct

* LayoutConverter no longer needs the depth argument (broken by rebase)

* Pooling of VP9 buffers, plus fix a memory leak on VP9

* Really wish VS could rename projects properly...

* Address feedback

* Remove using

* Catch OperationCanceledException

* Add licensing informations

* Add THIRDPARTY.md to release too

Co-authored-by: Thog <me@thog.eu>
2020-07-12 05:07:01 +02:00
gdkchan
f77694e4f7
Implement a new physical memory manager and replace DeviceMemory (#856)
* Implement a new physical memory manager and replace DeviceMemory

* Proper generic constraints

* Fix debug build

* Add memory tests

* New CPU memory manager and general code cleanup

* Remove host memory management from CPU project, use Ryujinx.Memory instead

* Fix tests

* Document exceptions on MemoryBlock

* Fix leak on unix memory allocation

* Proper disposal of some objects on tests

* Fix JitCache not being set as initialized

* GetRef without checks for 8-bits and 16-bits CAS

* Add MemoryBlock destructor

* Throw in separate method to improve codegen

* Address PR feedback

* QueryModified improvements

* Fix memory write tracking not marking all pages as modified in some cases

* Simplify MarkRegionAsModified

* Remove XML doc for ghost param

* Add back optimization to avoid useless buffer updates

* Add Ryujinx.Cpu project, move MemoryManager there and remove MemoryBlockWrapper

* Some nits

* Do not perform address translation when size is 0

* Address PR feedback and format NativeInterface class

* Remove ghost parameter description

* Update Ryujinx.Cpu to .NET Core 3.1

* Address PR feedback

* Fix build

* Return a well defined value for GetPhysicalAddress with invalid VA, and do not return unmapped ranges as modified

* Typo
2020-05-04 08:54:50 +10:00