Commit graph

32 commits

Author SHA1 Message Date
gdkchan
556be08c4e
Implement PM GetProcessInfo atmosphere extension (partially) (#2966) 2022-05-03 23:28:32 +02:00
gdkchan
95017b8c66
Support memory aliasing (#2954)
* Back to the origins: Make memory manager take guest PA rather than host address once again

* Direct mapping with alias support on Windows

* Fixes and remove more of the emulated shared memory

* Linux support

* Make shared and transfer memory not depend on SharedMemoryStorage

* More efficient view mapping on Windows (no more restricted to 4KB pages at a time)

* Handle potential access violations caused by partial unmap

* Implement host mapping using shared memory on Linux

* Add new GetPhysicalAddressChecked method, used to ensure the virtual address is mapped before address translation

Also align GetRef behaviour with software memory manager

* We don't need a mirrorable memory block for software memory manager mode

* Disable memory aliasing tests while we don't have shared memory support on Mac

* Shared memory & SIGBUS handler for macOS

* Fix typo + nits + re-enable memory tests

* Set MAP_JIT_DARWIN on x86 Mac too

* Add back the address space mirror

* Only set MAP_JIT_DARWIN if we are mapping as executable

* Disable aliasing tests again (still fails on Mac)

* Fix UnmapView4KB (by not casting size to int)

* Use ref counting on memory blocks to delay closing the shared memory handle until all blocks using it are disposed

* Address PR feedback

* Make RO hold a reference to the guest process memory manager to avoid early disposal

Co-authored-by: nastys <nastys@users.noreply.github.com>
2022-05-02 20:30:02 -03:00
MutantAura
ee174be57c
Mod loading from atmosphere SD directories (#3176)
* initial sd support

* GUI option

* alignment

* review changes
2022-03-06 22:12:01 +01:00
Alex Barney
3705c20668
Update LibHac to v0.16.0 (#3159) 2022-02-27 00:52:25 +01:00
Alex Barney
d317cfd639
Try to ensure save data always has a valid owner ID (#3057)
- Run the extra data fix in FixExtraData on non-system saves that have no owner ID.
- Set the owner ID in the dummy application control property if an application doesn't have a proper one available.
2022-02-02 14:49:49 -07:00
Alex Barney
f4bbc019b9
Update to LibHac 0.15.0 (#2986) 2022-01-12 12:22:19 +01:00
Alex Barney
aa932a6df1
Update to LibHac v0.14.3 (#2925)
* Update to LibHac v0.14.3

* Fix loading NCAs that don't have a data partition
2021-12-23 13:55:50 -03:00
Logan Stromberg
560ed5eebd
Don't blow up everything if a DLC file is moved or renamed. (#2867)
* Don't blow up everything if a DLC file is missing

* change comment

* More correctly setting the "enabled" check box on dlc dialog for the add-on NSP based on the enabled state of all NCAs in the package.

* Update Ryujinx.HLE/HOS/ApplicationLoader.cs

Co-authored-by: Ac_K <Acoustik666@gmail.com>

Co-authored-by: Logan Stromberg <lostromb@microsoft.com>
Co-authored-by: Ac_K <Acoustik666@gmail.com>
2021-11-29 15:21:27 -03:00
Alex Barney
dadc0e59da
Update to LibHac 0.13.1 (#2475)
* Update to LibHac 0.13.1

* Recreate directories for indexed saves if they're missing on emulator start
2021-08-12 23:56:24 +02:00
Mary
208ba1dde2 Revert LibHac update
Users are facing save destruction on failing extra data update apparently
2021-07-13 16:48:54 +02:00
Alex Barney
19afb3209c
Update to LibHac 0.13.1 (#2328)
Update the LibHac dependency to version 0.13.1. This brings a ton of improvements and changes such as:
- Refactor `FsSrv` to match the official refactoring done in FS.
- Change how the `Horizon` and `HorizonClient` classes are handled. Each client created represents a different process with its own process ID and client state.
- Add FS access control to handle permissions for FS service method calls.
- Add FS program registry to keep track of the program ID, location and permissions of each process.
- Add FS program index map info manager to track the program IDs and indexes of multi-application programs.
- Add all FS IPC interfaces.
- Rewrite `Fs.Fsa` code to be more accurate.
- Rewrite a lot of `FsSrv` code to be more accurate.
- Extend directory save data to store `SaveDataExtraData`
- Extend directory save data to lock the save directory to allow only one accessor at a time.
- Improve waiting and retrying when encountering access issues in `LocalFileSystem` and `DirectorySaveDataFileSystem`.
- More `IFileSystemProxy` methods should work now.
- Probably a bunch more stuff.

On the Ryujinx side:
- Forward most `IFileSystemProxy` methods to LibHac.
- Register programs and program index map info when launching an application.
- Remove hacks and workarounds for missing LibHac functionality.
- Recreate missing save data extra data found on emulator startup.
- Create system save data that wasn't indexed correctly on an older LibHac version.

`FsSrv` now enforces access control for each process. When a process tries to open a save data file system, FS reads the save's extra data to determine who the save owner is and if the caller has permission to open the save data. Previously-created save data did not have extra data created when the save was created.
With access control checks in place, this means that processes with no permissions (most games) wouldn't be able to access their own save data. The extra data can be partially created from data in the save data indexer, which should be enough for access control purposes.
2021-07-13 01:19:28 -07:00
riperiperi
54ea2285f0
POWER - Performance Optimizations With Extensive Ramifications (#2286)
* Refactoring of KMemoryManager class

* Replace some trivial uses of DRAM address with VA

* Get rid of GetDramAddressFromVa

* Abstracting more operations on derived page table class

* Run auto-format on KPageTableBase

* Managed to make TryConvertVaToPa private, few uses remains now

* Implement guest physical pages ref counting, remove manual freeing

* Make DoMmuOperation private and call new abstract methods only from the base class

* Pass pages count rather than size on Map/UnmapMemory

* Change memory managers to take host pointers

* Fix a guest memory leak and simplify KPageTable

* Expose new methods for host range query and mapping

* Some refactoring of MapPagesFromClientProcess to allow proper page ref counting and mapping without KPageLists

* Remove more uses of AddVaRangeToPageList, now only one remains (shared memory page checking)

* Add a SharedMemoryStorage class, will be useful for host mapping

* Sayonara AddVaRangeToPageList, you served us well

* Start to implement host memory mapping (WIP)

* Support memory tracking through host exception handling

* Fix some access violations from HLE service guest memory access and CPU

* Fix memory tracking

* Fix mapping list bugs, including a race and a error adding mapping ranges

* Simple page table for memory tracking

* Simple "volatile" region handle mode

* Update UBOs directly (experimental, rough)

* Fix the overlap check

* Only set non-modified buffers as volatile

* Fix some memory tracking issues

* Fix possible race in MapBufferFromClientProcess (block list updates were not locked)

* Write uniform update to memory immediately, only defer the buffer set.

* Fix some memory tracking issues

* Pass correct pages count on shared memory unmap

* Armeilleure Signal Handler v1 + Unix changes

Unix currently behaves like windows, rather than remapping physical

* Actually check if the host platform is unix

* Fix decommit on linux.

* Implement windows 10 placeholder shared memory, fix a buffer issue.

* Make PTC version something that will never match with master

* Remove testing variable for block count

* Add reference count for memory manager, fix dispose

Can still deadlock with OpenAL

* Add address validation, use page table for mapped check, add docs

Might clean up the page table traversing routines.

* Implement batched mapping/tracking.

* Move documentation, fix tests.

* Cleanup uniform buffer update stuff.

* Remove unnecessary assignment.

* Add unsafe host mapped memory switch

On by default. Would be good to turn this off for untrusted code (homebrew, exefs mods) and give the user the option to turn it on manually, though that requires some UI work.

* Remove C# exception handlers

They have issues due to current .NET limitations, so the meilleure one fully replaces them for now.

* Fix MapPhysicalMemory on the software MemoryManager.

* Null check for GetHostAddress, docs

* Add configuration for setting memory manager mode (not in UI yet)

* Add config to UI

* Fix type mismatch on Unix signal handler code emit

* Fix 6GB DRAM mode.

The size can be greater than `uint.MaxValue` when the DRAM is >4GB.

* Address some feedback.

* More detailed error if backing memory cannot be mapped.

* SetLastError on all OS functions for consistency

* Force pages dirty with UBO update instead of setting them directly.

Seems to be much faster across a few games. Need retesting.

* Rebase, configuration rework, fix mem tracking regression

* Fix race in FreePages

* Set memory managers null after decrementing ref count

* Remove readonly keyword, as this is now modified.

* Use a local variable for the signal handler rather than a register.

* Fix bug with buffer resize, and index/uniform buffer binding.

Should fix flickering in games.

* Add InvalidAccessHandler to MemoryTracking

Doesn't do anything yet

* Call invalid access handler on unmapped read/write.

Same rules as the regular memory manager.

* Make unsafe mapped memory its own MemoryManagerType

* Move FlushUboDirty into UpdateState.

* Buffer dirty cache, rather than ubo cache

Much cleaner, may be reusable for Inline2Memory updates.

* This doesn't return anything anymore.

* Add sigaction remove methods, correct a few function signatures.

* Return empty list of physical regions for size 0.

* Also on AddressSpaceManager

Co-authored-by: gdkchan <gab.dark.100@gmail.com>
2021-05-24 22:52:44 +02:00
Mary
bec67dbef7
misc: Move configuration management to the Ryujinx project (#2269)
* Decouple configuration from Ryujinx.HLE and Ryujinx.Input

* Move Configuration to the Ryujinx project
2021-05-16 17:12:14 +02:00
LDj3SNuD
57ea3f93a3
PPTC meets ExeFS Patching. (#1865)
* PPTC meets ExeFS Patching.

* InternalVersion = 1865

* Ready!

* Optimized the PtcProfiler Load/Save methods.
2021-05-13 20:05:15 +02:00
Ac_K
7344dee475
account: Adds AccountManager (#2184)
* account: Adds Account Manager

In a way to have Custom User Profiles merged in master faster, this PR adds a `AccountManager` class (based on `AccountUtils` class) and the following changes have been made:
- Adds a "default profile values" which were the old hardcoded ones.
- The image profile is moved to the Account service folder.
- The hardcoded UserId for the savedata is now using the `AccountManager` last opened one.
- The DeviceId in Mii service is changed to the right value (checked by REd sys:set call).

* Fix csproj

* Addresses gdkchan's comments

* Fix UserProfile fields

* Fix mii GetDeviceId()

* Update Ryujinx.HLE.csproj
2021-04-13 03:16:43 +02:00
Caian Benedicto
0c1ea1212a
Add the TamperMachine module for runtime mods and cheats (#1928)
* Add initial implementation of the Tamper Machine

* Implement Atmosphere opcodes 0, 4 and 9

* Add missing TamperCompilationException class

* Implement Atmosphere conditional and loop opcodes 1, 2 and 3

* Inplement input conditional opcode 8

* Add register store opcode A

* Implement extended pause/resume opcodes FF0 and FF1

* Implement extended log opcode FFF

* Implement extended register conditional opcode C0

* Refactor TamperProgram to an interface

* Moved Atmosphere classes to a separate subdirectory

* Fix OpProcCtrl class not setting process

* Implement extended register save/restore opcodes C1, C2 and C3

* Refactor code emitters to separate classes

* Supress memory access errors from the Tamper Machine

* Add debug information to tamper register and memory writes

* Add block stack check to Atmosphere Cheat compiler

* Add handheld input support to Tamper Machine

* Fix code styling

* Fix build id and cheat case mismatch

* Fix invalid immediate size selection

* Print build ids of the title

* Prevent Tamper Machine from change code regions

* Remove Atmosphere namespace

* Remove empty cheats from the list

* Prevent code modification without disabling the tampering

* Fix missing addressing mode in LoadRegisterWithMemory

* Fix wrong addressing in RegisterConditional

* Add name to the tamper machine thread

* Fix code styling
2021-03-27 15:12:05 +01:00
mageven
65eb9901f1
Allow modding AddOnContent RomFS (#2024)
Simplify some misc logic
2021-02-20 01:25:01 +01:00
Somebody Whoisbored
a03ab0c4a0
Fix exefs mod loading with unpacked games (#1857)
* Add the ability to layeredfs individual exefs with mod loader

* Address code style issues

* Further adjustments to the mod loading

* Update Ryujinx.HLE/HOS/ModLoader.cs

Co-authored-by: Ac_K <Acoustik666@gmail.com>

* Apply suggestions from code review

Co-authored-by: Ac_K <Acoustik666@gmail.com>

* Address issue with checking for NSO existence (and code style fixes)

Co-authored-by: Ac_K <Acoustik666@gmail.com>
2021-01-03 12:30:31 +01:00
Somebody Whoisbored
fb0db32338
Add the ability to add individual files exefs with mod loader (#1766)
Co-authored-by: Ac_K <Acoustik666@gmail.com>
2020-12-29 20:54:32 +01:00
Mary
863edae328
shader cache: Fix Linux boot issues (#1709)
* shader cache: Fix Linux boot issues

This rollback the init logic back to previous state, and replicate the
way PTC handle initialization.

* shader cache: set default state of ready for translation event to false

* Fix cpu unit tests
2020-11-17 22:40:19 +01:00
Mary
48f6570557
Salieri: shader cache (#1701)
Here come Salieri, my implementation of a disk shader cache!

"I'm sure you know why I named it that."
"It doesn't really mean anything."

This implementation collects shaders at runtime and cache them to be later compiled when starting a game.
2020-11-13 00:15:34 +01:00
Mary
33f8284bc0
hle/ui: Basic multi programs support (#1560)
* hos/gui: Add a check of NCA program index in titleid

This add a check to `ApplicationLoader` for the last 2 digits of the game TitleId who seems to be the NCA program index.
We currently return the last index, instead of the lower one.
Same check is added to ApplicationLibrary in the UI.

I've cleaned up both file too.

* hle: implement partial relaunch logic

TODO: make the emulator auto relauch.

* Handle auto relaunch

* hle: Unify update usage system

* hle: Implement support of multi programs in update system

* Add some documentation

* Address rip's comment

Co-authored-by: Ac_K <Acoustik666@gmail.com>
2020-09-21 13:45:30 +10:00
Alex Barney
1bb7fdaca4
Update to LibHac 0.12.0 (#1485)
* Update to LibHac 0.12.0

* Auto-formatting. Fixed a bug in SetApplicationCopyrightImage
2020-09-01 17:08:59 -03:00
mageven
b9398f1f3a
Allow launching with custom data directories (#1505)
* Allow launching with custom data directories

Don't load alternate keys when using custom directory

* Address gdkchan's comments

* Misc fixes to log levels

Added more enabled log levels by default

Moved successful config updation to Notice as
1. It's not a warning
2. Warnings could've been disabled by the config load and hence message
   would be lost
2020-08-30 18:51:53 +02:00
mageven
a33dc2f491
Improved Logger (#1292)
* Logger class changes only

Now compile-time checking is possible with the help of Nullable Value
types.

* Misc formatting

* Manual optimizations

PrintGuestLog
PrintGuestStackTrace
Surfaceflinger DequeueBuffer

* Reduce SendVibrationXX log level to Debug

* Add Notice log level

This level is always enabled and used to print system info, etc...
Also, rewrite LogColor to switch expression as colors are static

* Unify unhandled exception event handlers

* Print enabled LogLevels during init

* Re-add App Exit disposes in proper order

nit: switch case spacing

* Revert PrintGuestStackTrace to Info logs due to #1407

PrintGuestStackTrace is now called in some critical error handlers
so revert to old behavior as KThread isn't part of Guest.

* Batch replace Logger statements
2020-08-04 01:32:53 +02:00
mageven
16dafe6316
mods: Fix crash when loading via LoadCart (#1388) 2020-07-15 09:40:17 +10:00
mageven
189c0c9c72
Implement modding support (#1249)
* Implement Modding Support

* Executables: Rewrite to use contiguous mem and Spans

* Reorder ExeFs, Npdm, ControlData and SaveData calls

After discussion with gdkchan, it was decided it's best to call
LoadExeFs after all other loads are done as it starts the guest process.

* Build RomFs manually instead of Layering FS

Layered FS approach has considerable latency when building the final
romfs. So, we manually replace files in a single romfs instance.

* Add RomFs modding via storage file

* Fix and cleanup MemPatch

* Add dynamically loaded NRO patching

* Support exefs file replacement

* Rewrite ModLoader to use mods-search architecture

* Disable PPTC when exefs patches are detected

Disable PPTC on exefs replacements too

* Rewrite ModLoader, again

* Increased maintainability and matches Atmosphere closely
* Creates base mods structure if it doesn't exist
* Add Exefs partition replacement
* IPSwitch: Fix nsobid parsing

* Move mod logs to new LogClass

* Allow custom suffixes to title dirs again

* Address nits

* Add a per-App "Open Mods Directory" context menu item

Creates the path if not present.

* Normalize tooltips verbiage

* Use LocalStorage and remove unused namespaces
2020-07-09 14:31:15 +10:00
Mary
2c48750ff0
Fix compilation warnings and use new LibHac APIs for executable loading (#1350)
* Fix compilation warnings and use new LibHac APIs for executable loading

* Migrate NSO loader to the new reader and fix kip loader

* Fix CS0162 restore

* Remove extra return lines

* Address Moose's comment
2020-07-04 01:58:01 +02:00
Xpl0itR
2ed9db1fcd
Implement dlc management window (#1313)
* Implement dlc management window

* reduce repetition

* Implement per NCA toggling of DLC rather than per container
2020-06-23 10:32:07 +10:00
mageven
1c2af7ce92
Implement aoc:u and support loading AddOnContent (#1221)
* Initial rebased AddOnContent support

* Fix bounds calculation
* Use existing GameCard in VFS per Xpl0itR's suggestion
+ Add dummy IPurchaseEventManager per AcK's suggestion

* Support multiple containers

* Add option to selectively disable addons

* Import tickets from AOC FS

* Load all nsps in base directory automatically

* Revert LoadNsp renaming

Removes conflicts with Mods PR. Not much is lost, old names were fine.

* Address AcK's comments

* Address Thog's comments

Dispose opened nsp files
Fix potential bug by clearing metadata on load
2020-06-20 19:38:14 +02:00
LDj3SNuD
5e724cf24e
Add Profiled Persistent Translation Cache. (#769)
* Delete DelegateTypes.cs

* Delete DelegateCache.cs

* Add files via upload

* Update Horizon.cs

* Update Program.cs

* Update MainWindow.cs

* Update Aot.cs

* Update RelocEntry.cs

* Update Translator.cs

* Update MemoryManager.cs

* Update InstEmitMemoryHelper.cs

* Update Delegates.cs

* Nit.

* Nit.

* Nit.

* 10 fewer MSIL bytes for us

* Add comment. Nits.

* Update Translator.cs

* Update Aot.cs

* Nits.

* Opt..

* Opt..

* Opt..

* Opt..

* Allow to change compression level.

* Update MemoryManager.cs

* Update Translator.cs

* Manage corner cases during the save phase. Nits.

* Update Aot.cs

* Translator response tweak for Aot disabled. Nit.

* Nit.

* Nits.

* Create DelegateHelpers.cs

* Update Delegates.cs

* Nit.

* Nit.

* Nits.

* Fix due to #784.

* Fixes due to #757 & #841.

* Fix due to #846.

* Fix due to #847.

* Use MethodInfo for managed method calls.

Use IR methods instead of managed methods about Max/Min (S/U).
Follow-ups & Nits.

* Add missing exception messages.

Reintroduce slow path for Fmov_Vi.
Implement slow path for Fmov_Si.

* Switch to the new folder structure.

Nits.

* Impl. index-based relocation information. Impl. cache file version field.

* Nit.

* Address gdkchan comments.

Mainly:
- fixed cache file corruption issue on exit; - exposed a way to disable AOT on the GUI.

* Address AcK77 comment.

* Address Thealexbarney, jduncanator & emmauss comments.

Header magic, CpuId (FI) & Aot -> Ptc.

* Adaptation to the new application reloading system.

Improvements to the call system of managed methods.
Follow-ups.
Nits.

* Get the same boot times as on master when PTC is disabled.

* Profiled Aot.

* A32 support (#897).

* #975 support (1 of 2).

* #975 support (2 of 2).

* Rebase fix & nits.

* Some fixes and nits (still one bug left).

* One fix & nits.

* Tests fix (by gdk) & nits.

* Support translations not only in high quality and rejit.

Nits.

* Added possibility to skip translations and continue execution, using `ESC` key.

* Update SettingsWindow.cs

* Update GLRenderer.cs

* Update Ptc.cs

* Disabled Profiled PTC by default as requested in the past by gdk.

* Fix rejit bug. Increased number of parallel translations. Add stack unwinding stuffs support (1 of 2).

Nits.

* Add stack unwinding stuffs support (2 of 2). Tuned number of parallel translations.

* Restored the ability to assemble jumps with 8-bit offset when Profiled PTC is disabled or during profiling.

Modifications due to rebase.
Nits.

* Limited profiling of the functions to be translated to the addresses belonging to the range of static objects only.

* Nits.

* Nits.

* Update Delegates.cs

* Nit.

* Update InstEmitSimdArithmetic.cs

* Address riperiperi comments.

* Fixed the issue of unjustifiably longer boot times at the second boot than at the first boot, measured at the same time or reference point and with the same number of translated functions.

* Implemented a simple redundant load/save mechanism.

Halved the value of Decoder.MaxInstsPerFunction more appropriate for the current performance of the Translator.
Replaced by Logger.PrintError to Logger.PrintDebug in TexturePool.cs about the supposed invalid texture format to avoid the spawn of the log.
Nits.

* Nit.

Improved Logger.PrintError in TexturePool.cs to avoid log spawn.
Added missing code for FZ handling (in output) for fp max/min instructions (slow paths).

* Add configuration migration for PTC

Co-authored-by: Thog <me@thog.eu>
2020-06-16 20:28:02 +02:00
mageven
ba4830293e
Refactor out Application details from Horizon (#1236)
* Initial Application refactor

* Misc typo and access modifier fixes

* Clean unused namespaces

* Address gdkchan's comments

* Move ticket reading to common method

* Change IParentalControlService to use ApplicationLoader.ControlData
2020-05-15 03:16:46 -03:00