Add a new JIT compiler for CPU code (#693)
* Start of the ARMeilleure project
* Refactoring around the old IRAdapter, now renamed to PreAllocator
* Optimize the LowestBitSet method
* Add CLZ support and fix CLS implementation
* Add missing Equals and GetHashCode overrides on some structs, misc small tweaks
* Implement the ByteSwap IR instruction, and some refactoring on the assembler
* Implement the DivideUI IR instruction and fix 64-bits IDIV
* Correct constant operand type on CSINC
* Move division instructions implementation to InstEmitDiv
* Fix destination type for the ConditionalSelect IR instruction
* Implement UMULH and SMULH, with new IR instructions
* Fix some issues with shift instructions
* Fix constant types for BFM instructions
* Fix up new tests using the new V128 struct
* Update tests
* Move DIV tests to a separate file
* Add support for calls, and some instructions that depends on them
* Start adding support for SIMD & FP types, along with some of the related ARM instructions
* Fix some typos and the divide instruction with FP operands
* Fix wrong method call on Clz_V
* Implement ARM FP & SIMD move instructions, Saddlv_V, and misc. fixes
* Implement SIMD logical instructions and more misc. fixes
* Fix PSRAD x86 instruction encoding, TRN, UABD and UABDL implementations
* Implement float conversion instruction, merge in LDj3SNuD fixes, and some other misc. fixes
* Implement SIMD shift instruction and fix Dup_V
* Add SCVTF and UCVTF (vector, fixed-point) variants to the opcode table
* Fix check with tolerance on tester
* Implement FP & SIMD comparison instructions, and some fixes
* Update FCVT (Scalar) encoding on the table to support the Half-float variants
* Support passing V128 structs, some cleanup on the register allocator, merge LDj3SNuD fixes
* Use old memory access methods, made a start on SIMD memory insts support, some fixes
* Fix float constant passed to functions, save and restore non-volatile XMM registers, other fixes
* Fix arguments count with struct return values, other fixes
* More instructions
* Misc. fixes and integrate LDj3SNuD fixes
* Update tests
* Add a faster linear scan allocator, unwinding support on windows, and other changes
* Update Ryujinx.HLE
* Update Ryujinx.Graphics
* Fix V128 return pointer passing, RCX is clobbered
* Update Ryujinx.Tests
* Update ITimeZoneService
* Stop using GetFunctionPointer as that can't be called from native code, misc. fixes and tweaks
* Use generic GetFunctionPointerForDelegate method and other tweaks
* Some refactoring on the code generator, assert on invalid operations and use a separate enum for intrinsics
* Remove some unused code on the assembler
* Fix REX.W prefix regression on float conversion instructions, add some sort of profiler
* Add hardware capability detection
* Fix regression on Sha1h and revert Fcm** changes
* Add SSE2-only paths on vector extract and insert, some refactoring on the pre-allocator
* Fix silly mistake introduced on last commit on CpuId
* Generate inline stack probes when the stack allocation is too large
* Initial support for the System-V ABI
* Support multiple destination operands
* Fix SSE2 VectorInsert8 path, and other fixes
* Change placement of XMM callee save and restore code to match other compilers
* Rename Dest to Destination and Inst to Instruction
* Fix a regression related to calls and the V128 type
* Add an extra space on comments to match code style
* Some refactoring
* Fix vector insert FP32 SSE2 path
* Port over the ARM32 instructions
* Avoid memory protection races on JIT Cache
* Another fix on VectorInsert FP32 (thanks to LDj3SNuD
* Float operands don't need to use the same register when VEX is supported
* Add a new register allocator, higher quality code for hot code (tier up), and other tweaks
* Some nits, small improvements on the pre allocator
* CpuThreadState is gone
* Allow changing CPU emulators with a config entry
* Add runtime identifiers on the ARMeilleure project
* Allow switching between CPUs through a config entry (pt. 2)
* Change win10-x64 to win-x64 on projects
* Update the Ryujinx project to use ARMeilleure
* Ensure that the selected register is valid on the hybrid allocator
* Allow exiting on returns to 0 (should fix test regression)
* Remove register assignments for most used variables on the hybrid allocator
* Do not use fixed registers as spill temp
* Add missing namespace and remove unneeded using
* Address PR feedback
* Fix types, etc
* Enable AssumeStrictAbiCompliance by default
* Ensure that Spill and Fill don't load or store any more than necessary
2019-08-08 20:56:22 +02:00
|
|
|
using ARMeilleure.Memory;
|
|
|
|
using ARMeilleure.State;
|
|
|
|
using ARMeilleure.Translation;
|
2018-11-28 23:18:09 +01:00
|
|
|
using Ryujinx.Common;
|
2019-01-25 02:59:53 +01:00
|
|
|
using Ryujinx.HLE.Exceptions;
|
2018-12-18 06:33:36 +01:00
|
|
|
using Ryujinx.HLE.HOS.Kernel.Common;
|
|
|
|
using Ryujinx.HLE.HOS.Kernel.Memory;
|
|
|
|
using Ryujinx.HLE.HOS.Kernel.SupervisorCall;
|
|
|
|
using Ryujinx.HLE.HOS.Kernel.Threading;
|
2018-11-28 23:18:09 +01:00
|
|
|
using System;
|
|
|
|
using System.Collections.Generic;
|
|
|
|
using System.Linq;
|
|
|
|
using System.Threading;
|
|
|
|
|
2018-12-18 06:33:36 +01:00
|
|
|
namespace Ryujinx.HLE.HOS.Kernel.Process
|
2018-11-28 23:18:09 +01:00
|
|
|
{
|
|
|
|
class KProcess : KSynchronizationObject
|
|
|
|
{
|
|
|
|
public const int KernelVersionMajor = 10;
|
|
|
|
public const int KernelVersionMinor = 4;
|
|
|
|
public const int KernelVersionRevision = 0;
|
|
|
|
|
|
|
|
public const int KernelVersionPacked =
|
|
|
|
(KernelVersionMajor << 19) |
|
|
|
|
(KernelVersionMinor << 15) |
|
|
|
|
(KernelVersionRevision << 0);
|
|
|
|
|
2018-12-05 01:52:39 +01:00
|
|
|
public KMemoryManager MemoryManager { get; private set; }
|
2018-11-28 23:18:09 +01:00
|
|
|
|
2018-12-06 12:16:24 +01:00
|
|
|
private SortedDictionary<ulong, KTlsPageInfo> _fullTlsPages;
|
|
|
|
private SortedDictionary<ulong, KTlsPageInfo> _freeTlsPages;
|
2018-11-28 23:18:09 +01:00
|
|
|
|
2019-01-18 23:26:39 +01:00
|
|
|
public int DefaultCpuCore { get; set; }
|
2018-11-28 23:18:09 +01:00
|
|
|
|
|
|
|
public bool Debug { get; private set; }
|
|
|
|
|
|
|
|
public KResourceLimit ResourceLimit { get; private set; }
|
|
|
|
|
|
|
|
public ulong PersonalMmHeapPagesCount { get; private set; }
|
|
|
|
|
2018-12-06 12:16:24 +01:00
|
|
|
private ProcessState _state;
|
2018-11-28 23:18:09 +01:00
|
|
|
|
2018-12-06 12:16:24 +01:00
|
|
|
private object _processLock;
|
|
|
|
private object _threadingLock;
|
2018-11-28 23:18:09 +01:00
|
|
|
|
2018-12-05 01:52:39 +01:00
|
|
|
public KAddressArbiter AddressArbiter { get; private set; }
|
2018-11-28 23:18:09 +01:00
|
|
|
|
2018-12-05 01:52:39 +01:00
|
|
|
public long[] RandomEntropy { get; private set; }
|
2018-11-28 23:18:09 +01:00
|
|
|
|
2018-12-06 12:16:24 +01:00
|
|
|
private bool _signaled;
|
|
|
|
private bool _useSystemMemBlocks;
|
2018-11-28 23:18:09 +01:00
|
|
|
|
|
|
|
public string Name { get; private set; }
|
|
|
|
|
2018-12-06 12:16:24 +01:00
|
|
|
private int _threadCount;
|
2018-11-28 23:18:09 +01:00
|
|
|
|
|
|
|
public int MmuFlags { get; private set; }
|
|
|
|
|
2018-12-06 12:16:24 +01:00
|
|
|
private MemoryRegion _memRegion;
|
2018-11-28 23:18:09 +01:00
|
|
|
|
2018-12-05 01:52:39 +01:00
|
|
|
public KProcessCapabilities Capabilities { get; private set; }
|
2018-11-28 23:18:09 +01:00
|
|
|
|
2019-09-02 18:03:57 +02:00
|
|
|
public ulong TitleId { get; private set; }
|
|
|
|
public long Pid { get; private set; }
|
2018-11-28 23:18:09 +01:00
|
|
|
|
2018-12-06 12:16:24 +01:00
|
|
|
private long _creationTimestamp;
|
|
|
|
private ulong _entrypoint;
|
|
|
|
private ulong _imageSize;
|
|
|
|
private ulong _mainThreadStackSize;
|
|
|
|
private ulong _memoryUsageCapacity;
|
|
|
|
private int _category;
|
2018-11-28 23:18:09 +01:00
|
|
|
|
|
|
|
public KHandleTable HandleTable { get; private set; }
|
|
|
|
|
|
|
|
public ulong UserExceptionContextAddress { get; private set; }
|
|
|
|
|
2018-12-06 12:16:24 +01:00
|
|
|
private LinkedList<KThread> _threads;
|
2018-11-28 23:18:09 +01:00
|
|
|
|
|
|
|
public bool IsPaused { get; private set; }
|
|
|
|
|
2019-10-31 19:09:03 +01:00
|
|
|
public MemoryManager CpuMemory { get; private set; }
|
2018-11-28 23:18:09 +01:00
|
|
|
|
2019-10-31 19:09:03 +01:00
|
|
|
public Translator Translator { get; private set; }
|
2019-02-24 08:24:35 +01:00
|
|
|
|
2018-12-06 12:16:24 +01:00
|
|
|
private SvcHandler _svcHandler;
|
2018-11-28 23:18:09 +01:00
|
|
|
|
2019-02-24 08:24:35 +01:00
|
|
|
private Horizon _system;
|
|
|
|
|
2018-12-05 01:52:39 +01:00
|
|
|
public HleProcessDebugger Debugger { get; private set; }
|
2018-11-28 23:18:09 +01:00
|
|
|
|
2018-12-06 12:16:24 +01:00
|
|
|
public KProcess(Horizon system) : base(system)
|
2018-11-28 23:18:09 +01:00
|
|
|
{
|
2018-12-06 12:16:24 +01:00
|
|
|
_processLock = new object();
|
|
|
|
_threadingLock = new object();
|
2018-11-28 23:18:09 +01:00
|
|
|
|
2019-02-24 08:24:35 +01:00
|
|
|
_system = system;
|
2018-11-28 23:18:09 +01:00
|
|
|
|
2018-12-06 12:16:24 +01:00
|
|
|
AddressArbiter = new KAddressArbiter(system);
|
2018-11-28 23:18:09 +01:00
|
|
|
|
2018-12-06 12:16:24 +01:00
|
|
|
_fullTlsPages = new SortedDictionary<ulong, KTlsPageInfo>();
|
|
|
|
_freeTlsPages = new SortedDictionary<ulong, KTlsPageInfo>();
|
2018-11-28 23:18:09 +01:00
|
|
|
|
|
|
|
Capabilities = new KProcessCapabilities();
|
|
|
|
|
|
|
|
RandomEntropy = new long[KScheduler.CpuCoresCount];
|
|
|
|
|
2018-12-06 12:16:24 +01:00
|
|
|
_threads = new LinkedList<KThread>();
|
2018-11-28 23:18:09 +01:00
|
|
|
|
2018-12-06 12:16:24 +01:00
|
|
|
_svcHandler = new SvcHandler(system.Device, this);
|
2018-11-28 23:18:09 +01:00
|
|
|
|
|
|
|
Debugger = new HleProcessDebugger(this);
|
|
|
|
}
|
|
|
|
|
|
|
|
public KernelResult InitializeKip(
|
2018-12-06 12:16:24 +01:00
|
|
|
ProcessCreationInfo creationInfo,
|
|
|
|
int[] caps,
|
|
|
|
KPageList pageList,
|
|
|
|
KResourceLimit resourceLimit,
|
|
|
|
MemoryRegion memRegion)
|
2018-11-28 23:18:09 +01:00
|
|
|
{
|
2018-12-06 12:16:24 +01:00
|
|
|
ResourceLimit = resourceLimit;
|
|
|
|
_memRegion = memRegion;
|
2018-11-28 23:18:09 +01:00
|
|
|
|
2018-12-06 12:16:24 +01:00
|
|
|
AddressSpaceType addrSpaceType = (AddressSpaceType)((creationInfo.MmuFlags >> 1) & 7);
|
2018-11-28 23:18:09 +01:00
|
|
|
|
2019-02-24 08:24:35 +01:00
|
|
|
InitializeMemoryManager(addrSpaceType, memRegion);
|
|
|
|
|
2018-12-06 12:16:24 +01:00
|
|
|
bool aslrEnabled = ((creationInfo.MmuFlags >> 5) & 1) != 0;
|
2018-11-28 23:18:09 +01:00
|
|
|
|
2018-12-06 12:16:24 +01:00
|
|
|
ulong codeAddress = creationInfo.CodeAddress;
|
2018-11-28 23:18:09 +01:00
|
|
|
|
2018-12-06 12:16:24 +01:00
|
|
|
ulong codeSize = (ulong)creationInfo.CodePagesCount * KMemoryManager.PageSize;
|
2018-11-28 23:18:09 +01:00
|
|
|
|
2018-12-06 12:16:24 +01:00
|
|
|
KMemoryBlockAllocator memoryBlockAllocator = (MmuFlags & 0x40) != 0
|
2018-11-28 23:18:09 +01:00
|
|
|
? System.LargeMemoryBlockAllocator
|
|
|
|
: System.SmallMemoryBlockAllocator;
|
|
|
|
|
2018-12-06 12:16:24 +01:00
|
|
|
KernelResult result = MemoryManager.InitializeForProcess(
|
|
|
|
addrSpaceType,
|
|
|
|
aslrEnabled,
|
|
|
|
!aslrEnabled,
|
|
|
|
memRegion,
|
|
|
|
codeAddress,
|
|
|
|
codeSize,
|
|
|
|
memoryBlockAllocator);
|
2018-11-28 23:18:09 +01:00
|
|
|
|
2018-12-06 12:16:24 +01:00
|
|
|
if (result != KernelResult.Success)
|
2018-11-28 23:18:09 +01:00
|
|
|
{
|
2018-12-06 12:16:24 +01:00
|
|
|
return result;
|
2018-11-28 23:18:09 +01:00
|
|
|
}
|
|
|
|
|
2018-12-06 12:16:24 +01:00
|
|
|
if (!ValidateCodeAddressAndSize(codeAddress, codeSize))
|
2018-11-28 23:18:09 +01:00
|
|
|
{
|
|
|
|
return KernelResult.InvalidMemRange;
|
|
|
|
}
|
|
|
|
|
2018-12-06 12:16:24 +01:00
|
|
|
result = MemoryManager.MapPages(
|
|
|
|
codeAddress,
|
|
|
|
pageList,
|
2018-11-28 23:18:09 +01:00
|
|
|
MemoryState.CodeStatic,
|
|
|
|
MemoryPermission.None);
|
|
|
|
|
2018-12-06 12:16:24 +01:00
|
|
|
if (result != KernelResult.Success)
|
2018-11-28 23:18:09 +01:00
|
|
|
{
|
2018-12-06 12:16:24 +01:00
|
|
|
return result;
|
2018-11-28 23:18:09 +01:00
|
|
|
}
|
|
|
|
|
2018-12-06 12:16:24 +01:00
|
|
|
result = Capabilities.InitializeForKernel(caps, MemoryManager);
|
2018-11-28 23:18:09 +01:00
|
|
|
|
2018-12-06 12:16:24 +01:00
|
|
|
if (result != KernelResult.Success)
|
2018-11-28 23:18:09 +01:00
|
|
|
{
|
2018-12-06 12:16:24 +01:00
|
|
|
return result;
|
2018-11-28 23:18:09 +01:00
|
|
|
}
|
|
|
|
|
|
|
|
Pid = System.GetKipId();
|
|
|
|
|
|
|
|
if (Pid == 0 || (ulong)Pid >= Horizon.InitialProcessId)
|
|
|
|
{
|
|
|
|
throw new InvalidOperationException($"Invalid KIP Id {Pid}.");
|
|
|
|
}
|
|
|
|
|
2018-12-06 12:16:24 +01:00
|
|
|
result = ParseProcessInfo(creationInfo);
|
2018-11-28 23:18:09 +01:00
|
|
|
|
2018-12-06 12:16:24 +01:00
|
|
|
return result;
|
2018-11-28 23:18:09 +01:00
|
|
|
}
|
|
|
|
|
|
|
|
public KernelResult Initialize(
|
2018-12-06 12:16:24 +01:00
|
|
|
ProcessCreationInfo creationInfo,
|
|
|
|
int[] caps,
|
|
|
|
KResourceLimit resourceLimit,
|
|
|
|
MemoryRegion memRegion)
|
2018-11-28 23:18:09 +01:00
|
|
|
{
|
2018-12-06 12:16:24 +01:00
|
|
|
ResourceLimit = resourceLimit;
|
|
|
|
_memRegion = memRegion;
|
2018-11-28 23:18:09 +01:00
|
|
|
|
2018-12-06 12:16:24 +01:00
|
|
|
ulong personalMmHeapSize = GetPersonalMmHeapSize((ulong)creationInfo.PersonalMmHeapPagesCount, memRegion);
|
2018-11-28 23:18:09 +01:00
|
|
|
|
2018-12-06 12:16:24 +01:00
|
|
|
ulong codePagesCount = (ulong)creationInfo.CodePagesCount;
|
2018-11-28 23:18:09 +01:00
|
|
|
|
2018-12-06 12:16:24 +01:00
|
|
|
ulong neededSizeForProcess = personalMmHeapSize + codePagesCount * KMemoryManager.PageSize;
|
2018-11-28 23:18:09 +01:00
|
|
|
|
2018-12-06 12:16:24 +01:00
|
|
|
if (neededSizeForProcess != 0 && resourceLimit != null)
|
2018-11-28 23:18:09 +01:00
|
|
|
{
|
2018-12-06 12:16:24 +01:00
|
|
|
if (!resourceLimit.Reserve(LimitableResource.Memory, neededSizeForProcess))
|
2018-11-28 23:18:09 +01:00
|
|
|
{
|
|
|
|
return KernelResult.ResLimitExceeded;
|
|
|
|
}
|
|
|
|
}
|
|
|
|
|
|
|
|
void CleanUpForError()
|
|
|
|
{
|
2018-12-06 12:16:24 +01:00
|
|
|
if (neededSizeForProcess != 0 && resourceLimit != null)
|
2018-11-28 23:18:09 +01:00
|
|
|
{
|
2018-12-06 12:16:24 +01:00
|
|
|
resourceLimit.Release(LimitableResource.Memory, neededSizeForProcess);
|
2018-11-28 23:18:09 +01:00
|
|
|
}
|
|
|
|
}
|
|
|
|
|
2018-12-06 12:16:24 +01:00
|
|
|
PersonalMmHeapPagesCount = (ulong)creationInfo.PersonalMmHeapPagesCount;
|
2018-11-28 23:18:09 +01:00
|
|
|
|
2018-12-06 12:16:24 +01:00
|
|
|
KMemoryBlockAllocator memoryBlockAllocator;
|
2018-11-28 23:18:09 +01:00
|
|
|
|
|
|
|
if (PersonalMmHeapPagesCount != 0)
|
|
|
|
{
|
2018-12-06 12:16:24 +01:00
|
|
|
memoryBlockAllocator = new KMemoryBlockAllocator(PersonalMmHeapPagesCount * KMemoryManager.PageSize);
|
2018-11-28 23:18:09 +01:00
|
|
|
}
|
|
|
|
else
|
|
|
|
{
|
2018-12-06 12:16:24 +01:00
|
|
|
memoryBlockAllocator = (MmuFlags & 0x40) != 0
|
2018-11-28 23:18:09 +01:00
|
|
|
? System.LargeMemoryBlockAllocator
|
|
|
|
: System.SmallMemoryBlockAllocator;
|
|
|
|
}
|
|
|
|
|
2018-12-06 12:16:24 +01:00
|
|
|
AddressSpaceType addrSpaceType = (AddressSpaceType)((creationInfo.MmuFlags >> 1) & 7);
|
2018-11-28 23:18:09 +01:00
|
|
|
|
2019-02-24 08:24:35 +01:00
|
|
|
InitializeMemoryManager(addrSpaceType, memRegion);
|
|
|
|
|
2018-12-06 12:16:24 +01:00
|
|
|
bool aslrEnabled = ((creationInfo.MmuFlags >> 5) & 1) != 0;
|
2018-11-28 23:18:09 +01:00
|
|
|
|
2018-12-06 12:16:24 +01:00
|
|
|
ulong codeAddress = creationInfo.CodeAddress;
|
2018-11-28 23:18:09 +01:00
|
|
|
|
2018-12-06 12:16:24 +01:00
|
|
|
ulong codeSize = codePagesCount * KMemoryManager.PageSize;
|
2018-11-28 23:18:09 +01:00
|
|
|
|
2018-12-06 12:16:24 +01:00
|
|
|
KernelResult result = MemoryManager.InitializeForProcess(
|
|
|
|
addrSpaceType,
|
|
|
|
aslrEnabled,
|
|
|
|
!aslrEnabled,
|
|
|
|
memRegion,
|
|
|
|
codeAddress,
|
|
|
|
codeSize,
|
|
|
|
memoryBlockAllocator);
|
2018-11-28 23:18:09 +01:00
|
|
|
|
2018-12-06 12:16:24 +01:00
|
|
|
if (result != KernelResult.Success)
|
2018-11-28 23:18:09 +01:00
|
|
|
{
|
|
|
|
CleanUpForError();
|
|
|
|
|
2018-12-06 12:16:24 +01:00
|
|
|
return result;
|
2018-11-28 23:18:09 +01:00
|
|
|
}
|
|
|
|
|
2018-12-06 12:16:24 +01:00
|
|
|
if (!ValidateCodeAddressAndSize(codeAddress, codeSize))
|
2018-11-28 23:18:09 +01:00
|
|
|
{
|
|
|
|
CleanUpForError();
|
|
|
|
|
|
|
|
return KernelResult.InvalidMemRange;
|
|
|
|
}
|
|
|
|
|
2018-12-06 12:16:24 +01:00
|
|
|
result = MemoryManager.MapNewProcessCode(
|
|
|
|
codeAddress,
|
|
|
|
codePagesCount,
|
2018-11-28 23:18:09 +01:00
|
|
|
MemoryState.CodeStatic,
|
|
|
|
MemoryPermission.None);
|
|
|
|
|
2018-12-06 12:16:24 +01:00
|
|
|
if (result != KernelResult.Success)
|
2018-11-28 23:18:09 +01:00
|
|
|
{
|
|
|
|
CleanUpForError();
|
|
|
|
|
2018-12-06 12:16:24 +01:00
|
|
|
return result;
|
2018-11-28 23:18:09 +01:00
|
|
|
}
|
|
|
|
|
2018-12-06 12:16:24 +01:00
|
|
|
result = Capabilities.InitializeForUser(caps, MemoryManager);
|
2018-11-28 23:18:09 +01:00
|
|
|
|
2018-12-06 12:16:24 +01:00
|
|
|
if (result != KernelResult.Success)
|
2018-11-28 23:18:09 +01:00
|
|
|
{
|
|
|
|
CleanUpForError();
|
|
|
|
|
2018-12-06 12:16:24 +01:00
|
|
|
return result;
|
2018-11-28 23:18:09 +01:00
|
|
|
}
|
|
|
|
|
|
|
|
Pid = System.GetProcessId();
|
|
|
|
|
|
|
|
if (Pid == -1 || (ulong)Pid < Horizon.InitialProcessId)
|
|
|
|
{
|
|
|
|
throw new InvalidOperationException($"Invalid Process Id {Pid}.");
|
|
|
|
}
|
|
|
|
|
2018-12-06 12:16:24 +01:00
|
|
|
result = ParseProcessInfo(creationInfo);
|
2018-11-28 23:18:09 +01:00
|
|
|
|
2018-12-06 12:16:24 +01:00
|
|
|
if (result != KernelResult.Success)
|
2018-11-28 23:18:09 +01:00
|
|
|
{
|
|
|
|
CleanUpForError();
|
|
|
|
}
|
|
|
|
|
2018-12-06 12:16:24 +01:00
|
|
|
return result;
|
2018-11-28 23:18:09 +01:00
|
|
|
}
|
|
|
|
|
2018-12-06 12:16:24 +01:00
|
|
|
private bool ValidateCodeAddressAndSize(ulong address, ulong size)
|
2018-11-28 23:18:09 +01:00
|
|
|
{
|
2018-12-06 12:16:24 +01:00
|
|
|
ulong codeRegionStart;
|
|
|
|
ulong codeRegionSize;
|
2018-11-28 23:18:09 +01:00
|
|
|
|
|
|
|
switch (MemoryManager.AddrSpaceWidth)
|
|
|
|
{
|
|
|
|
case 32:
|
2018-12-06 12:16:24 +01:00
|
|
|
codeRegionStart = 0x200000;
|
|
|
|
codeRegionSize = 0x3fe00000;
|
2018-11-28 23:18:09 +01:00
|
|
|
break;
|
|
|
|
|
|
|
|
case 36:
|
2018-12-06 12:16:24 +01:00
|
|
|
codeRegionStart = 0x8000000;
|
|
|
|
codeRegionSize = 0x78000000;
|
2018-11-28 23:18:09 +01:00
|
|
|
break;
|
|
|
|
|
|
|
|
case 39:
|
2018-12-06 12:16:24 +01:00
|
|
|
codeRegionStart = 0x8000000;
|
|
|
|
codeRegionSize = 0x7ff8000000;
|
2018-11-28 23:18:09 +01:00
|
|
|
break;
|
|
|
|
|
|
|
|
default: throw new InvalidOperationException("Invalid address space width on memory manager.");
|
|
|
|
}
|
|
|
|
|
2018-12-06 12:16:24 +01:00
|
|
|
ulong endAddr = address + size;
|
2018-11-28 23:18:09 +01:00
|
|
|
|
2018-12-06 12:16:24 +01:00
|
|
|
ulong codeRegionEnd = codeRegionStart + codeRegionSize;
|
2018-11-28 23:18:09 +01:00
|
|
|
|
2018-12-06 12:16:24 +01:00
|
|
|
if (endAddr <= address ||
|
|
|
|
endAddr - 1 > codeRegionEnd - 1)
|
2018-11-28 23:18:09 +01:00
|
|
|
{
|
|
|
|
return false;
|
|
|
|
}
|
|
|
|
|
2018-12-06 12:16:24 +01:00
|
|
|
if (MemoryManager.InsideHeapRegion (address, size) ||
|
|
|
|
MemoryManager.InsideAliasRegion(address, size))
|
2018-11-28 23:18:09 +01:00
|
|
|
{
|
|
|
|
return false;
|
|
|
|
}
|
|
|
|
|
|
|
|
return true;
|
|
|
|
}
|
|
|
|
|
2018-12-06 12:16:24 +01:00
|
|
|
private KernelResult ParseProcessInfo(ProcessCreationInfo creationInfo)
|
2018-11-28 23:18:09 +01:00
|
|
|
{
|
2019-07-02 04:39:22 +02:00
|
|
|
// Ensure that the current kernel version is equal or above to the minimum required.
|
2018-12-06 12:16:24 +01:00
|
|
|
uint requiredKernelVersionMajor = (uint)Capabilities.KernelReleaseVersion >> 19;
|
|
|
|
uint requiredKernelVersionMinor = ((uint)Capabilities.KernelReleaseVersion >> 15) & 0xf;
|
2018-11-28 23:18:09 +01:00
|
|
|
|
|
|
|
if (System.EnableVersionChecks)
|
|
|
|
{
|
2018-12-06 12:16:24 +01:00
|
|
|
if (requiredKernelVersionMajor > KernelVersionMajor)
|
2018-11-28 23:18:09 +01:00
|
|
|
{
|
|
|
|
return KernelResult.InvalidCombination;
|
|
|
|
}
|
|
|
|
|
2018-12-06 12:16:24 +01:00
|
|
|
if (requiredKernelVersionMajor != KernelVersionMajor && requiredKernelVersionMajor < 3)
|
2018-11-28 23:18:09 +01:00
|
|
|
{
|
|
|
|
return KernelResult.InvalidCombination;
|
|
|
|
}
|
|
|
|
|
2018-12-06 12:16:24 +01:00
|
|
|
if (requiredKernelVersionMinor > KernelVersionMinor)
|
2018-11-28 23:18:09 +01:00
|
|
|
{
|
|
|
|
return KernelResult.InvalidCombination;
|
|
|
|
}
|
|
|
|
}
|
|
|
|
|
2018-12-06 12:16:24 +01:00
|
|
|
KernelResult result = AllocateThreadLocalStorage(out ulong userExceptionContextAddress);
|
2018-11-28 23:18:09 +01:00
|
|
|
|
2018-12-06 12:16:24 +01:00
|
|
|
if (result != KernelResult.Success)
|
2018-11-28 23:18:09 +01:00
|
|
|
{
|
2018-12-06 12:16:24 +01:00
|
|
|
return result;
|
2018-11-28 23:18:09 +01:00
|
|
|
}
|
|
|
|
|
2018-12-06 12:16:24 +01:00
|
|
|
UserExceptionContextAddress = userExceptionContextAddress;
|
2018-11-28 23:18:09 +01:00
|
|
|
|
2018-12-06 12:16:24 +01:00
|
|
|
MemoryHelper.FillWithZeros(CpuMemory, (long)userExceptionContextAddress, KTlsPageInfo.TlsEntrySize);
|
2018-11-28 23:18:09 +01:00
|
|
|
|
2018-12-06 12:16:24 +01:00
|
|
|
Name = creationInfo.Name;
|
2018-11-28 23:18:09 +01:00
|
|
|
|
2018-12-06 12:16:24 +01:00
|
|
|
_state = ProcessState.Created;
|
2018-11-28 23:18:09 +01:00
|
|
|
|
2018-12-06 12:16:24 +01:00
|
|
|
_creationTimestamp = PerformanceCounter.ElapsedMilliseconds;
|
2018-11-28 23:18:09 +01:00
|
|
|
|
2018-12-06 12:16:24 +01:00
|
|
|
MmuFlags = creationInfo.MmuFlags;
|
|
|
|
_category = creationInfo.Category;
|
|
|
|
TitleId = creationInfo.TitleId;
|
|
|
|
_entrypoint = creationInfo.CodeAddress;
|
|
|
|
_imageSize = (ulong)creationInfo.CodePagesCount * KMemoryManager.PageSize;
|
2018-11-28 23:18:09 +01:00
|
|
|
|
2018-12-06 12:16:24 +01:00
|
|
|
_useSystemMemBlocks = ((MmuFlags >> 6) & 1) != 0;
|
2018-11-28 23:18:09 +01:00
|
|
|
|
|
|
|
switch ((AddressSpaceType)((MmuFlags >> 1) & 7))
|
|
|
|
{
|
|
|
|
case AddressSpaceType.Addr32Bits:
|
|
|
|
case AddressSpaceType.Addr36Bits:
|
|
|
|
case AddressSpaceType.Addr39Bits:
|
2018-12-06 12:16:24 +01:00
|
|
|
_memoryUsageCapacity = MemoryManager.HeapRegionEnd -
|
2019-02-24 08:24:35 +01:00
|
|
|
MemoryManager.HeapRegionStart;
|
2018-11-28 23:18:09 +01:00
|
|
|
break;
|
|
|
|
|
|
|
|
case AddressSpaceType.Addr32BitsNoMap:
|
2018-12-06 12:16:24 +01:00
|
|
|
_memoryUsageCapacity = MemoryManager.HeapRegionEnd -
|
|
|
|
MemoryManager.HeapRegionStart +
|
|
|
|
MemoryManager.AliasRegionEnd -
|
|
|
|
MemoryManager.AliasRegionStart;
|
2018-11-28 23:18:09 +01:00
|
|
|
break;
|
|
|
|
|
|
|
|
default: throw new InvalidOperationException($"Invalid MMU flags value 0x{MmuFlags:x2}.");
|
|
|
|
}
|
|
|
|
|
|
|
|
GenerateRandomEntropy();
|
|
|
|
|
|
|
|
return KernelResult.Success;
|
|
|
|
}
|
|
|
|
|
2018-12-06 12:16:24 +01:00
|
|
|
public KernelResult AllocateThreadLocalStorage(out ulong address)
|
2018-11-28 23:18:09 +01:00
|
|
|
{
|
|
|
|
System.CriticalSection.Enter();
|
|
|
|
|
2018-12-06 12:16:24 +01:00
|
|
|
KernelResult result;
|
2018-11-28 23:18:09 +01:00
|
|
|
|
2018-12-06 12:16:24 +01:00
|
|
|
if (_freeTlsPages.Count > 0)
|
2018-11-28 23:18:09 +01:00
|
|
|
{
|
2019-07-02 04:39:22 +02:00
|
|
|
// If we have free TLS pages available, just use the first one.
|
2018-12-06 12:16:24 +01:00
|
|
|
KTlsPageInfo pageInfo = _freeTlsPages.Values.First();
|
2018-11-28 23:18:09 +01:00
|
|
|
|
2018-12-06 12:16:24 +01:00
|
|
|
if (!pageInfo.TryGetFreePage(out address))
|
2018-11-28 23:18:09 +01:00
|
|
|
{
|
|
|
|
throw new InvalidOperationException("Unexpected failure getting free TLS page!");
|
|
|
|
}
|
|
|
|
|
2018-12-06 12:16:24 +01:00
|
|
|
if (pageInfo.IsFull())
|
2018-11-28 23:18:09 +01:00
|
|
|
{
|
2018-12-06 12:16:24 +01:00
|
|
|
_freeTlsPages.Remove(pageInfo.PageAddr);
|
2018-11-28 23:18:09 +01:00
|
|
|
|
2018-12-06 12:16:24 +01:00
|
|
|
_fullTlsPages.Add(pageInfo.PageAddr, pageInfo);
|
2018-11-28 23:18:09 +01:00
|
|
|
}
|
|
|
|
|
2018-12-06 12:16:24 +01:00
|
|
|
result = KernelResult.Success;
|
2018-11-28 23:18:09 +01:00
|
|
|
}
|
|
|
|
else
|
|
|
|
{
|
2019-07-02 04:39:22 +02:00
|
|
|
// Otherwise, we need to create a new one.
|
2018-12-06 12:16:24 +01:00
|
|
|
result = AllocateTlsPage(out KTlsPageInfo pageInfo);
|
2018-11-28 23:18:09 +01:00
|
|
|
|
2018-12-06 12:16:24 +01:00
|
|
|
if (result == KernelResult.Success)
|
2018-11-28 23:18:09 +01:00
|
|
|
{
|
2018-12-06 12:16:24 +01:00
|
|
|
if (!pageInfo.TryGetFreePage(out address))
|
2018-11-28 23:18:09 +01:00
|
|
|
{
|
|
|
|
throw new InvalidOperationException("Unexpected failure getting free TLS page!");
|
|
|
|
}
|
|
|
|
|
2018-12-06 12:16:24 +01:00
|
|
|
_freeTlsPages.Add(pageInfo.PageAddr, pageInfo);
|
2018-11-28 23:18:09 +01:00
|
|
|
}
|
|
|
|
else
|
|
|
|
{
|
2018-12-06 12:16:24 +01:00
|
|
|
address = 0;
|
2018-11-28 23:18:09 +01:00
|
|
|
}
|
|
|
|
}
|
|
|
|
|
|
|
|
System.CriticalSection.Leave();
|
|
|
|
|
2018-12-06 12:16:24 +01:00
|
|
|
return result;
|
2018-11-28 23:18:09 +01:00
|
|
|
}
|
|
|
|
|
2018-12-06 12:16:24 +01:00
|
|
|
private KernelResult AllocateTlsPage(out KTlsPageInfo pageInfo)
|
2018-11-28 23:18:09 +01:00
|
|
|
{
|
2018-12-06 12:16:24 +01:00
|
|
|
pageInfo = default(KTlsPageInfo);
|
2018-11-28 23:18:09 +01:00
|
|
|
|
2018-12-06 12:16:24 +01:00
|
|
|
if (!System.UserSlabHeapPages.TryGetItem(out ulong tlsPagePa))
|
2018-11-28 23:18:09 +01:00
|
|
|
{
|
|
|
|
return KernelResult.OutOfMemory;
|
|
|
|
}
|
|
|
|
|
2018-12-06 12:16:24 +01:00
|
|
|
ulong regionStart = MemoryManager.TlsIoRegionStart;
|
|
|
|
ulong regionSize = MemoryManager.TlsIoRegionEnd - regionStart;
|
2018-11-28 23:18:09 +01:00
|
|
|
|
2018-12-06 12:16:24 +01:00
|
|
|
ulong regionPagesCount = regionSize / KMemoryManager.PageSize;
|
2018-11-28 23:18:09 +01:00
|
|
|
|
2018-12-06 12:16:24 +01:00
|
|
|
KernelResult result = MemoryManager.AllocateOrMapPa(
|
2018-11-28 23:18:09 +01:00
|
|
|
1,
|
|
|
|
KMemoryManager.PageSize,
|
2018-12-06 12:16:24 +01:00
|
|
|
tlsPagePa,
|
2018-11-28 23:18:09 +01:00
|
|
|
true,
|
2018-12-06 12:16:24 +01:00
|
|
|
regionStart,
|
|
|
|
regionPagesCount,
|
2018-11-28 23:18:09 +01:00
|
|
|
MemoryState.ThreadLocal,
|
|
|
|
MemoryPermission.ReadAndWrite,
|
2018-12-06 12:16:24 +01:00
|
|
|
out ulong tlsPageVa);
|
2018-11-28 23:18:09 +01:00
|
|
|
|
2018-12-06 12:16:24 +01:00
|
|
|
if (result != KernelResult.Success)
|
2018-11-28 23:18:09 +01:00
|
|
|
{
|
2018-12-06 12:16:24 +01:00
|
|
|
System.UserSlabHeapPages.Free(tlsPagePa);
|
2018-11-28 23:18:09 +01:00
|
|
|
}
|
|
|
|
else
|
|
|
|
{
|
2018-12-06 12:16:24 +01:00
|
|
|
pageInfo = new KTlsPageInfo(tlsPageVa);
|
2018-11-28 23:18:09 +01:00
|
|
|
|
2018-12-06 12:16:24 +01:00
|
|
|
MemoryHelper.FillWithZeros(CpuMemory, (long)tlsPageVa, KMemoryManager.PageSize);
|
2018-11-28 23:18:09 +01:00
|
|
|
}
|
|
|
|
|
2018-12-06 12:16:24 +01:00
|
|
|
return result;
|
2018-11-28 23:18:09 +01:00
|
|
|
}
|
|
|
|
|
2018-12-06 12:16:24 +01:00
|
|
|
public KernelResult FreeThreadLocalStorage(ulong tlsSlotAddr)
|
2018-11-28 23:18:09 +01:00
|
|
|
{
|
2018-12-06 12:16:24 +01:00
|
|
|
ulong tlsPageAddr = BitUtils.AlignDown(tlsSlotAddr, KMemoryManager.PageSize);
|
2018-11-28 23:18:09 +01:00
|
|
|
|
|
|
|
System.CriticalSection.Enter();
|
|
|
|
|
2018-12-06 12:16:24 +01:00
|
|
|
KernelResult result = KernelResult.Success;
|
2018-11-28 23:18:09 +01:00
|
|
|
|
2018-12-06 12:16:24 +01:00
|
|
|
KTlsPageInfo pageInfo = null;
|
2018-11-28 23:18:09 +01:00
|
|
|
|
2018-12-06 12:16:24 +01:00
|
|
|
if (_fullTlsPages.TryGetValue(tlsPageAddr, out pageInfo))
|
2018-11-28 23:18:09 +01:00
|
|
|
{
|
2019-07-02 04:39:22 +02:00
|
|
|
// TLS page was full, free slot and move to free pages tree.
|
2018-12-06 12:16:24 +01:00
|
|
|
_fullTlsPages.Remove(tlsPageAddr);
|
2018-11-28 23:18:09 +01:00
|
|
|
|
2018-12-06 12:16:24 +01:00
|
|
|
_freeTlsPages.Add(tlsPageAddr, pageInfo);
|
2018-11-28 23:18:09 +01:00
|
|
|
}
|
2018-12-06 12:16:24 +01:00
|
|
|
else if (!_freeTlsPages.TryGetValue(tlsPageAddr, out pageInfo))
|
2018-11-28 23:18:09 +01:00
|
|
|
{
|
2018-12-06 12:16:24 +01:00
|
|
|
result = KernelResult.InvalidAddress;
|
2018-11-28 23:18:09 +01:00
|
|
|
}
|
|
|
|
|
2018-12-06 12:16:24 +01:00
|
|
|
if (pageInfo != null)
|
2018-11-28 23:18:09 +01:00
|
|
|
{
|
2018-12-06 12:16:24 +01:00
|
|
|
pageInfo.FreeTlsSlot(tlsSlotAddr);
|
2018-11-28 23:18:09 +01:00
|
|
|
|
2018-12-06 12:16:24 +01:00
|
|
|
if (pageInfo.IsEmpty())
|
2018-11-28 23:18:09 +01:00
|
|
|
{
|
2019-07-02 04:39:22 +02:00
|
|
|
// TLS page is now empty, we should ensure it is removed
|
|
|
|
// from all trees, and free the memory it was using.
|
2018-12-06 12:16:24 +01:00
|
|
|
_freeTlsPages.Remove(tlsPageAddr);
|
2018-11-28 23:18:09 +01:00
|
|
|
|
|
|
|
System.CriticalSection.Leave();
|
|
|
|
|
2018-12-06 12:16:24 +01:00
|
|
|
FreeTlsPage(pageInfo);
|
2018-11-28 23:18:09 +01:00
|
|
|
|
|
|
|
return KernelResult.Success;
|
|
|
|
}
|
|
|
|
}
|
|
|
|
|
|
|
|
System.CriticalSection.Leave();
|
|
|
|
|
2018-12-06 12:16:24 +01:00
|
|
|
return result;
|
2018-11-28 23:18:09 +01:00
|
|
|
}
|
|
|
|
|
2018-12-06 12:16:24 +01:00
|
|
|
private KernelResult FreeTlsPage(KTlsPageInfo pageInfo)
|
2018-11-28 23:18:09 +01:00
|
|
|
{
|
2019-01-18 23:26:39 +01:00
|
|
|
if (!MemoryManager.ConvertVaToPa(pageInfo.PageAddr, out ulong tlsPagePa))
|
2018-11-28 23:18:09 +01:00
|
|
|
{
|
|
|
|
throw new InvalidOperationException("Unexpected failure translating virtual address to physical.");
|
|
|
|
}
|
|
|
|
|
2019-01-18 23:26:39 +01:00
|
|
|
KernelResult result = MemoryManager.UnmapForKernel(pageInfo.PageAddr, 1, MemoryState.ThreadLocal);
|
2018-11-28 23:18:09 +01:00
|
|
|
|
2018-12-06 12:16:24 +01:00
|
|
|
if (result == KernelResult.Success)
|
2018-11-28 23:18:09 +01:00
|
|
|
{
|
2018-12-06 12:16:24 +01:00
|
|
|
System.UserSlabHeapPages.Free(tlsPagePa);
|
2018-11-28 23:18:09 +01:00
|
|
|
}
|
|
|
|
|
2018-12-06 12:16:24 +01:00
|
|
|
return result;
|
2018-11-28 23:18:09 +01:00
|
|
|
}
|
|
|
|
|
|
|
|
private void GenerateRandomEntropy()
|
|
|
|
{
|
2019-07-02 04:39:22 +02:00
|
|
|
// TODO.
|
2018-11-28 23:18:09 +01:00
|
|
|
}
|
|
|
|
|
2018-12-06 12:16:24 +01:00
|
|
|
public KernelResult Start(int mainThreadPriority, ulong stackSize)
|
2018-11-28 23:18:09 +01:00
|
|
|
{
|
2018-12-06 12:16:24 +01:00
|
|
|
lock (_processLock)
|
2018-11-28 23:18:09 +01:00
|
|
|
{
|
2018-12-06 12:16:24 +01:00
|
|
|
if (_state > ProcessState.CreatedAttached)
|
2018-11-28 23:18:09 +01:00
|
|
|
{
|
|
|
|
return KernelResult.InvalidState;
|
|
|
|
}
|
|
|
|
|
|
|
|
if (ResourceLimit != null && !ResourceLimit.Reserve(LimitableResource.Thread, 1))
|
|
|
|
{
|
|
|
|
return KernelResult.ResLimitExceeded;
|
|
|
|
}
|
|
|
|
|
2018-12-06 12:16:24 +01:00
|
|
|
KResourceLimit threadResourceLimit = ResourceLimit;
|
|
|
|
KResourceLimit memoryResourceLimit = null;
|
2018-11-28 23:18:09 +01:00
|
|
|
|
2018-12-06 12:16:24 +01:00
|
|
|
if (_mainThreadStackSize != 0)
|
2018-11-28 23:18:09 +01:00
|
|
|
{
|
|
|
|
throw new InvalidOperationException("Trying to start a process with a invalid state!");
|
|
|
|
}
|
|
|
|
|
2018-12-06 12:16:24 +01:00
|
|
|
ulong stackSizeRounded = BitUtils.AlignUp(stackSize, KMemoryManager.PageSize);
|
2018-11-28 23:18:09 +01:00
|
|
|
|
2018-12-06 12:16:24 +01:00
|
|
|
ulong neededSize = stackSizeRounded + _imageSize;
|
2018-11-28 23:18:09 +01:00
|
|
|
|
2019-07-02 04:39:22 +02:00
|
|
|
// Check if the needed size for the code and the stack will fit on the
|
|
|
|
// memory usage capacity of this Process. Also check for possible overflow
|
|
|
|
// on the above addition.
|
2018-12-06 12:16:24 +01:00
|
|
|
if (neededSize > _memoryUsageCapacity ||
|
|
|
|
neededSize < stackSizeRounded)
|
2018-11-28 23:18:09 +01:00
|
|
|
{
|
2018-12-06 12:16:24 +01:00
|
|
|
threadResourceLimit?.Release(LimitableResource.Thread, 1);
|
2018-11-28 23:18:09 +01:00
|
|
|
|
|
|
|
return KernelResult.OutOfMemory;
|
|
|
|
}
|
|
|
|
|
2018-12-06 12:16:24 +01:00
|
|
|
if (stackSizeRounded != 0 && ResourceLimit != null)
|
2018-11-28 23:18:09 +01:00
|
|
|
{
|
2018-12-06 12:16:24 +01:00
|
|
|
memoryResourceLimit = ResourceLimit;
|
2018-11-28 23:18:09 +01:00
|
|
|
|
2018-12-06 12:16:24 +01:00
|
|
|
if (!memoryResourceLimit.Reserve(LimitableResource.Memory, stackSizeRounded))
|
2018-11-28 23:18:09 +01:00
|
|
|
{
|
2018-12-06 12:16:24 +01:00
|
|
|
threadResourceLimit?.Release(LimitableResource.Thread, 1);
|
2018-11-28 23:18:09 +01:00
|
|
|
|
|
|
|
return KernelResult.ResLimitExceeded;
|
|
|
|
}
|
|
|
|
}
|
|
|
|
|
2018-12-06 12:16:24 +01:00
|
|
|
KernelResult result;
|
2018-11-28 23:18:09 +01:00
|
|
|
|
2018-12-06 12:16:24 +01:00
|
|
|
KThread mainThread = null;
|
2018-11-28 23:18:09 +01:00
|
|
|
|
2018-12-06 12:16:24 +01:00
|
|
|
ulong stackTop = 0;
|
2018-11-28 23:18:09 +01:00
|
|
|
|
|
|
|
void CleanUpForError()
|
|
|
|
{
|
|
|
|
HandleTable.Destroy();
|
|
|
|
|
2019-01-18 23:26:39 +01:00
|
|
|
mainThread?.DecrementReferenceCount();
|
|
|
|
|
2018-12-06 12:16:24 +01:00
|
|
|
if (_mainThreadStackSize != 0)
|
2018-11-28 23:18:09 +01:00
|
|
|
{
|
2018-12-06 12:16:24 +01:00
|
|
|
ulong stackBottom = stackTop - _mainThreadStackSize;
|
2018-11-28 23:18:09 +01:00
|
|
|
|
2018-12-06 12:16:24 +01:00
|
|
|
ulong stackPagesCount = _mainThreadStackSize / KMemoryManager.PageSize;
|
2018-11-28 23:18:09 +01:00
|
|
|
|
2018-12-06 12:16:24 +01:00
|
|
|
MemoryManager.UnmapForKernel(stackBottom, stackPagesCount, MemoryState.Stack);
|
2019-01-18 23:26:39 +01:00
|
|
|
|
|
|
|
_mainThreadStackSize = 0;
|
2018-11-28 23:18:09 +01:00
|
|
|
}
|
|
|
|
|
2018-12-06 12:16:24 +01:00
|
|
|
memoryResourceLimit?.Release(LimitableResource.Memory, stackSizeRounded);
|
|
|
|
threadResourceLimit?.Release(LimitableResource.Thread, 1);
|
2018-11-28 23:18:09 +01:00
|
|
|
}
|
|
|
|
|
2018-12-06 12:16:24 +01:00
|
|
|
if (stackSizeRounded != 0)
|
2018-11-28 23:18:09 +01:00
|
|
|
{
|
2018-12-06 12:16:24 +01:00
|
|
|
ulong stackPagesCount = stackSizeRounded / KMemoryManager.PageSize;
|
2018-11-28 23:18:09 +01:00
|
|
|
|
2018-12-06 12:16:24 +01:00
|
|
|
ulong regionStart = MemoryManager.StackRegionStart;
|
|
|
|
ulong regionSize = MemoryManager.StackRegionEnd - regionStart;
|
2018-11-28 23:18:09 +01:00
|
|
|
|
2018-12-06 12:16:24 +01:00
|
|
|
ulong regionPagesCount = regionSize / KMemoryManager.PageSize;
|
2018-11-28 23:18:09 +01:00
|
|
|
|
2018-12-06 12:16:24 +01:00
|
|
|
result = MemoryManager.AllocateOrMapPa(
|
|
|
|
stackPagesCount,
|
2018-11-28 23:18:09 +01:00
|
|
|
KMemoryManager.PageSize,
|
|
|
|
0,
|
|
|
|
false,
|
2018-12-06 12:16:24 +01:00
|
|
|
regionStart,
|
|
|
|
regionPagesCount,
|
2018-11-28 23:18:09 +01:00
|
|
|
MemoryState.Stack,
|
|
|
|
MemoryPermission.ReadAndWrite,
|
2018-12-06 12:16:24 +01:00
|
|
|
out ulong stackBottom);
|
2018-11-28 23:18:09 +01:00
|
|
|
|
2018-12-06 12:16:24 +01:00
|
|
|
if (result != KernelResult.Success)
|
2018-11-28 23:18:09 +01:00
|
|
|
{
|
|
|
|
CleanUpForError();
|
|
|
|
|
2018-12-06 12:16:24 +01:00
|
|
|
return result;
|
2018-11-28 23:18:09 +01:00
|
|
|
}
|
|
|
|
|
2018-12-06 12:16:24 +01:00
|
|
|
_mainThreadStackSize += stackSizeRounded;
|
2018-11-28 23:18:09 +01:00
|
|
|
|
2018-12-06 12:16:24 +01:00
|
|
|
stackTop = stackBottom + stackSizeRounded;
|
2018-11-28 23:18:09 +01:00
|
|
|
}
|
|
|
|
|
2018-12-06 12:16:24 +01:00
|
|
|
ulong heapCapacity = _memoryUsageCapacity - _mainThreadStackSize - _imageSize;
|
2018-11-28 23:18:09 +01:00
|
|
|
|
2018-12-06 12:16:24 +01:00
|
|
|
result = MemoryManager.SetHeapCapacity(heapCapacity);
|
2018-11-28 23:18:09 +01:00
|
|
|
|
2018-12-06 12:16:24 +01:00
|
|
|
if (result != KernelResult.Success)
|
2018-11-28 23:18:09 +01:00
|
|
|
{
|
|
|
|
CleanUpForError();
|
|
|
|
|
2018-12-06 12:16:24 +01:00
|
|
|
return result;
|
2018-11-28 23:18:09 +01:00
|
|
|
}
|
|
|
|
|
|
|
|
HandleTable = new KHandleTable(System);
|
|
|
|
|
2018-12-06 12:16:24 +01:00
|
|
|
result = HandleTable.Initialize(Capabilities.HandleTableSize);
|
2018-11-28 23:18:09 +01:00
|
|
|
|
2018-12-06 12:16:24 +01:00
|
|
|
if (result != KernelResult.Success)
|
2018-11-28 23:18:09 +01:00
|
|
|
{
|
|
|
|
CleanUpForError();
|
|
|
|
|
2018-12-06 12:16:24 +01:00
|
|
|
return result;
|
2018-11-28 23:18:09 +01:00
|
|
|
}
|
|
|
|
|
2018-12-06 12:16:24 +01:00
|
|
|
mainThread = new KThread(System);
|
2018-11-28 23:18:09 +01:00
|
|
|
|
2018-12-06 12:16:24 +01:00
|
|
|
result = mainThread.Initialize(
|
|
|
|
_entrypoint,
|
2018-11-28 23:18:09 +01:00
|
|
|
0,
|
2018-12-06 12:16:24 +01:00
|
|
|
stackTop,
|
|
|
|
mainThreadPriority,
|
2018-11-28 23:18:09 +01:00
|
|
|
DefaultCpuCore,
|
|
|
|
this);
|
|
|
|
|
2018-12-06 12:16:24 +01:00
|
|
|
if (result != KernelResult.Success)
|
2018-11-28 23:18:09 +01:00
|
|
|
{
|
|
|
|
CleanUpForError();
|
|
|
|
|
2018-12-06 12:16:24 +01:00
|
|
|
return result;
|
2018-11-28 23:18:09 +01:00
|
|
|
}
|
|
|
|
|
2018-12-06 12:16:24 +01:00
|
|
|
result = HandleTable.GenerateHandle(mainThread, out int mainThreadHandle);
|
2018-11-28 23:18:09 +01:00
|
|
|
|
2018-12-06 12:16:24 +01:00
|
|
|
if (result != KernelResult.Success)
|
2018-11-28 23:18:09 +01:00
|
|
|
{
|
|
|
|
CleanUpForError();
|
|
|
|
|
2018-12-06 12:16:24 +01:00
|
|
|
return result;
|
2018-11-28 23:18:09 +01:00
|
|
|
}
|
|
|
|
|
2018-12-06 12:16:24 +01:00
|
|
|
mainThread.SetEntryArguments(0, mainThreadHandle);
|
2018-11-28 23:18:09 +01:00
|
|
|
|
2018-12-06 12:16:24 +01:00
|
|
|
ProcessState oldState = _state;
|
|
|
|
ProcessState newState = _state != ProcessState.Created
|
2018-11-28 23:18:09 +01:00
|
|
|
? ProcessState.Attached
|
|
|
|
: ProcessState.Started;
|
|
|
|
|
2018-12-06 12:16:24 +01:00
|
|
|
SetState(newState);
|
2018-11-28 23:18:09 +01:00
|
|
|
|
2019-07-02 04:39:22 +02:00
|
|
|
// TODO: We can't call KThread.Start from a non-guest thread.
|
|
|
|
// We will need to make some changes to allow the creation of
|
|
|
|
// dummy threads that will be used to initialize the current
|
|
|
|
// thread on KCoreContext so that GetCurrentThread doesn't fail.
|
2018-11-28 23:18:09 +01:00
|
|
|
/* Result = MainThread.Start();
|
|
|
|
|
|
|
|
if (Result != KernelResult.Success)
|
|
|
|
{
|
|
|
|
SetState(OldState);
|
|
|
|
|
|
|
|
CleanUpForError();
|
|
|
|
} */
|
|
|
|
|
2018-12-06 12:16:24 +01:00
|
|
|
mainThread.Reschedule(ThreadSchedState.Running);
|
2018-11-28 23:18:09 +01:00
|
|
|
|
2019-01-18 23:26:39 +01:00
|
|
|
if (result == KernelResult.Success)
|
|
|
|
{
|
|
|
|
mainThread.IncrementReferenceCount();
|
|
|
|
}
|
|
|
|
|
|
|
|
mainThread.DecrementReferenceCount();
|
|
|
|
|
2018-12-06 12:16:24 +01:00
|
|
|
return result;
|
2018-11-28 23:18:09 +01:00
|
|
|
}
|
|
|
|
}
|
|
|
|
|
2018-12-06 12:16:24 +01:00
|
|
|
private void SetState(ProcessState newState)
|
2018-11-28 23:18:09 +01:00
|
|
|
{
|
2018-12-06 12:16:24 +01:00
|
|
|
if (_state != newState)
|
2018-11-28 23:18:09 +01:00
|
|
|
{
|
2018-12-06 12:16:24 +01:00
|
|
|
_state = newState;
|
|
|
|
_signaled = true;
|
2018-11-28 23:18:09 +01:00
|
|
|
|
|
|
|
Signal();
|
|
|
|
}
|
|
|
|
}
|
|
|
|
|
|
|
|
public KernelResult InitializeThread(
|
2018-12-06 12:16:24 +01:00
|
|
|
KThread thread,
|
|
|
|
ulong entrypoint,
|
|
|
|
ulong argsPtr,
|
|
|
|
ulong stackTop,
|
|
|
|
int priority,
|
|
|
|
int cpuCore)
|
2018-11-28 23:18:09 +01:00
|
|
|
{
|
2018-12-06 12:16:24 +01:00
|
|
|
lock (_processLock)
|
2018-11-28 23:18:09 +01:00
|
|
|
{
|
2018-12-06 12:16:24 +01:00
|
|
|
return thread.Initialize(entrypoint, argsPtr, stackTop, priority, cpuCore, this);
|
2018-11-28 23:18:09 +01:00
|
|
|
}
|
|
|
|
}
|
|
|
|
|
2019-10-31 19:09:03 +01:00
|
|
|
public void SubscribeThreadEventHandlers(ARMeilleure.State.ExecutionContext context)
|
2018-11-28 23:18:09 +01:00
|
|
|
{
|
Add a new JIT compiler for CPU code (#693)
* Start of the ARMeilleure project
* Refactoring around the old IRAdapter, now renamed to PreAllocator
* Optimize the LowestBitSet method
* Add CLZ support and fix CLS implementation
* Add missing Equals and GetHashCode overrides on some structs, misc small tweaks
* Implement the ByteSwap IR instruction, and some refactoring on the assembler
* Implement the DivideUI IR instruction and fix 64-bits IDIV
* Correct constant operand type on CSINC
* Move division instructions implementation to InstEmitDiv
* Fix destination type for the ConditionalSelect IR instruction
* Implement UMULH and SMULH, with new IR instructions
* Fix some issues with shift instructions
* Fix constant types for BFM instructions
* Fix up new tests using the new V128 struct
* Update tests
* Move DIV tests to a separate file
* Add support for calls, and some instructions that depends on them
* Start adding support for SIMD & FP types, along with some of the related ARM instructions
* Fix some typos and the divide instruction with FP operands
* Fix wrong method call on Clz_V
* Implement ARM FP & SIMD move instructions, Saddlv_V, and misc. fixes
* Implement SIMD logical instructions and more misc. fixes
* Fix PSRAD x86 instruction encoding, TRN, UABD and UABDL implementations
* Implement float conversion instruction, merge in LDj3SNuD fixes, and some other misc. fixes
* Implement SIMD shift instruction and fix Dup_V
* Add SCVTF and UCVTF (vector, fixed-point) variants to the opcode table
* Fix check with tolerance on tester
* Implement FP & SIMD comparison instructions, and some fixes
* Update FCVT (Scalar) encoding on the table to support the Half-float variants
* Support passing V128 structs, some cleanup on the register allocator, merge LDj3SNuD fixes
* Use old memory access methods, made a start on SIMD memory insts support, some fixes
* Fix float constant passed to functions, save and restore non-volatile XMM registers, other fixes
* Fix arguments count with struct return values, other fixes
* More instructions
* Misc. fixes and integrate LDj3SNuD fixes
* Update tests
* Add a faster linear scan allocator, unwinding support on windows, and other changes
* Update Ryujinx.HLE
* Update Ryujinx.Graphics
* Fix V128 return pointer passing, RCX is clobbered
* Update Ryujinx.Tests
* Update ITimeZoneService
* Stop using GetFunctionPointer as that can't be called from native code, misc. fixes and tweaks
* Use generic GetFunctionPointerForDelegate method and other tweaks
* Some refactoring on the code generator, assert on invalid operations and use a separate enum for intrinsics
* Remove some unused code on the assembler
* Fix REX.W prefix regression on float conversion instructions, add some sort of profiler
* Add hardware capability detection
* Fix regression on Sha1h and revert Fcm** changes
* Add SSE2-only paths on vector extract and insert, some refactoring on the pre-allocator
* Fix silly mistake introduced on last commit on CpuId
* Generate inline stack probes when the stack allocation is too large
* Initial support for the System-V ABI
* Support multiple destination operands
* Fix SSE2 VectorInsert8 path, and other fixes
* Change placement of XMM callee save and restore code to match other compilers
* Rename Dest to Destination and Inst to Instruction
* Fix a regression related to calls and the V128 type
* Add an extra space on comments to match code style
* Some refactoring
* Fix vector insert FP32 SSE2 path
* Port over the ARM32 instructions
* Avoid memory protection races on JIT Cache
* Another fix on VectorInsert FP32 (thanks to LDj3SNuD
* Float operands don't need to use the same register when VEX is supported
* Add a new register allocator, higher quality code for hot code (tier up), and other tweaks
* Some nits, small improvements on the pre allocator
* CpuThreadState is gone
* Allow changing CPU emulators with a config entry
* Add runtime identifiers on the ARMeilleure project
* Allow switching between CPUs through a config entry (pt. 2)
* Change win10-x64 to win-x64 on projects
* Update the Ryujinx project to use ARMeilleure
* Ensure that the selected register is valid on the hybrid allocator
* Allow exiting on returns to 0 (should fix test regression)
* Remove register assignments for most used variables on the hybrid allocator
* Do not use fixed registers as spill temp
* Add missing namespace and remove unneeded using
* Address PR feedback
* Fix types, etc
* Enable AssumeStrictAbiCompliance by default
* Ensure that Spill and Fill don't load or store any more than necessary
2019-08-08 20:56:22 +02:00
|
|
|
context.Interrupt += InterruptHandler;
|
|
|
|
context.SupervisorCall += _svcHandler.SvcCall;
|
|
|
|
context.Undefined += UndefinedInstructionHandler;
|
2018-11-28 23:18:09 +01:00
|
|
|
}
|
|
|
|
|
|
|
|
private void InterruptHandler(object sender, EventArgs e)
|
|
|
|
{
|
|
|
|
System.Scheduler.ContextSwitch();
|
|
|
|
}
|
|
|
|
|
|
|
|
public void IncrementThreadCount()
|
|
|
|
{
|
2018-12-06 12:16:24 +01:00
|
|
|
Interlocked.Increment(ref _threadCount);
|
2018-11-28 23:18:09 +01:00
|
|
|
|
|
|
|
System.ThreadCounter.AddCount();
|
|
|
|
}
|
|
|
|
|
|
|
|
public void DecrementThreadCountAndTerminateIfZero()
|
|
|
|
{
|
|
|
|
System.ThreadCounter.Signal();
|
|
|
|
|
2018-12-06 12:16:24 +01:00
|
|
|
if (Interlocked.Decrement(ref _threadCount) == 0)
|
2018-11-28 23:18:09 +01:00
|
|
|
{
|
|
|
|
Terminate();
|
|
|
|
}
|
|
|
|
}
|
|
|
|
|
|
|
|
public ulong GetMemoryCapacity()
|
|
|
|
{
|
2018-12-06 12:16:24 +01:00
|
|
|
ulong totalCapacity = (ulong)ResourceLimit.GetRemainingValue(LimitableResource.Memory);
|
2018-11-28 23:18:09 +01:00
|
|
|
|
2018-12-06 12:16:24 +01:00
|
|
|
totalCapacity += MemoryManager.GetTotalHeapSize();
|
2018-11-28 23:18:09 +01:00
|
|
|
|
2018-12-06 12:16:24 +01:00
|
|
|
totalCapacity += GetPersonalMmHeapSize();
|
2018-11-28 23:18:09 +01:00
|
|
|
|
2018-12-06 12:16:24 +01:00
|
|
|
totalCapacity += _imageSize + _mainThreadStackSize;
|
2018-11-28 23:18:09 +01:00
|
|
|
|
2018-12-06 12:16:24 +01:00
|
|
|
if (totalCapacity <= _memoryUsageCapacity)
|
2018-11-28 23:18:09 +01:00
|
|
|
{
|
2018-12-06 12:16:24 +01:00
|
|
|
return totalCapacity;
|
2018-11-28 23:18:09 +01:00
|
|
|
}
|
|
|
|
|
2018-12-06 12:16:24 +01:00
|
|
|
return _memoryUsageCapacity;
|
2018-11-28 23:18:09 +01:00
|
|
|
}
|
|
|
|
|
|
|
|
public ulong GetMemoryUsage()
|
|
|
|
{
|
2018-12-06 12:16:24 +01:00
|
|
|
return _imageSize + _mainThreadStackSize + MemoryManager.GetTotalHeapSize() + GetPersonalMmHeapSize();
|
2018-11-28 23:18:09 +01:00
|
|
|
}
|
|
|
|
|
|
|
|
public ulong GetMemoryCapacityWithoutPersonalMmHeap()
|
|
|
|
{
|
|
|
|
return GetMemoryCapacity() - GetPersonalMmHeapSize();
|
|
|
|
}
|
|
|
|
|
|
|
|
public ulong GetMemoryUsageWithoutPersonalMmHeap()
|
|
|
|
{
|
|
|
|
return GetMemoryUsage() - GetPersonalMmHeapSize();
|
|
|
|
}
|
|
|
|
|
|
|
|
private ulong GetPersonalMmHeapSize()
|
|
|
|
{
|
2018-12-06 12:16:24 +01:00
|
|
|
return GetPersonalMmHeapSize(PersonalMmHeapPagesCount, _memRegion);
|
2018-11-28 23:18:09 +01:00
|
|
|
}
|
|
|
|
|
2018-12-06 12:16:24 +01:00
|
|
|
private static ulong GetPersonalMmHeapSize(ulong personalMmHeapPagesCount, MemoryRegion memRegion)
|
2018-11-28 23:18:09 +01:00
|
|
|
{
|
2018-12-06 12:16:24 +01:00
|
|
|
if (memRegion == MemoryRegion.Applet)
|
2018-11-28 23:18:09 +01:00
|
|
|
{
|
|
|
|
return 0;
|
|
|
|
}
|
|
|
|
|
2018-12-06 12:16:24 +01:00
|
|
|
return personalMmHeapPagesCount * KMemoryManager.PageSize;
|
2018-11-28 23:18:09 +01:00
|
|
|
}
|
|
|
|
|
2018-12-06 12:16:24 +01:00
|
|
|
public void AddThread(KThread thread)
|
2018-11-28 23:18:09 +01:00
|
|
|
{
|
2018-12-06 12:16:24 +01:00
|
|
|
lock (_threadingLock)
|
2018-11-28 23:18:09 +01:00
|
|
|
{
|
2018-12-06 12:16:24 +01:00
|
|
|
thread.ProcessListNode = _threads.AddLast(thread);
|
2018-11-28 23:18:09 +01:00
|
|
|
}
|
|
|
|
}
|
|
|
|
|
2018-12-06 12:16:24 +01:00
|
|
|
public void RemoveThread(KThread thread)
|
2018-11-28 23:18:09 +01:00
|
|
|
{
|
2018-12-06 12:16:24 +01:00
|
|
|
lock (_threadingLock)
|
2018-11-28 23:18:09 +01:00
|
|
|
{
|
2018-12-06 12:16:24 +01:00
|
|
|
_threads.Remove(thread.ProcessListNode);
|
2018-11-28 23:18:09 +01:00
|
|
|
}
|
|
|
|
}
|
|
|
|
|
2018-12-06 12:16:24 +01:00
|
|
|
public bool IsCpuCoreAllowed(int core)
|
2018-11-28 23:18:09 +01:00
|
|
|
{
|
2018-12-06 12:16:24 +01:00
|
|
|
return (Capabilities.AllowedCpuCoresMask & (1L << core)) != 0;
|
2018-11-28 23:18:09 +01:00
|
|
|
}
|
|
|
|
|
2018-12-06 12:16:24 +01:00
|
|
|
public bool IsPriorityAllowed(int priority)
|
2018-11-28 23:18:09 +01:00
|
|
|
{
|
2018-12-06 12:16:24 +01:00
|
|
|
return (Capabilities.AllowedThreadPriosMask & (1L << priority)) != 0;
|
2018-11-28 23:18:09 +01:00
|
|
|
}
|
|
|
|
|
|
|
|
public override bool IsSignaled()
|
|
|
|
{
|
2018-12-06 12:16:24 +01:00
|
|
|
return _signaled;
|
2018-11-28 23:18:09 +01:00
|
|
|
}
|
|
|
|
|
|
|
|
public KernelResult Terminate()
|
|
|
|
{
|
2018-12-06 12:16:24 +01:00
|
|
|
KernelResult result;
|
2018-11-28 23:18:09 +01:00
|
|
|
|
2018-12-06 12:16:24 +01:00
|
|
|
bool shallTerminate = false;
|
2018-11-28 23:18:09 +01:00
|
|
|
|
|
|
|
System.CriticalSection.Enter();
|
|
|
|
|
2018-12-06 12:16:24 +01:00
|
|
|
lock (_processLock)
|
2018-11-28 23:18:09 +01:00
|
|
|
{
|
2018-12-06 12:16:24 +01:00
|
|
|
if (_state >= ProcessState.Started)
|
2018-11-28 23:18:09 +01:00
|
|
|
{
|
2018-12-06 12:16:24 +01:00
|
|
|
if (_state == ProcessState.Started ||
|
|
|
|
_state == ProcessState.Crashed ||
|
|
|
|
_state == ProcessState.Attached ||
|
|
|
|
_state == ProcessState.DebugSuspended)
|
2018-11-28 23:18:09 +01:00
|
|
|
{
|
|
|
|
SetState(ProcessState.Exiting);
|
|
|
|
|
2018-12-06 12:16:24 +01:00
|
|
|
shallTerminate = true;
|
2018-11-28 23:18:09 +01:00
|
|
|
}
|
|
|
|
|
2018-12-06 12:16:24 +01:00
|
|
|
result = KernelResult.Success;
|
2018-11-28 23:18:09 +01:00
|
|
|
}
|
|
|
|
else
|
|
|
|
{
|
2018-12-06 12:16:24 +01:00
|
|
|
result = KernelResult.InvalidState;
|
2018-11-28 23:18:09 +01:00
|
|
|
}
|
|
|
|
}
|
|
|
|
|
|
|
|
System.CriticalSection.Leave();
|
|
|
|
|
2018-12-06 12:16:24 +01:00
|
|
|
if (shallTerminate)
|
2018-11-28 23:18:09 +01:00
|
|
|
{
|
2019-07-02 04:39:22 +02:00
|
|
|
// UnpauseAndTerminateAllThreadsExcept(System.Scheduler.GetCurrentThread());
|
2018-11-28 23:18:09 +01:00
|
|
|
|
|
|
|
HandleTable.Destroy();
|
|
|
|
|
|
|
|
SignalExitForDebugEvent();
|
|
|
|
SignalExit();
|
|
|
|
}
|
|
|
|
|
2018-12-06 12:16:24 +01:00
|
|
|
return result;
|
2018-11-28 23:18:09 +01:00
|
|
|
}
|
|
|
|
|
2018-12-06 12:16:24 +01:00
|
|
|
private void UnpauseAndTerminateAllThreadsExcept(KThread thread)
|
2018-11-28 23:18:09 +01:00
|
|
|
{
|
2019-07-02 04:39:22 +02:00
|
|
|
// TODO.
|
2018-11-28 23:18:09 +01:00
|
|
|
}
|
|
|
|
|
|
|
|
private void SignalExitForDebugEvent()
|
|
|
|
{
|
2019-07-02 04:39:22 +02:00
|
|
|
// TODO: Debug events.
|
2018-11-28 23:18:09 +01:00
|
|
|
}
|
|
|
|
|
|
|
|
private void SignalExit()
|
|
|
|
{
|
|
|
|
if (ResourceLimit != null)
|
|
|
|
{
|
|
|
|
ResourceLimit.Release(LimitableResource.Memory, GetMemoryUsage());
|
|
|
|
}
|
|
|
|
|
|
|
|
System.CriticalSection.Enter();
|
|
|
|
|
|
|
|
SetState(ProcessState.Exited);
|
|
|
|
|
|
|
|
System.CriticalSection.Leave();
|
|
|
|
}
|
|
|
|
|
|
|
|
public KernelResult ClearIfNotExited()
|
|
|
|
{
|
2018-12-06 12:16:24 +01:00
|
|
|
KernelResult result;
|
2018-11-28 23:18:09 +01:00
|
|
|
|
|
|
|
System.CriticalSection.Enter();
|
|
|
|
|
2018-12-06 12:16:24 +01:00
|
|
|
lock (_processLock)
|
2018-11-28 23:18:09 +01:00
|
|
|
{
|
2018-12-06 12:16:24 +01:00
|
|
|
if (_state != ProcessState.Exited && _signaled)
|
2018-11-28 23:18:09 +01:00
|
|
|
{
|
2018-12-06 12:16:24 +01:00
|
|
|
_signaled = false;
|
2018-11-28 23:18:09 +01:00
|
|
|
|
2018-12-06 12:16:24 +01:00
|
|
|
result = KernelResult.Success;
|
2018-11-28 23:18:09 +01:00
|
|
|
}
|
|
|
|
else
|
|
|
|
{
|
2018-12-06 12:16:24 +01:00
|
|
|
result = KernelResult.InvalidState;
|
2018-11-28 23:18:09 +01:00
|
|
|
}
|
|
|
|
}
|
|
|
|
|
|
|
|
System.CriticalSection.Leave();
|
|
|
|
|
2018-12-06 12:16:24 +01:00
|
|
|
return result;
|
2018-11-28 23:18:09 +01:00
|
|
|
}
|
|
|
|
|
|
|
|
public void StopAllThreads()
|
|
|
|
{
|
2018-12-06 12:16:24 +01:00
|
|
|
lock (_threadingLock)
|
2018-11-28 23:18:09 +01:00
|
|
|
{
|
2018-12-06 12:16:24 +01:00
|
|
|
foreach (KThread thread in _threads)
|
2018-11-28 23:18:09 +01:00
|
|
|
{
|
Add a new JIT compiler for CPU code (#693)
* Start of the ARMeilleure project
* Refactoring around the old IRAdapter, now renamed to PreAllocator
* Optimize the LowestBitSet method
* Add CLZ support and fix CLS implementation
* Add missing Equals and GetHashCode overrides on some structs, misc small tweaks
* Implement the ByteSwap IR instruction, and some refactoring on the assembler
* Implement the DivideUI IR instruction and fix 64-bits IDIV
* Correct constant operand type on CSINC
* Move division instructions implementation to InstEmitDiv
* Fix destination type for the ConditionalSelect IR instruction
* Implement UMULH and SMULH, with new IR instructions
* Fix some issues with shift instructions
* Fix constant types for BFM instructions
* Fix up new tests using the new V128 struct
* Update tests
* Move DIV tests to a separate file
* Add support for calls, and some instructions that depends on them
* Start adding support for SIMD & FP types, along with some of the related ARM instructions
* Fix some typos and the divide instruction with FP operands
* Fix wrong method call on Clz_V
* Implement ARM FP & SIMD move instructions, Saddlv_V, and misc. fixes
* Implement SIMD logical instructions and more misc. fixes
* Fix PSRAD x86 instruction encoding, TRN, UABD and UABDL implementations
* Implement float conversion instruction, merge in LDj3SNuD fixes, and some other misc. fixes
* Implement SIMD shift instruction and fix Dup_V
* Add SCVTF and UCVTF (vector, fixed-point) variants to the opcode table
* Fix check with tolerance on tester
* Implement FP & SIMD comparison instructions, and some fixes
* Update FCVT (Scalar) encoding on the table to support the Half-float variants
* Support passing V128 structs, some cleanup on the register allocator, merge LDj3SNuD fixes
* Use old memory access methods, made a start on SIMD memory insts support, some fixes
* Fix float constant passed to functions, save and restore non-volatile XMM registers, other fixes
* Fix arguments count with struct return values, other fixes
* More instructions
* Misc. fixes and integrate LDj3SNuD fixes
* Update tests
* Add a faster linear scan allocator, unwinding support on windows, and other changes
* Update Ryujinx.HLE
* Update Ryujinx.Graphics
* Fix V128 return pointer passing, RCX is clobbered
* Update Ryujinx.Tests
* Update ITimeZoneService
* Stop using GetFunctionPointer as that can't be called from native code, misc. fixes and tweaks
* Use generic GetFunctionPointerForDelegate method and other tweaks
* Some refactoring on the code generator, assert on invalid operations and use a separate enum for intrinsics
* Remove some unused code on the assembler
* Fix REX.W prefix regression on float conversion instructions, add some sort of profiler
* Add hardware capability detection
* Fix regression on Sha1h and revert Fcm** changes
* Add SSE2-only paths on vector extract and insert, some refactoring on the pre-allocator
* Fix silly mistake introduced on last commit on CpuId
* Generate inline stack probes when the stack allocation is too large
* Initial support for the System-V ABI
* Support multiple destination operands
* Fix SSE2 VectorInsert8 path, and other fixes
* Change placement of XMM callee save and restore code to match other compilers
* Rename Dest to Destination and Inst to Instruction
* Fix a regression related to calls and the V128 type
* Add an extra space on comments to match code style
* Some refactoring
* Fix vector insert FP32 SSE2 path
* Port over the ARM32 instructions
* Avoid memory protection races on JIT Cache
* Another fix on VectorInsert FP32 (thanks to LDj3SNuD
* Float operands don't need to use the same register when VEX is supported
* Add a new register allocator, higher quality code for hot code (tier up), and other tweaks
* Some nits, small improvements on the pre allocator
* CpuThreadState is gone
* Allow changing CPU emulators with a config entry
* Add runtime identifiers on the ARMeilleure project
* Allow switching between CPUs through a config entry (pt. 2)
* Change win10-x64 to win-x64 on projects
* Update the Ryujinx project to use ARMeilleure
* Ensure that the selected register is valid on the hybrid allocator
* Allow exiting on returns to 0 (should fix test regression)
* Remove register assignments for most used variables on the hybrid allocator
* Do not use fixed registers as spill temp
* Add missing namespace and remove unneeded using
* Address PR feedback
* Fix types, etc
* Enable AssumeStrictAbiCompliance by default
* Ensure that Spill and Fill don't load or store any more than necessary
2019-08-08 20:56:22 +02:00
|
|
|
thread.Context.Running = false;
|
2018-11-28 23:18:09 +01:00
|
|
|
|
Add a new JIT compiler for CPU code (#693)
* Start of the ARMeilleure project
* Refactoring around the old IRAdapter, now renamed to PreAllocator
* Optimize the LowestBitSet method
* Add CLZ support and fix CLS implementation
* Add missing Equals and GetHashCode overrides on some structs, misc small tweaks
* Implement the ByteSwap IR instruction, and some refactoring on the assembler
* Implement the DivideUI IR instruction and fix 64-bits IDIV
* Correct constant operand type on CSINC
* Move division instructions implementation to InstEmitDiv
* Fix destination type for the ConditionalSelect IR instruction
* Implement UMULH and SMULH, with new IR instructions
* Fix some issues with shift instructions
* Fix constant types for BFM instructions
* Fix up new tests using the new V128 struct
* Update tests
* Move DIV tests to a separate file
* Add support for calls, and some instructions that depends on them
* Start adding support for SIMD & FP types, along with some of the related ARM instructions
* Fix some typos and the divide instruction with FP operands
* Fix wrong method call on Clz_V
* Implement ARM FP & SIMD move instructions, Saddlv_V, and misc. fixes
* Implement SIMD logical instructions and more misc. fixes
* Fix PSRAD x86 instruction encoding, TRN, UABD and UABDL implementations
* Implement float conversion instruction, merge in LDj3SNuD fixes, and some other misc. fixes
* Implement SIMD shift instruction and fix Dup_V
* Add SCVTF and UCVTF (vector, fixed-point) variants to the opcode table
* Fix check with tolerance on tester
* Implement FP & SIMD comparison instructions, and some fixes
* Update FCVT (Scalar) encoding on the table to support the Half-float variants
* Support passing V128 structs, some cleanup on the register allocator, merge LDj3SNuD fixes
* Use old memory access methods, made a start on SIMD memory insts support, some fixes
* Fix float constant passed to functions, save and restore non-volatile XMM registers, other fixes
* Fix arguments count with struct return values, other fixes
* More instructions
* Misc. fixes and integrate LDj3SNuD fixes
* Update tests
* Add a faster linear scan allocator, unwinding support on windows, and other changes
* Update Ryujinx.HLE
* Update Ryujinx.Graphics
* Fix V128 return pointer passing, RCX is clobbered
* Update Ryujinx.Tests
* Update ITimeZoneService
* Stop using GetFunctionPointer as that can't be called from native code, misc. fixes and tweaks
* Use generic GetFunctionPointerForDelegate method and other tweaks
* Some refactoring on the code generator, assert on invalid operations and use a separate enum for intrinsics
* Remove some unused code on the assembler
* Fix REX.W prefix regression on float conversion instructions, add some sort of profiler
* Add hardware capability detection
* Fix regression on Sha1h and revert Fcm** changes
* Add SSE2-only paths on vector extract and insert, some refactoring on the pre-allocator
* Fix silly mistake introduced on last commit on CpuId
* Generate inline stack probes when the stack allocation is too large
* Initial support for the System-V ABI
* Support multiple destination operands
* Fix SSE2 VectorInsert8 path, and other fixes
* Change placement of XMM callee save and restore code to match other compilers
* Rename Dest to Destination and Inst to Instruction
* Fix a regression related to calls and the V128 type
* Add an extra space on comments to match code style
* Some refactoring
* Fix vector insert FP32 SSE2 path
* Port over the ARM32 instructions
* Avoid memory protection races on JIT Cache
* Another fix on VectorInsert FP32 (thanks to LDj3SNuD
* Float operands don't need to use the same register when VEX is supported
* Add a new register allocator, higher quality code for hot code (tier up), and other tweaks
* Some nits, small improvements on the pre allocator
* CpuThreadState is gone
* Allow changing CPU emulators with a config entry
* Add runtime identifiers on the ARMeilleure project
* Allow switching between CPUs through a config entry (pt. 2)
* Change win10-x64 to win-x64 on projects
* Update the Ryujinx project to use ARMeilleure
* Ensure that the selected register is valid on the hybrid allocator
* Allow exiting on returns to 0 (should fix test regression)
* Remove register assignments for most used variables on the hybrid allocator
* Do not use fixed registers as spill temp
* Add missing namespace and remove unneeded using
* Address PR feedback
* Fix types, etc
* Enable AssumeStrictAbiCompliance by default
* Ensure that Spill and Fill don't load or store any more than necessary
2019-08-08 20:56:22 +02:00
|
|
|
System.Scheduler.CoreManager.Set(thread.HostThread);
|
2018-11-28 23:18:09 +01:00
|
|
|
}
|
|
|
|
}
|
|
|
|
}
|
|
|
|
|
2019-02-24 08:24:35 +01:00
|
|
|
private void InitializeMemoryManager(AddressSpaceType addrSpaceType, MemoryRegion memRegion)
|
2018-11-28 23:18:09 +01:00
|
|
|
{
|
2019-02-24 08:24:35 +01:00
|
|
|
int addrSpaceBits;
|
|
|
|
|
|
|
|
switch (addrSpaceType)
|
|
|
|
{
|
|
|
|
case AddressSpaceType.Addr32Bits: addrSpaceBits = 32; break;
|
|
|
|
case AddressSpaceType.Addr36Bits: addrSpaceBits = 36; break;
|
|
|
|
case AddressSpaceType.Addr32BitsNoMap: addrSpaceBits = 32; break;
|
|
|
|
case AddressSpaceType.Addr39Bits: addrSpaceBits = 39; break;
|
|
|
|
|
|
|
|
default: throw new ArgumentException(nameof(addrSpaceType));
|
|
|
|
}
|
|
|
|
|
|
|
|
bool useFlatPageTable = memRegion == MemoryRegion.Application;
|
|
|
|
|
2019-10-31 19:09:03 +01:00
|
|
|
CpuMemory = new MemoryManager(_system.Device.Memory.RamPointer, addrSpaceBits, useFlatPageTable);
|
2019-02-24 08:24:35 +01:00
|
|
|
|
2019-10-31 19:09:03 +01:00
|
|
|
Translator = new Translator(CpuMemory);
|
2019-02-24 08:24:35 +01:00
|
|
|
|
Add a new JIT compiler for CPU code (#693)
* Start of the ARMeilleure project
* Refactoring around the old IRAdapter, now renamed to PreAllocator
* Optimize the LowestBitSet method
* Add CLZ support and fix CLS implementation
* Add missing Equals and GetHashCode overrides on some structs, misc small tweaks
* Implement the ByteSwap IR instruction, and some refactoring on the assembler
* Implement the DivideUI IR instruction and fix 64-bits IDIV
* Correct constant operand type on CSINC
* Move division instructions implementation to InstEmitDiv
* Fix destination type for the ConditionalSelect IR instruction
* Implement UMULH and SMULH, with new IR instructions
* Fix some issues with shift instructions
* Fix constant types for BFM instructions
* Fix up new tests using the new V128 struct
* Update tests
* Move DIV tests to a separate file
* Add support for calls, and some instructions that depends on them
* Start adding support for SIMD & FP types, along with some of the related ARM instructions
* Fix some typos and the divide instruction with FP operands
* Fix wrong method call on Clz_V
* Implement ARM FP & SIMD move instructions, Saddlv_V, and misc. fixes
* Implement SIMD logical instructions and more misc. fixes
* Fix PSRAD x86 instruction encoding, TRN, UABD and UABDL implementations
* Implement float conversion instruction, merge in LDj3SNuD fixes, and some other misc. fixes
* Implement SIMD shift instruction and fix Dup_V
* Add SCVTF and UCVTF (vector, fixed-point) variants to the opcode table
* Fix check with tolerance on tester
* Implement FP & SIMD comparison instructions, and some fixes
* Update FCVT (Scalar) encoding on the table to support the Half-float variants
* Support passing V128 structs, some cleanup on the register allocator, merge LDj3SNuD fixes
* Use old memory access methods, made a start on SIMD memory insts support, some fixes
* Fix float constant passed to functions, save and restore non-volatile XMM registers, other fixes
* Fix arguments count with struct return values, other fixes
* More instructions
* Misc. fixes and integrate LDj3SNuD fixes
* Update tests
* Add a faster linear scan allocator, unwinding support on windows, and other changes
* Update Ryujinx.HLE
* Update Ryujinx.Graphics
* Fix V128 return pointer passing, RCX is clobbered
* Update Ryujinx.Tests
* Update ITimeZoneService
* Stop using GetFunctionPointer as that can't be called from native code, misc. fixes and tweaks
* Use generic GetFunctionPointerForDelegate method and other tweaks
* Some refactoring on the code generator, assert on invalid operations and use a separate enum for intrinsics
* Remove some unused code on the assembler
* Fix REX.W prefix regression on float conversion instructions, add some sort of profiler
* Add hardware capability detection
* Fix regression on Sha1h and revert Fcm** changes
* Add SSE2-only paths on vector extract and insert, some refactoring on the pre-allocator
* Fix silly mistake introduced on last commit on CpuId
* Generate inline stack probes when the stack allocation is too large
* Initial support for the System-V ABI
* Support multiple destination operands
* Fix SSE2 VectorInsert8 path, and other fixes
* Change placement of XMM callee save and restore code to match other compilers
* Rename Dest to Destination and Inst to Instruction
* Fix a regression related to calls and the V128 type
* Add an extra space on comments to match code style
* Some refactoring
* Fix vector insert FP32 SSE2 path
* Port over the ARM32 instructions
* Avoid memory protection races on JIT Cache
* Another fix on VectorInsert FP32 (thanks to LDj3SNuD
* Float operands don't need to use the same register when VEX is supported
* Add a new register allocator, higher quality code for hot code (tier up), and other tweaks
* Some nits, small improvements on the pre allocator
* CpuThreadState is gone
* Allow changing CPU emulators with a config entry
* Add runtime identifiers on the ARMeilleure project
* Allow switching between CPUs through a config entry (pt. 2)
* Change win10-x64 to win-x64 on projects
* Update the Ryujinx project to use ARMeilleure
* Ensure that the selected register is valid on the hybrid allocator
* Allow exiting on returns to 0 (should fix test regression)
* Remove register assignments for most used variables on the hybrid allocator
* Do not use fixed registers as spill temp
* Add missing namespace and remove unneeded using
* Address PR feedback
* Fix types, etc
* Enable AssumeStrictAbiCompliance by default
* Ensure that Spill and Fill don't load or store any more than necessary
2019-08-08 20:56:22 +02:00
|
|
|
MemoryManager = new KMemoryManager(_system, CpuMemory);
|
2018-11-28 23:18:09 +01:00
|
|
|
}
|
|
|
|
|
|
|
|
public void PrintCurrentThreadStackTrace()
|
|
|
|
{
|
|
|
|
System.Scheduler.GetCurrentThread().PrintGuestStackTrace();
|
|
|
|
}
|
|
|
|
|
2019-01-25 02:59:53 +01:00
|
|
|
private void UndefinedInstructionHandler(object sender, InstUndefinedEventArgs e)
|
|
|
|
{
|
Add a new JIT compiler for CPU code (#693)
* Start of the ARMeilleure project
* Refactoring around the old IRAdapter, now renamed to PreAllocator
* Optimize the LowestBitSet method
* Add CLZ support and fix CLS implementation
* Add missing Equals and GetHashCode overrides on some structs, misc small tweaks
* Implement the ByteSwap IR instruction, and some refactoring on the assembler
* Implement the DivideUI IR instruction and fix 64-bits IDIV
* Correct constant operand type on CSINC
* Move division instructions implementation to InstEmitDiv
* Fix destination type for the ConditionalSelect IR instruction
* Implement UMULH and SMULH, with new IR instructions
* Fix some issues with shift instructions
* Fix constant types for BFM instructions
* Fix up new tests using the new V128 struct
* Update tests
* Move DIV tests to a separate file
* Add support for calls, and some instructions that depends on them
* Start adding support for SIMD & FP types, along with some of the related ARM instructions
* Fix some typos and the divide instruction with FP operands
* Fix wrong method call on Clz_V
* Implement ARM FP & SIMD move instructions, Saddlv_V, and misc. fixes
* Implement SIMD logical instructions and more misc. fixes
* Fix PSRAD x86 instruction encoding, TRN, UABD and UABDL implementations
* Implement float conversion instruction, merge in LDj3SNuD fixes, and some other misc. fixes
* Implement SIMD shift instruction and fix Dup_V
* Add SCVTF and UCVTF (vector, fixed-point) variants to the opcode table
* Fix check with tolerance on tester
* Implement FP & SIMD comparison instructions, and some fixes
* Update FCVT (Scalar) encoding on the table to support the Half-float variants
* Support passing V128 structs, some cleanup on the register allocator, merge LDj3SNuD fixes
* Use old memory access methods, made a start on SIMD memory insts support, some fixes
* Fix float constant passed to functions, save and restore non-volatile XMM registers, other fixes
* Fix arguments count with struct return values, other fixes
* More instructions
* Misc. fixes and integrate LDj3SNuD fixes
* Update tests
* Add a faster linear scan allocator, unwinding support on windows, and other changes
* Update Ryujinx.HLE
* Update Ryujinx.Graphics
* Fix V128 return pointer passing, RCX is clobbered
* Update Ryujinx.Tests
* Update ITimeZoneService
* Stop using GetFunctionPointer as that can't be called from native code, misc. fixes and tweaks
* Use generic GetFunctionPointerForDelegate method and other tweaks
* Some refactoring on the code generator, assert on invalid operations and use a separate enum for intrinsics
* Remove some unused code on the assembler
* Fix REX.W prefix regression on float conversion instructions, add some sort of profiler
* Add hardware capability detection
* Fix regression on Sha1h and revert Fcm** changes
* Add SSE2-only paths on vector extract and insert, some refactoring on the pre-allocator
* Fix silly mistake introduced on last commit on CpuId
* Generate inline stack probes when the stack allocation is too large
* Initial support for the System-V ABI
* Support multiple destination operands
* Fix SSE2 VectorInsert8 path, and other fixes
* Change placement of XMM callee save and restore code to match other compilers
* Rename Dest to Destination and Inst to Instruction
* Fix a regression related to calls and the V128 type
* Add an extra space on comments to match code style
* Some refactoring
* Fix vector insert FP32 SSE2 path
* Port over the ARM32 instructions
* Avoid memory protection races on JIT Cache
* Another fix on VectorInsert FP32 (thanks to LDj3SNuD
* Float operands don't need to use the same register when VEX is supported
* Add a new register allocator, higher quality code for hot code (tier up), and other tweaks
* Some nits, small improvements on the pre allocator
* CpuThreadState is gone
* Allow changing CPU emulators with a config entry
* Add runtime identifiers on the ARMeilleure project
* Allow switching between CPUs through a config entry (pt. 2)
* Change win10-x64 to win-x64 on projects
* Update the Ryujinx project to use ARMeilleure
* Ensure that the selected register is valid on the hybrid allocator
* Allow exiting on returns to 0 (should fix test regression)
* Remove register assignments for most used variables on the hybrid allocator
* Do not use fixed registers as spill temp
* Add missing namespace and remove unneeded using
* Address PR feedback
* Fix types, etc
* Enable AssumeStrictAbiCompliance by default
* Ensure that Spill and Fill don't load or store any more than necessary
2019-08-08 20:56:22 +02:00
|
|
|
throw new UndefinedInstructionException(e.Address, e.OpCode);
|
2019-01-25 02:59:53 +01:00
|
|
|
}
|
2018-11-28 23:18:09 +01:00
|
|
|
}
|
|
|
|
}
|