Commit graph

107 commits

Author SHA1 Message Date
merry
dc063eac83
ARMeilleure: Implement single stepping (#3133)
* Decoder: Implement SingleInstruction decoder mode

* Translator: Implement Step

* DecoderMode: Rename Normal to MultipleBlocks
2022-02-22 11:11:42 -03:00
Berkan Diler
644b497df1
Collapse AsSpan().Slice(..) calls into AsSpan(..) (#3145)
* Collapse AsSpan().Slice(..) calls into AsSpan(..)

Less code and a bit faster

* Collapse an Array.Clear(array, 0, array.Length) call to Array.Clear(array)
2022-02-22 10:32:10 -03:00
gdkchan
f2087ca29e
PPTC version increment (#3139) 2022-02-17 23:52:42 -03:00
gdkchan
92d166ecb7
Enable CPU JIT cache invalidation (#2965)
* Enable CPU JIT cache invalidation

* Invalidate cache on IC IVAU
2022-02-18 02:53:18 +01:00
merry
98e05ee4b7
ARMeilleure: Thumb support (All T16 instructions) (#3105)
* Decoders: Add InITBlock argument

* OpCodeTable: Minor cleanup

* OpCodeTable: Remove existing thumb instruction implementations

* OpCodeTable: Prepare for thumb instructions

* OpCodeTables: Improve thumb fast lookup

* Tests: Prepare for thumb tests

* T16: Implement BX

* T16: Implement LSL/LSR/ASR (imm)

* T16: Implement ADDS, SUBS (reg)

* T16: Implement ADDS, SUBS (3-bit immediate)

* T16: Implement MOVS, CMP, ADDS, SUBS (8-bit immediate)

* T16: Implement ANDS, EORS, LSLS, LSRS, ASRS, ADCS, SBCS, RORS, TST, NEGS, CMP, CMN, ORRS, MULS, BICS, MVNS (low registers)

* T16: Implement ADD, CMP, MOV (high reg)

* T16: Implement BLX (reg)

* T16: Implement LDR (literal)

* T16: Implement {LDR,STR}{,H,B,SB,SH} (register)

* T16: Implement {LDR,STR}{,B,H} (immediate)

* T16: Implement LDR/STR (SP)

* T16: Implement ADR

* T16: Implement Add to SP (immediate)

* T16: Implement ADD/SUB (SP)

* T16: Implement SXTH, SXTB, UXTH, UTXB

* T16: Implement CBZ, CBNZ

* T16: Implement PUSH, POP

* T16: Implement REV, REV16, REVSH

* T16: Implement NOP

* T16: Implement LDM, STM

* T16: Implement SVC

* T16: Implement B (conditional)

* T16: Implement B (unconditional)

* T16: Implement IT

* fixup! T16: Implement ADD/SUB (SP)

* fixup! T16: Implement Add to SP (immediate)

* fixup! T16: Implement IT

* CpuTestThumb: Add randomized tests

* Remove inITBlock argument

* Address nits

* Use index to handle IfThenBlockState

* Reduce line noise

* fixup

* nit
2022-02-17 19:39:45 -03:00
gdkchan
bd412afb9f
Fix small precision error on CPU reciprocal estimate instructions (#3061)
* Fix small precision error on CPU reciprocal estimate instructions

* PPTC version bump
2022-01-29 23:59:34 +01:00
gdkchan
f3bfd799e1
Fix calls passing V128 values on Linux (#3034)
* Fix calls passing V128 values on Linux

* PPTC version bump
2022-01-24 11:23:24 +01:00
gdkchan
f0824fde9f
Add host CPU memory barriers for DMB/DSB and ordered load/store (#3015)
* Add host CPU memory barriers for DMB/DSB and ordered load/store

* PPTC version bump

* Revert to old barrier order
2022-01-21 12:47:34 -03:00
sharmander
60f7cba30a
Implement FCVTNS (Scalar GP) (#2953)
* Implement FCVTNS (Scalar GP)

* Update Ptc Version
2022-01-19 22:21:44 -03:00
sharmander
e5f7ff1eee
CPU - Implement FCVTMS (Vector) (#2937)
* Add FCVTMS_V Implementation to Armeilleure

* Fix opcode designation

* Add tests

* Amend Ptc version

* Fix OpCode / Tests

* Create Math.Floor helper method + Update implementation

* Address gdk comments

* Re-address gdk comments

* Update ARMeilleure/Decoders/OpCodeTable.cs

Co-authored-by: gdkchan <gab.dark.100@gmail.com>

* Update Tests to use 2S (4S) and 2D

Co-authored-by: gdkchan <gab.dark.100@gmail.com>
2022-01-04 16:45:28 -03:00
Piyachet Kanda
3e2f89b4fd
Implement UHADD8 instruction (#2908)
* Implement UHADD8 instruction along with a test unit

* Update PTC revision number
2021-12-08 17:05:59 -03:00
Mary
f39fce8f54
misc: Migrate usage of RuntimeInformation to OperatingSystem (#2901)
Very basic migration across the codebase.
2021-12-04 20:02:30 -03:00
FICTURE7
fbf40424f4
Add an early TailMerge pass (#2721)
* Add an early `TailMerge` pass

Some translations can have a lot of guest calls and since for each guest
call there is a call guard which may return. This can produce a lot of
epilogue code for returns. This pass merges the epilogue into a single
block.

```
Using filter 'hcq'.
Using metric 'code size'.

Total diff: -1648111 (-7.19 %) (bytes):
  Base: 22913847
  Diff: 21265736

Improved: 4567, regressed: 14, unchanged: 144
```

* Set PTC version

* Address feedback

* Handle `void` returning functions

* Actually handle `void` returning functions

* Fix `RegisterToLocal` logging
2021-10-18 19:51:22 -03:00
FICTURE7
ecc64c934d
Add Operand.Label support to Assembler (#2680)
* Add `Operand.Label` support to `Assembler`

This adds label support to `Assembler` and enables branch tightening
when compiling with relocatables. Jump management and patching has been
moved to the `Assembler`.

* Move instruction table to `Assembler.Table`

* Set PTC internal version

* Rename `Assembler.Table` to `AssemblerTable`
2021-10-05 14:04:55 -03:00
riperiperi
1ae690ba2f
Use normal memory store path for DC ZVA (#2693)
Seems like this is used as an optimized way to clear memory in homebrew applications. Unfortunately, calling the software fallback method every 8 bytes was not very optimal.

The existing EmitStore is used by passing in ZR as the register to get a 0 write.
2021-09-29 01:21:30 +02:00
FICTURE7
0d23504e30
Fix PTC count table relocation patching (#2666)
Fix an issue introduced in #2190 where by 2 different count table entry
addresses were used for LCQ functions. E.g:

```asm
 .L1:
   mov rbp,COUNT_TABLE_0   ;; This gets an address.
   mov ebp,[rbp]
   lea esi,[rbp+1]
   mov rdi,COUNT_TABLE_1   ;; This gets another address.
   mov [rdi],esi
   cmp ebp,64h
   je near .L34
```

This caused LCQ functions to not tier up when they're loaded from the
PTC cache. This does not happen when they're freshly compiled.

This PR fixes the issue by ensuring only a single counter is created per
translation.
2021-09-29 00:28:34 +02:00
FICTURE7
a9343c9364
Refactor PtcInfo (#2625)
* Refactor `PtcInfo`

This change reduces the coupling of `PtcInfo` by moving relocation
tracking to the backend. `RelocEntry`s remains as `RelocEntry`s through
out the pipeline until it actually needs to be written to the PTC
streams. Keeping this representation makes inspecting and manipulating
relocations after compilations less painful. This is something I needed
to do to patch relocations to 0 to diff dumps.

Contributes to #1125.

* Turn `Symbol` & `RelocInfo` into readonly structs

* Add documentation to `CompiledFunction`

* Remove `Compiler.Compile<T>`

Remove `Compiler.Compile<T>` and replace it by `Map<T>` of the
`CompiledFunction` returned.
2021-09-14 01:23:37 +02:00
FICTURE7
f2a7b300c4
Fix type mismatch in BitwiseAnd simplification (#2571)
* Fix type mismatch in `BitwiseAnd` simplification

`TryEliminateBitwiseAnd` would turn the `BitwiseAnd` operation into a
copy of the wrong type. E.g:

Before `Simplification`:
```llvm
i64 %0 = BitwiseAnd i64 0x0, %1
```

After `Simplication`:
```llvm
i64 %0 = Copy i32 0x0
```

Since the with the changes in #2515, we iterate in reverse order and
`Simplication`, `ConstantFolding` does not indicate if it modified
the CFG, the second pass to "retype" the copy into the proper
destination type does not happen.

This also blocked copy propagation since its destination type did not
match with its source type. But in the cases I've seen, the
`PreAllocator` would insert a copy for the propagated constant, which
results in no diffs.

Since the copy remained as is, asserts are fired when generating it.

* Set PPTC version
2021-08-20 14:42:00 -03:00
FICTURE7
22b2cb39af
Reduce JIT GC allocations (#2515)
* Turn `MemoryOperand` into a struct

* Remove `IntrinsicOperation`

* Remove `PhiNode`

* Remove `Node`

* Turn `Operand` into a struct

* Turn `Operation` into a struct

* Clean up pool management methods

* Add `Arena` allocator

* Move `OperationHelper` to `Operation.Factory`

* Move `OperandHelper` to `Operand.Factory`

* Optimize `Operation` a bit

* Fix `Arena` initialization

* Rename `NativeList<T>` to `ArenaList<T>`

* Reduce `Operand` size from 88 to 56 bytes

* Reduce `Operation` size from 56 to 40 bytes

* Add optimistic interning of Register & Constant operands

* Optimize `RegisterUsage` pass a bit

* Optimize `RemoveUnusedNodes` pass a bit

Iterating in reverse-order allows killing dependency chains in a single
pass.

* Fix PPTC symbols

* Optimize `BasicBlock` a bit

Reduce allocations from `_successor` & `DominanceFrontiers`

* Fix `Operation` resize

* Make `Arena` expandable

Change the arena allocator to be expandable by allocating in pages, with
some of them being pooled. Currently 32 pages are pooled. An LRU removal
mechanism should probably be added to it.

Apparently MHR can allocate bitmaps large enough to exceed the 16MB
limit for the type.

* Move `Arena` & `ArenaList` to `Common`

* Remove `ThreadStaticPool` & co

* Add `PhiOperation`

* Reduce `Operand` size from 56 from 48 bytes

* Add linear-probing to `Operand` intern table

* Optimize `HybridAllocator` a bit

* Add `Allocators` class

* Tune `ArenaAllocator` sizes

* Add page removal mechanism to `ArenaAllocator`

Remove pages which have not been used for more than 5s after each reset.

I am on fence if this would be better using a Gen2 callback object like
the one in System.Buffers.ArrayPool<T>, to trim the pool. Because right
now if a large translation happens, the pages will be freed only after a
reset. This reset may not happen for a while because no new translation
is hit, but the arena base sizes are rather small.

* Fix `OOM` when allocating larger than page size in `ArenaAllocator`

Tweak resizing mechanism for Operand.Uses and Assignemnts.

* Optimize `Optimizer` a bit

* Optimize `Operand.Add<T>/Remove<T>` a bit

* Clean up `PreAllocator`

* Fix phi insertion order

Reduce codegen diffs.

* Fix code alignment

* Use new heuristics for degree of parallelism

* Suppress warnings

* Address gdkchan's feedback

Renamed `GetValue()` to `GetValueUnsafe()` to make it more clear that
`Operand.Value` should usually not be modified directly.

* Add fast path to `ArenaAllocator`

* Assembly for `ArenaAllocator.Allocate(ulong)`:

  .L0:
    mov rax, [rcx+0x18]
    lea r8, [rax+rdx]
    cmp r8, [rcx+0x10]
    ja short .L2
  .L1:
    mov rdx, [rcx+8]
    add rax, [rdx+8]
    mov [rcx+0x18], r8
    ret
  .L2:
    jmp ArenaAllocator.AllocateSlow(UInt64)

  A few variable/field had to be changed to ulong so that RyuJIT avoids
  emitting zero-extends.

* Implement a new heuristic to free pooled pages.

  If an arena is used often, it is more likely that its pages will be
  needed, so the pages are kept for longer (e.g: during PPTC rebuild or
  burst sof compilations). If is not used often, then it is more likely
  that its pages will not be needed (e.g: after PPTC rebuild or bursts
  of compilations).

* Address riperiperi's feedback

* Use `EqualityComparer<T>` in `IntrusiveList<T>`

Avoids a potential GC hole in `Equals(T, T)`.
2021-08-17 15:08:34 -03:00
FICTURE7
9d7627af64
Add multi-level function table (#2228)
* Add AddressTable<T>

* Use AddressTable<T> for dispatch

* Remove JumpTable & co.

* Add fallback for out of range addresses

* Add PPTC support

* Add documentation to `AddressTable<T>`

* Make AddressTable<T> configurable

* Fix table walk

* Fix IsMapped check

* Remove CountTableCapacity

* Add PPTC support for fast path

* Rename IsMapped to IsValid

* Remove stale comment

* Change format of address in exception message

* Add TranslatorStubs

* Split DispatchStub

Avoids recompilation of stubs during tests.

* Add hint for 64bit or 32bit

* Add documentation to `Symbol`

* Add documentation to `TranslatorStubs`

Make `TranslatorStubs` disposable as well.

* Add documentation to `SymbolType`

* Add `AddressTableEventSource` to monitor function table size

Add an EventSource which measures the amount of unmanaged bytes
allocated by AddressTable<T> instances.

 dotnet-counters monitor -n Ryujinx --counters ARMeilleure

* Add `AllowLcqInFunctionTable` optimization toggle

This is to reduce the impact this change has on the test duration.
Before everytime a test was ran, the FunctionTable would be initialized
and populated so that the newly compiled test would get registered to
it.

* Implement unmanaged dispatcher

Uses the DispatchStub to dispatch into the next translation, which
allows execution to stay in unmanaged for longer and skips a
ConcurrentDictionary look up when the target translation has been
registered to the FunctionTable.

* Remove redundant null check

* Tune levels of FunctionTable

Uses 5 levels instead of 4 and change unit of AddressTableEventSource
from KB to MB.

* Use 64-bit function table

Improves codegen for direct branches:

    mov qword [rax+0x408],0x10603560
 -  mov rcx,sub_10603560_OFFSET
 -  mov ecx,[rcx]
 -  mov ecx,ecx
 -  mov rdx,JIT_CACHE_BASE
 -  add rdx,rcx
 +  mov rcx,sub_10603560
 +  mov rdx,[rcx]
    mov rcx,rax

Improves codegen for dispatch stub:

    and rax,byte +0x1f
 -  mov eax,[rcx+rax*4]
 -  mov eax,eax
 -  mov rcx,JIT_CACHE_BASE
 -  lea rax,[rcx+rax]
 +  mov rax,[rcx+rax*8]
    mov rcx,rbx

* Remove `JitCacheSymbol` & `JitCache.Offset`

* Turn `Translator.Translate` into an instance method

We do not have to add more parameter to this method and related ones as
new structures are added & needed for translation.

* Add symbol only when PTC is enabled

Address LDj3SNuD's feedback

* Change `NativeContext.Running` to a 32-bit integer

* Fix PageTable symbol for host mapped
2021-05-29 18:06:28 -03:00
riperiperi
54ea2285f0
POWER - Performance Optimizations With Extensive Ramifications (#2286)
* Refactoring of KMemoryManager class

* Replace some trivial uses of DRAM address with VA

* Get rid of GetDramAddressFromVa

* Abstracting more operations on derived page table class

* Run auto-format on KPageTableBase

* Managed to make TryConvertVaToPa private, few uses remains now

* Implement guest physical pages ref counting, remove manual freeing

* Make DoMmuOperation private and call new abstract methods only from the base class

* Pass pages count rather than size on Map/UnmapMemory

* Change memory managers to take host pointers

* Fix a guest memory leak and simplify KPageTable

* Expose new methods for host range query and mapping

* Some refactoring of MapPagesFromClientProcess to allow proper page ref counting and mapping without KPageLists

* Remove more uses of AddVaRangeToPageList, now only one remains (shared memory page checking)

* Add a SharedMemoryStorage class, will be useful for host mapping

* Sayonara AddVaRangeToPageList, you served us well

* Start to implement host memory mapping (WIP)

* Support memory tracking through host exception handling

* Fix some access violations from HLE service guest memory access and CPU

* Fix memory tracking

* Fix mapping list bugs, including a race and a error adding mapping ranges

* Simple page table for memory tracking

* Simple "volatile" region handle mode

* Update UBOs directly (experimental, rough)

* Fix the overlap check

* Only set non-modified buffers as volatile

* Fix some memory tracking issues

* Fix possible race in MapBufferFromClientProcess (block list updates were not locked)

* Write uniform update to memory immediately, only defer the buffer set.

* Fix some memory tracking issues

* Pass correct pages count on shared memory unmap

* Armeilleure Signal Handler v1 + Unix changes

Unix currently behaves like windows, rather than remapping physical

* Actually check if the host platform is unix

* Fix decommit on linux.

* Implement windows 10 placeholder shared memory, fix a buffer issue.

* Make PTC version something that will never match with master

* Remove testing variable for block count

* Add reference count for memory manager, fix dispose

Can still deadlock with OpenAL

* Add address validation, use page table for mapped check, add docs

Might clean up the page table traversing routines.

* Implement batched mapping/tracking.

* Move documentation, fix tests.

* Cleanup uniform buffer update stuff.

* Remove unnecessary assignment.

* Add unsafe host mapped memory switch

On by default. Would be good to turn this off for untrusted code (homebrew, exefs mods) and give the user the option to turn it on manually, though that requires some UI work.

* Remove C# exception handlers

They have issues due to current .NET limitations, so the meilleure one fully replaces them for now.

* Fix MapPhysicalMemory on the software MemoryManager.

* Null check for GetHostAddress, docs

* Add configuration for setting memory manager mode (not in UI yet)

* Add config to UI

* Fix type mismatch on Unix signal handler code emit

* Fix 6GB DRAM mode.

The size can be greater than `uint.MaxValue` when the DRAM is >4GB.

* Address some feedback.

* More detailed error if backing memory cannot be mapped.

* SetLastError on all OS functions for consistency

* Force pages dirty with UBO update instead of setting them directly.

Seems to be much faster across a few games. Need retesting.

* Rebase, configuration rework, fix mem tracking regression

* Fix race in FreePages

* Set memory managers null after decrementing ref count

* Remove readonly keyword, as this is now modified.

* Use a local variable for the signal handler rather than a register.

* Fix bug with buffer resize, and index/uniform buffer binding.

Should fix flickering in games.

* Add InvalidAccessHandler to MemoryTracking

Doesn't do anything yet

* Call invalid access handler on unmapped read/write.

Same rules as the regular memory manager.

* Make unsafe mapped memory its own MemoryManagerType

* Move FlushUboDirty into UpdateState.

* Buffer dirty cache, rather than ubo cache

Much cleaner, may be reusable for Inline2Memory updates.

* This doesn't return anything anymore.

* Add sigaction remove methods, correct a few function signatures.

* Return empty list of physical regions for size 0.

* Also on AddressSpaceManager

Co-authored-by: gdkchan <gab.dark.100@gmail.com>
2021-05-24 22:52:44 +02:00
gdkchan
fb65f392d1
Improve accuracy of reciprocal step instructions (#2305)
* Improve accuracy of reciprocal step instructions

* Fix small mistake on RECPE rounding, nits, PTC version bump
2021-05-24 20:20:07 +10:00
FICTURE7
65ac00833a
Use branch instead of tailcall for recursive calls (#2282)
* Use branch instead of tailcall for recursive calls

Use a branch instead of doing a tailcall for recursive calls. This
avoids having to store the dispatch address, setting up the epilogue and
keeps guest registers in host registers for longer.

The rejit check is moved down into the entry block so that the rejit
behaviour remains the same as before.

* Set PTC version

Co-authored-by: gdkchan <gab.dark.100@gmail.com>
2021-05-20 09:31:45 -03:00
FICTURE7
0181068016
Add BIC/ORR Vd.T, #imm fast path (#2279)
* Add fast path for BIC Vd.T, #imm

* Add fast path for ORR Vd.T, #imm

* Set PTC version

* Fixup Exception to InvalidOperationException
2021-05-20 09:09:17 -03:00
FICTURE7
c805542b29
Allow LocalVariable to be assigned more than once (#2288)
* Allow `LocalVariable` to be assigned more than once

This allows us to write flow controls like loops and if-elses with
LocalVariables participating in phi nodes.

* Add `GetLocalNumber` to operand
2021-05-17 01:54:53 +02:00
gdkchan
e318022b89
Fold constant offsets and group constant addresses (#2285)
* Fold constant offsets and group constant addresses

* PPTC version bump
2021-05-13 21:26:57 +02:00
LDj3SNuD
57ea3f93a3
PPTC meets ExeFS Patching. (#1865)
* PPTC meets ExeFS Patching.

* InternalVersion = 1865

* Ready!

* Optimized the PtcProfiler Load/Save methods.
2021-05-13 20:05:15 +02:00
FICTURE7
89791ba68d
Add inlined on translation call counting (#2190)
* Add EntryTable<TEntry>

* Add on translation call counting

* Add Counter

* Add PPTC support

* Make Counter a generic & use a 32-bit counter instead

* Return false on overflow

* Set PPTC version

* Print more information about the rejit queue

* Make Counter<T> disposable

* Remove Block.TailCall since it is not used anymore

* Apply suggestions from code review

Address gdkchan's feedback

Co-authored-by: gdkchan <gab.dark.100@gmail.com>

* Fix more stale docs

* Remove rejit requests queue logging

* Make Counter<T> finalizable

Most certainly quite an odd use case.

* Make EntryTable<T>.TryAllocate set entry to default

* Re-trigger CI

* Dispose Counters before they hit the finalizer queue

* Re-trigger CI

Just for good measure...

* Make EntryTable<T> expandable

* EntryTable is now expandable instead of being a fixed slab.
* Remove EntryTable<T>.TryAllocate
* Remove Counter<T>.TryCreate

Address LDj3SNuD's feedback

* Apply suggestions from code review

Address LDj3SNuD's feedback

Co-authored-by: LDj3SNuD <35856442+LDj3SNuD@users.noreply.github.com>

* Remove useless return

* POH approach, but the sequel

* Revert "POH approach, but the sequel"

This reverts commit 5f5abaa247.

The sequel got shelved

* Add extra documentation

Co-authored-by: gdkchan <gab.dark.100@gmail.com>
Co-authored-by: LDj3SNuD <35856442+LDj3SNuD@users.noreply.github.com>
2021-04-18 23:43:53 +02:00
LDj3SNuD
90163087a0
PPTC vs. giant ExeFS. (#2168)
* PPTC vs. giant ExeFS.

* InternalVersion = 2168

* Add new heuristic algorithm for calculating the number of threads for parallel translations that also takes into account the user's free physical memory and not just the number of CPU cores.

* Nit.

* Add an outer Header structure and add the hashes for both this new structure and the existing "inner" Header structure.

* InternalVersion = 2169
2021-04-13 03:24:36 +02:00
FICTURE7
98ed81e4cd
Improve StoreToContext emission (#2155)
* Improve StoreToContext emission

Hoist StoreToContext in dynamic branch fast & slow paths out into
their predecessor.

Reduces register pressure, code size and compile time because we're
throwing less stuff down the pipeline.

* Set PTC internal version

* Turn EmitDynamicTableCall private

* Re-trigger CI
2021-04-02 19:54:23 +02:00
FICTURE7
8b3eba7e13
Reduce allocation during SSA construction (#2162)
* Reduce allocation during SSA construction

* Re-trigger CI
2021-04-02 19:26:16 +02:00
mageven
69f8722e79
Fix inconsistencies in progress reporting (#2129)
Signal and setup events correctly
Eliminate possible races
Use a single event
Mark volatiles and reduce scope of waithandles
Common handler
100ms -> 50ms
2021-03-22 19:40:07 +01:00
mageven
ca5d8e58dd
Add progress reporting to PTC and Shader Cache (#2057)
* UI changes

* Add progress reporting to PTC & ShaderCache

* Account for null events and expand docs

Co-authored-by: Joshi234 <46032261+Joshi234@users.noreply.github.com>
2021-03-03 01:39:36 +01:00
LDj3SNuD
bcbf240d2e
PPTC: Fix unwanted propagation of a relocatable constant in a specific case. (#1990)
* Fix unwanted propagation of a relocatable constant in a specific case.

* Ptc.InternalVersion = 1990

* Nit to retrigger the Checks.
2021-02-23 13:15:45 +01:00
mageven
9bda7b4699
Implement VCNT instruction (#1963)
* Implement VCNT based on AArch64 CNT

Add tests

* Update PTC version

* Address LDj's comments

* Explicit size in encoding
* Tighter tests
* Replace SoftFallback with IR helper

Co-authored-by: LDj3SNuD <35856442+LDj3SNuD@users.noreply.github.com>

* Reduce one BitwiseAnd from IR fallback

Based on popcount64b from https://en.wikipedia.org/wiki/Hamming_weight#Efficient_implementation

* Rename parameter and add assert

Co-authored-by: LDj3SNuD <35856442+LDj3SNuD@users.noreply.github.com>

Co-authored-by: LDj3SNuD <35856442+LDj3SNuD@users.noreply.github.com>
2021-02-22 16:26:13 +01:00
LDj3SNuD
dc0adb533d
PPTC & Pool Enhancements. (#1968)
* PPTC & Pool Enhancements.

* Avoid buffer allocations in CodeGenContext.GetCode(). Avoid stream allocations in PTC.PtcInfo.

Refactoring/nits.

* Use XXHash128, for Ptc.Load & Ptc.Save, x10 faster than Md5.

* Why not a nice Span.

* Added a simple PtcFormatter library for deserialization/serialization, which does not require reflection, in use at PtcJumpTable and PtcProfiler; improves maintainability and simplicity/readability of affected code.

* Nits.

* Revert #1987.

* Revert "Revert #1987."

This reverts commit 998be765cf.
2021-02-22 03:23:48 +01:00
FICTURE7
1586880114
Turn Copy into Fill in HybridAllocator (#2010)
* Turn Copy into Fill in HybridAllocator

* Set PTC internal verison
2021-02-21 18:33:59 +01:00
gdkchan
9d82d27df2
Fix memory tracking performance regression (#2026)
* Fix memory tracking performance regression

* Set PTC version
2021-02-17 09:16:20 +11:00
gdkchan
715b605e95
Validate CPU virtual addresses on access (#1987)
* Enable PTE null checks again

* Do address validation on EmitPtPointerLoad, and make it branchless

* PTC version increment

* Mask of pointer tag for exclusive access

* Move mask to the correct place

Co-authored-by: LDj3SNuD <35856442+LDj3SNuD@users.noreply.github.com>
2021-02-16 19:04:19 +01:00
sharmander
40797a1283
Optimization | Modify Add (Integer) Instruction to use LEA instead. (#1971)
* Optimization | Modify Add Instruction to use LEA instead.

Currently, the add instruction requires 4 registers to take place. By using LEA, we can effectively perform the same working using 3 registers, reducing memory spills and improving translation efficiency.

* Fix IsSameOperandDestSrc1 Check for Add

* Use LEA if Dest != SRC1

* Update IsSameOperandDestSrc1 to account for Cases where Dest and Src1 can be same for add

* Fix error in logic

* Typo

* Add paranthesis for clarity

* Compare registers as requested.

* Cleanup if statement, use same comparison method as generateCopy

* Make change as recommended by gdk

* Perform check only when Add calls are made

* use ensure sametype for lea, fix else

* Update comment

* Update version #
2021-02-08 10:49:46 +11:00
gdkchan
ee28ccebf4
Disable partial JIT invalidation on unmap (#1991) 2021-02-08 10:25:14 +11:00
gdkchan
dcce407071
Lower precision of estimate instruction results to match Arm behavior (#1943)
* Lower precision of estimate instruction results to match Arm behavior

* PTC version update

* Nits
2021-01-28 10:23:00 +11:00
mageven
c19cfca183
Implement PRFM (register variant) as NOP (#1956)
* Implement PRFM (register variant) as NOP

Fix typo pfrm -> prfm
Add comments to distinguish variants

* Increment PTC version
2021-01-26 16:09:27 +11:00
FICTURE7
ddf1105bcb
Add VCLZ.* fast path (#1917)
* Add VCLZ fast path

* Add VCLZ.8B/16B SSSE3 fast path
* Add VCLZ.4H/8H SSSE3 fast path
* Add VCLZ.2S/4S SSE2 fast path

* Improve CLZ.4H/8H fast path

* Improve CLZ.2S/4S fast path

* Set PPTC version
2021-01-25 10:01:25 +11:00
LDj3SNuD
68f6b79fd3
Add a simple Pools Limiter. (#1830)
* Added support for offline invalidation, via PPTC, of low cq translations replaced by high cq translations; both on a single run and between runs.

Added invalidation of .cache files in the event of reuse on a different user operating system.

Added .info and .cache files invalidation in case of a failed stream decompression.

Nits.

* InternalVersion = 1712;

* Nits.

* Address comment.

* Get rid of BinaryFormatter.

Nits.

* Move Ptc.LoadTranslations().

Nits.

* Nits.

* Fixed corner cases (in case backup copies have to be used). Added save logs.

* Not core fixes.

* Complement to the previous commit. Added load logs. Removed BinaryFormatter leftovers.

* Add LoadTranslations log.

* Nits.

* Removed the search and management of LowCq overlapping functions.

* Final increment of .info and .cache flags.

* Nit.

* Free up memory allocated by Pools during any PPTC translations at boot time.

* Nit due to rebase.

* Add a simple Pools Limiter.

* Nits.

* Fix missing JumpTable.RegisterFunction() due to rebase.

Clear MemoryStreams as soon as possible, when they are no longer needed.

* Code cleaning.

* Nit for retrigger Checks.

* Update Ptc.cs

* Contextual refactoring of Translator. Ignore resetting of pools for DirectCallStubs.

* Nit for retrigger Checks.
2021-01-12 19:04:02 +01:00
LDj3SNuD
430ba6da65
CPU (A64): Add Pmull_V Inst. with Clmul fast path for the "1/2D -> 1Q" variant & Sse fast path and slow path for both the "8/16B -> 8H" and "1/2D -> 1Q" variants; with Test. (#1817)
* Add Pmull_V Sse fast path only, both "8/16B -> 8H" and "1/2D -> 1Q" variants; with Test.

* Add Clmul fast path for the 128 bits variant.

* Small optimisation (save 60 instructions) for the Sse fast path about the 128 bits variant.

* Add slow path, both variants. Fix V128 Shl/Shr when shift = 0.

* A32: Add Vmull_I P64 variant (slow path); not tested.

* A32: Add Vmull_I_P8_P64 Test and fix P64 variant.
2021-01-04 23:45:54 +01:00
Ac_K
5b9c876155 Hotfix for #1814 2020-12-24 04:44:39 +01:00
LDj3SNuD
2502f1f07f
Free up memory allocated by Pools during any PPTC translations at boot time. (#1814)
* Added support for offline invalidation, via PPTC, of low cq translations replaced by high cq translations; both on a single run and between runs.

Added invalidation of .cache files in the event of reuse on a different user operating system.

Added .info and .cache files invalidation in case of a failed stream decompression.

Nits.

* InternalVersion = 1712;

* Nits.

* Address comment.

* Get rid of BinaryFormatter.

Nits.

* Move Ptc.LoadTranslations().

Nits.

* Nits.

* Fixed corner cases (in case backup copies have to be used). Added save logs.

* Not core fixes.

* Complement to the previous commit. Added load logs. Removed BinaryFormatter leftovers.

* Add LoadTranslations log.

* Nits.

* Removed the search and management of LowCq overlapping functions.

* Final increment of .info and .cache flags.

* Nit.

* Free up memory allocated by Pools during any PPTC translations at boot time.

* Nit due to rebase.
2020-12-24 03:58:36 +01:00
LDj3SNuD
8a33e884f8
Fix Vnmls_S fast path (F64: losing input d value). Fix Vnmla_S & Vnmls_S slow paths (using fused inst.s). Fix Vfma_V slow path not using StandardFPSCRValue(). (#1775)
* Fix Vnmls_S fast path (F64: losing input d value). Fix Vnmla_S & Vnmls_S slow paths (using fused inst.s).

Add Vfma_S & Vfms_S Fma fast paths.
Add Vfnma_S inst. with Fma/Sse fast paths and slow path.
Add Vfnms_S Sse fast path.

Add Tests for affected inst.s.

Nits.

* InternalVersion = 1775

* Nits.

* Fix Vfma_V slow path not using StandardFPSCRValue().

* Nit: Fix Vfma_V order.

* Add Vfms_V Sse fast path and slow path.

* Add Vfma_V and Vfms_V Test.
2020-12-17 20:43:41 +01:00
LDj3SNuD
b5c215111d
PPTC Follow-up. (#1712)
* Added support for offline invalidation, via PPTC, of low cq translations replaced by high cq translations; both on a single run and between runs.

Added invalidation of .cache files in the event of reuse on a different user operating system.

Added .info and .cache files invalidation in case of a failed stream decompression.

Nits.

* InternalVersion = 1712;

* Nits.

* Address comment.

* Get rid of BinaryFormatter.

Nits.

* Move Ptc.LoadTranslations().

Nits.

* Nits.

* Fixed corner cases (in case backup copies have to be used). Added save logs.

* Not core fixes.

* Complement to the previous commit. Added load logs. Removed BinaryFormatter leftovers.

* Add LoadTranslations log.

* Nits.

* Removed the search and management of LowCq overlapping functions.

* Final increment of .info and .cache flags.

* Nit.

* GetIndirectFunctionAddress(): Validate that writing actually takes place in dynamic table memory range (and not elsewhere).

* Fix Ptc.UpdateInfo() due to rebase.

* Nit for retrigger Checks.

* Nit for retrigger Checks.
2020-12-17 20:32:09 +01:00