Commit Graph

2159 Commits

Author SHA1 Message Date
PixelyIon
f882b613bc Fix .hook section being allocated without any hooked symbols
Due to the trampoline and save/load context functions, `GetHookSectionSize` returned a non-zero size for when there were no hooked symbols supplied to it. This is problematic as it isn't required and hooking is currently not stable so it can lead to crashes or freezes in certain titles.
2023-01-11 00:13:15 +05:30
PixelyIon
3fa314f6cb Always print thread IDs rather than handles for SVC logs
Handles are rather arbitrary and difficult to reference, as a result, we've moved to thread IDs across the board for logs.
2023-01-11 00:13:15 +05:30
PixelyIon
e192d4e5c1 Warn when RemoveThread is called on a non-inserted thread 2023-01-11 00:13:15 +05:30
PixelyIon
3a6f205e6f Clear insertThreadOnResume in RemoveThread
A thread can be paused while it is in a synchronization primitive which will do `RemoveThread`, we need to update the state of `insertThreadOnResume` in this case by clearing it so it isn't incorrectly reinserted on resuming the thread.
2023-01-11 00:13:15 +05:30
PixelyIon
7fef849594 Make UpdateCore's locking coreMigrationMutex requirement explicit
`Scheduler::UpdateCore` implicitly depended on `KThread::coreMigrationMutex` being locked during calls to it, this requirement has now been made explicit to avoid confusion.
2023-01-11 00:13:15 +05:30
PixelyIon
c4b4532222 Check waitThread rather than waitMutex during condvar timeouts
When a timeout occurs in `ConditionVariableWait`, we used to check `waitMutex` which is cleared by `MutexUnlock` but when we hit the CAS case in `ConditionVariableSignal` then we don't clear `waitMutex`. It's far more reliable to check `waitThread` as an indication for if the thread has already been unlocked as it's cleared at the start of `ConditionVariableWait` and would implicitly stay cleared in the CAS case while being set in `MutexLock` and being unset in `MutexUnlock`.
2023-01-11 00:13:15 +05:30
PixelyIon
2525bafe06 Consolidate thread yielding in Scheduler
There's multiple locations where a thread is yielded in the scheduler and all of them repeat the code of checking for `pendingYield` and signalling with an optional optimization of checking if the thread being yielded is the calling thread.

All this functionality has now been consolidated into `Scheduler::YieldThread` which checks for `pendingYield` and does the calling thread yield optimization. This should lead to better readability and better performance in cases where `UpdatePriority` would signal the calling thread.
2023-01-11 00:13:15 +05:30
PixelyIon
8b973a3de3 Always set forceYield for running threads in PauseThread
`forceYield` was incorrectly not set when pausing running threads if the thread already had `pendingYield` set. This could lead to cases where `Rotate` would later throw an exception due to it being unset.
2023-01-11 00:13:15 +05:30
PixelyIon
6645692288 Don't block while inserting paused threads
Blocking while inserting a paused thread can lead to deadlocks where the inserting thread later resumes the paused thread.

Co-authored-by: Billy Laws <blaws05@gmail.com>
2023-01-11 00:13:15 +05:30
Billy Laws
643f4cf864 Ensure thread doesn't migrate during InsertThread
As we didn't hold `coreMigrationMutex`, the thread could simply migrate during `InsertThread` which would lead to the thread potentially never waking up as it's been inserted on a non-resident core.

Co-authored-by: PixelyIon <pixelyion@protonmail.com>
2023-01-11 00:13:15 +05:30
PixelyIon
7f7352ed59 Recalculate highest-priority waiters during cvar/address signaling
`SignalToAddress`/`ConditionVariableSignal` need to wake waiters in priority order, while threads are inserted in order this doesn't remain the case as priority updates don't reinsert the thread into `syncWaiters`. 

It was determined that reinsertion into `syncWaiters` would be fairly complex due to locking the `syncWaitersMutex` with the thread's mutexes. To avoid this, this commit instead sorts waiters by priority at signal time to always wake threads in the right order.
2023-01-11 00:13:15 +05:30
PixelyIon
626008d8e2 Fix WaitForAddress timeout mutex deadlock
Calling `WaitSchedule` inside the block where `syncWaiterMutex` is locked causes a race with other threads which lock the core mutex and `syncWaiterMutex` together. This commit moves the `WaitSchedule` outside the block while simply setting a flag to wait later similar to `ConditionVariableWait`'s timeout case.

Co-authored-by: Billy Laws <blaws05@gmail.com>
2023-01-11 00:13:15 +05:30
PixelyIon
4df3c98225 Add double-insertion debug check to InsertThread
This is a cause for a large amount of scheduler bugs so we should generally check for this on debug builds as it is a fairly easy way to check for issues for some performance cost.
2023-01-11 00:13:15 +05:30
PixelyIon
5694c9b34b Rename KThread::waitKey to KThread::waitMutex
It was determined that `waitKey` is too ambiguous when waiter members are used for both mutexes and condition variables.
2023-01-11 00:13:15 +05:30
PixelyIon
91bb8d231a Rename ConditionalVariable -> ConditionVariable
"Conditional Variable" is a typo which was propagated through the codebase, it has been corrected to "Condition Variable".
2023-01-11 00:13:15 +05:30
PixelyIon
f487d81769 Refactor Condition Variable Waiting/Signalling
The way we handled waking/timeouts of condition variables was fairly inaccurate to HOS as we moved locking of the mutex to the waker thread which could change the order of operations and would cause what were functionally spurious wakeups for all awoken threads.

This commit fixes it by doing all locks on the waker thread and only awakening the waiter thread once the condition variable was signalled and the mutex was unlocked. In addition, this fixes races between a timeout and a signal that could lead to double-insertion as a result of a refactor of how timeouts work in the new system.
2023-01-11 00:13:15 +05:30
PixelyIon
1eb4eec103 Allow locking external thread in MutexLock
We want the ability to lock mutexes on behalf of other threads to refactor condition variables to match HOS on waking behavior.
2023-01-11 00:13:15 +05:30
PixelyIon
6bbe9de881 Fix result returned by MutexLock
`MutexLock` incorrectly returned `InvalidCurrentMemory` for cases where the userspace value didn't match the expected value. It's been corrected to return no error in those cases while preserving the error code for usage in `ConditionalVariableWait`.
2023-01-11 00:13:15 +05:30
PixelyIon
08ef88b156 Add early-timeout path for WaitForAddress 2023-01-11 00:13:15 +05:30
PixelyIon
d0c56235f4 Read address atomically in WaitForAddress
We didn't read the values for arbitration atomically in all cases as we should have, this consolidates the reading of the value and uses the value across all cases.
2023-01-11 00:13:15 +05:30
PixelyIon
e8a1bd1aad Fix WaitForAddress timeout signal race
A race could occur from the timeout path in `WaitForAddress` taking place at the same time as `SignalToAddress` has been caused, this causes a deadlock due to double-insertion.
2023-01-11 00:13:15 +05:30
Billy Laws
0f1d97fe2c Update edge supporter names 2023-01-08 21:35:14 +00:00
Billy Laws
31fb6d30eb Fake maxwell occlusion query results 2023-01-08 19:30:52 +00:00
Billy Laws
a92c26531e Keep holes in descriptors for unsupported bindings 2023-01-08 19:30:52 +00:00
Billy Laws
81d82008c7 Pre-signal suspend ticks event 2023-01-08 19:30:52 +00:00
Billy Laws
3e5992e366 Update hades 2023-01-08 19:30:52 +00:00
Billy Laws
45bbf3bb2a Fix indirect draws with direct buffers
We need to wait on the GPFIFO manually as we won't hit the traps when accesing the indirect params with direct as we usually would.
2023-01-08 19:30:52 +00:00
Billy Laws
68ad052cb1 Add geometry passthrough shader support for vertex layer writes 2023-01-08 19:30:52 +00:00
Billy Laws
ec519a7d52 Return null texture on encountering unmapped textures 2023-01-08 19:30:52 +00:00
Billy Laws
97e127153b Make shader trap mutex recursive
There are cases there we hit a shader trap within the GPU, by making it recursive we avoid deadlocking on reads within the GPU.
2023-01-08 19:30:52 +00:00
Billy Laws
1a6165f74d Fix GetReadOnlyBackingSpan for non-direct buffers
This was missed in the initial implementation
2023-01-08 19:30:52 +00:00
Billy Laws
4e5141f879 Fix missed attempt increment in spinlock
Should hog CPU slightly less and correctly yield now
2023-01-08 19:30:52 +00:00
Billy Laws
35a46acbb1 Determine storage buffer alignment dynamically 2023-01-08 19:30:52 +00:00
Billy Laws
12d80fe6c2 Use a shared mutex for GPU VMM to avoid deadlocks
Two reads need to be able to occur simultanously or deadlocks ccan occur (e.g read traps to wait on GPU but GPU needs to read).
2023-01-08 19:30:52 +00:00
Billy Laws
28b2a7a8a1 Dynamically apply GPU turbo clocks only when GPU submissions are queued
Allows for the GPU to clock down in cases where it's idle for most of the time, while still forcing maximum clocks when we care.
2023-01-08 19:30:52 +00:00
Billy Laws
81f3ff348c Transition memory handling from memfd to anonymous shared mappings
Memfd mappings are incompatible with KGSL user memory importing on older kernels, transition to shared anon mappings to avoid this.
2023-01-08 19:30:52 +00:00
Billy Laws
cc3c869b9f Attempt to signal the vsync event at present time if possible
Some games rely on the vsync event to schedule frames, by matching its timing with presentation we can reduce needless waiting as the game will immediely be able to queue the next frame after presentation.
2023-01-08 19:30:52 +00:00
Billy Laws
918a493a45 Implement wfi and setReference GPFIFO barriers 2023-01-08 19:30:52 +00:00
Billy Laws
7315ba04e6 Fixup optional flattenable binder obj structure 2023-01-08 19:30:52 +00:00
Billy Laws
90e21b0ca1 Split syncpoints into host-guest pairs
This allows for the presentation engine to grab the presentation image early when direct buffers are in use, since it'll handle sync on its own using semaphores it doesn't need to wait for GPU execution.
2023-01-08 19:30:52 +00:00
Billy Laws
966c31810a Return appropriate fences in surfaceflinger queue buffer 2023-01-08 19:30:52 +00:00
Billy Laws
afef6c5123 Always populate all colour attachments
This better follow the Vulkan spec, which doesn't mention anything about writes to OOB attachments, only those marked as unused.
2023-01-08 19:30:52 +00:00
Billy Laws
3571737392 Reset maxwell3d quick bind state before adding subpasses to executor
If a submission happens during the call to addsubpass we could end up with invalid quick bind state, move this to to before to prevent that.
2023-01-08 19:30:52 +00:00
Billy Laws
3d31ade35f Implement an alternative buffer path using direct memory importing
By importing guest memory directly onto the host GPU we can avoid many of the complexities that occur with memory tracking as well as the heavy performance overhead in some situations. Since it's still desired to support the traditional buffer method, as it's faster in some cases and more widely supported, most of the exposed buffer methods have been split into two variants with just a small amount of shared code. While in most cases the code is simpler, one area with more complexity is handling CPU accesses that need to be sequenced, since we don't have any place we can easily apply writes to on the GPFIFO thread that wont also impact the buffer on the GPU, to solve this, when the GPU is actively using a buffer's contents, an interval list is used to keep track of any GPFIO-written regions on the CPU and any CPU reads to them will instead be directed to a shadow of the buffer with just those writes applied. Once the GPU has finished using buffer contents the shadow can then be removed as all writes will have been done by the GPU.

The main caveat of this is that it requires tying host sync to guest sync, this can reduce performance in games which double buffer command buffers as it prevents us from fully saturating the CPU with the GPFIFO thread.
2023-01-08 19:30:52 +00:00
Billy Laws
b3f7e990cc Allow for tying guest GPU sync operations to host GPU sync
This is necessary for the upcoming direct buffer support, as in order to use guest buffers directly without trapping we need to recreate any guest GPU sync on the host GPU. This avoids the guest thinking work is done that isn't and overwriting in-use buffer contents.
2023-01-08 19:30:52 +00:00
Billy Laws
89c6fab1cb Implement a way to check if the command record thread is idle
Useful for debugging and testing
2023-01-08 19:30:52 +00:00
Billy Laws
c67f27e914 Add a setting to control the maximum number of accumulated GPU cmds
This helps to keep the GPU fed when processing large command buffers which don't have any syncpoints to force a flush inbetween.
2023-01-08 19:30:52 +00:00
Billy Laws
77214a98dd Add a setting to force maximum GPU clocks on KGSL devices 2023-01-08 19:30:52 +00:00
Billy Laws
83ecc33a77 Update adrenotools 2023-01-08 19:30:52 +00:00
Billy Laws
3ecaedd71e Add adrenotools direct mapping support 2023-01-08 19:30:52 +00:00