Commit Graph

425 Commits

Author SHA1 Message Date
PixelyIon
33f1a3e1b3 Optimize Service Function Lookup
This moves from using std::function with a this pointer binding (which would likely cause a heap allocation) to returning the this pointer in a structure which implements operator() to do the call with it. It also moves to using const char* for strings from std::string_view which was pointless in this scenario due to it's usage being limited to being a C-string for the most part, it also integrates the class name directly into the string which allows us to avoid runtime string concatenation in libfmt and RTTI for finding the class name.
2021-03-23 02:40:02 +05:30
PixelyIon
2b4a094cd8 Logger Function Prefix Support 2021-03-23 02:40:02 +05:30
Billy Laws
6c6e665569 Implement Perfetto Tracing
This commit implements tracing Skyline using https://perfetto.dev/ for SVCs, IPC, Scheduler and Presentation
2021-03-23 02:40:02 +05:30
PixelyIon
1155394d58 Improve KMemory + Fix Service Implementation Warning
* Improve KMemory Comments
* Add parameter prefix 'p-' to `KPrivateMemory::UpdatePermission`
* Fix the missing trailing double quote in missing service prints, this was due to `stringName` being padded with extra 0s
2021-03-23 02:40:02 +05:30
Billy Laws
d8d4b4c9e0 Address review feedback 2021-03-21 18:56:31 +05:30
Billy Laws
74cd0bc2c3 Various timesrv cleanups and comments
Mainly just adapts the rest of time to add some things missed in the
initial commit as they required TZ, everything else is just renames from
switchbrew and comments.
2021-03-21 18:56:31 +05:30
Billy Laws
ba418976b0 Implement full timezone service support
This serves as an extension to the initial time commit and combined
they provide a complete implementation of everything application facing
in time.

psc:ITimeZoneService and glue:ITimeZoneService are used to convert
between POSIX and calendar times according to the device location.
Timezone binaries are used during the conversion, details of them can
be read about in the previous commit.

This is based off my own glue RE and Thog's time RE.
2021-03-21 18:56:31 +05:30
Billy Laws
34cb0b49e8 Implement the entirety of time services
This reimplements our time backend to be significantly more accurate to
the real PSC and provides complete implementations for every time IPC
allowing many newer games to work properly.

Time is unique in its use of glue services, the core sysmodule is fully
isolated and doesn't interface with any other services. Glue is instead
used where that is needed (e.g. for fetching settings), this distinction
is also present in our implementation.

Another unique feature of time is its global state, as time is
calibrated from the start of the service its state cannot be lost as
that would result in the application offsetting time incorrectly
whenever it closed a session.

A large proportion of this is based off of Thog's 9.0.0 PSC reversing.
2021-03-21 18:56:31 +05:30
Billy Laws
cb9f5f144e Pull in tzcode submodule and add tzdata assets
These are used for timezone conversions between POSIX and calander time.

Tzdata is in exactly the same format as HOS to allow loading sysarchives
in the future if needed. See its README for more info.

Details on tzcode can be found in its own repo, there are several
changes done Vs the base release to allow for HOS compat.
2021-03-21 18:56:31 +05:30
Billy Laws
63c54207d2 Add AndroidAsset{Backing,FileSystem} for accessing built-in assets
These only implement the subset of VFS needed for time, implementing
more is difficult due to some issues in the AAsset API which make
support quite ugly. The abstract asset filesystem can be accessed by
services through the OS class allowing other implementations to be used
in the future.
2021-03-21 18:56:31 +05:30
Billy Laws
a0f6cc161c Add a UUID struct for holding and generating RFC4122 UUIDs
Time uses UUIDs for clock identification.
2021-03-21 18:56:31 +05:30
Billy Laws
937e3d6f68 Add endiannes swapping helper functions
Avoid using compiler functions manually to create a cleaner interface,
and allow swapping the endianness of arbritary lengths.
2021-03-21 18:56:31 +05:30
Billy Laws
def9f96b02 Add a ResultValue helper class to simplify result handling
This is a very common usecase for result handling and allows much
cleaner code.
2021-03-21 18:56:31 +05:30
PixelyIon
e90dd672af Fix Clock Rescaling Bug (Introduced in #135)
There was a mistake in the code-style refactor where the signature in the instruction encoding of `MRS` was set to `0xD54` instead of `0xD53` which would cause a SIGILL (Illegal Instruction) for devices which had their HW timer frequency equivalent to the Switch (19.2MHz) as a modified `MRS` would be deployed there. This issue should not affect devices which perform clock rescaling as the `MRS` instruction there is encoded by the assembler.
2021-03-10 15:13:57 +05:30
sspacelynx
75b769ca1d Add NSP ticket extraction support 2021-03-07 23:16:36 +05:30
sspacelynx
77d8d1bee1 Add support for XCI titlekey decryption 2021-03-07 22:55:59 +05:30
sspacelynx
666df1eb43 Add XCI file format support 2021-03-07 22:55:59 +05:30
Billy Laws
a8dafc10e0 Fix backing read size check
I buggered this one up and broke all reads during the refactor, so fix
that.
2021-03-06 20:29:55 +05:30
Billy Laws
6390561f0f Fixup VFS error checking
Many users of VFS didn't check for nullptr or 0 results leading to
various potential issues, to mitigate this introduce error checking to
VFS by default. The original variants can still be used through the
*Unchecked family of functions.
2021-03-06 18:58:04 +05:30
Billy Laws
00d0586d1f Correctly set size in CtrEncryptedBacking
This bug caused crashes with the improved error checking.
2021-03-06 18:58:04 +05:30
Billy Laws
5f942e2dff Split VFS implementations from API
This allows better validation and simplified default argument handling.
Could also be useful in the future when we switch to proper VFS error
reporting.
2021-03-06 18:58:04 +05:30
Billy Laws
48acb6d369 Address a bunch of issues detected by clang-tidy 2021-03-06 18:58:04 +05:30
Billy Laws
9d5138bef1 Support Ioctl3 without the inline output buffer
This is used by Project Diva for GET_VA_REGIONS
2021-03-05 23:54:32 +05:30
Billy Laws
c489da610e Add locking to GPU VMM and fix a few codestyle issues
As VMM can be accessed by nvdrv and the GPFIFO thread at the same
time locking is needed to prevent races.
2021-03-05 23:54:32 +05:30
Billy Laws
c1aec00ed1 Rework GPFIFO pushing to optimise performance and accuracy
* Pushbuffer data is now stored in a member buffer to avoid reallocating
  it for each pushbuffer which hampered performance before.
* Don't prefetch pushbuffers as it puts unnecessary load on the guest
  thread that is better suited for the GPFIFO thread.
* Clean up some misc code to avoid pointless casts of a 4 byte object
  and handle GPFIFO control opcodes.
2021-03-05 23:54:32 +05:30
Billy Laws
78cdb1eeb4 Add locking to nvhost-ctrl syncpoint events and sync with switchbrew
NvHostEvents were renamed to SyncpointEvents which is a much clearer
name that more accurately describes them. Locking is needed as IOCTLs
can be called asynchronously and so event registration and signalling
can race.
2021-03-05 23:54:32 +05:30
PixelyIon
c282276b74 Address CR Comments 2 (#132) + Fix Several Scheduler Bugs
The following scheduler bugs were fixed:
* It was assumed that all non-cooperative `Rotate` calls were from a preemptive yield and changed the state of `KThread::isPreempted` incorrectly which could lead to UB, an example of a scenario with it would be:
* * Preemptive thread A gets a signal to yield from cooperative thread B due to it being ready to schedule and higher priority
* * A complies with this request but there's an assumption that the signal was actually from it's preemption timer therefore it doesn't reset it (As it isn't required if the timer was responsible for the signal)
* * A receives the actual preemption signal a while later, causing UB as the signal handler is invoked twice
* `Scheduler::UpdatePriority`
* * A check for `currentIt == core->queue.begin()` existed which caused an incorrect early return 
* * The preemption timer was armed correctly when a priority transition from cooperative priority -> preemption priority occurred but not disarmed when a transition from preemption priority -> cooperative priority occurred
* * The timer was unnecessarily disarmed in the case of updating the priority of a non-running thread, this isn't as much a bug as it is just pointless
* Priority inheritance in `KProcess::MutexLock` is fundamentally broken as it performs UB with `waitThread` being accessed prior to being assigned
* When a thread sets its own priority using  `SvcSetThreadCoreMask` and its current core is no longer in the affinity mask, it wouldn't actually move to the new thread until the next time the thread is load balanced
2021-03-05 14:55:34 +05:30
PixelyIon
fe5061a8e0 Address CR Comments (#132) + Change Core Migration API
This addresses all CR comments including more codebase-wide changes arising from certain review comments like proper usage of its/it's and consistent contraction of it is into it's. 

An overhaul was made to the presentation and formatting of `KThread.h` and `LoadBalance` works has been superseded by `GetOptimalCoreForThread` which can be used alongside `InsertThread` or `MigrateToCore`. It makes the API far more atomic and neater. This was a major point of contention for the design prior, it's simplified some code and potentially improved performance.
2021-03-05 14:55:34 +05:30
PixelyIon
0ea02f2d56 Fix Non-Cooperative Core Migration + Fix yieldWithCoreMigration + Improve Mutex Locking in ConditionalVariableWait
The case of a thread not being in the core queue during a non-cooperative core affinity change would break things as the thread was non-conditionally removed and inserted, this has been fixed by adding a check to see if the thread exists in the core's queue prior to migration. In addition, `yieldWithCoreMigration` was broken by the previous commit as the fallthrough was intentional and removing it cause core migration without a yield which led to breakage in certain circumstances. The mutex locking logic was also improved in `ConditionalVariableWait` to use atomics in a more effective manner with less atomic operations being performed overall.
2021-03-05 14:55:34 +05:30
PixelyIon
0233916489 Increase Code Region Size to 4GiB
The code region's size was previously set at the same value as it is for 36-bit ASes, this value is inadequate for certain larger games and needed to be expanded. We've chosen 4GiB as the new value which should easily encompass all Switch games.
2021-03-05 14:55:34 +05:30
PixelyIon
11c5f50d37 Use likelihood attributes in NCE + Fix System Version + SVC improvements
The SVCs improvements are as follows:
* Make SVC logs more concise for:
* * `SleepThread`
* * `ClearEvent`
* * `CloseHandle`
* * `ResetSignal`
* * `WaitSynchronization` (Special case for single handle)
* * `ArbitrateLock`
* * `ArbitrateUnlock`
* * `WaitProcessWideKeyAtomic`
* * `SignalProcessWideKey`
* Fix unintentional fallthrough into `yieldWithoutCoreMigration` from `yieldWithCoreMigration` in `SleepThread`
* Return `result::InvalidState` when an unsignalled handle is reset in `ResetSignal`
* Return `Result{}` (Success) in `CancelSynchronization`
* Do not return `result::InvalidCurrentMemory` in `ArbitrateLock` as it's not a failure condition
* Make `count` in `WaitProcessWideKeyAtomic` a `i32` from a `u32`, zero and all negative values result in waking all waiters
2021-03-05 14:55:34 +05:30
PixelyIon
1884d98163 Implement Address Arbiter
The entirety of the address arbiter is implemented in this commit, all three arbitration types: `WaitIfLessThan`, `DecrementAndWaitIfLessThan` and `WaitIfEqual`, and all three signal types: `Signal`, `SignalAndIncrementIfEqual` and `SignalAndModifyBasedOnWaitingThreadCountIfEqual` have been implemented. 
This allows any application which uses levent (Light Events) to function which includes titles such as ARMS.
2021-03-05 14:55:34 +05:30
PixelyIon
20bdda6a63 Support Core Migration for Running External Thread
We did not support migration of threads which were running in a non-cooperative manner, this was partially due to the dependence on per-core conditional variables rather than per-thread which made this harder to do programmatically. This has been fixed by moving to per-thread cvars and therefore the limitation can be removed, this feature is used by Unity games.
2021-03-05 14:55:34 +05:30
◱ PixelyIon
861d7e9eb2 Fix SvcClearEvent resetting behaviour
SvcClearEvent previously set the `signalled` flag directly rather than
calling `ResetSignal`, which skipped the locking necessary to make it
globally visible. Switch it to use `ResetSignal` to fix this.
2021-03-05 14:55:34 +05:30
◱ PixelyIon
a621408b9c Move to using ASCII separators for Logger
We've moved to using RS and GS from ASCII as delimiters rather than
'\n' and '|', this allows more robust parsing and increases the
readability of the log files
2021-03-05 14:55:34 +05:30
◱ PixelyIon
31db70f1d4 Lock decryption in CtrEncryptedBacking
This prevents a race where two threads could read at the same time and
end up using the wrong IV leading to garbage data being read. This
caused crashes in several games including Celeste.
2021-03-05 14:55:34 +05:30
Billy Laws
ce0e032255 Avoid constantly signalling audren DSP done event
This was causing a significant amount of sched thrashing and pinning a
core to 100% as games constantly updated audren, now change it to only
signal on buffer release.
2021-03-05 14:55:34 +05:30
◱ PixelyIon
7e7a792dc5 Use per-thread scheduler condvars and clean up AS filters
Per-thread condvars previously caused issues due to an audren bug, now
that's fixed they have a clear performance benefit.
2021-03-05 14:55:34 +05:30
PixelyIon
2bbf526419 Fix Logger Settings + Use Java 8 + Update Kotlin + Extract Native SOs 2021-03-05 14:55:34 +05:30
◱ PixelyIon
80302cf1ad Update NDK, Gradle and dependencies + Improve Settings API + Migrate to PugiXML 2021-03-05 14:55:34 +05:30
◱ PixelyIon
1f48fdd4a5 Fix Thread Insertion Optimization + Revert Per-Thread Scheduler Conditions 2021-03-05 14:55:34 +05:30
◱ PixelyIon
d5d133372f Fix Clean Exiting + Optimize Core Queues + Optimize Thread Insertion + Implement HID SendVibrationValue 2021-03-05 14:55:34 +05:30
◱ PixelyIon
98b1fd9056 Optimize Scheduler/IPC/HID + Fix Various Bugs
* Optimize Scheduler With Per-Thread Scheduler Conditions
* Optimize IPC by yielding
* Optimize HID Vibration
* Fix Priority Inheritance
* Fix `KThread` Start/Kill/Signal Races 
* Fix `YieldPending` Races in `StartThread` & `SvcHandler` 
* Fix POSIX Time -> NN CalendarTime Conversion 
* Fix HID `TouchScreen`/`NPad` Activation
2021-03-05 14:55:34 +05:30
◱ PixelyIon
ef52e22cef Improve Synchronization SVCs + Fix TLS Page Allocation Race + Fix KProcess::GetHandle<KObject> 2021-03-05 14:55:34 +05:30
◱ PixelyIon
ebadc1d1e1 Generate Stack Traces + More Robust Terminate Handler + Exit Process on Signal in Guest 2021-03-05 14:55:34 +05:30
◱ PixelyIon
14dbb5305a Fix Priority Queue + Cooperative Yielding + Conditional Variable Timeouts 2021-03-05 14:55:34 +05:30
◱ PixelyIon
33bbfb9fb7 Implement Conditional Variables 2021-03-05 14:55:34 +05:30
◱ PixelyIon
7079f11add Implement PI-Mutexes + Optimize InsertThread 2021-03-05 14:55:34 +05:30
◱ PixelyIon
7ba7cd2394 Support Priority & Affinity Mask Changes 2021-03-05 14:55:34 +05:30
◱ PixelyIon
f41bcd1e22 Implement Preemptive Scheduling 2021-03-05 14:55:34 +05:30
◱ PixelyIon
cf000f5750 Implement Cooperative Scheduling With Load Balancing 2021-03-05 14:55:34 +05:30
◱ PixelyIon
8564edcb16 Address CR Comments #2 2020-11-23 11:44:43 +05:30
◱ PixelyIon
fbf9f06244 Address CR Comments + Fix Clock Rescaling 2020-11-22 23:56:17 +05:30
◱ PixelyIon
a3dd759a1c Fix Circular Queue Destructor + Memory Infos + Improve Priority Documentation 2020-11-22 23:56:17 +05:30
Billy Laws
7167393e3c Only initialise maxEntry for active input devices
This caused the menus in Sonic Mania to be nonfunctional, futhermore,
default init is not ran for the input structs so the default max
definition in CommonHeader never actually applied.
2020-11-22 23:56:17 +05:30
◱ PixelyIon
bc5c094860 Fix CircularQueue + NPDM ACI0 & Kernel Capability Parsing 2020-11-22 23:56:17 +05:30
◱ PixelyIon
a5927c5c7b Fix Native-Initiated Exiting 2020-11-22 23:56:17 +05:30
◱ PixelyIon
d155e9cd71 Complete Exceptional Signal Handler Implementation + Fix More Destruction Behavior 2020-11-22 23:56:17 +05:30
Billy Laws
8bf08ed66f Fix CircularQueue and improve debug logging + exefs loading
CircularQueue was looping around too early resulting in the wrong
pushbuffers being used. The debug logging is useful for interpreting the
GPU method call logs.

Exefs loading was changed to check if an NSO exists before trying to
read it, preventing exceptions that get annoying while debugging.
2020-11-22 23:56:17 +05:30
Billy Laws
c161ef0cac Various accuracy improvements in services
* 'Fix' memory accounting to not measure reserved regions
* Fix some copy bugs introduced by switch to span
* Correct remap the behaviour of Modify so it actually works
2020-11-22 23:56:17 +05:30
Billy Laws
c7e5202042 Rework GPU VMM variable naming 2020-11-22 23:56:17 +05:30
Billy Laws
4c9d453008 Update formatter config for new AS and reformat 2020-11-22 23:56:17 +05:30
◱ PixelyIon
668f623256 Implement Exceptional Signal Handler + Fix Destruction Behavior
An exceptional signal handler allows us to convert an OS signal into a C++ exception, this allows us to alleviate a lot of crashes that would otherwise occur from signals being thrown during execution of games and be able to handle them gracefully.
2020-11-22 23:56:17 +05:30
◱ PixelyIon
3cde568c51 Run Guest on Main Emulator Thread + Remove Mutex/GroupMutex + Introduce PresentationEngine 2020-11-22 23:56:17 +05:30
◱ PixelyIon
779884edcf Introduce TID to Logger + Fix VMM Bug + Fix NSO Backtrace + Improve Logger 2020-11-22 23:56:17 +05:30
Billy Laws
39f0345ac7 Fix bugs introduced by refactoring
GPU VMM was mistakenly checking the alignment of the PA rather than the
VA and NvMap::Free was not accounting for the handle index starting from
one.
2020-11-22 23:56:17 +05:30
◱ PixelyIon
c65c91e1bc Implement NPDM, Core Mask SVCs + Fix VMM bug + Introduce Verbose Log Level 2020-11-22 23:56:17 +05:30
◱ PixelyIon
324381908b Implement SvcMap/UnmapPhysicalMemory + Fix W-Register Writing + Improve Accuracy of SvcGetInfo 2020-11-22 23:56:17 +05:30
◱ PixelyIon
657beea070 JVM Auto-Attach + Fix Thread Exiting + Fix Thread Signal Handler 2020-11-22 23:56:17 +05:30
◱ PixelyIon
6f2cd41470 Move .patch to start of executable (Pre-Patching) 2020-11-22 23:56:17 +05:30
◱ PixelyIon
369bd469f6 Dynamic Guest Memory Base Allocation 2020-11-22 23:56:17 +05:30
◱ PixelyIon
cffbfc8034 Skip Saving Callee-Saved Registers + Fix GetMemoryObject 2020-11-22 23:56:17 +05:30
Billy Laws
31efb5e930 Correct an NSO loader bug and use the correct address space extents 2020-11-22 23:56:17 +05:30
Billy Laws
17feb68eb5 Fix audren voice sample copying 2020-11-22 23:56:17 +05:30
◱ PixelyIon
745ea19f42 Use u64s for FmtCast + Remove Functional Casts + Fix VMM Bugs 2020-11-22 23:56:17 +05:30
◱ PixelyIon
1db76dee1e NCE3: In-Process Guest Execution 2020-11-22 23:56:17 +05:30
◱ PixelyIon
90127740f0 Rework NCE/KThread/KProcess + Remove Guest Process 2020-11-22 23:56:17 +05:30
◱ PixelyIon
878cb24389 Make fmt::ptr implicit for raw pointers + Update Submodules 2020-11-22 23:56:17 +05:30
◱ PixelyIon
02f3e37c4f Remove KProcess Memory Functions 2020-11-22 23:56:17 +05:30
◱ PixelyIon
60e82e6af0 Rework VMM + Adapt KMemory Objects to be in-process
Note: This commit isn't functional on it's own, it will require the rest of NCE3 to work
2020-11-22 23:56:17 +05:30
Willi Ye
7b13f2d387 Add visibility toggle for osc 2020-11-12 22:19:55 +05:30
Willi Ye
f479aeb4ac Add comments 2020-11-12 22:19:55 +05:30
Willi Ye
7526a985fb Use property delegate to handle preferences
* Add option to disable joystick recentering
2020-11-12 22:19:55 +05:30
Willi Ye
5c4aa95da6 Add on screen controls layout edit settings 2020-11-12 22:19:55 +05:30
Willi Ye
22140defae Rewrite adapter to handle any layout 2020-11-12 22:19:55 +05:30
Willi Ye
e023dbbf0a Add joystick press and general clean up 2020-11-12 22:19:55 +05:30
Willi Ye
3057e4b29a Add on screen controls
* Fix missing default constructor for dialog fragments
2020-11-12 22:19:55 +05:30
Billy Laws
85d5dd3619
Extend NvServices and implement IDirectory (#107)
* Fix alignment handling in NvHostAsGpu::AllocSpace

* Implement Ioctl{2,3} ioctls

These were added in HOS 3.0.0 in order to ease handling ioctl buffers.

* Introduce support for GPU address space remapping

* Fix nvdrv and am service bugs

Syncpoints are supposed to be allocated from ID 1, they were allocated
at 0 before. The ioctl functions were also missing from the service map

* Fix friend:u service name

* Stub NVGPU_IOCTL_CHANNEL_SET_TIMESLICE

* Stub IManagerForApplication::CheckAvailability

* Add OsFileSystem Directory support and add a size field to directory entries

The size field will be needed by the incoming HOS IDirectory support.

* Implement support for IDirectory

This is used by applications to list the contents of a directory.

* Address feedback
2020-11-03 15:10:42 +05:30
◱ PixelyIon
7ad86ec46f Improve span::as_string and other minor fixes 2020-10-02 15:28:48 +00:00
◱ PixelyIon
97ac45d83b Update Logger to use NDK Logger APIs + Improve Backing API + Fix FDSAN issues 2020-10-02 15:28:48 +00:00
◱ PixelyIon
4070686897 Refactor Comments + Other Minor Fixes 2020-10-02 15:28:48 +00:00
◱ PixelyIon
429af1990a Equal -> Brace Initializer + Remove Constexpr Auto for Integers 2020-10-02 15:28:48 +00:00
◱ PixelyIon
2764bd7c96 Use Vector for Kernel Handles + Remove Redundant Includes 2020-10-02 15:28:48 +00:00
◱ PixelyIon
20559c5dca Introduce Custom Span Class + IPC Buffer -> Span 2020-10-02 15:28:48 +00:00
◱ PixelyIon
4d6ae9aa26 Constexpr Maps for Service Functions 2020-10-02 15:28:48 +00:00
◱ PixelyIon
157c54f918 Implement a few HID Functions + Fix FAB handler 2020-10-02 15:28:48 +00:00
◱ PixelyIon
4970e58999 Address CR Comments (#102) 2020-09-20 20:07:33 +00:00
◱ PixelyIon
bb2c31264d Implement IOCTL2 & IOCTL3 2020-09-20 20:07:33 +00:00
◱ PixelyIon
a5fece8020 NVDRV IOCTL Refactor
Buffer -> Span + All buffers as arguments + Return -> NvStatus + Print Service Names + Function Names
2020-09-20 20:07:33 +00:00
◱ PixelyIon
70d67ef563 Constexpr Maps for NvDevice IOCTLs 2020-09-20 20:07:33 +00:00