Add the 'desynced GPU thread' mode.

It's a relatively big commit (less big with -w), but it's hard to test
any of this separately...

The basic problem is that in netplay or movies, the state of the CPU
must be deterministic, including when the game receives notification
that the GPU has processed FIFO data.  Dual core mode notifies the game
whenever the GPU thread actually gets around to doing the work, so it
isn't deterministic.  Single core mode is because it notifies the game
'instantly' (after processing the data synchronously), but it's too slow
for many systems and games.

My old dc-netplay branch worked as follows: everything worked as normal
except the state of the CP registers was a lie, and the CPU thread only
delivered results when idle detection triggered (waiting for the GPU if
they weren't ready at that point).  Usually, a game is idle iff all the
work for the frame has been done, except for a small amount of work
depending on the GPU result, so neither the CPU or the GPU waiting on
the other affected performance much.  However, it's possible that the
game could be waiting for some earlier interrupt, and any of several
games which, for whatever reason, never went into a detectable idle
(even when I tried to improve the detection) would never receive results
at all.  (The current method should have better compatibility, but it
also has slightly higher overhead and breaks some other things, so I
want to reimplement this, hopefully with less impact on the code, in the
future.)

With this commit, the basic idea is that the CPU thread acts as if the
work has been done instantly, like single core mode, but actually hands
it off asynchronously to the GPU thread (after backing up some data that
the game might change in memory before it's actually done).  Since the
work isn't done, any feedback from the GPU to the CPU, such as real
XFB/EFB copies (virtual are OK), EFB pokes, performance queries, etc. is
broken; but most games work with these options disabled, and there is no
need to try to detect what the CPU thread is doing.

Technically: when the flag g_use_deterministic_gpu_thread (currently
stuck on) is on, the CPU thread calls RunGpu like in single core mode.
This function synchronously copies the data from the FIFO to the
internal video buffer and updates the CP registers, interrupts, etc.
However, instead of the regular ReadDataFromFifo followed by running the
opcode decoder, it runs ReadDataFromFifoOnCPU ->
OpcodeDecoder_Preprocess, which relatively quickly scans through the
FIFO data, detects SetFinish calls etc., which are immediately fired,
and saves certain associated data from memory (e.g. display lists) in
AuxBuffers (a parallel stream to the main FIFO, which is a bit slow at
the moment), before handing the data off to the GPU thread to actually
render.  That makes up the bulk of this commit.

In various circumstances, including the aforementioned EFB pokes and
performance queries as well as swap requests (i.e. the end of a frame -
we don't want the CPU potentially pumping out frames too quickly and the
GPU falling behind*), SyncGPU is called to wait for actual completion.

The overhead mainly comes from OpcodeDecoder_Preprocess (which is,
again, synchronous), as well as the actual copying.

Currently, display lists and such are escrowed from main memory even
though they usually won't change over the course of a frame, and
textures are not even though they might, resulting in a small chance of
graphical glitches.  When the texture locking (i.e. fault on write) code
lands, I can make this all correct and maybe a little faster.

* This suggests an alternate determinism method of just delaying results
until a short time before the end of each frame.  For all I know this
might mostly work - I haven't tried it - but if any significant work
hinges on the competion of render to texture etc., the frame will be
missed.

Source/Core/VideoCommon/OpcodeDecoding.cpp

@@ -24,6 +24,7 @@
 #include "VideoCommon/DataReader.h"
 #include "VideoCommon/Fifo.h"
 #include "VideoCommon/OpcodeDecoding.h"
+#include "VideoCommon/PixelEngine.h"
 #include "VideoCommon/Statistics.h"
 #include "VideoCommon/VertexLoaderManager.h"
 #include "VideoCommon/VideoCommon.h"
@@ -36,7 +37,12 @@ bool g_bRecordFifoData = false;
 static u32 InterpretDisplayList(u32 address, u32 size)
 {
 	u8* old_pVideoData = g_video_buffer_read_ptr;
-	u8* startAddress = Memory::GetPointer(address);
+	u8* startAddress;
+
+	if (g_use_deterministic_gpu_thread)
+		startAddress = (u8*) PopFifoAuxBuffer(size);
+	else
+		startAddress = Memory::GetPointer(address);
 
 	u32 cycles = 0;
 
@@ -62,11 +68,29 @@ static u32 InterpretDisplayList(u32 address, u32 size)
 	return cycles;
 }
 
+static void InterpretDisplayListPreprocess(u32 address, u32 size)
+{
+	u8* old_read_ptr = g_video_buffer_pp_read_ptr;
+	u8* startAddress = Memory::GetPointer(address);
+
+	PushFifoAuxBuffer(startAddress, size);
+
+	if (startAddress != nullptr)
+	{
+		g_video_buffer_pp_read_ptr = startAddress;
+
+		u8 *end = startAddress + size;
+		OpcodeDecoder_Preprocess(end);
+	}
+
+	g_video_buffer_pp_read_ptr = old_read_ptr;
+}
+
 static void UnknownOpcode(u8 cmd_byte, void *buffer, bool preprocess)
 {
 	// TODO(Omega): Maybe dump FIFO to file on this error
 	std::string temp = StringFromFormat(
-		"GFX FIFO: Unknown Opcode (0x%x @ %p).\n"
+		"GFX FIFO: Unknown Opcode (0x%x @ %p, preprocessing=%s).\n"
 		"This means one of the following:\n"
 		"* The emulated GPU got desynced, disabling dual core can help\n"
 		"* Command stream corrupted by some spurious memory bug\n"
@@ -74,7 +98,8 @@ static void UnknownOpcode(u8 cmd_byte, void *buffer, bool preprocess)
 		"* Some other sort of bug\n\n"
 		"Dolphin will now likely crash or hang. Enjoy." ,
 		cmd_byte,
-		buffer);
+		buffer,
+		preprocess ? "yes" : "no");
 	Host_SysMessage(temp.c_str());
 	INFO_LOG(VIDEO, "%s", temp.c_str());
 	{
@@ -104,14 +129,16 @@ static void UnknownOpcode(u8 cmd_byte, void *buffer, bool preprocess)
 	}
 }
 
+template <bool is_preprocess, u8** bufp>
 static u32 Decode(u8* end)
 {
-	u8 *opcodeStart = g_video_buffer_read_ptr;
-	if (g_video_buffer_read_ptr == end)
+	u8 *opcodeStart = *bufp;
+	if (*bufp == end)
 		return 0;
 
-	u8 cmd_byte = DataReadU8();
+	u8 cmd_byte = DataRead<u8>(bufp);
 	u32 cycles;
+	int refarray;
 	switch (cmd_byte)
 	{
 	case GX_NOP:
@@ -120,64 +147,72 @@ static u32 Decode(u8* end)
 
 	case GX_LOAD_CP_REG: //0x08
 		{
-			if (end - g_video_buffer_read_ptr < 1 + 4)
+			if (end - *bufp < 1 + 4)
 				return 0;
 			cycles = 12;
-			u8 sub_cmd = DataReadU8();
-			u32 value = DataReadU32();
-			LoadCPReg(sub_cmd, value);
-			INCSTAT(stats.thisFrame.numCPLoads);
+			u8 sub_cmd = DataRead<u8>(bufp);
+			u32 value = DataRead<u32>(bufp);
+			LoadCPReg(sub_cmd, value, is_preprocess);
+			if (!is_preprocess)
+				INCSTAT(stats.thisFrame.numCPLoads);
 		}
 		break;
 
 	case GX_LOAD_XF_REG:
 		{
-			if (end - g_video_buffer_read_ptr < 4)
+			if (end - *bufp < 4)
 				return 0;
-			u32 Cmd2 = DataReadU32();
+			u32 Cmd2 = DataRead<u32>(bufp);
 			int transfer_size = ((Cmd2 >> 16) & 15) + 1;
-			if ((size_t) (end - g_video_buffer_read_ptr) < transfer_size * sizeof(u32))
+			if ((size_t) (end - *bufp) < transfer_size * sizeof(u32))
 				return 0;
 			cycles = 18 + 6 * transfer_size;
-			u32 xf_address = Cmd2 & 0xFFFF;
-			LoadXFReg(transfer_size, xf_address);
+			if (!is_preprocess)
+			{
+				u32 xf_address = Cmd2 & 0xFFFF;
+				LoadXFReg(transfer_size, xf_address);
 
-			INCSTAT(stats.thisFrame.numXFLoads);
+				INCSTAT(stats.thisFrame.numXFLoads);
+			}
+			else
+			{
+				*bufp += transfer_size * sizeof(u32);
+			}
 		}
 		break;
 
 	case GX_LOAD_INDX_A: //used for position matrices
-		if (end - g_video_buffer_read_ptr < 4)
-			return 0;
-		cycles = 6;
-		LoadIndexedXF(DataReadU32(), 0xC);
-		break;
+		refarray = 0xC;
+		goto load_indx;
 	case GX_LOAD_INDX_B: //used for normal matrices
-		if (end - g_video_buffer_read_ptr < 4)
-			return 0;
-		cycles = 6;
-		LoadIndexedXF(DataReadU32(), 0xD);
-		break;
+		refarray = 0xD;
+		goto load_indx;
 	case GX_LOAD_INDX_C: //used for postmatrices
-		if (end - g_video_buffer_read_ptr < 4)
-			return 0;
-		cycles = 6;
-		LoadIndexedXF(DataReadU32(), 0xE);
-		break;
+		refarray = 0xE;
+		goto load_indx;
 	case GX_LOAD_INDX_D: //used for lights
-		if (end - g_video_buffer_read_ptr < 4)
+		refarray = 0xF;
+		goto load_indx;
+	load_indx:
+		if (end - *bufp < 4)
 			return 0;
 		cycles = 6;
-		LoadIndexedXF(DataReadU32(), 0xF);
+		if (is_preprocess)
+			PreprocessIndexedXF(DataRead<u32>(bufp), refarray);
+		else
+			LoadIndexedXF(DataRead<u32>(bufp), refarray);
 		break;
 
 	case GX_CMD_CALL_DL:
 		{
-			if (end - g_video_buffer_read_ptr < 8)
+			if (end - *bufp < 8)
 				return 0;
-			u32 address = DataReadU32();
-			u32 count = DataReadU32();
-			cycles = 6 + InterpretDisplayList(address, count);
+			u32 address = DataRead<u32>(bufp);
+			u32 count = DataRead<u32>(bufp);
+			if (is_preprocess)
+				InterpretDisplayListPreprocess(address, count);
+			else
+				cycles = 6 + InterpretDisplayList(address, count);
 		}
 		break;
 
@@ -195,12 +230,19 @@ static u32 Decode(u8* end)
 		// In skipped_frame case: We have to let BP writes through because they set
 		// tokens and stuff.  TODO: Call a much simplified LoadBPReg instead.
 		{
-			if (end - g_video_buffer_read_ptr < 4)
+			if (end - *bufp < 4)
 				return 0;
 			cycles = 12;
-			u32 bp_cmd = DataReadU32();
-			LoadBPReg(bp_cmd);
-			INCSTAT(stats.thisFrame.numBPLoads);
+			u32 bp_cmd = DataRead<u32>(bufp);
+			if (is_preprocess)
+			{
+				LoadBPRegPreprocess(bp_cmd);
+			}
+			else
+			{
+				LoadBPReg(bp_cmd);
+				INCSTAT(stats.thisFrame.numBPLoads);
+			}
 		}
 		break;
 
@@ -210,33 +252,43 @@ static u32 Decode(u8* end)
 		{
 			cycles = 1600;
 			// load vertices
-			if (end - g_video_buffer_read_ptr < 2)
+			if (end - *bufp < 2)
 				return 0;
-			u16 numVertices = DataReadU16();
+			u16 num_vertices = DataRead<u16>(bufp);
 
-			if (!VertexLoaderManager::RunVertices(
-				cmd_byte & GX_VAT_MASK,   // Vertex loader index (0 - 7)
-				(cmd_byte & GX_PRIMITIVE_MASK) >> GX_PRIMITIVE_SHIFT,
-				numVertices,
-				end - g_video_buffer_read_ptr,
-				g_bSkipCurrentFrame))
+			if (is_preprocess)
 			{
-				return 0;
+				size_t size = num_vertices * VertexLoaderManager::GetVertexSize(cmd_byte & GX_VAT_MASK, is_preprocess);
+				if ((size_t) (end - *bufp) < size)
+					return 0;
+				*bufp += size;
+			}
+			else
+			{
+				if (!VertexLoaderManager::RunVertices(
+					cmd_byte & GX_VAT_MASK,   // Vertex loader index (0 - 7)
+					(cmd_byte & GX_PRIMITIVE_MASK) >> GX_PRIMITIVE_SHIFT,
+					num_vertices,
+					end - *bufp,
+					g_bSkipCurrentFrame))
+					return 0;
 			}
 		}
 		else
 		{
-			UnknownOpcode(cmd_byte, opcodeStart, false);
+			UnknownOpcode(cmd_byte, opcodeStart, is_preprocess);
 			cycles = 1;
 		}
 		break;
 	}
 
 	// Display lists get added directly into the FIFO stream
-	if (g_bRecordFifoData && cmd_byte != GX_CMD_CALL_DL)
-		FifoRecorder::GetInstance().WriteGPCommand(opcodeStart, u32(g_video_buffer_read_ptr - opcodeStart));
+	if (!is_preprocess && g_bRecordFifoData && cmd_byte != GX_CMD_CALL_DL)
+		FifoRecorder::GetInstance().WriteGPCommand(opcodeStart, u32(*bufp - opcodeStart));
 
-	return cycles;
+	// In is_preprocess mode, we don't actually care about cycles, at least for
+	// now... make sure the compiler realizes that.
+	return is_preprocess ? 1 : cycles;
 }
 
 void OpcodeDecoder_Init()
@@ -255,7 +307,7 @@ u32 OpcodeDecoder_Run(u8* end)
 	while (true)
 	{
 		u8* old = g_video_buffer_read_ptr;
-		u32 cycles = Decode(end);
+		u32 cycles = Decode</*is_preprocess*/ false, &g_video_buffer_read_ptr>(end);
 		if (cycles == 0)
 		{
 			g_video_buffer_read_ptr = old;
@@ -265,3 +317,17 @@ u32 OpcodeDecoder_Run(u8* end)
 	}
 	return totalCycles;
 }
+
+void OpcodeDecoder_Preprocess(u8 *end)
+{
+	while (true)
+	{
+		u8* old = g_video_buffer_pp_read_ptr;
+		u32 cycles = Decode</*is_preprocess*/ true, &g_video_buffer_pp_read_ptr>(end);
+		if (cycles == 0)
+		{
+			g_video_buffer_pp_read_ptr = old;
+			break;
+		}
+	}
+}