fusee-launcher/report/fusee_gelee.md

## Vulnerability Disclosure: Fusée Gelée

This report documents Fusée Gelée, a coldboot vulnerability that allows full,
unauthenticated arbitrary code execution from an early bootROM context via Tegra
Recovery Mode (RCM) on NVIDIA's Tegra line of embedded processors.  As this
vulnerability allows arbitrary code execution on the Boot and Power Management
Processor (BPMP) before any lock-outs take effect, this vulnerability compromises
the entire root-of-trust for each processor, and allows exfiltration of secrets
e.g. burned into device fuses.

 Quick vitals:                       | |
--------------------|--------------------------------------------------------
 *Reporter:*      | Katherine Temkin (@ktemkin)
 *Affiliation:*   | ReSwitched (https://reswitched.tech)
 *E-mail:*        | k@ktemkin.com
 *Affects:*       | Tegra SoCs, independent of software stack
 *Versions:*      | believed to affect Tegra SoCs released prior to the T186 / X2
 *Impact:*        | early bootROM code execution with no software requirements, which can lead to full compromise of on-device secrets where USB access is possible
 *Disclosure*     | public disclosure planned for June 15th, 2018

#### Vulnerability Summary

The USB software stack provided inside the boot instruction rom (IROM/bootROM)
contains a copy operation whose length can be controlled by an attacker. By
carefully constructing a USB control request, an attacker can leverage this
vulnerability to copy the contents of an attacker-controlled buffer over the
active execution stack, gaining control of the Boot and Power Management
processor (BPMP) before any lock-outs or privilege reductions occur. This
execution can then be used to exfiltrate secrets and to load arbitrary code onto
the main CPU Complex (CCPLEX) "application processors" at the highest possible
level of privilege (typically as the TrustZone Secure Monitor at PL3/EL3).


#### Public Disclosure Notice

This vulnerability is notable due to the significant number and variety of
devices affected, the severity of the issue, and the immutability of the relevant
code on devices already delivered to end users. This vulnerability report
is provided as a courtesy to help aid remediation efforts, guide communication,
and minimize impact to users.

As other groups appear to have this or an equivalent exploit--
[including a group who claims they will be selling access to an implementation of such an exploit](http://team-xecuter.com/team-xecuter-coming-to-your-nintendo-switch-console/)--
it is the author and the ReSwitched team's belief that prompt public disclosure
best serves the public interest. By minimizing the information asymmetry between
the general public and exploit-holders and notifying the public, users will be
able to best assess how this vulnerability impacts their personal threat models.

Accordingly, ReSwitched anticipates public disclosure of this vulnerability:
  * If another group releases an implementation of the identified
     vulnerability; or
  * On June 15th, 2018, whichever comes first.

### Vulnerability Details

The core of the Tegra boot process is approximated by the following block of
pseudo-code, as obtained by reverse-engineering an IROM extracted from a
vulnerable T210 system:

```C
// If this is a warmboot (from "sleep"), restore the saved state from RAM.
if (read_scratch0_bit(1)) {
  restore_warmboot_image(&load_addr);
}
// Otherwise, bootstrap the processor.
else
{
  // Allow recovery mode to be forced by a PMC scratch bit or physical straps.
  force_recovery = check_for_rcm_straps() || read_scratch0_bit(2);

  // Determine whether to use USB2 or USB3 for RCM.
  determine_rcm_usb_version(&usb_version);
  usb_ops = set_up_usb_ops(usb_version);
  usb_ops->initialize();

  // If we're not forcing recovery, attempt to load an image from boot media.
  if (!force_recovery)
  {
    // If we succeeded, don't fall back into recovery mode.
    if (read_boot_configuration_and_images(&load_addr) == SUCCESS) {
      goto boot_complete;
    }
  }

  // In all other conditions
  if (read_boot_images_via_usb_rcm(<snip>, &load_addr) != SUCCESS) {
     /* load address is poisoned here */
  }
}

boot_complete:
  /* apply lock-outs, and boot the program at address load_address  */
```

Tegra processors include a USB Recovery Mode (RCM), which we can observe to be activated under a number of conditions:
  * If the processor fails to find a valid Boot Control Table (BCT) + bootloader on its boot media;
  * If processor straps are pulled to a particular value e.g. by holding a button combination; or
  * If the processor is rebooted after a particular value is written into a power management controller scratch register.

USB recovery mode is present in all devices, including devices that have been
production secured. To ensure that USB recovery mode does not allow unauthenticated
communications, RCM requires all recovery commands be signed using either RSA
or via AES-CMAC.

The bootloader's implementation of the Tegra RCM protocol is simple, and exists
to allow loading a small piece of code (called the *miniloader* or *applet*) into
the bootloader's local Instruction RAM (IRAM). In a typical application, this
*applet* is `nvtboot-recovery`, a stub which allows further USB communications to
bootstrap a system or to allow system provisioning.

The RCM process is approximated by the following pseudo-code, again obtained via
reverse engineering a dumped IROM from a T210:

```C
// Significantly simplified for clarity, with error checking omitted where unimportant.
while (1) {
  // Repeatedly handle USB standard events on the control endpoint EP0.
  usb_ops->handle_control_requests(current_dma_buffer);

  // Try to send the device ID over the main USB data pipe until we succeed.
  if ( rcm_send_device_id() == USB_NOT_CONFIGURED ) {
    usb_initialized = 0;
  }
  // Once we've made a USB connection, accept RCM commands on EP1.
  else {
    usb_initialized = 1;

    // Read a full RCM command and any associated payload into a global buffer.
    // (Error checking omitted for brevity.)
    rcm_read_command_and_payload();

    // Validate the received RCM command; e.g. by checking for signatures
    // in RSA or AES_CMAC mode, or by trivially succeeding if we're not in
    // a secure mode.
    rc = rcm_validate_command();
    if (rc != VALIDATION_PASS) {
      return rc;
    }

    // Handle the received and validated command.
    // For a "load miniloader" command, this sanity checks the (validated)
    // miniloader image and takes steps to prevent re-use of signed data not
    // intended to be used as an RCM command.
    rcm_handle_command_complete(...);
  }
}
```

It is important to note that a full RCM command *and its associated payload*
are read into 1) a global buffer, and 2) the target load address, respectively,
before any signature checking is done. This effectively grants the attacker a
narrow window in which they control a large region of unvalidated memory.

The largest vulnerability surface area occurs in the `rcm_read_command_and_payload`
function, which accepts the RCM command and payload packets via a USB bulk endpoint.
For our purposes, this endpoint is essentially a simple pipe for conveyance
of blocks of binary data separate from standard USB communications.

The `rcm_read_command_and_payload` function actually contains several issues--
of which exactly one is known to be exploitable:

```C
uint32_t total_rxd  = 0;
uint32_t total_to_rx = 0x400;

// Loop until we've received our full command and payload.
while (total_rxd < total_to_rx) {
  // Switch between two DMA buffers, so the USB is never DMA'ing into the same
  // buffer that we're processing.
  active_buffer = next_buffer;
  next_buffer   = switch_dma_buffers();

  // Start a USB DMA transaction on the RCM bulk endpoint, which will hopefully
  // receive data from the host in the background as we copy.
  usb_ops->start_nonblocking_bulk_read(active_buffer, 0x1000);

  // If we're in the first 680-bytes we're receiving, this is part of the RCM
  // command, and we should read it into the command buffer.
  if ( total_rxd < 680 ) {
    /* copy data from the DMA buffer into the RCM command buffer until we've
       read a full 680-byte RCM command */

    // Once we've received the first four bytes of the RCM command,
    // use that to figure out how much data should be received.
    if ( total_rxd >= 4 )
    {
      // validate:
      //  -- the command won't exceed our total RAM
      //     (680 here, 0x30000 in upper IRAM)
      //  -- the command is >= 0x400 bytes
      //  -- the size ends in 8
      if ( rcm_command_buffer[0] >= 0x302A8u
              || rcm_command_buffer[0] < 0x400u
              || (rcm_command_buffer[0] & 0xF) != 8 ) {
        return ERROR_INVALID_SIZE;
      } else  {
        left_to_rx = *((uint32_t *)rcm_command_buffer);
      }
    }
  }

  /* copy any data _past_ the command into a separate payload
     buffer at 0x40010000 */
  /* -code omitted for brevity - */

  // Wait for the DMA transaction to complete.
  // [This is, again, simplified to convey concepts.]
  while(!usb_ops->bulk_read_complete()) {

    // While we're blocking, it's still important that we respond to standard
    // USB packets on the control endpoint, so do that here.
    usb_ops->handle_control_requests(next_buffer);
  }
}
```

Astute readers will notice an issue unrelated to the Fusée Gelée exploit: this
code fails to properly ensure DMA buffers are being used exclusively for a single
operation. This results in an interesting race condition in which a DMA buffer
can be simultaneously used to handle a control request and a RCM bulk transfer.
This can break the flow of RCM, but as both operations contain untrusted data,
this issue poses no security risk.

To find the actual vulnerability, we must delve deeper, into the code that handles
standard USB control requests. The core of this code is responsible for responding
to USB control requests. A *control request* is initiated when the host sends a
setup packet, of the following form:

Field     | &nbsp; &nbsp; &nbsp; &nbsp; Size  &nbsp; &nbsp;| Description
----------|:----:|-----
direction | 1b   | if '1', the device should respond with data
type      | 2b   | specifies whether this request is of a standard type or not
recipient | 5b   | encodes the context in which this request should be considered; <br /> for example, is this about a `DEVICE` or about an `ENDPOINT`?
request   | 8b   | specifies the request number
value     | 16b  | argument to the request
index     | 16b  | argument to the request
length    | 16b  | specifies the maximum amount of data to be transferred

As an example, the host can request the status of a device by issuing a
`GET_STATUS` request, at which point the device would be expected to respond with
a short setup packet. Of particular note is the `length` field of the request,
which should *limit* -- but not exclusively determine-- the *maximum* amount of
data that should be included in the response. Per the specification, the device
should respond with either the *amount of data specified* or the *amount of data
available*, whichever is less.

The bootloader's implementation of this behavior is conceptually implemented as
follows:

```C

// Temporary, automatic variables, located on the stack.
uint16_t status;
void *data_to_tx;

// The amount of data available to transmit.
uint16_t size_to_tx   = 0;

 // The amount of data the USB host requested.
uint16_t length_read  = setup_packet.length;

/* Lots of handler cases have omitted for brevity. */

// Handle GET_STATUS requests.
if (setup_packet.request == REQUEST_GET_STATUS)
{
  // If this is asking for the DEVICE's status, respond accordingly.
  if(setup_packet.recipient == RECIPIENT_DEVICE) {
      status     = get_usb_device_status();
      size_to_tx = sizeof(status);
  }
  // Otherwise, respond with the ENDPOINT status.
  else if (setup_packet.recipient == RECIPIENT_ENDPOINT){
      status     = get_usb_endpoint_status(setup_packet.index);
      size_to_tx = length_read; // <-- This is a critical error!
  }
  else {
    /* ... */
  }

  // Send the status value, which we'll copy from the stack variable 'status'.
  data_to_tx = &status;
}

// Copy the data we have into our DMA buffer for transmission.
// For a GET_STATUS request, this copies data from the stack into our DMA buffer.
memcpy(dma_buffer, data_to_tx, size_to_tx);

// If the host requested less data than we have, only send the amount requested.
// This effectively selects min(size_to_tx, length_read).
if (length_read < size_to_tx) {
  size_to_tx = length_read;
}

// Transmit the response we've constructed back to the host.
respond_to_control_request(dma_buffer, length_to_send);
```

In most cases, the handler correctly limits the length of the transmitted
responses to the amount it has available, per the USB specification. However,
in a few notable cases, the length is *incorrectly always set to the amount
requested* by the host:
  * When issuing a `GET_CONFIGURATION` request with a `DEVICE` recipient.
  * When issuing a `GET_INTERFACE` request with a `INTERFACE` recipient.
  * When issuing a `GET_STATUS` request with a `ENDPOINT` recipient.

This is a critical security error, as the host can request up to 65,535 bytes per
control request. In cases where this is loaded directly into `size_to_tx`, this
value directly sets the extent of the `memcpy` that follows-- and thus can copy
up to 65,535 bytes into the currently selected `dma_buffer`. As the DMA buffers
used for the USB stack are each comparatively short, this can result in a _very_
significant buffer overflow.

To validate that the vulnerability is present on a given device, one can try
issuing an oversized request and watch as the device responds. Pictured below is
the response generated when sending a oversized `GET_STATUS` control request
with an `ENDPOINT` recipient to a T124:

![Reading a chunk of stack memory from a K1](stack_read.png)

A compliant device should generate a two-byte response to a `GET_STATUS` request--
but the affected Tegra responds with significantly longer response. This is a clear
indication that we've run into the vulnerability described above.

To really understand the impact of this vulnerability, it helps to understand
the memory layout used by the bootROM. For our proof-of-concept, we'll consider
the layout used by the T210 variant of the affected bootROM:

![Bootrom memory layout](mem_layout.png)

The major memory regions relevant to this vulnerability are as follows:
  * The bootROM's *execution stack* grows downward from `0x40010000`; so the
    execution stack is located in the memory *immediately preceding* that address.
  * The DMA buffers used for USB are located at `0x40005000` and `0x40009000`,
    respectively. Because the USB stack alternates between these two buffers
    once per USB transfer, the host effectively can control which DMA buffer
    is in use by sending USB transfers.
  * Once the bootloader's RCM code receives a 680-byte command, it begins to store
    received data in a section of upper IRAM located at address `0x40010000`, and can
    store up to `0x30000` bytes of payload. This address is notable, as it is immediately
    past the end of the active execution stack.

Of particular note is the adjacency of the bootROM's *execution stack* and the
attacker-controlled *RCM payload*. Consider the behavior of the previous pseudo-code
segment on receipt of a `GET_STATUS` request to the `ENDPOINT` with an
excessive length. The resulting memcpy:
  * copies *up to* 65,535 bytes total;
  * sources data from a region *starting at the status variable on the stack*
    and extending significantly past the stack -- effectively copying mostly
    *from the attacker-controllable RCM payload buffer*
  * targets a buffer starting either `0x40005000` or `0x40009000`, at the
    attacker's discretion, reaching addresses of up to `0x40014fff` or `0x40018fff`

This is a powerful copy primitive, as it copies *from attacker controlled memory*
and into a region that *includes the entire execution stack*:

![Effect of the vulnerability memcpy](copy_span.png)

This would be a powerful exploit on any platform; but this is a particularly devastating
attack in the bootROM environment, which does not:
  * Use common attack mitigations such as stack canaries, ostensibly to reduce
    complexity and save limited IRAM and IROM space.
  * Apply memory protections, so the entire stack and all attacker
    controlled buffers can be read from, written to, and executed from.
  * Employ typical 'application-processor' mitigation strategies such as ASLR.

Accordingly, we now have:
  1. The capability to load arbitrary payloads into memory via RCM, as RCM only
     validates command signatures once payload receipt is complete.
  2. The ability to copy attacker-controlled values over the execution stack,
     overwriting return addresses and redirecting execution to a location of our
     choice.

Together, these two abilities give us a full arbitrary-code execution exploit at
a critical point in the Tegra's start-up process. As control flow is hijacked
before return from `read_boot_images_via_usb_rcm`, none of the "lock-out"
operations that precede normal startup are executed. This means, for example,
that the T210 fuses-- and the keydata stored within them-- are accessible from
the attack payload, and the bootROM is not yet protected.

#### Exploit Execution

The Fusée Launcher PoC exploits the vulnerability described on the T210 via a
careful sequence of interactions:
  1. The device is started in RCM mode. Device specifics will differ, but this
     is often via a key-combination held on startup.
  2. A host computer is allowed to enumerate the RCM device normally.
  3. The host reads the RCM device's ID by reading 16 bytes from the EP1 IN.
  4. The host builds an exploit payload, which is comprised of:
      1. An RCM command that includes a maximum length, ensuring that we can send
         as much payload as possible without completing receipt of the RCM payload.
         Only the length of this command is used prior to validation; so we can
         submit an RCM command that starts with a maximum length of 0x30298, but
         which fills the remaining 676 bytes of the RCM command with any value.
      2. A set of values with which to overwrite the stack. As stack return address
         locations vary across the series, it's recommended that a large block
         composed of a single entry-point address be repeated a significant number
         of times, so one can effectively replace the entire stack with that address.
      3. The program to be executed ("final payload") is appended, ensuring that its
         position in the binary matches the entry-point from the previous step.
      4. The payload is padded to be evenly divisible by the 0x1000 block size to
         ensure the active block is not overwritten by the "DMA dual-use" bug
         described above.
  5. The exploit payload is sent to the device over EP1 OUT, tracking the number of
     0x1000-byte "blocks" that have been sent to the device. If this number is _even_,
     the next write will be issued to the lower DMA buffer (`0x40005000`); otherwise,
     it will be issued to the upper DMA buffer (`0x40009000`).
  6. If the next write would target the lower DMA buffer, issue another write
     of a full 0x1000 bytes to move the target to the upper DMA buffer, reducing
     the total amount of data to be copied.
  7. Trigger the vulnerable memcpy by sending a `GET_STATUS` `IN` control
     request with an `ENDPOINT` recipient, and a length long enough to smash the
     desired stack region, and preferably not longer than required.

A simple host program that triggers this vulnerability is included with this
report: see `fusee-launcher.py`. Note the restrictions on its function in the
following section.


### Proof of Concept

Included with this report is a set of three files:
  * `fusee-launcher.py` -- The main proof-of-concept accompanying this report.
    This python script is designed to launch a simple binary payload in the
    described bootROM context via the exploit.
  * `intermezzo.bin` -- This small stub is designed to relocate a payload from
    a higher load address to the standard RCM load address of `0x40010000`. This
    allows standard RCM payloads (such as `nvtboot-recover.bin`) to be executed.
  * `fusee.bin` -- An example payload for the Nintendo Switch, a representative
    and well-secured device based on a T210. This payload will print information
    from the device's fuses and protected IROM to the display, demonstrating that
    early bootROM execution has been achieved.

**Support note:** Many host-OS driver stacks are reluctant to issue unreasonably
large control requests. Accordingly, the current proof-of-concept includes code
designed to work in the following environments:
  * **64-bit linux via `xhci_hcd`**. The proof-of-concept can manually submit
    large control requests, but does not work with the common `ehci_hcd` drivers
    due to driver limitations. A rough rule of thumb is that a connection via a
    blue / USB3 SuperSpeed port will almost always be handled by `xhci_hcd`.
  * **macOS**. The exploit works out of the box with no surprises or restrictions
    on modern macOS.

Windows support would require addition of a custom kernel module, and thus was
beyond the scope of a simple proof-of-concept.

To use this proof-of-concept on a Nintendo Switch:
  1. Set up an Linux or macOS environment that meets the criteria above, and
     which has a working `python3` and `pyusb` as well as `libusb` installed.
  2. Connect the Switch to your host PC with a USB A -> USB C cable.
  3. Boot the Switch in RCM mode. There are three ways to do this, but the first--
     unseating its eMMC board-- is likely the most straightforward:
      1. Ensure the Switch cannot boot off its eMMC. The most straightforward way to
         to this is to open the back cover and remove the socketed eMMC board; corrupting
         the BCT or bootloader on the eMMC boot partition would also work.
      2. Trigger the RCM straps. Hold VOL_UP and short pin 10 on the right
         JoyCon connector to ground while engaging the power button.
      3. Set bit 2 of PMC scratch register zero. On modern firmwares, this requires
         EL3 or pre-sleep BPMP execution.
  4. Run the `fusee-launcher.py` with an argument of `fusee.bin`. (This requires
     `intermezzo.bin` to be located in the same folder as `fusee-launcher.py`.)

    ```
    sudo python3 ./fusee-launcher.py fusee.bin
    ```

If everything functions correctly, your Switch should be displaying a collection
of fuse and protected-IROM information:

![exploit working](switch_hax.jpg)


### Recommended Mitigations

In this case, the recommended mitigation is to correct the USB control request
handler such that it always correctly constrains the length to be transmitted.
This has to be handled according to the type of device:
  * **For a device already in consumer hands**, no solution is proposed.
    Unfortunately, access to the fuses needed to configure the device's ipatches
    was blocked when the ODM_PRODUCTION fuse was burned, so no bootROM update
    is possible. It is suggested that consumers be made aware of the situation
    so they can move to other devices, where possible.
  * **For new devices**, the correct solution is likely to introduce an
    new ipatch or new ipatches that limits the size of control request responses.

It seems likely that OEMs producing T210-based devices may move to T214 solutions;
it is the hope of the author that the T214's bootROM shares immunity with
the T186. If not, patching the above is a recommended modification to the mask ROM
and/or ipatches of the T214, as well.