(MTL S01E04) Primitives pt.1: Buffers

Metal is a powerful framework for GPU-based computing and rendering, necessitating the storage of various data such as images, parameters, and objects. In this episode, I’ll delve into the solutions Metal offers for this, specifically focusing on how to utilize MTLBuffer.

What is `MTLBuffer`

Essentially, a buffer is a designated area in GPU memory (ARM architectures may feature shared memory). A buffer does not maintain any information about the structure of the data it holds, making it your responsibility to ensure that data structures are consistent between the CPU and GPU sides.

Look inside

You can utilize the Frame Capture tool in Xcode to inspect the contents of your buffers, which I will detail further in future episodes.

1 - Press the M button and wait for Xcode to collect GPU data.

2 - Go to the Bound Resources section of a draw or dispatch call you wish to examine.

3 - Double-click a buffer to select it and view its contents. Xcode will display the data using the structure assigned to the buffer in the linked GPU function.

This is an overview of what buffers are and how to examine their contents. In the following sections, I’ll explain how to create them on the CPU side.

Explicit buffers

To explicitly create a buffer, you need to instantiate a `MTLBuffer`. There are several scenarios where you must declare your buffer explicitly:

• To store the results of a GPU function,
• To share identical data across multiple calls,
• When managing a substantial amount of data,
• When you need to access the contents of the buffer.

Creating

If you need to create a buffer and initialize it with data, consider the following example (assuming you already have a `device` object of type `MTLDevice`):

let content = [simd_float2](                        // (1)
    repeating: simd_float2(0, 0), 
    count: 1024)
let bufferFromData = device.makeBuffer(             // (2)
    bytes: content,                                 // (3)
    length: 1024 * MemoryLayout<simd_float2>.size,  // (4)
    options: [.storageModeShared])                  // (5)
bufferFromData?.label = "Buffer A"                  // (6)

1. Initialize with default data — it could be anything you need (meshes, structures, vectors, numbers, etc.).
2. Create a buffer in the `device`’s memory.
3. Pass a pointer to the data to be uploaded.
4. Specify the length of the data.
5. Set buffer options. Use `.storageModeShared` for CPU uploads.
6. Optionally name the buffer for easier management.

For buffers that do not need to be shared with the CPU, you can create a private buffer:

let privateBuffer = device.makeBuffer(
    length: 1024 * MemoryLayout<simd_float2>.size,
    options: [.storageModePrivate])

Here, `.storageModePrivate` means the buffer is accessible only from the GPU, allowing Metal to optimise its usage.

Additionally, `.storageModeMemoryless` is used for creating buffers that are shared between calls within one encoder, offering further optimisation.

Using

To bind your buffer to an index for a specific encoder, use the following syntax:

encoder.setBuffer(buffer, offset: 0, index: 0)

• `encoder` refers to a compute encoder; the syntax may vary slightly for rendering encoders.
• `buffer` is the buffer object you wish to use.
• `offset` is the starting point within your buffer from which the GPU function will access data.
• `index` corresponds to the buffer’s index within the GPU function (e.g., `[[buffer(0)]]`).

Implicit buffers

Often, we need to provide our GPU functions with parameters that change frequently, calculated on the CPU side or inputted by the user. If there’s no need to store results back into these parameters from the GPU, creating a buffer object might seem like unnecessary extra steps. Fortunately, Metal allows you to pass data directly to the encoder, and it handles buffer creation behind the scenes.

var value = simd_float2(42, 42)
encoder.setBytes(
    &value, 
    length: MemoryLayout<simd_float2>.size, 
    index: 2)

In this example, we transfer the data stored in `value` to the GPU side and bind the implicitly created buffer to index 2.

Accessing buffers

We’ve previously discussed how to access a buffer’s content on the GPU side and how to pass data to a buffer at creation. But what if we need to download computation results back to the CPU side?

As previously mentioned, you initially need a buffer with the `.storageModeShared` option for direct CPU access. If this isn’t the case, you could create a temporary buffer with this setting and transfer the contents from a private buffer using a blit operation. Afterward, you can resolve a pointer to its content, which can be used for both writing and reading, but remember to manage necessary synchronization between the CPU and GPU.

let rawParticles = buffer
    .contents()
    .assumingMemoryBound(to: SomeType.self)

While using `.contents()` is sufficient to access the raw memory (as might be common in Objective-C), Swift requires you to specify the data type explicitly to facilitate easier manipulation of the raw memory.

Conclusion

We’ve delved into the essentials of creating and managing MTLBuffers, explored scenarios that necessitate direct buffer manipulation for tasks such as storing GPU computations and sharing data between function calls. Additionally, we examined techniques for efficiently transferring data to the GPU and retrieving computational results back to the CPU.