(MTL S01E08) Rendering pt.2: Render Pipeline

We discussed the full rendering pipeline in Metal in the last episode, so by now, you should have understanding of its stages and underlying principles. In this episode, I’ll guide you through preparing the essential resources to run it. We’ll focus on the CPU side only, saving shaders for the next two episodes.

Overview

What do we need to perform rendering in Metal?

1. Prepare all necessary resources (buffers, textures, etc.).
2. Create a render encoder.
3. Set up a pipeline state.
4. Bind the required resources.
5. Call the draw function.
6. End encoding.

Render encoder

Descriptor

For creating a render pass encoder, we need to initialize its descriptor:

let renderPassDescriptor = MTLRenderPassDescriptor()           // (1)
renderPassDescriptor.colorAttachments[0].texture = dstTexture  // (2)
renderPassDescriptor.colorAttachments[0].loadAction = .clear   // (3)
renderPassDescriptor.colorAttachments[0].clearColor = .init(   // (4)
   red: 0.5, green: 0.5, blue: 0.5, alpha: 1.0)
renderPassDescriptor.colorAttachments[0].storeAction = .store  // (5)

1. Create an instance of the descriptor.
2. Set an output texture to the `0` color attachment. Multiple color attachments, are possible, but here we use just one texture. Also depth and stencil attachments could be set there.

NOTE: If you have multiple color attachments of different sizes, the final render will appear only in the intersecting area of their rectangles.

3. Define an action for loading the texture, which could be:

• .load — Initializes the output buffer with the content of the attached texture.
• .dontCare — Leaves the output buffer uninitialized, which may contain any existing memory data (often amusingly random). Use this if you’re confident the entire visible area will be covered by your rendering, as it boosts performance by skipping texture loading or filling.
• .clear — Fills the entire output buffer with a color, which we’ll set in step (4).

4. Set the color to initialize the output buffer.

Creating render encoder

While an empty render encoder might seem boring, it can still be useful for filling a texture with a color. It also provides a solid starting point for adding draw calls later.

let renderPassDescriptor = MTLRenderPassDescriptor()         
renderPassDescriptor.colorAttachments[0].texture = dstTexture
renderPassDescriptor.colorAttachments[0].loadAction = .clear 
renderPassDescriptor.colorAttachments[0].clearColor = .init( 
   red: 0.5, green: 0.5, blue: 0.5, alpha: 1.0)
renderPassDescriptor.colorAttachments[0].storeAction = .store

// ⬇ NEW CODE ⬇
guard let encoder = commandBuffer.makeRenderCommandEncoder(descriptor: renderPassDescriptor) 
else { return }
renderEncoder.endEncoding()

Assume we already have a `commandBuffer` to create an encoder. When finished with the encoder, we must call `.endEncoding` to close it.

You can reuse the same encoder for similar operations — such as computing, blitting, or rendering — provided they share the same target attachments. This practice reduces overhead by minimizing resource loading and related costs.

Draw Call

To render any content beyond simple color fills with a render encoder, you’ll need to use shaders, defined in an `MTLRenderPipelineState` (covered in the next section). Then, you can call methods like `.drawPrimitives` to draw the content.

let renderPassDescriptor = MTLRenderPassDescriptor()         
renderPassDescriptor.colorAttachments[0].texture = dstTexture
renderPassDescriptor.colorAttachments[0].loadAction = .clear 
renderPassDescriptor.colorAttachments[0].clearColor = .init( 
   red: 0.5, green: 0.5, blue: 0.5, alpha: 1.0)
renderPassDescriptor.colorAttachments[0].storeAction = .store

guard let encoder = commandBuffer.makeRenderCommandEncoder(descriptor: renderPassDescriptor) 
else { return }

// ⬇ NEW CODE ⬇
encoder.setRenderPipelineState(pipeline)
encoder.drawPrimitives(type: .triangleStrip, vertexStart: 0, vertexCount: 4)
// ⬆ NEW CODE ⬆

encoder.endEncoding()

This approach could be sufficient if you compute all vertex positions in the vertex shader and all pixel values in the fragment shader, which are set in the `pipeline`. However, more commonly, you’ll work with a preloaded mesh. This requires a different draw method and additional parameters for rendering, meaning you’ll need to pass these parameters to the shaders.

Set up resources

If your shader requires arguments, you’ll need to pass in some data. We’ve covered this in the episodes on buffers and textures, but let’s quickly recap:

let renderPassDescriptor = MTLRenderPassDescriptor()         
renderPassDescriptor.colorAttachments[0].texture = dstTexture
renderPassDescriptor.colorAttachments[0].loadAction = .clear 
renderPassDescriptor.colorAttachments[0].clearColor = .init( 
   red: 0.5, green: 0.5, blue: 0.5, alpha: 1.0)
renderPassDescriptor.colorAttachments[0].storeAction = .store


guard let encoder = commandBuffer.makeRenderCommandEncoder(descriptor: renderPassDescriptor) 
else { return }

encoder.setRenderPipelineState(pipeline)
// ⬇ NEW CODE ⬇
encoder.setFragmentBytes(&someColor, length: MemoryLayout<simd_float4>.size, index: 0) // (1)
encoder.setFragmentTexture(someTexture, index: 0)                                      // (2)

encoder.setVertexBuffer(vertices, offset: 0, index: 0)                                 // (3)
encoder.setVertexBytes(&imageSize, length: MemoryLayout<simd_float2>.size, index: 2)   // (4)
// ⬆ NEW CODE ⬆
encoder.drawPrimitives(type: .triangleStrip, vertexStart: 0, vertexCount: 4)

encoder.endEncoding()

NOTE: We’ll assume `someColor`, `someTexture`, and `imageSize` have been initialized before. Remember, these articles are meant to help you understand the underlying principles, not serve as a step-by-step, copy-paste tutorial. So, try to look beyond the code details and focus on the core concepts.

1. Passing `someColor` to the fragment shader: Metal will create a buffer implicitly. This works fine if the value changes every frame, but for long-lasting constant values (especially if they consume significant memory), I recommend using explicit buffers. Note that this implicit approach is limited to 4KB — beyond that, you’ll need a buffer object.
2. Passing `someTexture` to the fragment shader: You can also pass a sampler with `setFragmentSamplerState` if additional control is needed. These methods are similarly available for vertex shaders.
3. Passing `vertices` to the vertex shader: Here, we use an explicit buffer. Generally, geometry is preloaded, but for small, dynamic data, you can use an implicit buffer with `setVertexBytes`.
4. Passing `imageSize` to the vertex buffers: Technically the same as step (1) but applied to the vertex shader.

drawIndexedPrimitives

As mentioned, preloaded geometry often contains various attributes such as positions, texture coordinates, normals, etc. — stored in different structures or even separate buffers. Additionally, some attributes (like vertex color) can be updated programmatically in real-time, so vertex parameters might be spread across multiple buffers. Manually managing each buffer can be cumbersome, so Metal offers an alternative: using `drawIndexedPrimitives`.

//...
renderEncoder.drawIndexedPrimitives(type: .triangle,
                                    indexCount: mesh.indices.count,
                                    indexType: .uint16,
                                    indexBuffer: indices,
                                    indexBufferOffset: 0)
//...

To use this, you first need to map all parameters from different buffers to attributes within a structure, which is then passed to your vertex shader as an input argument `[[stage_in]]`. This mapping is set up when creating the `MTLRenderPipelineState` — its descriptor includes a `vertexDescriptor` parameter where you assign an instance of `MTLVertexDescriptor`. I’ll explain this further in the section on pipeline state.

NOTE: These buffers must still be bound to the vertex shader at the same indices specified in `MTLVertexDescriptor`. In the vertex shader, you don’t need explicit arguments for these indices, but keep in mind that they’re already taken by the mesh.

You may have noticed that we’re using an `indexBuffer` here. This buffer contains vertex indices in the correct order to construct a mesh using the specified primitive type. Typically, it’s included with the mesh data.

Primitives

The possible primitives were discussed in the previous episode, but here’s a quick recap just in case:

Viewport and Scissor

Another important, though not always necessary, set of parameters includes the viewport and scissor area.

Viewport is the rectangular area where your objects are visible, like viewing through an aquarium. Set it using the following:

//...
encoder.setViewport(
   MTLViewport(originX: 20, originY: 40,  // (1)
               width: 600, height: 400,   // (2)
               znear: -1, zfar: 1))       // (3)
//...

1. The origin of the viewport in target (texture) coordinates.
2. The size of the viewport in target coordinates.
3. The near and far planes.

NOTE: The viewport can extend beyond the target size; for example, the origin can be negative, and the size can exceed the target dimensions.

Scissor defines a rectangular area within the target where rendering is allowed; anything outside this box will not be drawn.

encoder.setScissorRect(MTLScissorRect(x: 100, y: 200, width: 600, height: 400))

NOTE: The entire scissor area must fit within your target!

MTLRenderPipelineState

Earlier, we set up a render encoder and applied a pipeline state, but we didn’t initialize it. Let’s do that now. A pipeline state can include GPU functions and additional parameters, such as attachment formats, vertex attributes, function constants, and more.

Shaders

// (1)
let library = device.makeDefaultLibrary()
// (2)
let vertexFunction = library?.makeFunction(name: vertex)
let fragmentFunction = library?.makeFunction(name: fragment)

// (3)
let descriptor = MTLRenderdescriptor()
// (4)
descriptor.vertexFunction = vertexFunction
descriptor.fragmentFunction = fragmentFunction

// (5)
let pipelineState = try device.makeRenderPipelineState(descriptor: descriptor)

1. Create a default library. The default library is a Metal library, typically built at compile time from your `.metal` files and included in your app’s bundle. In most cases, this is what you’ll use.
2. Create instances of vertex and fragment functions from the library. The second argument, `constantValues`, acts as an alternative to `#ifdef`, allowing you to alter code at compile time without runtime branching.
3. Create an instance of `MTLRenderPipelineDescriptor`.
4. Set the shader functions to be used in the pipeline state.
5. Create the pipeline state based on the descriptor.

Attachments

The pipeline state must have the same pixel formats for attachments as those specified in the encoder where it’s used.

let library = device.makeDefaultLibrary()
let vertexFunction = library?.makeFunction(name: vertex)
let fragmentFunction = library?.makeFunction(name: fragment)

let descriptor = MTLRenderdescriptor()
descriptor.vertexFunction = vertexFunction
descriptor.fragmentFunction = fragmentFunction

// ⬇ NEW CODE ⬇
descriptor.colorAttachments[0].pixelFormat = .rgba8Unorm
// ⬆ NEW CODE ⬆

let pipelineState = try device.makeRenderPipelineState(descriptor: descriptor)

Blending

You can enable blending (`.blendingEnabled`) or set its equation for each attachment individually:

let library = device.makeDefaultLibrary()
let vertexFunction = library?.makeFunction(name: vertex)
let fragmentFunction = library?.makeFunction(name: fragment)

let descriptor = MTLRenderdescriptor()
descriptor.vertexFunction = vertexFunction
descriptor.fragmentFunction = fragmentFunction

descriptor.colorAttachments[0].pixelFormat = .rgba8Unorm
// ⬇ NEW CODE ⬇
descriptor.colorAttachments[0].blendingEnabled = true
descriptor.colorAttachments[0].rgbBlendOperation = .add
descriptor.colorAttachments[0].alphaBlendOperation = .add
descriptor.colorAttachments[0].sourceAlphaBlendFactor = .one
descriptor.colorAttachments[0].sourceRGBBlendFactor = .one
descriptor.colorAttachments[0].destinationRGBBlendFactor = .oneMinusSourceAlpha
descriptor.colorAttachments[0].destinationAlphaBlendFactor = .oneMinusSourceAlpha
// ⬆ NEW CODE ⬆

let pipelineState = try device.makeRenderPipelineState(descriptor: descriptor)

By default, Metal uses standard alpha blending, but you can customize it using various operations and factors. For a full list of options, refer to the documentation.

Here’s how blending works:

• `.rgbBlendOperation` — specifies the operation Metal performs on color channels.
• `.alphaBlendOperation` — specifies the operation Metal performs on alpha channels.
• `.sourceAlphaBlendFactor` — multiplier for the source alpha.
• `.sourceRGBBlendFactor` — multiplier for the source RGB.
• `.destinationAlphaBlendFactor` — multiplier for the destination alpha.
• `.destinationRGBBlendFactor` — multiplier for the destination RGB.

This setup defines the following equation:

resRGB = 1.0 * srcRGB + (1.0 - srcA) * dstRGB;
resA   = 1.0 * srcA   + (1.0 - srcA) * dstA;
          |           |      \ 
          |           \       destination blend factor
          \            blend operation
           source  blend factor

Vertex Attributes

As mentioned earlier, we may need to map vertex data from multiple buffers to the input structure of the vertex shader. In Metal, this approach is similar to OpenGL’s Vertex Array Objects (VAOs):

let vertexDescriptor = MTLVertexDescriptor()
vertexDescriptor.attributes[0].format = MTLVertexFormat.float2             // (1)
vertexDescriptor.attributes[0].offset = 0                                  // (2)
vertexDescriptor.attributes[0].bufferIndex = 0                             // (3)

vertexDescriptor.layouts[0].stride = MemoryLayout<SIMD2<Float32>>.size     // (4)
vertexDescriptor.layouts[0].stepRate = 1                                   // (5)
vertexDescriptor.layouts[0].stepFunction = MTLVertexStepFunction.perVertex // (6)

//...
// At pipeline state descriptor setup:
descriptor.vertexDescriptor = vertexDescriptor

1. Format of the target vertex attribute.
2. Starting position of the data within the buffer.
3. The index where the associated buffer will be bound.
4. Byte stride between the starts of each structure within the buffer.
5. Step rate for presenting data to the vertex function (default is `1`; relevant for instance-based steps).
6. Frequency of vertex data fetching for the vertex function.

In this example, we have one buffer and one attribute, but consider these cases:

• If there are two attributes in the same buffer, you would use `.attribute[0]` and `.attribute[1]` with a single `.layout[0]`.
• If the attributes are in separate buffers, you would have `.attribute[0]` and `.attribute[1]`, along with `.layout[0]` and `.layout[1]`, where the indices in layout correspond to the buffer indices (point 3 above).

Conclusion

• You’ve learned how to set up a render pipeline state, configure a render encoder, and call your shaders — ready to build your own image editor!
• Vertex attributes can be used either with straight access to buffers or with vertex descriptor.
• For experimenting with shaders, primitives, and basic rendering, KodeLife is a convenient tool.