Allocating and freeing a scratch buffer inside a multi-stream loop

This loop pipelines chunks across four non-default streams using pinned host buffers, so copies and kernels from different chunks should overlap. Each iteration needs a temporary device buffer between stage1 and stage2. How does the per-iteration cudaMalloc/cudaFree affect the pipeline, and what is the idiomatic fix?

pipeline.cu
for (int i = 0; i < numChunks; ++i) {
    cudaStream_t s = streams[i % 4];
    size_t off = i * chunkElems;

    float* scratch;
    cudaMalloc(&scratch, chunkBytes);

    cudaMemcpyAsync(d_in + off, h_in + off, chunkBytes, cudaMemcpyHostToDevice, s);
    stage1<<<grid, block, 0, s>>>(d_in + off, scratch);
    stage2<<<grid, block, 0, s>>>(scratch, d_out + off);
    cudaMemcpyAsync(h_out + off, d_out + off, chunkBytes, cudaMemcpyDeviceToHost, s);

    cudaFree(scratch);
}

34 of 46 in GPU / CUDA