Medium
Histogram binning with hist[bin]++ versus atomicAdd
Thousands of threads each increment one of 16 bins in a histogram held in global memory, chosen by their input value; many threads land on the same bin at once. Compare the two kernels below.
kernel.cu
// Kernel A: no atomics
__global__ void histNaive(int* hist, const int* data, int n) {
int i = blockIdx.x * blockDim.x + threadIdx.x;
if (i < n) {
int bin = data[i] % 16;
hist[bin]++; // read-modify-write, not atomic
}
}
// Kernel B: atomics
__global__ void histAtomic(int* hist, const int* data, int n) {
int i = blockIdx.x * blockDim.x + threadIdx.x;
if (i < n) {
int bin = data[i] % 16;
atomicAdd(&hist[bin], 1);
}
}15 of 46 in GPU / CUDA