An 8192 x 8192 SGEMM on an A100 compared with the datasheet

An engineer benchmarks cublasSgemm on square 8192 x 8192 FP32 matrices on an A100 with a CUDA 11 or later toolkit, using a freshly created cuBLAS handle with default settings. It sustains about 18.5 TFLOPS. The A100 datasheet lists 156 TFLOPS for 'TF32 Tensor Core' and 19.5 TFLOPS for 'FP32'. What accounts for the gap between 18.5 and 156?

35 of 46 in GPU / CUDA