Figure 1:
Architecture of the CUDA program and these memories.