performance: add low latency hint for exec queues #794
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
When workloads submit compute kernels, it's required to keep low submission latency. Impact of submission overhead is significant in a burst model, when application submits short-running kernels continuously. Signed-off-by: Szymon Morek <[email protected]>
When workloads submit compute kernels, it's required to keep low submission latency. Impact of submission overhead is significant in a burst model, when application submits short-running kernels continuously. Signed-off-by: Szymon Morek <[email protected]>
intel-lab-lkp
pushed a commit
to intel-lab-lkp/linux
that referenced
this pull request
Jan 9, 2025
Allow user to provide a low latency hint per exec queue. When set,
KMD sends a hint to GuC which results in special handling for this
exec queue. SLPC will ramp the GT frequency aggressively every time
it switches to this exec queue.
We need to enable the use of SLPC Compute strategy during init, but
it will apply only to exec queues that set this bit during exec queue
creation.
Improvement with this approach as below:
Before,
:~$ NEOReadDebugKeys=1 EnableDirectSubmission=0 clpeak --kernel-latency
Platform: Intel(R) OpenCL Graphics
Device: Intel(R) Graphics [0xe20b]
Driver version : 24.52.0 (Linux x64)
Compute units : 160
Clock frequency : 2850 MHz
Kernel launch latency : 283.16 us
After,
:~$ NEOReadDebugKeys=1 EnableDirectSubmission=0 clpeak --kernel-latency
Platform: Intel(R) OpenCL Graphics
Device: Intel(R) Graphics [0xe20b]
Driver version : 24.52.0 (Linux x64)
Compute units : 160
Clock frequency : 2850 MHz
Kernel launch latency : 63.38 us
UMD will indicate low latency hint with flag as mentioned below,
* struct drm_xe_exec_queue_create exec_queue_create = {
* .flags = DRM_XE_EXEC_QUEUE_LOW_LATENCY_HINT or 0
* .extensions = 0,
* .vm_id = vm,
* .num_bb_per_exec = 1,
* .num_eng_per_bb = 1,
* .instances = to_user_pointer(&instance),
* };
* ioctl(fd, DRM_IOCTL_XE_EXEC_QUEUE_CREATE, &exec_queue_create);
Link to UMD PR : intel/compute-runtime#794
Note: There is outstanding issue on guc side to be not able to switch to max
frequency as per strategy indicated by KMD, so for experminet/test result
hardcoding apporch was taken and passed to guc as policy. Effort on debugging
from guc side is going on in parallel.
V2:
- DRM_XE_EXEC_QUEUE_LOW_LATENCY_HINT 1 is already planned for other hint(Szymon)
- Add motivation to description (Lucas)
Cc:[email protected]
Cc:[email protected]
Cc:Michal Mrozek <[email protected]>
Cc:Szymon Morek <[email protected]>
Cc:José Roberto de Souza <[email protected]>
Signed-off-by: Tejas Upadhyay <[email protected]>
When workloads submit compute kernels, it's required to keep low submission latency. Impact of submission overhead is significant in a burst model, when application submits short-running kernels continuously. Signed-off-by: Szymon Morek <[email protected]>
intel-lab-lkp
pushed a commit
to intel-lab-lkp/linux
that referenced
this pull request
Jan 10, 2025
Allow user to provide a low latency hint per exec queue. When set,
KMD sends a hint to GuC which results in special handling for this
exec queue. SLPC will ramp the GT frequency aggressively every time
it switches to this exec queue.
We need to enable the use of SLPC Compute strategy during init, but
it will apply only to exec queues that set this bit during exec queue
creation.
Improvement with this approach as below:
Before,
:~$ NEOReadDebugKeys=1 EnableDirectSubmission=0 clpeak --kernel-latency
Platform: Intel(R) OpenCL Graphics
Device: Intel(R) Graphics [0xe20b]
Driver version : 24.52.0 (Linux x64)
Compute units : 160
Clock frequency : 2850 MHz
Kernel launch latency : 283.16 us
After,
:~$ NEOReadDebugKeys=1 EnableDirectSubmission=0 clpeak --kernel-latency
Platform: Intel(R) OpenCL Graphics
Device: Intel(R) Graphics [0xe20b]
Driver version : 24.52.0 (Linux x64)
Compute units : 160
Clock frequency : 2850 MHz
Kernel launch latency : 63.38 us
UMD will indicate low latency hint with flag as mentioned below,
* struct drm_xe_exec_queue_create exec_queue_create = {
* .flags = DRM_XE_EXEC_QUEUE_LOW_LATENCY_HINT or 0
* .extensions = 0,
* .vm_id = vm,
* .num_bb_per_exec = 1,
* .num_eng_per_bb = 1,
* .instances = to_user_pointer(&instance),
* };
* ioctl(fd, DRM_IOCTL_XE_EXEC_QUEUE_CREATE, &exec_queue_create);
Link to UMD PR : intel/compute-runtime#794
Note: There is outstanding issue on guc side to be not able to switch to max
frequency as per strategy indicated by KMD, so for experminet/test result
hardcoding apporch was taken and passed to guc as policy. Effort on debugging
from guc side is going on in parallel.
V3:
- Conver user flag to kernel internal flag and use (Oak)
- Support query config for use to check kernel support (Jose)
- Dont need to take runtime pm (Vinay)
V2:
- DRM_XE_EXEC_QUEUE_LOW_LATENCY_HINT 1 is already planned for other hint(Szymon)
- Add motivation to description (Lucas)
Signed-off-by: Tejas Upadhyay <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
When workloads submit compute kernels, it's required
to keep low submission latency.
Impact of submission overhead is significant in a burst model,
when application submits short-running kernels continuously.
Signed-off-by: Szymon Morek [email protected]