Skip to content

fix(rocjitsu): detect VGPR WAW races #8180

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
lialan wants to merge 1 commit into from
Closed

fix(rocjitsu): detect VGPR WAW races #8180

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 9 additions & 7 deletions emulation/rocjitsu/docs/race-detector.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -76,7 +76,7 @@ RJ_RACE=1 build/tools/rocjitsu/rocjitsu --config configs/gfx950_cdna4.json -- /t
You should see output:

```
RACE type=LDS reg=508 wave=0 lane=0 wg=0,0,0 conflict=unknown
RACE type=LDS access=read reg=508 wave=0 lane=0 wg=0,0,0 conflict=unknown
Race on LDS byte 508 [workgroup (0, 0, 0), wave 0, lane 0]
==> ds_write_b32 v0, v1 ; <-- wave 1
v_sub_u32_e32 v1, 0, v0
Expand Down Expand Up @@ -127,7 +127,8 @@ races. Some examples:
## What this plugin detects

- **VGPR races**: a vector register is read before a pending global or LDS load
has completed (`s_waitcnt vmcnt` / `s_waitcnt lgkmcnt` insufficient).
has completed (`s_waitcnt vmcnt` / `s_waitcnt lgkmcnt` insufficient), or an
instruction writes a VGPR that still has an active pending load.
- **SGPR races**: a scalar register is read before a pending scalar load has
completed (`s_waitcnt lgkmcnt` insufficient).
- **LDS races**: an LDS byte is read or written by one wave while another wave
Expand Down Expand Up @@ -155,7 +156,9 @@ operations are in flight. When an instruction in the emulator accesses an LDS
byte, there is a check to see what memory events are still in flight that
read/write that byte, from the perspective of the accessing thread. In this way,
RAW (read-after-write) and WAR (write-after-read) hazards can be detected.
Similar logic applies for VGPR and SGPR accesses.
Similar logic applies for VGPR and SGPR accesses. VGPR writes are also checked
against outstanding loads to catch write-after-write hazards where a later ALU
write may be clobbered by a still-pending memory result.

**LDS race detection** uses coarse-grained counters (one per 16-byte chunk) for
fast-path checks, with interval-based overlap scanning as a fallback. Live
Expand Down Expand Up @@ -266,10 +269,9 @@ ctest --test-dir build -R "RaceTest"
(missing `s_waitcnt` and `s_barrier`). It does not detect inter-workgroup
races, races between dispatches, or host-device synchronization issues.

- **No WAW detection**: write-after-write hazards are not currently flagged.
This includes both LDS WAW (two waves writing to the same LDS byte without a
barrier) and VGPR WAW (an ALU instruction overwriting a register that has a
pending global load).
- **No LDS WAW detection**: LDS write-after-write hazards are not currently
flagged. VGPR write-after-load hazards are checked, but two waves writing the
same LDS byte without a barrier are still not reported.

- **Kernel name resolution**: kernel names in race reports may show as `"?"` if
symbol information is not available in the code object.
Expand Down
29 changes: 27 additions & 2 deletions emulation/rocjitsu/lib/python/amdisa/codegen/_generator.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2029,6 +2029,33 @@ def gen_encodings(self) -> None:
public_members.append(func_decl)
class_func_impls.append(func_body)

_enc_upper = inst_enc.enc_name.upper()
_dpp_struct, _dpp8_struct = self._vop_dpp_struct_names(_enc_upper)
if _enc_upper in ('ENC_VOP1', 'ENC_VOP2'):
public_members.append(
cgen.Line(
'std::optional<VgprWriteHookFilter> vgpr_write_hook_filter('
'uint32_t wf_size, uint64_t exec) const override {'
' return amdgpu::vop_write_hook_filter('
'inst_.src0, wf_size, exec, dpp_ctrl_, dpp_row_mask_,'
' dpp_bank_mask_, dpp_bound_ctrl_, sdwa_dst_sel_,'
' sdwa_dst_unused_); }'
)
)
elif _dpp_struct and _enc_upper in (
'ENC_VOP3',
'ENC_VOP3P',
'VOP3_SDST_ENC',
):
public_members.append(
cgen.Line(
'std::optional<VgprWriteHookFilter> vgpr_write_hook_filter('
'uint32_t wf_size, uint64_t exec) const override {'
' return amdgpu::dpp_write_hook_filter('
'inst_.src0, wf_size, exec, dpp_ctrl_, dpp_row_mask_,'
' dpp_bank_mask_, dpp_bound_ctrl_); }'
)
)
class_members.extend(public_members)
class_members.append(
cgen.Statement(f'using OpEncoding = {inst_enc.fmt_enc_name}MachineInst')
Expand All @@ -2041,8 +2068,6 @@ def gen_encodings(self) -> None:
class_members.append(cgen.Statement('std::string owned_mnemonic_'))
# VOP encoding bases store DPP control fields.
# apply_dpp() is a free function in dpp_sdwa_ops.h.
_enc_upper = inst_enc.enc_name.upper()
_dpp_struct, _dpp8_struct = self._vop_dpp_struct_names(_enc_upper)
if (
_dpp_struct
or _dpp8_struct
Expand Down
38 changes: 38 additions & 0 deletions emulation/rocjitsu/lib/python/amdisa/tests/test_generator_profile_gates.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1280,6 +1280,44 @@ def test_generated_dpp_cleanup_uses_full_write_mask_for_dpp16(
assert 'dpp_bound_ctrl_, dpp_fi_' in body


def test_vop_encoding_bases_publish_architectural_write_hook_filter(
amdgpu_generated_root: Path,
):
arches = (
'cdna1',
'cdna2',
'cdna3',
'cdna4',
'rdna1',
'rdna2',
'rdna3',
'rdna3_5',
'rdna4',
'gfx1250',
)
for arch in arches:
encodings = (amdgpu_generated_root / arch / 'encodings.h').read_text()
for class_name, next_class in (('Vop1', 'Vopc'), ('Vop2', 'Vop3p')):
start = encodings.index(f'class {class_name} :')
end = encodings.index(f'class {next_class} :', start)
body = encodings[start:end]
assert 'vgpr_write_hook_filter' in body, arch
assert 'amdgpu::vop_write_hook_filter(' in body, arch
assert 'sdwa_dst_unused_' in body, arch

for arch in ('rdna3', 'rdna3_5', 'rdna4', 'gfx1250'):
encodings = (amdgpu_generated_root / arch / 'encodings.h').read_text()
classes = ('Vop3', 'Vop3p', 'Vop3SdstEnc')
for class_name, next_class in zip(classes, classes[1:] + ('}',)):
start = encodings.index(f'class {class_name} :')
end = encodings.index(
f'class {next_class} :' if next_class != '}' else '\n};', start
)
body = encodings[start:end]
assert 'vgpr_write_hook_filter' in body, (arch, class_name)
assert 'amdgpu::dpp_write_hook_filter(' in body, (arch, class_name)


def test_rdna1_2_generated_vopc_dpp_is_explicitly_unsupported(
amdgpu_generated_root: Path,
):
Expand Down
12 changes: 12 additions & 0 deletions emulation/rocjitsu/lib/rocjitsu/src/rocjitsu/isa/arch/amdgpu/cdna1/encodings.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -476,6 +476,12 @@ class Vop1 : public IsaInstruction<Isa> {
public:
Vop1(std::string_view mnemonic, const Vop1MachineInst *inst, ExecuteFn exec_fn);
void implicit_uses(RegisterSet &uses) const override;
std::optional<VgprWriteHookFilter> vgpr_write_hook_filter(uint32_t wf_size,
uint64_t exec) const override {
return amdgpu::vop_write_hook_filter(inst_.src0, wf_size, exec, dpp_ctrl_, dpp_row_mask_,
dpp_bank_mask_, dpp_bound_ctrl_, sdwa_dst_sel_,
sdwa_dst_unused_);
}
bool default_encoding();
bool has_lit();
bool has_dpp();
Expand Down Expand Up @@ -533,6 +539,12 @@ class Vop2 : public IsaInstruction<Isa> {
public:
Vop2(std::string_view mnemonic, const Vop2MachineInst *inst, ExecuteFn exec_fn);
void implicit_uses(RegisterSet &uses) const override;
std::optional<VgprWriteHookFilter> vgpr_write_hook_filter(uint32_t wf_size,
uint64_t exec) const override {
return amdgpu::vop_write_hook_filter(inst_.src0, wf_size, exec, dpp_ctrl_, dpp_row_mask_,
dpp_bank_mask_, dpp_bound_ctrl_, sdwa_dst_sel_,
sdwa_dst_unused_);
}
bool default_encoding();
bool has_lit();
bool has_dpp();
Expand Down
12 changes: 12 additions & 0 deletions emulation/rocjitsu/lib/rocjitsu/src/rocjitsu/isa/arch/amdgpu/cdna2/encodings.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -460,6 +460,12 @@ class Vop1 : public IsaInstruction<Isa> {
public:
Vop1(std::string_view mnemonic, const Vop1MachineInst *inst, ExecuteFn exec_fn);
void implicit_uses(RegisterSet &uses) const override;
std::optional<VgprWriteHookFilter> vgpr_write_hook_filter(uint32_t wf_size,
uint64_t exec) const override {
return amdgpu::vop_write_hook_filter(inst_.src0, wf_size, exec, dpp_ctrl_, dpp_row_mask_,
dpp_bank_mask_, dpp_bound_ctrl_, sdwa_dst_sel_,
sdwa_dst_unused_);
}
bool default_encoding();
bool has_lit();
bool has_dpp();
Expand Down Expand Up @@ -517,6 +523,12 @@ class Vop2 : public IsaInstruction<Isa> {
public:
Vop2(std::string_view mnemonic, const Vop2MachineInst *inst, ExecuteFn exec_fn);
void implicit_uses(RegisterSet &uses) const override;
std::optional<VgprWriteHookFilter> vgpr_write_hook_filter(uint32_t wf_size,
uint64_t exec) const override {
return amdgpu::vop_write_hook_filter(inst_.src0, wf_size, exec, dpp_ctrl_, dpp_row_mask_,
dpp_bank_mask_, dpp_bound_ctrl_, sdwa_dst_sel_,
sdwa_dst_unused_);
}
bool default_encoding();
bool has_lit();
bool has_dpp();
Expand Down
12 changes: 12 additions & 0 deletions emulation/rocjitsu/lib/rocjitsu/src/rocjitsu/isa/arch/amdgpu/cdna3/encodings.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -448,6 +448,12 @@ class Vop1 : public IsaInstruction<Isa> {
public:
Vop1(std::string_view mnemonic, const Vop1MachineInst *inst, ExecuteFn exec_fn);
void implicit_uses(RegisterSet &uses) const override;
std::optional<VgprWriteHookFilter> vgpr_write_hook_filter(uint32_t wf_size,
uint64_t exec) const override {
return amdgpu::vop_write_hook_filter(inst_.src0, wf_size, exec, dpp_ctrl_, dpp_row_mask_,
dpp_bank_mask_, dpp_bound_ctrl_, sdwa_dst_sel_,
sdwa_dst_unused_);
}
bool default_encoding();
bool has_lit();
bool has_dpp();
Expand Down Expand Up @@ -505,6 +511,12 @@ class Vop2 : public IsaInstruction<Isa> {
public:
Vop2(std::string_view mnemonic, const Vop2MachineInst *inst, ExecuteFn exec_fn);
void implicit_uses(RegisterSet &uses) const override;
std::optional<VgprWriteHookFilter> vgpr_write_hook_filter(uint32_t wf_size,
uint64_t exec) const override {
return amdgpu::vop_write_hook_filter(inst_.src0, wf_size, exec, dpp_ctrl_, dpp_row_mask_,
dpp_bank_mask_, dpp_bound_ctrl_, sdwa_dst_sel_,
sdwa_dst_unused_);
}
bool default_encoding();
bool has_lit();
bool has_dpp();
Expand Down
12 changes: 12 additions & 0 deletions emulation/rocjitsu/lib/rocjitsu/src/rocjitsu/isa/arch/amdgpu/cdna4/encodings.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -450,6 +450,12 @@ class Vop1 : public IsaInstruction<Isa> {
public:
Vop1(std::string_view mnemonic, const Vop1MachineInst *inst, ExecuteFn exec_fn);
void implicit_uses(RegisterSet &uses) const override;
std::optional<VgprWriteHookFilter> vgpr_write_hook_filter(uint32_t wf_size,
uint64_t exec) const override {
return amdgpu::vop_write_hook_filter(inst_.src0, wf_size, exec, dpp_ctrl_, dpp_row_mask_,
dpp_bank_mask_, dpp_bound_ctrl_, sdwa_dst_sel_,
sdwa_dst_unused_);
}
bool default_encoding();
bool has_lit();
bool has_dpp();
Expand Down Expand Up @@ -507,6 +513,12 @@ class Vop2 : public IsaInstruction<Isa> {
public:
Vop2(std::string_view mnemonic, const Vop2MachineInst *inst, ExecuteFn exec_fn);
void implicit_uses(RegisterSet &uses) const override;
std::optional<VgprWriteHookFilter> vgpr_write_hook_filter(uint32_t wf_size,
uint64_t exec) const override {
return amdgpu::vop_write_hook_filter(inst_.src0, wf_size, exec, dpp_ctrl_, dpp_row_mask_,
dpp_bank_mask_, dpp_bound_ctrl_, sdwa_dst_sel_,
sdwa_dst_unused_);
}
bool default_encoding();
bool has_lit();
bool has_dpp();
Expand Down
27 changes: 27 additions & 0 deletions emulation/rocjitsu/lib/rocjitsu/src/rocjitsu/isa/arch/amdgpu/gfx1250/encodings.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -434,6 +434,12 @@ class Vop1 : public IsaInstruction<Isa> {
public:
Vop1(std::string_view mnemonic, const Vop1MachineInst *inst, ExecuteFn exec_fn);
void implicit_uses(RegisterSet &uses) const override;
std::optional<VgprWriteHookFilter> vgpr_write_hook_filter(uint32_t wf_size,
uint64_t exec) const override {
return amdgpu::vop_write_hook_filter(inst_.src0, wf_size, exec, dpp_ctrl_, dpp_row_mask_,
dpp_bank_mask_, dpp_bound_ctrl_, sdwa_dst_sel_,
sdwa_dst_unused_);
}
bool default_encoding();
bool has_lit();
bool has_lit64();
Expand Down Expand Up @@ -496,6 +502,12 @@ class Vop2 : public IsaInstruction<Isa> {
public:
Vop2(std::string_view mnemonic, const Vop2MachineInst *inst, ExecuteFn exec_fn);
void implicit_uses(RegisterSet &uses) const override;
std::optional<VgprWriteHookFilter> vgpr_write_hook_filter(uint32_t wf_size,
uint64_t exec) const override {
return amdgpu::vop_write_hook_filter(inst_.src0, wf_size, exec, dpp_ctrl_, dpp_row_mask_,
dpp_bank_mask_, dpp_bound_ctrl_, sdwa_dst_sel_,
sdwa_dst_unused_);
}
bool default_encoding();
bool has_lit();
bool has_lit64();
Expand Down Expand Up @@ -530,6 +542,11 @@ class Vop3 : public IsaInstruction<Isa> {
public:
Vop3(std::string_view mnemonic, const Vop3MachineInst *inst, ExecuteFn exec_fn);
void implicit_uses(RegisterSet &uses) const override;
std::optional<VgprWriteHookFilter> vgpr_write_hook_filter(uint32_t wf_size,
uint64_t exec) const override {
return amdgpu::dpp_write_hook_filter(inst_.src0, wf_size, exec, dpp_ctrl_, dpp_row_mask_,
dpp_bank_mask_, dpp_bound_ctrl_);
}
bool has_lit_0();
bool has_lit_1();
bool has_lit_0_and_has_lit_1();
Expand All @@ -555,6 +572,11 @@ class Vop3p : public IsaInstruction<Isa> {
public:
Vop3p(std::string_view mnemonic, const Vop3pMachineInst *inst, ExecuteFn exec_fn);
void implicit_uses(RegisterSet &uses) const override;
std::optional<VgprWriteHookFilter> vgpr_write_hook_filter(uint32_t wf_size,
uint64_t exec) const override {
return amdgpu::dpp_write_hook_filter(inst_.src0, wf_size, exec, dpp_ctrl_, dpp_row_mask_,
dpp_bank_mask_, dpp_bound_ctrl_);
}
bool has_lit_0();
bool has_lit_1();
bool has_lit_0_and_has_lit_1();
Expand Down Expand Up @@ -626,6 +648,11 @@ class Vop3SdstEnc : public IsaInstruction<Isa> {
public:
Vop3SdstEnc(std::string_view mnemonic, const Vop3SdstEncMachineInst *inst, ExecuteFn exec_fn);
void implicit_uses(RegisterSet &uses) const override;
std::optional<VgprWriteHookFilter> vgpr_write_hook_filter(uint32_t wf_size,
uint64_t exec) const override {
return amdgpu::dpp_write_hook_filter(inst_.src0, wf_size, exec, dpp_ctrl_, dpp_row_mask_,
dpp_bank_mask_, dpp_bound_ctrl_);
}
bool has_lit_0();
bool has_lit_1();
bool has_lit_0_and_has_lit_1();
Expand Down
12 changes: 12 additions & 0 deletions emulation/rocjitsu/lib/rocjitsu/src/rocjitsu/isa/arch/amdgpu/rdna1/encodings.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -425,6 +425,12 @@ class Vop1 : public IsaInstruction<Isa> {
public:
Vop1(std::string_view mnemonic, const Vop1MachineInst *inst, ExecuteFn exec_fn);
void implicit_uses(RegisterSet &uses) const override;
std::optional<VgprWriteHookFilter> vgpr_write_hook_filter(uint32_t wf_size,
uint64_t exec) const override {
return amdgpu::vop_write_hook_filter(inst_.src0, wf_size, exec, dpp_ctrl_, dpp_row_mask_,
dpp_bank_mask_, dpp_bound_ctrl_, sdwa_dst_sel_,
sdwa_dst_unused_);
}
bool default_encoding();
bool has_lit();
using OpEncoding = Vop1MachineInst;
Expand Down Expand Up @@ -480,6 +486,12 @@ class Vop2 : public IsaInstruction<Isa> {
public:
Vop2(std::string_view mnemonic, const Vop2MachineInst *inst, ExecuteFn exec_fn);
void implicit_uses(RegisterSet &uses) const override;
std::optional<VgprWriteHookFilter> vgpr_write_hook_filter(uint32_t wf_size,
uint64_t exec) const override {
return amdgpu::vop_write_hook_filter(inst_.src0, wf_size, exec, dpp_ctrl_, dpp_row_mask_,
dpp_bank_mask_, dpp_bound_ctrl_, sdwa_dst_sel_,
sdwa_dst_unused_);
}
bool default_encoding();
bool has_lit();
bool hasImpliedLiteral();
Expand Down
12 changes: 12 additions & 0 deletions emulation/rocjitsu/lib/rocjitsu/src/rocjitsu/isa/arch/amdgpu/rdna2/encodings.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -421,6 +421,12 @@ class Vop1 : public IsaInstruction<Isa> {
public:
Vop1(std::string_view mnemonic, const Vop1MachineInst *inst, ExecuteFn exec_fn);
void implicit_uses(RegisterSet &uses) const override;
std::optional<VgprWriteHookFilter> vgpr_write_hook_filter(uint32_t wf_size,
uint64_t exec) const override {
return amdgpu::vop_write_hook_filter(inst_.src0, wf_size, exec, dpp_ctrl_, dpp_row_mask_,
dpp_bank_mask_, dpp_bound_ctrl_, sdwa_dst_sel_,
sdwa_dst_unused_);
}
bool default_encoding();
bool has_lit();
using OpEncoding = Vop1MachineInst;
Expand Down Expand Up @@ -476,6 +482,12 @@ class Vop2 : public IsaInstruction<Isa> {
public:
Vop2(std::string_view mnemonic, const Vop2MachineInst *inst, ExecuteFn exec_fn);
void implicit_uses(RegisterSet &uses) const override;
std::optional<VgprWriteHookFilter> vgpr_write_hook_filter(uint32_t wf_size,
uint64_t exec) const override {
return amdgpu::vop_write_hook_filter(inst_.src0, wf_size, exec, dpp_ctrl_, dpp_row_mask_,
dpp_bank_mask_, dpp_bound_ctrl_, sdwa_dst_sel_,
sdwa_dst_unused_);
}
bool default_encoding();
bool has_lit();
bool hasImpliedLiteral();
Expand Down
Loading
Loading