A new IR op, Vec4VfpuMultiDot, does four vfpu_dot()s against one vector,
the other operands being the rows or Vec4s of a 4x4 block of registers.
The frontend emits it for vmmul and vtfm/vhtfm, and for runs of vdot that
fit it, like a matrix times a vector written out.
On ARM64 it calls a routine that has the dots in lanes rather than the
terms, so nothing is horizontal and nothing goes through the integer
registers. The block comes in with LD1, or LD4 for the transposed form,
when none of it is in host registers. The IR interpreter runs it through
vfpu_dot(); the other backends don't get it yet.
The CPU fuzzer gets runs of vdot shaped for it, and finds no difference
between the cores on them.
There's a routine per number of terms, so vdot.t runs don't pay for a
fourth. The vector's significands are worked out once and multiplied by
element, and the rounding at the end is UCVTF's in round-to-nearest,
which is what games run in; the integer version stays for the other
modes. The routine saves nothing, and the call site keeps only those
registers it clobbers that hold guest state.
JitDotBench, ns per dot: vtfm4.q 3.40, vmmul.q 3.12, a run of vdot.t
2.83, against 4.6 for single dots.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
- Copy propagation through an FPR temp didn't stop when an instruction
rewrote the temp in place (it compared an FPR number against the +32
offset reg), so later reads lost that write.
- A read of the temp in both operands only had src1 replaced, yet the copy
into the temp was still removed.
- The replacement matched operands by number without checking their type,
so a StoreFloat whose GPR address had the temp's number got its address
replaced (IRVTEMP_PFX_S and IRTEMP_0 are both 192).
- A write to lanes 1-3 of a Vec4 temp wasn't noticed.
- IRReadsFromFPRs stopped after the F operands, missing Vec4Scale's vector.
- Exits and barriers didn't count as reading everything, so a write to a
real reg could be moved above an exit.
- Load32Linked and Store32Conditional were removed when their reg was
overwritten unread, losing LLBIT and the store.
Also fixes an off-by-one in the vec src3 read check. The unit test now
reports every failing case instead of stopping at the first.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
