- Fixes#12082
The bit-exact VFPU dot product is the default everywhere and fast enough,
so the plain multiply-add paths in the interpreter and the IR frontend go,
along with the AccurateVdot setting, --accurate-vdot, the ForceAccurateVdot
compat flag and the Vec4Dot IR op, which nothing emits any more.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
With LSX, the Vec2 pack and unpack ops went to the interpreter, as they
addressed the lanes of a pair one by one. They're now an interleave or a
pick of halfwords in the pair's register.
That makes pairs, so mapping a quad whose halves are mapped pairs (two
vs2i.s feeding a vi2f.q) joins them with VILVL_D rather than storing both
and loading the quad back, like the ARM64 JIT's ZIP1 (3d20d51c70).
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The ops from 108e66f2d1, so both now get moreVec4Ops. With LSX they're
vector code: vfpu_minmax_key() of each lane picked by VBITSEL, a compare
ANDed with 1.0, and VFFINT with a scale. RISC-V and LoongArch without LSX
do them lane by lane, vslt and vsge as a compare into a GPR rather than
through fpcond.
Vec4VfpuMultiDot stays ARM64-only, so the frontend still doesn't emit it
for these.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
As on RISC-V: the accurate dot products called the IR interpreter. Now the JIT calls a routine
generated at startup, vfpu_dot_lsx() with LSX (the operands go in v8 and v9) and vfpu_dot_scalar()
without (through mips->vfpuDotArgs), with inf and NaN going to vfpu_dot_reference(). With LSX, a
result that lands in one of the operands' Vec4s goes in with VEXTRINS.
VFPUDotJit covers both versions.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
A fused multiply-add rounds once where the IR interpreter rounds twice, so
vdot, vhdp, vtfm and vmmul gave different low bits. Multiply and add
separately, in the IR interpreter's order. (LoongArch's LSX path already
did, in the pairwise order of the x86 and ARM64 JITs.)
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
GenerateArm64VfpuDot() generates vfpu_dot() as integer code, once per JIT, to
be called with BL. The operands and result go through mips->vfpuDotArgs and
vfpuDotResult, and the routine preserves every register but x30, so a call
site is two stores, the BL and a load, with nothing flushed. Inf and NaN
operands go to vfpu_dot_reference(), with the caller-saved registers saved.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The prefix fields were decoded by hand in the interpreter, the IR frontend,
JitState, MIPSState and the disassembler, with three copies of the constant
table. They now share small decoders and one table, in MIPS.h. The backends'
Vec4Shuffle lane decoding gets a helper of its own in IRInst.h.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The reg cache asserts when one reg is mapped as a vector and as a scalar, so
the overlap check has to come first, like the other ops do it. Found by
CPUEquivalence under qemu on CI.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
With LSX a lane group can be mapped as one vector reg, and F() then returns
the same reg for every lane, so the per-lane code wrote only lane 0 (vs2i,
vus2i). Also drop the stale aliasing TODOs; each path reads its sources first.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Vec4ClampToZero and Vec2ClampToZero only ever fed Vec4Pack31To8 and
Vec2Pack31To16, for vi2uc and vi2us. The packs now clamp negative lanes
to zero themselves, which saves an op and a vector temp, and lets x64
clamp with PACKUSWB's saturation after an arithmetic shift.
While at it, RISC-V compiles Vec2Unpack16To31, Vec2Pack31To16 and
Vec4Pack32To8, and LoongArch Vec2Unpack16To31, Vec2Pack31To16 and the
non-LSX Vec4Pack32To8, all of which went to the IR interpreter.
LoongArch's Vec2Pack32To16 and Vec2Unpack16To32 now take their scalar
path with LSX too, instead of falling back.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
