New IR ops do them four lanes at a time where the backend has them, the
ARM64 JIT and the IR interpreter so far. vmin and vmax compare by
vfpu_minmax_key() in NEON, which took about twelve instructions a lane
through the integer registers before; vslt and vsge are a compare and an
AND with 1.0.
vi2f becomes one SCVTF with fractional bits.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
A new IR op, Vec4VfpuMultiDot, does four vfpu_dot()s against one vector,
the other operands being the rows or Vec4s of a 4x4 block of registers.
The frontend emits it for vmmul and vtfm/vhtfm, and for runs of vdot that
fit it, like a matrix times a vector written out.
On ARM64 it calls a routine that has the dots in lanes rather than the
terms, so nothing is horizontal and nothing goes through the integer
registers. The block comes in with LD1, or LD4 for the transposed form,
when none of it is in host registers. The IR interpreter runs it through
vfpu_dot(); the other backends don't get it yet.
The CPU fuzzer gets runs of vdot shaped for it, and finds no difference
between the cores on them.
There's a routine per number of terms, so vdot.t runs don't pay for a
fourth. The vector's significands are worked out once and multiplied by
element, and the rounding at the end is UCVTF's in round-to-nearest,
which is what games run in; the integer version stays for the other
modes. The routine saves nothing, and the call site keeps only those
registers it clobbers that hold guest state.
JitDotBench, ns per dot: vtfm4.q 3.40, vmmul.q 3.12, a run of vdot.t
2.83, against 4.6 for single dots.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The operands go in q0 and q1 and the result comes back in w1. The routine saves
nothing: the call site flushes whatever guest state is in the few registers it
clobbers (q4-q6, w0-w3), which come late in the allocation order, so usually
there's nothing to do. That's about 20 bytes a call, with nothing going through
memory, and as fast as having the whole thing inline, which made a vmmul.q about
5 KB of code. Inf and NaN go through a routine that calls vfpu_dot_reference().
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
GenerateArm64VfpuDot() generates vfpu_dot() as integer code, once per JIT, to
be called with BL. The operands and result go through mips->vfpuDotArgs and
vfpuDotResult, and the routine preserves every register but x30, so a call
site is two stores, the BL and a load, with nothing flushed. Inf and NaN
operands go to vfpu_dot_reference(), with the caller-saved registers saved.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The prefix fields were decoded by hand in the interpreter, the IR frontend,
JitState, MIPSState and the disassembler, with three copies of the constant
table. They now share small decoders and one table, in MIPS.h. The backends'
Vec4Shuffle lane decoding gets a helper of its own in IRInst.h.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Vec4ClampToZero and Vec2ClampToZero only ever fed Vec4Pack31To8 and
Vec2Pack31To16, for vi2uc and vi2us. The packs now clamp negative lanes
to zero themselves, which saves an op and a vector temp, and lets x64
clamp with PACKUSWB's saturation after an arithmetic shift.
While at it, RISC-V compiles Vec2Unpack16To31, Vec2Pack31To16 and
Vec4Pack32To8, and LoongArch Vec2Unpack16To31, Vec2Pack31To16 and the
non-LSX Vec4Pack32To8, all of which went to the IR interpreter.
LoongArch's Vec2Pack32To16 and Vec2Unpack16To32 now take their scalar
path with LSX too, instead of falling back.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
