Commit Graph
123213 Commits
Author SHA1 Message Date
Richard Henderson 3b5257c860 target/arm: Implement EXTQ for SME2p1/SVE2p1
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20250704142112.1018902-90-richard.henderson@linaro.org
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2025-07-04 15:52:22 +01:00
Richard Henderson a4a49a31f4 target/arm: Implement DUPQ for SME2p1/SVE2p1
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20250704142112.1018902-89-richard.henderson@linaro.org
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2025-07-04 15:52:22 +01:00
Richard Henderson 47810f99e8 target/arm: Implement CNTP (predicate as counter) for SME2/SVE2p1
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20250704142112.1018902-88-richard.henderson@linaro.org
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2025-07-04 15:52:22 +01:00
Richard Henderson 4fddbdf934 target/arm: Implement BFMLSLB{L, T} for SME2/SVE2p1
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20250704142112.1018902-87-richard.henderson@linaro.org
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2025-07-04 15:52:22 +01:00
Richard Henderson 1de7ecfc12 target/arm: Implement FADDQV, F{MIN, MAX}{NM}QV for SVE2p1
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20250704142112.1018902-86-richard.henderson@linaro.org
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2025-07-04 15:52:22 +01:00
Richard Henderson 5b334d17e6 target/arm: Implement ANDQV, ORQV, EORQV for SVE2p1
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20250704142112.1018902-85-richard.henderson@linaro.org
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2025-07-04 15:52:22 +01:00
Richard Henderson ab6bf3d93d target/arm: Implement SME2 SEL
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20250704142112.1018902-84-richard.henderson@linaro.org
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2025-07-04 15:52:22 +01:00
Richard Henderson 16fe3bb942 target/arm: Implement SVE2p1 PEXT
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20250704142112.1018902-83-richard.henderson@linaro.org
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2025-07-04 15:52:22 +01:00
Richard Henderson 8f7e127b66 target/arm: Implement {ADD, SMIN, SMAX, UMIN, UMAX}QV for SVE2p1
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20250704142112.1018902-82-richard.henderson@linaro.org
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2025-07-04 15:52:22 +01:00
Richard Henderson b7094bc252 target/arm: Implement SVE2p1 PTRUE (predicate as counter)
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20250704142112.1018902-81-richard.henderson@linaro.org
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2025-07-04 15:52:22 +01:00
Richard Henderson f9cb0ac8ba target/arm: Implement SVE2p1 WHILE (predicate as counter)
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20250704142112.1018902-80-richard.henderson@linaro.org
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2025-07-04 15:52:22 +01:00
Richard Henderson a3dde8e382 target/arm: Implement SVE2p1 WHILE (predicate pair)
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20250704142112.1018902-79-richard.henderson@linaro.org
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2025-07-04 15:52:22 +01:00
Richard Henderson b61f453053 target/arm: Enable PSEL for SVE2p1
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20250704142112.1018902-78-richard.henderson@linaro.org
This instruction is present in both SME(1) and SVE2.1 extensions.

Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2025-07-04 15:52:22 +01:00
Richard Henderson 54e260bbfc target/arm: Split trans_WHILE to lt and gt
Use TRANS_FEAT to select the correct predicate.
Pass the helper and a boolean to do_WHILE.

Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20250704142112.1018902-77-richard.henderson@linaro.org
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2025-07-04 15:52:22 +01:00
Richard Henderson f96fd13c6e target/arm: Move scale by esz into helper_sve_while*
Change the API to pass element count rather than bit count.
This will be helpful later for predicate as counter.

Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20250704142112.1018902-76-richard.henderson@linaro.org
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2025-07-04 15:52:22 +01:00
Richard Henderson 3fd8ce4294 target/arm: Split out do_whileg from helper_sve_whileg
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Message-id: 20250704142112.1018902-75-richard.henderson@linaro.org
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2025-07-04 15:52:22 +01:00
Richard Henderson 56ddef7794 target/arm: Split out do_whilel from helper_sve_whilel
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20250704142112.1018902-74-richard.henderson@linaro.org
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2025-07-04 15:52:22 +01:00
Richard Henderson 91afd009ba target/arm: Expand do_zero inline
Expand to memset plus the return value, when used.

Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Message-id: 20250704142112.1018902-73-richard.henderson@linaro.org
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2025-07-04 15:52:22 +01:00
Richard Henderson 18914dfba7 target/arm: Fold predtest_ones into helper_sve_brkns
Merge predtest_ones into its only caller.

Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20250704142112.1018902-72-richard.henderson@linaro.org
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2025-07-04 15:52:22 +01:00
Richard Henderson bcbe82a293 target/arm: Introduce pred_count_test
For WHILE, we have the count of enabled predicates, so we don't
need to search to compute the PredTest result.  Reuse the logic
that will shortly be required for counted predicates.

Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Message-id: 20250704142112.1018902-71-richard.henderson@linaro.org
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2025-07-04 15:52:22 +01:00
Richard Henderson f8f65ebc0a target/arm: Implement SME2p1 Multiple Zero
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20250704142112.1018902-70-richard.henderson@linaro.org
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2025-07-04 15:52:22 +01:00
Richard Henderson e9b743947b target/arm: Implement FCLAMP for SME2, SVE2p1
This is the single vector version within SVE decode space.

Tested-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20250704142112.1018902-69-richard.henderson@linaro.org
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2025-07-04 15:52:22 +01:00
Richard Henderson c1317025d8 target/arm: Enable SCLAMP, UCLAMP for SVE2p1
These instructions are present in both SME(1) and SVE2.1 extensions.

Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20250704142112.1018902-68-richard.henderson@linaro.org
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2025-07-04 15:52:22 +01:00
Richard Henderson 8b61eff8e7 target/arm: Implement SME2 FCLAMP, SCLAMP, UCLAMP
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20250704142112.1018902-67-richard.henderson@linaro.org
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2025-07-04 15:52:22 +01:00
Richard Henderson 7e27088cfc target/arm: Implement SME2 ZIP, UZP (two registers)
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20250704142112.1018902-66-richard.henderson@linaro.org
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2025-07-04 15:52:22 +01:00
Richard Henderson 70ad5b9fb1 target/arm: Implement SME2 SQRSHR, UQRSHR, SQRSHRN
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Message-id: 20250704142112.1018902-65-richard.henderson@linaro.org
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2025-07-04 15:52:22 +01:00
Richard Henderson 9a7d445c09 target/arm: Move do_urshr, do_srshr to vec_internal.h
Unify two copies of these inline functions.

Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20250704142112.1018902-64-richard.henderson@linaro.org
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2025-07-04 15:52:22 +01:00
Richard Henderson a3a019c0ef target/arm: Implement SME2 ZIP, UZP (four registers)
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20250704142112.1018902-63-richard.henderson@linaro.org
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2025-07-04 15:52:22 +01:00
Richard Henderson 2ccea62157 target/arm: Implement SME2 SUNPK, UUNPK
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Message-id: 20250704142112.1018902-62-richard.henderson@linaro.org
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2025-07-04 15:52:22 +01:00
Richard Henderson 10168d3912 target/arm: Implement SQCVTN, UQCVTN, SQCVTUN for SME2/SVE2p1
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20250704142112.1018902-61-richard.henderson@linaro.org
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2025-07-04 15:52:22 +01:00
Richard Henderson bde57978ce target/arm: Implement SME2 SQCVT, UQCVT, SQCVTU
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Message-id: 20250704142112.1018902-60-richard.henderson@linaro.org
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2025-07-04 15:52:22 +01:00
Richard Henderson 63b2c8de71 target/arm: Use do_[us]sat_[bhs] in sve_helper.c
Replace and remove do_sat_bhs.
This avoids multiple repetitions of INT*_MIN/MAX.

Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20250704142112.1018902-59-richard.henderson@linaro.org
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2025-07-04 15:52:22 +01:00
Richard Henderson 88b7337311 target/arm: Introduce do_[us]sat_[bhs] macros
Inputs are a wider type of indeterminate sign.

Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20250704142112.1018902-58-richard.henderson@linaro.org
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2025-07-04 15:52:22 +01:00
Richard Henderson 2065b1420d target/arm: Implement SME2 FRINTN, FRINTP, FRINTM, FRINTA
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20250704142112.1018902-57-richard.henderson@linaro.org
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2025-07-04 15:52:22 +01:00
Richard Henderson 41167be429 target/arm: Implement SME2 SCVTF, UCVTF
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20250704142112.1018902-56-richard.henderson@linaro.org
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2025-07-04 15:52:22 +01:00
Richard Henderson 91a11f7468 target/arm: Implement SME2 FCVTZS, FCVTZU
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20250704142112.1018902-55-richard.henderson@linaro.org
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2025-07-04 15:52:22 +01:00
Richard Henderson f73ef7f90f target/arm: Implement SME2 FCVT (widening), FCVTL
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20250704142112.1018902-54-richard.henderson@linaro.org
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2025-07-04 15:52:22 +01:00
Richard Henderson 465d36db0e target/arm: Implement SME2 BFCVT, BFCVTN, FCVT, FCVTN
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20250704142112.1018902-53-richard.henderson@linaro.org
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2025-07-04 15:52:22 +01:00
Richard Henderson ccb512d5b5 target/arm: Implement SME2 ADD/SUB (array accumulator)
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20250704142112.1018902-52-richard.henderson@linaro.org
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2025-07-04 15:52:22 +01:00
Richard Henderson c4f514e2b0 target/arm: Implement SME2 FADD, FSUB, BFADD, BFSUB
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20250704142112.1018902-51-richard.henderson@linaro.org
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2025-07-04 15:52:22 +01:00
Richard Henderson 57ef00819e target/arm: Implement SME2 BFMLA, BFMLS
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20250704142112.1018902-50-richard.henderson@linaro.org
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2025-07-04 15:52:22 +01:00
Richard Henderson 77a15fa72a target/arm: Implement SME2 FMLA, FMLS
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20250704142112.1018902-49-richard.henderson@linaro.org
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2025-07-04 15:52:22 +01:00
Richard Henderson e0646fc93e target/arm: Rename gvec_fml[as]_[hs] with _nf_ infix
Emphasize the non-fused nature of these multiply-add.
Matches other helpers such as gvec_rsqrts_nf_[hs].

Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20250704142112.1018902-48-richard.henderson@linaro.org
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2025-07-04 15:52:22 +01:00
Richard Henderson 0a151896eb target/arm: Implement SME2 SMLAL, SMLSL, UMLAL, UMLSL
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20250704142112.1018902-47-richard.henderson@linaro.org
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2025-07-04 15:52:22 +01:00
Richard Henderson 279cd4d312 target/arm: Implement SME2 SVDOT, UVDOT, SUVDOT, USVDOT
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20250704142112.1018902-46-richard.henderson@linaro.org
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2025-07-04 15:52:22 +01:00
Richard Henderson 26c3bafbf3 target/arm: Implement SDOT, UDOT (2-way) for SME2/SVE2p1
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20250704142112.1018902-45-richard.henderson@linaro.org
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2025-07-04 15:52:22 +01:00
Richard Henderson 050ce4fb64 target/arm: Tighten USDOT (vectors) decode
Rename to USDOT_zzzz_4s and force size=2 during decode.

Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20250704142112.1018902-44-richard.henderson@linaro.org
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2025-07-04 15:52:22 +01:00
Richard Henderson a49089cd60 target/arm: Rename SVE SDOT and UDOT patterns
Emphasize the 4-way nature of these dot products.

Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20250704142112.1018902-43-richard.henderson@linaro.org
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2025-07-04 15:52:22 +01:00
Richard Henderson 9fce675a8e target/arm: Implemement SME2 SDOT, UDOT, USDOT, SUDOT
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Message-id: 20250704142112.1018902-42-richard.henderson@linaro.org
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2025-07-04 15:52:22 +01:00
Richard Henderson f7186b4288 target/arm: Rename helper_gvec_*dot_[bh] to *_4[bh]
Emphasize that these are 4-way dot products.

Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20250704142112.1018902-41-richard.henderson@linaro.org
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2025-07-04 15:52:22 +01:00