Commit Graph

123590 Commits

Author SHA1 Message Date
Paolo Bonzini e1c689c218 meson, cargo: require Rust 1.83.0
Reviewed-by: Zhao Liu <zhao1.liu@intel.com>
Link: https://lore.kernel.org/r/20250908105005.2119297-4-pbonzini@redhat.com
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2025-09-17 19:00:56 +02:00
Paolo Bonzini 091f115ea5 configure: bump Meson to 1.9.0 for use with Rust
Meson 1.9.0 provides mixed linking of Rust and C objects.  As a side effect,
this also allows adding dependencies with "sources: ..." files to Rust crates
that use structured_sources().

It can also clean up up the meson.build files for Rust noticeably, but due
to an issue with doctests (see https://github.com/mesonbuild/meson/pull/14973)
that will have to wait for 1.9.1.

Reviewed-by: Zhao Liu <zhao1.liu@intel.com>
Link: https://lore.kernel.org/r/20250908105005.2119297-3-pbonzini@redhat.com
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2025-09-17 19:00:56 +02:00
Paolo Bonzini 614fff7c0a ci: temporarily remove rust from Ubuntu
This is for the purpose of getting an easy-to-use base for future
development.  The plan is:
- that Debian will require trixie to enable Rust usage
- that Ubuntu will backport 1.83 to its 22.04 and 24.04 versions
  (https://bugs.launchpad.net/ubuntu/+source/rustc-1.83/+bug/2120318)

Marc-André is working on adding Rust to other CI jobs.

Reviewed-by: Zhao Liu <zhao1.liu@intel.com>
Link: https://lore.kernel.org/r/20250908105005.2119297-2-pbonzini@redhat.com
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2025-09-17 19:00:56 +02:00
Paolo Bonzini b422a7bff6 tcg/user: do not set exit_request gratuitously
Whenever user-mode emulation needs to go all the way out of the cpu
exec loop, it uses cpu_exit(), which already sets cpu->exit_request.

Therefore, there is no need for tcg_kick_vcpu_thread() to set
cpu->exit_request again outside system emulation.

Reviewed-by: Igor Mammedov <imammedo@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2025-09-17 19:00:56 +02:00
Paolo Bonzini d5e33b5f8f accel: make all calls to qemu_process_cpu_events look the same
There is no reason for some accelerators to use qemu_process_cpu_events_common
(which is separated from qemu_process_cpu_events() specifically for round
robin TCG).  They can also check for events directly on the first pass through
the loop, instead of setting cpu->exit_request to true.

Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2025-09-17 19:00:56 +02:00
Paolo Bonzini 9a191d3782 cpus: clear exit_request in qemu_process_cpu_events
Make the code common to all accelerators: after seeing cpu->exit_request
set to true, accelerator code needs to reach qemu_process_cpu_events_common().

So for the common cases where they use qemu_process_cpu_events(), go ahead and
clear it in there.  Note that the cheap qatomic_set() is enough because
at this point the thread has taken the BQL; qatomic_set_mb() is not needed.
In particular, this is the ordering of the communication between
I/O and vCPU threads is always the same.

In the I/O thread:

(a) store other memory locations that will be checked if cpu->exit_request
    or cpu->interrupt_request is 1 (for example cpu->stop or cpu->work_list
    for cpu->exit_request)

(b) cpu_exit(): store-release cpu->exit_request, or
(b) cpu_interrupt(): store-release cpu->interrupt_request

>>> at this point, cpu->halt_cond is broadcast and the BQL released

(c) do the accelerator-specific kick (e.g. write icount_decr for TCG,
    pthread_kill for KVM, etc.)

In the vCPU thread instead the opposite order is respected:

(c) the accelerator's execution loop exits thanks to the kick

(b) then the inner execution loop checks cpu->interrupt_request
    and cpu->exit_request.  If needed cpu->interrupt_request is
    converted into cpu->exit_request when work is needed outside
    the execution loop.

(a) then the other memory locations are checked.  Some may need to
    be read under the BQL, but the vCPU thread may also take other
    locks (e.g. for queued work items) or none at all.

qatomic_set_mb() would only be needed if the halt sleep was done
outside the BQL (though in that case, cpu->exit_request probably
would be replaced by a QemuEvent or something like that).

Reviewed-by: Igor Mammedov <imammedo@redhat.com>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2025-09-17 19:00:56 +02:00
Paolo Bonzini 758e5de501 bsd-user, linux-user: introduce qemu_process_cpu_events
Add a user-mode emulation version of the function.  More will be
added later, for now it is just process_queued_cpu_work.

Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2025-09-17 19:00:56 +02:00
Paolo Bonzini 871de7078f treewide: rename qemu_wait_io_event/qemu_wait_io_event_common
Do so before extending it to the user-mode emulators, where there is no
such thing as an "I/O thread".

Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2025-09-17 19:00:55 +02:00
Paolo Bonzini f8217ae54e cpus: properly kick CPUs out of inner execution loop
Now that cpu_exit() actually kicks all accelerators, use it whenever
the message to another thread is processed in qemu_wait_io_event().

Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2025-09-17 19:00:55 +02:00
Paolo Bonzini dcb46ecb2e cpus: remove TCG-ism from cpu_exit()
Now that TCG has its own kick function, make cpu_exit() do the right kick
for all accelerators.

Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Reviewed-by: Igor Mammedov <imammedo@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2025-09-17 19:00:55 +02:00
Paolo Bonzini 61d996da50 accel/tcg: inline cpu_exit()
Right now, cpu_exit() is not usable from all accelerators because it
includes a TCG-specific thread kick.  In fact, cpu_exit() doubles as
the TCG thread-kick via tcg_kick_vcpu_thread().

In preparation for changing that, inline cpu_exit() into
tcg_kick_vcpu_thread().  The direction of the calls can then be
reversed, with an accelerator-independent cpu_exit() calling into
qemu_vcpu_kick() rather than the opposite.

Reviewed-by: Igor Mammedov <imammedo@redhat.com>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2025-09-17 19:00:55 +02:00
Paolo Bonzini 9cf342b491 accel/tcg: create a thread-kick function for TCG
Round-robin TCG is calling into cpu_exit() directly.  In preparation
for making cpu_exit() usable from all accelerators, define a generic
thread-kick function for TCG which is used directly in the multi-threaded
case, and through CPU_FOREACH in the round-robin case.

Use it also for user-mode emulation, and take the occasion to move
the implementation to accel/tcg/user-exec.c.

Reviewed-by: Igor Mammedov <imammedo@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2025-09-17 19:00:55 +02:00
Paolo Bonzini f084ff128b accel: use atomic accesses for exit_request
CPU threads write exit_request as a "note to self" that they need to
go out to a slow path.  This write happens out of the BQL and can be
a data race with another threads' cpu_exit(); use atomic accesses
consistently.

While at it, change the source argument from int ("1") to bool ("true").

Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Reviewed-by: Peter Xu <peterx@redhat.com>
Reviewed-by: Igor Mammedov <imammedo@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2025-09-17 19:00:55 +02:00
Paolo Bonzini ac6c8a390b accel: use store_release/load_acquire for cross-thread exit_request
Reads and writes cpu->exit_request do not use a load-acquire/store-release
pair right now, but this means that cpu_exit() may not write cpu->exit_request
after any flags that are read by the vCPU thread.

Probably everything is protected one way or the other by the BQL, because
cpu->exit_request leads to the slow path, where the CPU thread often takes
the BQL (for example, to go to sleep by waiting on the BQL-protected
cpu->halt_cond); but it's not clear, so use load-acquire/store-release
consistently.

Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Reviewed-by: Peter Xu <peterx@redhat.com>
Reviewed-by: Igor Mammedov <imammedo@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2025-09-17 19:00:55 +02:00
Paolo Bonzini 9e1ecd4aaa cpus: document that qemu_cpu_kick() can be used for BQL-less operation
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2025-09-17 19:00:55 +02:00
Paolo Bonzini 27e76d0101 cpu-common: use atomic access for interrupt_request
Writes to interrupt_request used non-atomic accesses, but there are a
few cases where the access was not protected by the BQL.  Now that
there is a full set of helpers, it's easier to guarantee that
interrupt_request accesses are fully atomic, so just drop the
requirement instead of fixing them.

Reviewed-by: Igor Mammedov <imammedo@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2025-09-17 19:00:55 +02:00
Paolo Bonzini 602d5ebba2 treewide: clear bits of cs->interrupt_request with cpu_reset_interrupt()
Open coding cpu_reset_interrupt() can cause bugs if the BQL is not
taken, for example i386 has the call chain kvm_cpu_exec() ->
kvm_put_vcpu_events() -> kvm_arch_put_registers().

Reviewed-by: Igor Mammedov <imammedo@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2025-09-17 19:00:55 +02:00
Paolo Bonzini 11a73c6ea3 user-exec: remove cpu_interrupt() stub
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2025-09-17 19:00:55 +02:00
Paolo Bonzini bd1cefdd9f target-arm: remove uses of cpu_interrupt() for user-mode emulation
Arm leaves around some functions that use cpu_interrupt(), even for
user-mode emulation when the code is unreachable.  Pull out the
system-mode implementation to a separate file, and add stubs for
CONFIG_USER_ONLY.

Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2025-09-17 19:00:55 +02:00
Paolo Bonzini 3efe1a0f60 target/i386: limit a20 to system emulation
It is not used by user-mode emulation and is the only caller of
cpu_interrupt() in qemu-i386 and qemu-x86_64.

Reviewed-by: Igor Mammedov <imammedo@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2025-09-17 19:00:55 +02:00
Paolo Bonzini a445d3b85c target/sparc: limit cpu_check_irqs to system emulation
It is not used by user-mode emulation and is the only caller of
cpu_interrupt() in qemu-sparc* binaries.

Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2025-09-17 19:00:55 +02:00
Paolo Bonzini 588ffa75eb target/ppc: limit cpu_interrupt_exittb to system emulation
It is not used by user-mode emulation and is the only caller of
cpu_interrupt() in qemu-ppc* binaries.

Reviewed-by: Igor Mammedov <imammedo@redhat.com>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2025-09-17 19:00:55 +02:00
Richard Henderson 2e66fb24a0 Merge tag 'pull-misc-20250916' of https://gitlab.com/rth7680/qemu into staging
.gitmodules: move u-boot mirrors to qemu-project-mirrors
tests/functional/x86_64: Accept a few locked pages in test_memlock.py

# -----BEGIN PGP SIGNATURE-----
#
# iQFRBAABCgA7FiEEekgeeIaLTbaoWgXAZN846K9+IV8FAmjJmVwdHHJpY2hhcmQu
# aGVuZGVyc29uQGxpbmFyby5vcmcACgkQZN846K9+IV+RMQf/e2ScmOiJaD/5ba5i
# RPZneCUOV0AkVSpAwxKX9M1Gsroz/NZefVp9yh2wPdcoAPnLjoFJaJuEv1jEPPVh
# /RApI2NehuqWTzndY0dci6vV6YzxLoMZsQWoBRz0l3SfzVq0YuePSlV+YkTMSfgX
# z9ocwPwGG0AEYvEllVPq5UVHZlEoE4BE23QyHtyuXRwjqo3BHjJKQXBK4hA5vAYQ
# gbWiFs3QJ+fsKQk7Z2KJDc6k4H/Oql4T2bILGjQRb9DG4yR1TIvwIsbOj2dFHVrs
# 6HdP5e6PweuHGc8mdFEHJFgZrdA8exxwT0lq/5vYxvjf86nick+z3XrhOXWoHq6G
# L6OgLg==
# =YBDc
# -----END PGP SIGNATURE-----
# gpg: Signature made Tue 16 Sep 2025 10:07:40 AM PDT
# gpg:                using RSA key 7A481E78868B4DB6A85A05C064DF38E8AF7E215F
# gpg:                issuer "richard.henderson@linaro.org"
# gpg: Good signature from "Richard Henderson <richard.henderson@linaro.org>" [ultimate]

* tag 'pull-misc-20250916' of https://gitlab.com/rth7680/qemu:
  .gitmodules: move u-boot mirrors to qemu-project-mirrors
  tests/functional/x86_64: Accept a few locked pages in test_memlock.py

Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
2025-09-16 10:10:51 -07:00
Richard Henderson 41511ed734 Merge tag 'python-pull-request' of https://gitlab.com/jsnow/qemu into staging
Python Pull Request

Python 3.14 support & synchronize with python-qemu-qmp repo

# -----BEGIN PGP SIGNATURE-----
#
# iQIzBAABCgAdFiEE+ber27ys35W+dsvQfe+BBqr8OQ4FAmjJjxIACgkQfe+BBqr8
# OQ48aA/+JRRIEN8LMbNDRvPTTkvCxstSAb2q8yA+8ccWg0H+EGcewjd+oCoPOqjC
# SwIMAGYJ6Dv2LW6c+rK6VjKw1Da8J9WgEpKmfoWu+1Pef8odU5PoRhAvvZdMq+Eh
# Kqk0r1f87fTiZK1gCBhBUIO0oTroOYxDvIYV0B6UFDPArL8jJ5eTpGLCVAYuk8tH
# MuzQD0IcxCBoraOx9vqVMbKIHwMH/m9pJ2IqINzIStpLoFgT1d5V9CoKXImMVXmF
# XovcMWQzFz1a/lm0ybSAzhgXcpW/vNjstb1IcrigYjQWXU6S+/bRpq17c2WqAJtG
# 78Dal7heSjpvWyyCCii+QO+BegH53Mgz3W+aQN7+fkcepjivVYy8tnxOrSjJR+pX
# DqRhMNSc4CrLvJH4BOHKUsJaWMxjd4oJiNhUmhJ7MxZhPTHZvERsOo9kpoJo4eTw
# GhRV98FnJbotgs2kjQpSBF8FDj9LZqPwTfMuEU2NUsIB9o7/Iqj36RDe9L+2r9Ch
# 2UKhnUg58y4eYFoC4CO8yCfjsR6HzLdqiVaDhcu5pdQM0Dw1pxrSIHb6faNmSLL5
# v0brhgJGujWt6wAc2c3ASMf8qpWkBrlVfHybodOB2cUDcRgNk85M/s41PnGShqBZ
# Qq7VW9zR4sejwof9dTwYKuwsNzxzFdS2nLwPPkud5aDngrLsNn0=
# =jZpa
# -----END PGP SIGNATURE-----
# gpg: Signature made Tue 16 Sep 2025 09:23:46 AM PDT
# gpg:                using RSA key F9B7ABDBBCACDF95BE76CBD07DEF8106AAFC390E
# gpg: Good signature from "John Snow (John Huston) <jsnow@redhat.com>" [unknown]
# gpg: WARNING: This key is not certified with a trusted signature!
# gpg:          There is no indication that the signature belongs to the owner.
# Primary key fingerprint: FAEB 9711 A12C F475 812F  18F2 88A9 064D 1835 61EB
#      Subkey fingerprint: F9B7 ABDB BCAC DF95 BE76  CBD0 7DEF 8106 AAFC 390E

* tag 'python-pull-request' of https://gitlab.com/jsnow/qemu:
  iotests/check: always enable all python warnings
  iotests/151: ensure subprocesses are cleaned up
  iotests/147: ensure temporary sockets are closed before exiting
  python: ensure QEMUQtestProtocol closes its socket
  iotests: drop compat for old version context manager
  python: synchronize qemu.qmp documentation
  python: backport 'avoid creating additional event loops per thread'
  python: backport 'Remove deprecated get_event_loop calls'
  python: backport 'qmp-tui: Do not crash if optional dependencies are not met'
  python: backport 'qmp-shell-wrap: handle missing binary gracefully'
  python: backport 'make require() preserve async-ness'
  python: backport 'feat: allow setting read buffer limit'
  python: backport 'qmp-shell: add common_parser()'
  python: backport 'Use @asynciocontextmanager'
  python: backport 'drop Python3.6 workarounds'
  python: backport 'protocol: adjust logging name when changing client name'
  python: backport 'kick event queue on legacy event_pull()'
  python: backport 'EventListener: add __repr__ method'
  python: backport 'Change error classes to have better repr methods'

Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
2025-09-16 10:10:29 -07:00
Richard Henderson 5bf071485a Merge tag 'mem-staging-pull-request' of https://gitlab.com/peterx/qemu into staging
Memory pull for 10.2

- Peter's fix on flatview_access_allowed()
- Peter's fix on MR circular ref

# -----BEGIN PGP SIGNATURE-----
#
# iIgEABYKADAWIQS5GE3CDMRX2s990ak7X8zN86vXBgUCaMg4oxIccGV0ZXJ4QHJl
# ZGhhdC5jb20ACgkQO1/MzfOr1wYeLAD+LQ44LdRTjbdlAbDjSNnCorfEBFUmNysK
# St4ut4Z9ZzAA+gK8DO12zc41Oi51NaBdD+X0s94DCV4UFl4Cz1D8HoIF
# =hAUJ
# -----END PGP SIGNATURE-----
# gpg: Signature made Mon 15 Sep 2025 09:02:43 AM PDT
# gpg:                using EDDSA key B9184DC20CC457DACF7DD1A93B5FCCCDF3ABD706
# gpg:                issuer "peterx@redhat.com"
# gpg: Good signature from "Peter Xu <xzpeter@gmail.com>" [unknown]
# gpg:                 aka "Peter Xu <peterx@redhat.com>" [unknown]
# gpg: WARNING: The key's User ID is not certified with a trusted signature!
# gpg:          There is no indication that the signature belongs to the owner.
# Primary key fingerprint: B918 4DC2 0CC4 57DA CF7D  D1A9 3B5F CCCD F3AB D706

* tag 'mem-staging-pull-request' of https://gitlab.com/peterx/qemu:
  memory: Fix leaks due to owner-shared MRs circular references
  memory: Fix addr/len for flatview_access_allowed()

Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
2025-09-16 10:09:59 -07:00
Alex Bennée a11d1847d5 .gitmodules: move u-boot mirrors to qemu-project-mirrors
To continue our GitLab Open Source Program license we need to pass an
automated license check for all repos under qemu-project. While U-Boot
is clearly GPLv2 rather than fight with the automated validation
script just move the mirror across to a separate project.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Suggested-by: Daniel P. Berrangé <berrange@redhat.com>
Cc: qemu-stable@nongnu.org
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-ID: <20250908141911.2546063-1-alex.bennee@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
2025-09-16 09:58:20 -07:00
Richard Henderson aa3c761c70 tests/functional/x86_64: Accept a few locked pages in test_memlock.py
Startup of libgcrypt locks a small pool of pages -- by default 16k.
Testing for zero locked pages is isn't correct, while testing for
32k is a decent compromise.

Reviewed-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
2025-09-16 09:55:52 -07:00
Daniel P. Berrangé 9a494d8353 iotests/check: always enable all python warnings
Of most importance is that this gives us a heads-up if anything
we rely on has been deprecated. The default python behaviour
only emits a warning if triggered from __main__ which is very
limited.

Setting the env variable further ensures that any python child
processes will also display warnings.

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2025-09-15 14:36:02 -04:00
Daniel P. Berrangé 2b2fb25c2a iotests/151: ensure subprocesses are cleaned up
The iotest 151 creates a bunch of subprocesses, with their stdout
connected to a pipe but never reads any data from them and does
not gurantee the processes are killed on cleanup.

This triggers resource leak warnings from python when the
subprocess.Popen object is garbage collected.

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2025-09-15 14:36:02 -04:00
Daniel P. Berrangé d4d0ebfcc9 iotests/147: ensure temporary sockets are closed before exiting
This avoids the python resource leak detector from issuing warnings
in the iotests.

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2025-09-15 14:36:01 -04:00
Daniel P. Berrangé 6ccb48ffc1 python: ensure QEMUQtestProtocol closes its socket
While QEMUQtestMachine closes the socket that was passed to
QEMUQtestProtocol, the python resource leak manager still
believes that the copy QEMUQtestProtocol holds is open. We
must explicitly call close to avoid this leak warnnig.

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2025-09-15 14:36:01 -04:00
Daniel P. Berrangé 82c7cb93c7 iotests: drop compat for old version context manager
Our minimum python is now 3.9, so back compat with prior
python versions is no longer required.

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2025-09-15 14:36:01 -04:00
John Snow f414048e32 python: synchronize qemu.qmp documentation
This patch collects comments and documentation changes from many commits
in the python-qemu-qmp repository; bringing the qemu.git copy in
bit-identical alignment with the standalone library *except* for several
copyright messages that reference the "LICENSE" file which is, for QEMU,
named "COPYING" instead and are therefore left unchanged.

Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2025-09-15 14:36:01 -04:00
John Snow 85f223e5b0 python: backport 'avoid creating additional event loops per thread'
This commit is two backports squashed into one to avoid regressions.

python: *really* remove get_event_loop

A prior commit, aa1ff990, switched away from using get_event_loop *by
default*, but this is not good enough to avoid deprecation warnings as
`asyncio.get_event_loop_policy().get_event_loop()` is *also*
deprecated. Replace this mechanism with explicit calls to
asyncio.get_new_loop() and revise the cleanup mechanisms in __del__ to
match.

python: avoid creating additional event loops per thread

"Too hasty by far!", commit 21ce2ee4 attempted to avoid deprecated
behavior altogether by calling new_event_loop() directly if there was no
loop currently running, but this has the unfortunate side effect of
potentially creating multiple event loops per thread if tests
instantiate multiple QMP connections in a single thread. This behavior
is apparently not well-defined and causes problems in some, but not all,
combinations of Python interpreter version and platform environment.

Partially revert to Daniel Berrange's original patch, which calls
get_event_loop and simply suppresses the deprecation warning in
Python<=3.13. This time, however, additionally register new loops
created with new_event_loop() so that future calls to get_event_loop()
will return the loop already created.

Reported-by: Richard W.M. Jones <rjones@redhat.com>
Reported-by: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: John Snow <jsnow@redhat.com>
cherry picked from commit python-qemu-qmp@21ce2ee4f2df87efe84a27b9c5112487f4670622
cherry picked from commit python-qemu-qmp@c08fb82b38212956ccffc03fc6d015c3979f42fe
Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2025-09-15 14:36:01 -04:00
John Snow 5d99044d09 python: backport 'Remove deprecated get_event_loop calls'
This method was deprecated in 3.12 because it ordinarily should not be
used from coroutines; if there is not a currently running event loop,
this automatically creates a new event loop - which is usually not what
you want from code that would ever run in the bottom half.

In our case, we do want this behavior in two places:

(1) The synchronous shim, for convenience: this allows fully sync
programs to use QEMUMonitorProtocol() without needing to set up an event
loop beforehand. This is intentional to fully box in the async
complexities into the legacy sync shim.

(2) The qmp_tui shell; instead of relying on asyncio.run to create and
run an asyncio program, we need to be able to pass the current asyncio
loop to urwid setup functions. For convenience, again, we create one if
one is not present to simplify the creation of the TUI appliance.

The remaining user of get_event_loop() was in fact one of the erroneous
users that should not have been using this function: if there's no
running event loop inside of a coroutine, you're in big trouble :)

Signed-off-by: John Snow <jsnow@redhat.com>
cherry picked from commit python-qemu-qmp@aa1ff9907603a3033296027e1bd021133df86ef1
Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2025-09-15 14:36:01 -04:00
John Snow fd0ed46d4e python: backport 'qmp-tui: Do not crash if optional dependencies are not met'
Based on the discussion at https://github.com/pypa/pip/issues/9726 -
even though the setuptools documentation implies that it is possible to
guard script execution with optional dependency groups, this is not true
in practice with the scripts generated by pip.

Just do the simple thing and guard the import statements.

Signed-off-by: John Snow <jsnow@redhat.com>
cherry picked from commit python-qemu-qmp@df520dcacf9a75dd4c82ab1129768de4128b554c
Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2025-09-15 14:36:01 -04:00
John Snow fcaeeb7653 python: backport 'qmp-shell-wrap: handle missing binary gracefully'
Signed-off-by: John Snow <jsnow@redhat.com>
cherry picked from commit python-qemu-qmp@9c889dcbd58817b0c917a9d2dd16161f48ac8203
Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2025-09-15 14:36:01 -04:00
John Snow a50b8572f0 python: backport 'make require() preserve async-ness'
This is not strictly needed functionality-wise, but doing this allows
sphinx to see which decorated methods are async. Without this, sphinx
misses the "async" classifier on generated docs, which ... for an async
library, isn't great.

It does make an already gnarly function even gnarlier, though.

So, what's going on here?

A synchronous function (like require() before this patch) can return a
coroutine that can be awaited on, for example:

  def some_func():
      return asyncio.task(asyncio.sleep(5))

  async def some_async_func():
      await some_func()

However, this function is not considered to be an "async" function in
the eyes of the abstract syntax tree. Specifically,
some_func.__code__.co_flags will not be set with CO_COROUTINE.

The interpreter uses this flag to know if it's legal to use "await" from
within the body of the function. Since this function is just wrapping
another function, it doesn't matter much for the decorator, but sphinx
uses the stdlib inspect.iscoroutinefunction() to determine when to add
the "async" prefix in generated output. This function uses the presence
of CO_COROUTINE.

So, in order to preserve the "async" flag for docs, the require()
decorator needs to differentiate based on whether it is decorating a
sync or async function and use a different wrapping mechanism
accordingly.

Phew.

Signed-off-by: John Snow <jsnow@redhat.com>
cherry picked from commit python-qemu-qmp@40aa9699d619849f528032aa456dd061a4afa957
Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2025-09-15 14:36:01 -04:00
Adam Dorsey 653f501434 python: backport 'feat: allow setting read buffer limit'
Expose the limit parameter of the underlying StreamReader and StreamWriter
instances.

This is helpful for the use case of transferring files in and out of a VM
via the QEMU guest agent's guest-file-open, guest-file-read, guest-file-write,
and guest-file-close methods, as it allows pushing the buffer size up to the
guest agent's limit of 48MB per transfer.

Signed-off-by: Adam Dorsey <adam@dorseys.email>
cherry picked from commit python-qemu-qmp@9ba6a698344eb3b570fa4864e906c54042824cd6
cherry picked from commit python-qemu-qmp@e4d0d3f835d82283ee0e48438d1b154e18303491
[Squashed in linter fixups. --js]
Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2025-09-15 14:36:01 -04:00
John Snow 8fd9ccebd9 python: backport 'qmp-shell: add common_parser()'
Signed-off-by: John Snow <jsnow@redhat.com>
cherry picked from commit python-qemu-qmp@20a88c2471f37d10520b2409046d59e1d0f1e905
Signed-off-by: John Snow <jsnow@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2025-09-15 14:36:01 -04:00
John Snow 0408b8d7a0 python: backport 'Use @asynciocontextmanager'
This removes a non-idiomatic use of a "coroutine callback" in favor of
something a bit more standardized.

Signed-off-by: John Snow <jsnow@redhat.com>
cherry picked from commit python-qemu-qmp@commit 97f7ffa3be17a50544b52767d14b6fd478c07b9e
Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2025-09-15 14:36:01 -04:00
John Snow f9d2e0a3bd python: backport 'drop Python3.6 workarounds'
Now that the minimum version is 3.7, drop some of the 3.6-specific hacks
we've been carrying. A single remaining compatibility hack concerning
3.6's lack of @asynccontextmanager is addressed in the following commit.

Signed-off-by: John Snow <jsnow@redhat.com>
cherry picked from commit python-qemu-qmp@3e8e34e594cfc6b707e6f67959166acde4b421b8
Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2025-09-15 14:36:01 -04:00
John Snow 094ded5227 python: backport 'protocol: adjust logging name when changing client name'
The client name is mutable, so the logging name should also change to
reflect it when it changes.

Signed-off-by: John Snow <jsnow@redhat.com>
cherry picked from commit python-qemu-qmp@e10b73c633ce138ba30bc8beccd2ab31989eaf3d
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2025-09-15 14:36:01 -04:00
John Snow 1e343714bf python: backport 'kick event queue on legacy event_pull()'
This corrects an oversight in qmp-shell operation where new events will
not accumulate in the event queue when pressing "enter" with an empty
command buffer, so no new events show up.

Reported-by: Jag Raman <jag.raman@oracle.com>
Signed-off-by: John Snow <jsnow@redhat.com>
cherry picked from commit python-qemu-qmp@0443582d16cf9efd52b2c41a7b5be7af42c856cd
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2025-09-15 14:36:01 -04:00
John Snow cb0e438040 python: backport 'EventListener: add __repr__ method'
When the object is not stateful, this repr method prints what you'd
expect. In cases where there are pending events, the output is augmented
to illustrate that.

The object itself has no idea if it's "active" or not, so it cannot
convey that information.

Signed-off-by: John Snow <jsnow@redhat.com>
cherry picked from commit python-qemu-qmp@8a6f2e136dae395fec8aa5fd77487cfe12d9e05e
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2025-09-15 14:36:01 -04:00
John Snow 2d26741fc5 python: backport 'Change error classes to have better repr methods'
By passing all of the arguments to the base class and overriding the
__str__ method when we want a different "human readable" message that
isn't just printing the list of arguments, we can ensure that all custom
error classes have a reasonable __repr__ implementation.

In the case of ExecuteError, the pseudo-field that isn't actually
correlated to an input argument can be re-imagined as a read-only
property; this forces consistency in the class and makes the repr output
more obviously correct.

Signed-off-by: John Snow <jsnow@redhat.com>
cherry picked from commit python-qemu-qmp@afdb7893f3b34212da4259b7202973f9a8cb85b3
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2025-09-15 14:36:01 -04:00
Peter Xu ac7a892fd3 memory: Fix leaks due to owner-shared MRs circular references
Currently, QEMU refcounts the MR by always taking it from the owner.

It's common that one object will have multiple MR objects embeded in the
object itself.  All the MRs in this case share the same lifespan of the
owner object.

It's also common that in the instance_init() of an object, MR A can be a
container of MR B, C, D, by using memory_region_add_subregion*() set of
memory region APIs.

Now we have a circular reference issue, as when adding subregions for MR A,
we essentially incremented the owner's refcount within the instance_init(),
meaning the object will be self-boosted and its refcount can never go down
to zero if the MRs won't get detached properly before object's finalize().

Delete subregions within object's finalize() won't work either, because
finalize() will be invoked only if the refcount goes to zero first.  What
is worse, object_finalize() will do object_property_del_all() first before
object_deinit().  Since embeded MRs will be properties of the owner object,
it means they'll be freed _before_ the owner's finalize().

To fix that, teach memory API to stop refcount on MRs that share the same
owner.  Because if they share the lifecycle of the owner, then they share
the same lifecycle between themselves, hence the refcount doesn't help but
only introduce troubles.

Meanwhile, allow auto-detachments of MRs during finalize() of MRs even
against its container, as long as they belong to the same owner.

The latter is needed because now it's possible to have MRs' finalize()
happen in any order when they share the same lifespan with a same owner.
In this case, we should allow finalize() to happen in any order of either
the parent or child MR.  Loose the mr->container check in MR's finalize()
to allow auto-detach.  Double check it shares the same owner.

Proper document this behavior in code.

This patch is heavily based on the work done by Akihiko Odaki:

https://lore.kernel.org/r/CAFEAcA8DV40fGsci76r4yeP1P-SP_QjNRDD2OzPxjx5wRs0GEg@mail.gmail.com

Cc: Akihiko Odaki <odaki@rsg.ci.i.u-tokyo.ac.jp>
Cc: Paolo Bonzini <pbonzini@redhat.com>
Reviewed-by: Clément Mathieu--Drif <clement.mathieu--drif@eviden.com>
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Tested-by: Peter Maydell <peter.maydell@linaro.org>
Link: https://lore.kernel.org/r/20250826221750.285242-1-peterx@redhat.com
Signed-off-by: Peter Xu <peterx@redhat.com>
2025-09-15 12:00:12 -04:00
Peter Xu c75847f9eb memory: Fix addr/len for flatview_access_allowed()
flatview_access_allowed() should pass in the address offset of the memory
region, rather than the global address space. Shouldn't be a major issue
yet, since the addr is only used in an error log.

Cc: Philippe Mathieu-Daudé <philmd@linaro.org>
Fixes: 3ab6fdc91b ("softmmu/physmem: Introduce MemTxAttrs::memory field and MEMTX_ACCESS_ERROR")
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Reviewed-by: David Hildenbrand <david@redhat.com>
Link: https://lore.kernel.org/r/20250903142932.1038765-1-peterx@redhat.com
Signed-off-by: Peter Xu <peterx@redhat.com>
2025-09-15 12:00:12 -04:00
Peter Maydell 190d5d7fd7 Merge tag 'pull-request-2025-09-09' of https://gitlab.com/thuth/qemu into staging
* Silence warnings from the undefined-behaviour sanitizer
* Many small improvements to various functional tests
* Remove remainders from storing avocado artifacts in the Gitlab CI
* Keep more meson log files as artifacts in the Gitlab CI instead
* Re-enable -fzero-call-used-regs on OpenBSD

# -----BEGIN PGP SIGNATURE-----
#
# iQJFBAABCgAvFiEEJ7iIR+7gJQEY8+q5LtnXdP5wLbUFAmjAMK4RHHRodXRoQHJl
# ZGhhdC5jb20ACgkQLtnXdP5wLbX4ohAAl3AomPjCeCUEbwJqD0I8eSUeKKDNGbbI
# pwyEjg+e1nptqT7RVeS2EMKUAGT5dasZqjoMRMNS+PywCzDjkKPIjAZdatPMKMx2
# YK56qVaMcUKSDdpb/P091Bn8LLBX8kx8J0TpiRCvGH5KsflamMW7sVrAdn9X2lrM
# yTFN65asbbgfRWDW8qzXxX7JHdQZ1xwJiWSiJicTHzLRSoA4Ud6ymQxJyESgDUGs
# 44j1ieCrA7itbcUSIzYmEbcw0BgjaXSOUMXkUyZZ4GtQA7q5IVi+6iJm/3lbKEvA
# Wu3zPk1FeK6iyVGSn/fcaQfGEjTUI7zbOLN0Ub1ob9N3jO5z7EdUe7DjY2BF6L8y
# 4KYjWOcCWYL5BGNdi6ilaIk8l5sB1Vw/wIONdgqtcBJs0syiSzGqLmVttHIQlmI3
# 4tADDnINAUYi7T7q2/0F9VZB89TY7W7zAYrypTMzTIbGwEHgBj+4kq5DsplTfkg4
# LG+Vplv04NAmdgPndCj/AQ9y9ZtFjmZwuF0drLOSXFGzzfTv0g5YT3HQBbQ3gHsM
# tjeU5RSwHjr3OfvNWE1U/CIbu0Qa3CJcnco8JP5NIGCw8I0lHOnqsVq/1EC6PnGM
# 3QYvSd/z4jMO+5pXlMUQ52Lc7IRVTL8SVJf723gPV9TcV/EwLYtlv2s41GVqPwLM
# e+KxAirDD9c=
# =fJQL
# -----END PGP SIGNATURE-----
# gpg: Signature made Tue 09 Sep 2025 14:50:38 BST
# gpg:                using RSA key 27B88847EEE0250118F3EAB92ED9D774FE702DB5
# gpg:                issuer "thuth@redhat.com"
# gpg: Good signature from "Thomas Huth <th.huth@gmx.de>" [full]
# gpg:                 aka "Thomas Huth <thuth@redhat.com>" [full]
# gpg:                 aka "Thomas Huth <huth@tuxfamily.org>" [full]
# gpg:                 aka "Thomas Huth <th.huth@posteo.de>" [undefined]
# Primary key fingerprint: 27B8 8847 EEE0 2501 18F3  EAB9 2ED9 D774 FE70 2DB5

* tag 'pull-request-2025-09-09' of https://gitlab.com/thuth/qemu: (23 commits)
  tests/functional: purge scratch dir on test startup
  tests/functional: avoid tearDown failure when QEMU dies
  tests/functional: avoid duplicate messages on failures
  tests/functional: fix infinite loop on console EOF
  tests/functional: add vm param to cmd.py helpers
  tests/functional: return output from cmd.py helpers
  gitlab: prevent duplicated meson log artifacts in test jobs
  gitlab: include all junit XML files from meson
  gitlab: always include entire of meson-logs directory
  gitlab: replace avocado results files with meson results files
  tests/functional/arm: Update test ASPEED SDK v09.07 for AST2700 vbootrom
  tests/functional/arm: Update test ASPEED SDK v09.07 for AST2600
  tests/functional/arm: Update test ASPEED SDK v09.07 for AST2500
  tests/functional/arm: Update test ASPEED SDK v03.02 for AST1030
  tests/functional: handle URLError when fetching assets
  tests/functional: fix formatting of exception args
  tests/functional: enable force refresh of cached assets
  tests/functional/m68k: Avoid ResourceWarning in the nextcube test
  ui/vnc: Fix crash when specifying [vnc] without id in the config file
  system/physmem: Silence warning from ubsan
  ...

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2025-09-11 12:41:01 +01:00
Peter Maydell 7aba16fdd8 Merge tag 'pull-vfio-20250908' of https://github.com/legoater/qemu into staging
vfio queue:

* Large refactor of the VFIO code to use QOM casts and follow the
  current coding style guidelines
* Removal of the deprecated vfio-platform, vfio-calxeda-xgmac and
  vfio-amd-xgbe devices
* Fail with error if dma_max_mappings limit is hit
* Added 'x-pci-class-code' property to vfio-user-pci device
* Added a new helper to retrieve a VFIOPCIDevice from a VFIODevice
* Fixed IGD OpRegion detection

# -----BEGIN PGP SIGNATURE-----
#
# iQIzBAABCAAdFiEEoPZlSPBIlev+awtgUaNDx8/77KEFAmi/CfMACgkQUaNDx8/7
# 7KFkbhAAmbIxFxqLzRO3vnsSRGMkBnvZiQdz/b/iHHV8Uj6eJ6Vr5YxoR2c4XQqz
# kkU1+sBBezIWWWQ1dwRW19oObFp8ZvZKWT5AELqSXu/Svdh/kcJ99JnZa+JoK+r9
# rMTFEyjKOc0p9IDI1EIT6wJSqtUi2nXm3w1M4O1qkC17KriLtNLxtxEPCc6l6Ks4
# 0SEDddcVBhk8T0Jb8bHuRoaXqCER11bH07xzCQWTI7tW3cebX6622+F+H72/GvL9
# Gzs/61xTnoiIyGMs+jKM1hoURF0Fvv24zaHmSImxZX1vvF+ezeaRl2equ/H6AbDl
# 0+ShqRIj4XChRe0jf+08UDMor9sjRZYRw7NDSkM7hf9kS2VU/44FutAeOL0qAjb4
# oQqIAA1XBqDUW+5SzuzF2t2idBokLKH+d6qEmTTaMQEMrCDzv6Rx41e+27S3Qmai
# Ir+D37S57rrUvQFT6asCu6SOIrUPQHPZHDZkod7VPFTNsJkIaHzlViThzip4+CxO
# WbO8j0qXYq6OZ4C99GngkosT750DHoLrH5CKujNS4qHAqjisH145xkChM6LwFpzO
# o6SlBJg9mmMAsV7hHONk9x1GyESXHsoECQhjXrR7Yibv5ffPapEm/ruG3/gnZ5jA
# AgnQLrbKrla5AX8UsGi/Pz75BqadWBC0uABtYV+A0XTEluMC+0Q=
# =8KqI
# -----END PGP SIGNATURE-----
# gpg: Signature made Mon 08 Sep 2025 17:53:07 BST
# gpg:                using RSA key A0F66548F04895EBFE6B0B6051A343C7CFFBECA1
# gpg: Good signature from "Cédric Le Goater <clg@redhat.com>" [full]
# gpg:                 aka "Cédric Le Goater <clg@kaod.org>" [full]
# Primary key fingerprint: A0F6 6548 F048 95EB FE6B  0B60 51A3 43C7 CFFB ECA1

* tag 'pull-vfio-20250908' of https://github.com/legoater/qemu: (31 commits)
  vfio/pci.h: rename VFIOPCIDevice pdev field to parent_obj
  s390x/s390-pci-vfio.c: use QOM casts where appropriate
  vfio-user/pci.c: use QOM casts where appropriate
  vfio/igd.c: use QOM casts where appropriate
  vfio/cpr.c: use QOM casts where appropriate
  vfio/pci-quirks.c: use QOM casts where appropriate
  vfio/pci.c: use QOM casts where appropriate
  vfio/pci.h: update VFIOPCIDevice declaration
  vfio-user/pci.c: rename VFIOUserPCIDevice device field to parent_obj
  vfio-user/pci.c: use QOM casts where appropriate
  vfio-user/pci.c: update VFIOUserPCIDevice declaration
  vfio-user/container.h: rename VFIOUserContainer bcontainer field to parent_obj
  vfio/container.c: use QOM casts where appropriate
  vfio-user/container.h: update VFIOUserContainer declaration
  vfio/vfio-container.h: rename VFIOContainer bcontainer field to parent_obj
  vfio/spapr.c: use QOM casts where appropriate
  ppc/spapr_pci_vfio.c: use QOM casts where appropriate
  hw/vfio/container.c: use QOM casts where appropriate
  hw/vfio/cpr-legacy.c: use QOM casts where appropriate
  vfio/vfio-container.h: update VFIOContainer declaration
  ...

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2025-09-11 12:40:38 +01:00