hw/cxl/cxl-mailbox-utils.c | 368 +++++++++++++++------- include/hw/cxl/cxl_device.h | 76 +++++ include/hw/cxl/cxl_mailbox.h | 1 + include/hw/pci-bridge/cxl_upstream_port.h | 9 + 4 files changed, 347 insertions(+), 107 deletions(-)
This patch series refactor existing support for Identify Switch Device
and Get Physical Port State by utilizing physical ports (USP & DSP)
information stored during enumeration.
Additionally, it introduces new support for Physical Port Control
of FM-API based physical switch command set as per CXL spec r3.2
Table 8-230:Physical Switch. It primarily constitutes two logic:
-Assert-Deassert PERST: Assert PERST involves physical port to be in
hold reset phase for minimum 100ms. No other physical port control
request are entertained until Deassert PERST command for the given
port is issued.
-Reset PPB: cold reset of physical port (completing enter->hold->exit phases).
Tested using libcxl-mi interface[1]:
All active ports and all opcodes per active port is tested. Also, tested
against possible edge cases manually since the interface currently dosen't
support run time input.
Example topology (1 USP + 3 DSP's->switch with 2 CXLType3 devices connected
to 2 DSP's):
FM="-object memory-backend-file,id=cxl-mem1,mem-path=$TMP_DIR/t3_cxl1.raw,size=256M \
-object memory-backend-file,id=cxl-lsa1,mem-path=$TMP_DIR/t3_lsa1.raw,size=1M \
-object memory-backend-file,id=cxl-mem2,mem-path=$TMP_DIR/t3_cxl2.raw,size=512M \
-object memory-backend-file,id=cxl-lsa2,mem-path=$TMP_DIR/t3_lsa2.raw,size=512M \
-device pxb-cxl,bus_nr=12,bus=pcie.0,id=cxl.1,hdm_for_passthrough=true \
-device cxl-rp,port=0,bus=cxl.1,id=cxl_rp_port0,chassis=0,slot=2 \
-device cxl-upstream,port=2,sn=1234,bus=cxl_rp_port0,id=us0,addr=0.0,multifunction=on, \
-device cxl-switch-mailbox-cci,bus=cxl_rp_port0,addr=0.1,target=us0 \
-device cxl-downstream,port=0,bus=us0,id=swport0,chassis=0,slot=4 \
-device cxl-downstream,port=1,bus=us0,id=swport1,chassis=0,slot=5 \
-device cxl-downstream,port=3,bus=us0,id=swport2,chassis=0,slot=6 \
-device cxl-type3,bus=swport0,memdev=cxl-mem1,id=cxl-pmem1,lsa=cxl-lsa1,sn=3 \
-device cxl-type3,bus=swport2,memdev=cxl-mem2,id=cxl-pmem2,lsa=cxl-lsa2,sn=4 \
-machine cxl-fmw.0.targets.0=cxl.1,cxl-fmw.0.size=4G,cxl-fmw.0.interleave-granularity=1k \
-device i2c_mctp_cxl,bus=aspeed.i2c.bus.0,address=4,target=us0 \
-device i2c_mctp_cxl,bus=aspeed.i2c.bus.0,address=5,target=cxl-pmem1 \
-device i2c_mctp_cxl,bus=aspeed.i2c.bus.0,address=6,target=cxl-pmem2 \
-device virtio-rng-pci,bus=swport1"
Multiple Qemu Topologies tested:
-without any devices connected to downstream ports.
-with virtio-rng-pci devices connected to downstream ports.
-with CXLType3 devices connected to downstream ports.
-with different unique values of ports (both upstream and downstream).
Changes from v2->v3:
-cxl_set_port_type(): optimized storing of strucutre members.
-namespace defines instead of enum.
-Calculating size for active_port_bitmask than hardcoding to 0x20.
-Defined struct phy_port directly inside struct CXLUpstreamPort as pports.
-Renamed struct pperst to struct CXLPhyPortPerst.
-Optimized perst member initializations for ports inside
cxl_initialize_usp_mctpcci() using active_port_bitmask.
[1] https://github.com/computexpresslink/libcxlmi/commit/35fe68bd9a31469f832a87694d7b18d2d50be5b8
The patches are generated against the Johnathan's tree
https://gitlab.com/jic23/qemu.git and branch cxl-2025-07-03.
Signed-off-by: Arpit Kumar <arpit1.kumar@samsung.com>
Arpit Kumar (2):
hw/cxl: Refactored Identify Switch Device & Get Physical Port State
hw/cxl: Add Physical Port Control (Opcode 5102h)
hw/cxl/cxl-mailbox-utils.c | 368 +++++++++++++++-------
include/hw/cxl/cxl_device.h | 76 +++++
include/hw/cxl/cxl_mailbox.h | 1 +
include/hw/pci-bridge/cxl_upstream_port.h | 9 +
4 files changed, 347 insertions(+), 107 deletions(-)
--
2.34.1
On Thu, 4 Sep 2025 18:49:02 +0530 Arpit Kumar <arpit1.kumar@samsung.com> wrote: > This patch series refactor existing support for Identify Switch Device > and Get Physical Port State by utilizing physical ports (USP & DSP) > information stored during enumeration. > > Additionally, it introduces new support for Physical Port Control > of FM-API based physical switch command set as per CXL spec r3.2 > Table 8-230:Physical Switch. It primarily constitutes two logic: > -Assert-Deassert PERST: Assert PERST involves physical port to be in > hold reset phase for minimum 100ms. No other physical port control > request are entertained until Deassert PERST command for the given > port is issued. > -Reset PPB: cold reset of physical port (completing enter->hold->exit phases). > > Tested using libcxl-mi interface[1]: > All active ports and all opcodes per active port is tested. Also, tested > against possible edge cases manually since the interface currently dosen't > support run time input. > > Example topology (1 USP + 3 DSP's->switch with 2 CXLType3 devices connected > to 2 DSP's): > FM="-object memory-backend-file,id=cxl-mem1,mem-path=$TMP_DIR/t3_cxl1.raw,size=256M \ > -object memory-backend-file,id=cxl-lsa1,mem-path=$TMP_DIR/t3_lsa1.raw,size=1M \ > -object memory-backend-file,id=cxl-mem2,mem-path=$TMP_DIR/t3_cxl2.raw,size=512M \ > -object memory-backend-file,id=cxl-lsa2,mem-path=$TMP_DIR/t3_lsa2.raw,size=512M \ > -device pxb-cxl,bus_nr=12,bus=pcie.0,id=cxl.1,hdm_for_passthrough=true \ > -device cxl-rp,port=0,bus=cxl.1,id=cxl_rp_port0,chassis=0,slot=2 \ > -device cxl-upstream,port=2,sn=1234,bus=cxl_rp_port0,id=us0,addr=0.0,multifunction=on, \ > -device cxl-switch-mailbox-cci,bus=cxl_rp_port0,addr=0.1,target=us0 \ > -device cxl-downstream,port=0,bus=us0,id=swport0,chassis=0,slot=4 \ > -device cxl-downstream,port=1,bus=us0,id=swport1,chassis=0,slot=5 \ > -device cxl-downstream,port=3,bus=us0,id=swport2,chassis=0,slot=6 \ > -device cxl-type3,bus=swport0,memdev=cxl-mem1,id=cxl-pmem1,lsa=cxl-lsa1,sn=3 \ > -device cxl-type3,bus=swport2,memdev=cxl-mem2,id=cxl-pmem2,lsa=cxl-lsa2,sn=4 \ > -machine cxl-fmw.0.targets.0=cxl.1,cxl-fmw.0.size=4G,cxl-fmw.0.interleave-granularity=1k \ > -device i2c_mctp_cxl,bus=aspeed.i2c.bus.0,address=4,target=us0 \ > -device i2c_mctp_cxl,bus=aspeed.i2c.bus.0,address=5,target=cxl-pmem1 \ > -device i2c_mctp_cxl,bus=aspeed.i2c.bus.0,address=6,target=cxl-pmem2 \ > -device virtio-rng-pci,bus=swport1" > > Multiple Qemu Topologies tested: > -without any devices connected to downstream ports. > -with virtio-rng-pci devices connected to downstream ports. > -with CXLType3 devices connected to downstream ports. > -with different unique values of ports (both upstream and downstream). > > Changes from v2->v3: > -cxl_set_port_type(): optimized storing of strucutre members. > -namespace defines instead of enum. > -Calculating size for active_port_bitmask than hardcoding to 0x20. > -Defined struct phy_port directly inside struct CXLUpstreamPort as pports. > -Renamed struct pperst to struct CXLPhyPortPerst. > -Optimized perst member initializations for ports inside > cxl_initialize_usp_mctpcci() using active_port_bitmask. > > [1] https://github.com/computexpresslink/libcxlmi/commit/35fe68bd9a31469f832a87694d7b18d2d50be5b8 > > The patches are generated against the Johnathan's tree > https://gitlab.com/jic23/qemu.git and branch cxl-2025-07-03. > > Signed-off-by: Arpit Kumar <arpit1.kumar@samsung.com> Hi Arpit, I'll have a go (probably next week) at rebasing this rather earlier in my tree as I'd like to get this upstream without it having a dependency on the MCTP support. That means bring it up with the switch-cci / pcie mailbox CCI and squashing the MCTP bit into the patch that brings that support up later in my tree. I do plan to fix up the remaining 'feature' gap on the FMAPI/MCTP/USB emulation which is that it's ignoring the MTU to the host and so not breaking messages up as it should. Linux doesn't care but maybe some other OS will. Not entirely sure when I'll get to that though and I'd like to move your work forward before that. Jonathan > > Arpit Kumar (2): > hw/cxl: Refactored Identify Switch Device & Get Physical Port State > hw/cxl: Add Physical Port Control (Opcode 5102h) > > hw/cxl/cxl-mailbox-utils.c | 368 +++++++++++++++------- > include/hw/cxl/cxl_device.h | 76 +++++ > include/hw/cxl/cxl_mailbox.h | 1 + > include/hw/pci-bridge/cxl_upstream_port.h | 9 + > 4 files changed, 347 insertions(+), 107 deletions(-) >
On 05/09/25 05:12PM, Jonathan Cameron wrote: >On Thu, 4 Sep 2025 18:49:02 +0530 >Arpit Kumar <arpit1.kumar@samsung.com> wrote: > >> This patch series refactor existing support for Identify Switch Device >> and Get Physical Port State by utilizing physical ports (USP & DSP) >> information stored during enumeration. >> >> Additionally, it introduces new support for Physical Port Control >> of FM-API based physical switch command set as per CXL spec r3.2 >> Table 8-230:Physical Switch. It primarily constitutes two logic: >> -Assert-Deassert PERST: Assert PERST involves physical port to be in >> hold reset phase for minimum 100ms. No other physical port control >> request are entertained until Deassert PERST command for the given >> port is issued. >> -Reset PPB: cold reset of physical port (completing enter->hold->exit phases). >> >> Tested using libcxl-mi interface[1]: >> All active ports and all opcodes per active port is tested. Also, tested >> against possible edge cases manually since the interface currently dosen't >> support run time input. >> >> Example topology (1 USP + 3 DSP's->switch with 2 CXLType3 devices connected >> to 2 DSP's): >> FM="-object memory-backend-file,id=cxl-mem1,mem-path=$TMP_DIR/t3_cxl1.raw,size=256M \ >> -object memory-backend-file,id=cxl-lsa1,mem-path=$TMP_DIR/t3_lsa1.raw,size=1M \ >> -object memory-backend-file,id=cxl-mem2,mem-path=$TMP_DIR/t3_cxl2.raw,size=512M \ >> -object memory-backend-file,id=cxl-lsa2,mem-path=$TMP_DIR/t3_lsa2.raw,size=512M \ >> -device pxb-cxl,bus_nr=12,bus=pcie.0,id=cxl.1,hdm_for_passthrough=true \ >> -device cxl-rp,port=0,bus=cxl.1,id=cxl_rp_port0,chassis=0,slot=2 \ >> -device cxl-upstream,port=2,sn=1234,bus=cxl_rp_port0,id=us0,addr=0.0,multifunction=on, \ >> -device cxl-switch-mailbox-cci,bus=cxl_rp_port0,addr=0.1,target=us0 \ >> -device cxl-downstream,port=0,bus=us0,id=swport0,chassis=0,slot=4 \ >> -device cxl-downstream,port=1,bus=us0,id=swport1,chassis=0,slot=5 \ >> -device cxl-downstream,port=3,bus=us0,id=swport2,chassis=0,slot=6 \ >> -device cxl-type3,bus=swport0,memdev=cxl-mem1,id=cxl-pmem1,lsa=cxl-lsa1,sn=3 \ >> -device cxl-type3,bus=swport2,memdev=cxl-mem2,id=cxl-pmem2,lsa=cxl-lsa2,sn=4 \ >> -machine cxl-fmw.0.targets.0=cxl.1,cxl-fmw.0.size=4G,cxl-fmw.0.interleave-granularity=1k \ >> -device i2c_mctp_cxl,bus=aspeed.i2c.bus.0,address=4,target=us0 \ >> -device i2c_mctp_cxl,bus=aspeed.i2c.bus.0,address=5,target=cxl-pmem1 \ >> -device i2c_mctp_cxl,bus=aspeed.i2c.bus.0,address=6,target=cxl-pmem2 \ >> -device virtio-rng-pci,bus=swport1" >> >> Multiple Qemu Topologies tested: >> -without any devices connected to downstream ports. >> -with virtio-rng-pci devices connected to downstream ports. >> -with CXLType3 devices connected to downstream ports. >> -with different unique values of ports (both upstream and downstream). >> >> Changes from v2->v3: >> -cxl_set_port_type(): optimized storing of strucutre members. >> -namespace defines instead of enum. >> -Calculating size for active_port_bitmask than hardcoding to 0x20. >> -Defined struct phy_port directly inside struct CXLUpstreamPort as pports. >> -Renamed struct pperst to struct CXLPhyPortPerst. >> -Optimized perst member initializations for ports inside >> cxl_initialize_usp_mctpcci() using active_port_bitmask. >> >> [1] https://github.com/computexpresslink/libcxlmi/commit/35fe68bd9a31469f832a87694d7b18d2d50be5b8 >> >> The patches are generated against the Johnathan's tree >> https://gitlab.com/jic23/qemu.git and branch cxl-2025-07-03. >> >> Signed-off-by: Arpit Kumar <arpit1.kumar@samsung.com> > >Hi Arpit, > >I'll have a go (probably next week) at rebasing this rather earlier in my tree as I'd >like to get this upstream without it having a dependency on the MCTP support. > >That means bring it up with the switch-cci / pcie mailbox CCI and squashing >the MCTP bit into the patch that brings that support up later in my tree. > >I do plan to fix up the remaining 'feature' gap on the FMAPI/MCTP/USB >emulation which is that it's ignoring the MTU to the host and so not >breaking messages up as it should. Linux doesn't care but maybe some other >OS will. Not entirely sure when I'll get to that though and I'd like to >move your work forward before that. > >Jonathan > > Hi Jonathan, Thanks for the review comments! As per my understanding from CXL spec r3.2 Table 8-215: Physical Port Control request is allowed only for switch FM interface and is prohibited for switch-cci/pcie mailbox CCI. However, if possible, should I be using cxl_initialize_mailbox_swcci() to initialize my perst members? Thanks, Arpit. >> >> Arpit Kumar (2): >> hw/cxl: Refactored Identify Switch Device & Get Physical Port State >> hw/cxl: Add Physical Port Control (Opcode 5102h) >> >> hw/cxl/cxl-mailbox-utils.c | 368 +++++++++++++++------- >> include/hw/cxl/cxl_device.h | 76 +++++ >> include/hw/cxl/cxl_mailbox.h | 1 + >> include/hw/pci-bridge/cxl_upstream_port.h | 9 + >> 4 files changed, 347 insertions(+), 107 deletions(-) >> >
On Mon, 8 Sep 2025 18:52:11 +0530 Arpit Kumar <arpit1.kumar@samsung.com> wrote: > On 05/09/25 05:12PM, Jonathan Cameron wrote: > >On Thu, 4 Sep 2025 18:49:02 +0530 > >Arpit Kumar <arpit1.kumar@samsung.com> wrote: > > > >> This patch series refactor existing support for Identify Switch Device > >> and Get Physical Port State by utilizing physical ports (USP & DSP) > >> information stored during enumeration. > >> > >> Additionally, it introduces new support for Physical Port Control > >> of FM-API based physical switch command set as per CXL spec r3.2 > >> Table 8-230:Physical Switch. It primarily constitutes two logic: > >> -Assert-Deassert PERST: Assert PERST involves physical port to be in > >> hold reset phase for minimum 100ms. No other physical port control > >> request are entertained until Deassert PERST command for the given > >> port is issued. > >> -Reset PPB: cold reset of physical port (completing enter->hold->exit phases). > >> > >> Tested using libcxl-mi interface[1]: > >> All active ports and all opcodes per active port is tested. Also, tested > >> against possible edge cases manually since the interface currently dosen't > >> support run time input. > >> > >> Example topology (1 USP + 3 DSP's->switch with 2 CXLType3 devices connected > >> to 2 DSP's): > >> FM="-object memory-backend-file,id=cxl-mem1,mem-path=$TMP_DIR/t3_cxl1.raw,size=256M \ > >> -object memory-backend-file,id=cxl-lsa1,mem-path=$TMP_DIR/t3_lsa1.raw,size=1M \ > >> -object memory-backend-file,id=cxl-mem2,mem-path=$TMP_DIR/t3_cxl2.raw,size=512M \ > >> -object memory-backend-file,id=cxl-lsa2,mem-path=$TMP_DIR/t3_lsa2.raw,size=512M \ > >> -device pxb-cxl,bus_nr=12,bus=pcie.0,id=cxl.1,hdm_for_passthrough=true \ > >> -device cxl-rp,port=0,bus=cxl.1,id=cxl_rp_port0,chassis=0,slot=2 \ > >> -device cxl-upstream,port=2,sn=1234,bus=cxl_rp_port0,id=us0,addr=0.0,multifunction=on, \ > >> -device cxl-switch-mailbox-cci,bus=cxl_rp_port0,addr=0.1,target=us0 \ > >> -device cxl-downstream,port=0,bus=us0,id=swport0,chassis=0,slot=4 \ > >> -device cxl-downstream,port=1,bus=us0,id=swport1,chassis=0,slot=5 \ > >> -device cxl-downstream,port=3,bus=us0,id=swport2,chassis=0,slot=6 \ > >> -device cxl-type3,bus=swport0,memdev=cxl-mem1,id=cxl-pmem1,lsa=cxl-lsa1,sn=3 \ > >> -device cxl-type3,bus=swport2,memdev=cxl-mem2,id=cxl-pmem2,lsa=cxl-lsa2,sn=4 \ > >> -machine cxl-fmw.0.targets.0=cxl.1,cxl-fmw.0.size=4G,cxl-fmw.0.interleave-granularity=1k \ > >> -device i2c_mctp_cxl,bus=aspeed.i2c.bus.0,address=4,target=us0 \ > >> -device i2c_mctp_cxl,bus=aspeed.i2c.bus.0,address=5,target=cxl-pmem1 \ > >> -device i2c_mctp_cxl,bus=aspeed.i2c.bus.0,address=6,target=cxl-pmem2 \ > >> -device virtio-rng-pci,bus=swport1" > >> > >> Multiple Qemu Topologies tested: > >> -without any devices connected to downstream ports. > >> -with virtio-rng-pci devices connected to downstream ports. > >> -with CXLType3 devices connected to downstream ports. > >> -with different unique values of ports (both upstream and downstream). > >> > >> Changes from v2->v3: > >> -cxl_set_port_type(): optimized storing of strucutre members. > >> -namespace defines instead of enum. > >> -Calculating size for active_port_bitmask than hardcoding to 0x20. > >> -Defined struct phy_port directly inside struct CXLUpstreamPort as pports. > >> -Renamed struct pperst to struct CXLPhyPortPerst. > >> -Optimized perst member initializations for ports inside > >> cxl_initialize_usp_mctpcci() using active_port_bitmask. > >> > >> [1] https://github.com/computexpresslink/libcxlmi/commit/35fe68bd9a31469f832a87694d7b18d2d50be5b8 > >> > >> The patches are generated against the Johnathan's tree > >> https://gitlab.com/jic23/qemu.git and branch cxl-2025-07-03. > >> > >> Signed-off-by: Arpit Kumar <arpit1.kumar@samsung.com> > > > >Hi Arpit, > > > >I'll have a go (probably next week) at rebasing this rather earlier in my tree as I'd > >like to get this upstream without it having a dependency on the MCTP support. > > > >That means bring it up with the switch-cci / pcie mailbox CCI and squashing > >the MCTP bit into the patch that brings that support up later in my tree. > > > >I do plan to fix up the remaining 'feature' gap on the FMAPI/MCTP/USB > >emulation which is that it's ignoring the MTU to the host and so not > >breaking messages up as it should. Linux doesn't care but maybe some other > >OS will. Not entirely sure when I'll get to that though and I'd like to > >move your work forward before that. > > > >Jonathan > > > > > Hi Jonathan, > Thanks for the review comments! > > As per my understanding from CXL spec r3.2 Table 8-215: Physical Port Control I guess table 8-230? > request is allowed only for switch FM interface and is prohibited for > switch-cci/pcie mailbox CCI. However, if possible, should I be using > cxl_initialize_mailbox_swcci() to initialize my perst members? I always find that table really hard to understand. I'm not sure what intent is wrt to "host mailbox" in that table. I can't immediately think why the FM API CCI (thing that was switch cci in earlier specs) wouldn't support this particular command. You are supposed to be able to control anything via that which is controllable for MCTP. Only exceptions should be the few things to do with the MCTP interface itself. Just to add to the confusion, the entry for Get Physical Port State in the host mailbox column is "MSW = mandatory on MCTP-based CCIs for all switches that support the FM API MCTP message type" which is odd given that mailbox isn't mctp based. Gut feeling is that it can be implemented on the 'switch CCI'. I'd like to do that for now just because the MCTP stuff is going to take longer to upstream and I'd rather your series was not queued up behind it. Jonathan > > Thanks, > Arpit. > >> > >> Arpit Kumar (2): > >> hw/cxl: Refactored Identify Switch Device & Get Physical Port State > >> hw/cxl: Add Physical Port Control (Opcode 5102h) > >> > >> hw/cxl/cxl-mailbox-utils.c | 368 +++++++++++++++------- > >> include/hw/cxl/cxl_device.h | 76 +++++ > >> include/hw/cxl/cxl_mailbox.h | 1 + > >> include/hw/pci-bridge/cxl_upstream_port.h | 9 + > >> 4 files changed, 347 insertions(+), 107 deletions(-) > >> > > >
© 2016 - 2026 Red Hat, Inc.