[PATCH 00/11] UPI topology discovery for SKX/ICX/SPR

[alexander . antonov]

From: Alexander Antonov <alexander.antonov@xxxxxxxxxxxxxxx>

Starting with the Intel Xeon Scalable family, Intel processors use a
coherent interconnect for scaling to multiple sockets, known as Intel
Ultra Path Interconnect (Intel UPI). The Intel UPI technology provides
a cache-coherent socket-to-socket external communication interface between
processors.

Unfortunately, users have little idea on how sockets are connected to
each other. This patchset extends the current mapping implementation by
adding sysfs attributes for UPI PMON blocks in the same manner as was
done for IIO blocks. These attributes reveal the topology of the system
by showing which how sockets and UPI controllers are connected to each
other.

Example for a 4-socket Sapphire Rapids server:

# tail /sys/devices/uncore_upi_*/die*
==> /sys/devices/uncore_upi_0/die0 <==
upi_1,die_1

==> /sys/devices/uncore_upi_0/die1 <==
upi_0,die_3

==> /sys/devices/uncore_upi_0/die2 <==
upi_1,die_3

==> /sys/devices/uncore_upi_0/die3 <==
upi_0,die_1

==> /sys/devices/uncore_upi_1/die0 <==
upi_1,die_2

==> /sys/devices/uncore_upi_1/die1 <==
upi_0,die_0

==> /sys/devices/uncore_upi_1/die2 <==
upi_1,die_0

==> /sys/devices/uncore_upi_1/die3 <==
upi_0,die_2

==> /sys/devices/uncore_upi_2/die0 <==
upi_2,die_3

==> /sys/devices/uncore_upi_2/die1 <==
upi_2,die_2

==> /sys/devices/uncore_upi_2/die2 <==
upi_2,die_1

==> /sys/devices/uncore_upi_2/die3 <==
upi_2,die_0

The following output:
# cat /sys/devices/uncore_upi_X/dieY
upi_Z,die_W
means that UPI link number X on die Y is connected to UPI link Z on die W.

Changes are splitted into two separate parts:
1. kernel part - adds and fills new sysfs attributes for UPI PMON blocks
2. perf-tool part - enables UPI topology info and per-link UPI metrics for
'perf iostat' mode. See example below.

The 'perf iostat list' will be extend to show IIO and UPI mappings, for example:

# perf iostat list
S0-uncore_iio_0<0000:00>
S1-uncore_iio_0<0000:80>
S0-uncore_iio_1<0000:17>
S1-uncore_iio_1<0000:85>
S0-uncore_iio_2<0000:3a>
S1-uncore_iio_2<0000:ae>
S0-uncore_iio_3<0000:5d>
S1-uncore_iio_3<0000:d7>
UPI Link 0 on Socket 0 -> UPI Link 1 on Socket 1
UPI Link 0 on Socket 1 -> UPI Link 1 on Socket 0
UPI Link 1 on Socket 0 -> UPI Link 0 on Socket 1
UPI Link 1 on Socket 1 -> UPI Link 0 on Socket 0

The 'perf iostat upi' mode will be added to show per-link UPI metrics:
# perf iostat upi
                                            link            Outgoing Data(GB)      Outgoing Non-Data(GB)
UPI Link 0 on Socket 0 -> UPI Link 1 on Socket 1                    0                      0
UPI Link 0 on Socket 1 -> UPI Link 1 on Socket 0                    0                      0
UPI Link 1 on Socket 0 -> UPI Link 0 on Socket 1                    0                      0
UPI Link 1 on Socket 1 -> UPI Link 0 on Socket 0                    0                      0

       1.437023089 seconds time elapsed

Alexander Antonov (11):
  perf/x86/intel/uncore: Generalize IIO topology support
  perf/x86/intel/uncore: Introduce UPI topology type
  perf/x86/intel/uncore: Clear attr_update properly
  perf/x86/intel/uncore: Disable I/O stacks to PMU mapping on ICX-D
  perf/x86/intel/uncore: Generalize get_topology() for SKX PMUs
  perf/x86/intel/uncore: Enable UPI topology discovery for Skylake
    Server
  perf/x86/intel/uncore: Get UPI NodeID and GroupID
  perf/x86/intel/uncore: Enable UPI topology discovery for Icelake
    Server
  perf/x86/intel/uncore: Enable UPI topology discovery for Sapphire
    Rapids
  perf/x86/intel/uncore: Update sysfs-devices-mapping file
  perf/x86/intel/uncore: Make set_mapping() procedure void

 .../ABI/testing/sysfs-devices-mapping         |  30 +-
 arch/x86/events/intel/uncore.h                |  24 +-
 arch/x86/events/intel/uncore_snbep.c          | 492 +++++++++++++++---
 3 files changed, 480 insertions(+), 66 deletions(-)


base-commit: ee6050c8af96bba2f81e8b0793a1fc2f998fcd20
-- 
2.25.1