Skip to content

[GSD-13402] [Bug Report] Three ZES extensions are advertised on Arc Pro B60 (BMG) but every corresponding call returns UNSUPPORTED_FEATURE or yields nothing #993

Description

@AlexanderScha

Pre-submission Checklist

GPU Hardware

Intel Arc Pro B60 (8086:e211, BMG G21) — 8 chips on 4 × MAXSUN Arc Pro B60 Dual 48G boards

DRI Devices Information

$ ls -la /dev/dri/by-path/
lrwxrwxrwx 1 root root  8 Sep  6 11:57 pci-0000:18:00.0-card   -> ../card1
lrwxrwxrwx 1 root root 13 Sep  6 11:57 pci-0000:18:00.0-render -> ../renderD128
lrwxrwxrwx 1 root root  8 Sep  6 11:57 pci-0000:1c:00.0-card   -> ../card2
lrwxrwxrwx 1 root root 13 Sep  6 11:57 pci-0000:1c:00.0-render -> ../renderD129
lrwxrwxrwx 1 root root  8 Sep  6 11:57 pci-0000:36:00.0-card   -> ../card3
lrwxrwxrwx 1 root root 13 Sep  6 11:57 pci-0000:36:00.0-render -> ../renderD130
lrwxrwxrwx 1 root root  8 Sep  6 11:57 pci-0000:3a:00.0-card   -> ../card4
lrwxrwxrwx 1 root root 13 Sep  6 11:57 pci-0000:3a:00.0-render -> ../renderD131
lrwxrwxrwx 1 root root  8 Sep  6 11:57 pci-0000:54:00.0-card   -> ../card5
lrwxrwxrwx 1 root root 13 Sep  6 11:57 pci-0000:54:00.0-render -> ../renderD132
lrwxrwxrwx 1 root root  8 Sep  6 11:57 pci-0000:58:00.0-card   -> ../card6
lrwxrwxrwx 1 root root 13 Sep  6 11:57 pci-0000:58:00.0-render -> ../renderD133
lrwxrwxrwx 1 root root  8 Sep  6 11:57 pci-0000:72:00.0-card   -> ../card7
lrwxrwxrwx 1 root root 13 Sep  6 11:57 pci-0000:72:00.0-render -> ../renderD134
lrwxrwxrwx 1 root root  8 Sep  6 11:57 pci-0000:76:00.0-card   -> ../card8
lrwxrwxrwx 1 root root 13 Sep  6 11:57 pci-0000:76:00.0-render -> ../renderD135
(0000:05:00.0 is the ASPEED BMC graphics, card9)

GPU Detailed Information (lspci output)

$ sudo lspci -vvv -k -s 0000:18:00.0
18:00.0 VGA compatible controller: Intel Corporation Battlemage G21 [Arc Pro B60] (prog-if 00 [VGA controller])
	Subsystem: Zycoo Co., Ltd Device 806c
	NUMA node: 0
	IOMMU group: 55
	Region 0: Memory at 9e000000 (64-bit, non-prefetchable) [size=16M]
	Region 2: Memory at 207000000000 (64-bit, prefetchable) [size=32G]
	Capabilities: [40] Vendor Specific Information: Intel Capabilities v1
	Capabilities: [70] Express (v2) Endpoint, IntMsgNum 0
		DevCap:	MaxPayload 256 bytes, PhantFunc 0, Latency L0s unlimited, L1 unlimited
		LnkCap:	Port #0, Speed 2.5GT/s, Width x1, ASPM L0s L1
		LnkSta:	Speed 2.5GT/s, Width x1
		DevCap2: Completion Timeout: Range B, TimeoutDis+ NROPrPrP- LTR+
			 AtomicOpsCap: 32bit- 64bit- 128bitCAS-
		LnkCap2: Supported Link Speeds: 2.5GT/s, Crosslink- Retimer- 2Retimers- DRS-
		LnkCtl2: Target Link Speed: 2.5GT/s
	Capabilities: [ac] MSI: Enable+ Count=1/1 Maskable+ 64bit+
	Capabilities: [d0] Power Management version 3
	Capabilities: [100 v1] Alternative Routing-ID Interpretation (ARI)
	Capabilities: [110 v1] Null
	Capabilities: [200 v1] Address Translation Service (ATS)
	Capabilities: [420 v1] Physical Resizable BAR
		BAR 2: current size: 32GB, supported: 256MB 512MB 1GB 2GB 4GB 8GB 16GB 32GB
	Capabilities: [400 v1] Latency Tolerance Reporting
	Kernel driver in use: xe
	Kernel modules: xe

Driver Version

26.31.39395.13

Installed GPU Driver Packages

ii  intel-opencl-icd            26.31.39395.13-1~26.04~ppa1   amd64
ii  libze-intel-gpu1            26.31.39395.13-1~26.04~ppa1   amd64
ii  libze-intel-gpu-raytracing  1.2.4-1~26.04~ppa3            amd64
ii  libze1:amd64                1.32.0-1~26.04~ppa1           amd64
ii  libze-dev:amd64             1.32.0-1~26.04~ppa1           amd64
ii  libigc2                     2.40.13+ds1-1~26.04           amd64
ii  libigdfcl2                  2.40.13+ds1-1~26.04           amd64
ii  libigdgmm12:amd64           22.10.1-1~26.04~ppa1          amd64
ii  ocl-icd-libopencl1:amd64    2.3.4-1                       amd64
ii  clinfo                      3.0.25.02.14-1build1          amd64

Driver Installation Details

  • Installation method: Ubuntu PPA kobuk-team/intel-graphics (the source of the GPU stack on this node)
  • Installed with apt from that PPA: libze-intel-gpu1, libze1, intel-opencl-icd and friends
  • Kernel driver: in-tree xe from the distribution kernel 7.0.0-31-generic (no DKMS)
  • Custom kernel parameters: intel_iommu=on iommu=pt (no i915/xe-specific parameters)

Linux Distribution

Other (please specify below)

Other Linux Distribution

Ubuntu 26.04.1 LTS

Kernel Version & Boot Parameters

$ uname -r
7.0.0-31-generic

$ cat /proc/cmdline
BOOT_IMAGE=/boot/vmlinuz-7.0.0-31-generic root=UUID=... ro quiet splash
intel_iommu=on iommu=pt loglevel=3 crashkernel=2G-4G:320M,...

$ lsmod | grep -E 'i915|xe'
xe                   4308992  48
drm_ttm_helper         20480  1 xe
ttm                   135168  2 drm_ttm_helper,xe
drm_suballoc_helper    24576  1 xe
drm_buddy              28672  1 xe
gpu_sched              69632  1 xe
drm_gpuvm              57344  1 xe
drm_exec               12288  2 drm_gpuvm,xe
drm_display_helper    303104  1 xe
cec                   106496  2 drm_display_helper,xe

Actual Behavior

zesDriverGetExtensionProperties() advertises 17 ZES extensions on this driver. For three of them the corresponding calls do not work on any of the eight Arc Pro B60 chips — identically as an unprivileged user and as root:

  • ZES_extension_ras_state (v65537)zesDeviceEnumRasErrorSets() returns ZE_RESULT_SUCCESS with count = 0 on every device. No RAS error sets exist, so no error category can ever be queried.
  • ZES_extension_mem_bandwidth_counter_bits_properties (v65536)zesMemoryGetBandwidth() returns 0x78000003 (ZE_RESULT_ERROR_UNSUPPORTED_FEATURE).
  • ZES_intel_memory_page_offline (v65536) — the offline-page state cannot be retrieved; the same failure occurs as root.

Running as root changes nothing, so this is not a permissions problem.

Expected Behavior

Either the calls work, or the extensions are not advertised on hardware where they are not implemented.

The extension list exists so that a consumer can decide whether calling is worthwhile. As it stands, the list says "yes" and the call says "unsupported" — a caller that checks first, which is the documented use of the list, is misled.

If the intent is that an extension may be present while unimplemented per-device, a distinguishable status would let callers tell "not supported on this device" apart from "supported but this call failed".

Reproduction Rate

Always reproduces - 100%

Steps to Reproduce

  1. Install driver 26.31.39395.13 on Ubuntu 26.04 with kernel 7.0.0-31-generic (xe driver).
  2. Build the reproducer below: gcc -O2 -o zes_ext_repro zes_ext_repro.c -lze_loader
  3. Run it: ./zes_ext_repro
  4. Run as root: sudo ./zes_ext_repro — output is identical.
  5. Observe: the three extensions appear in the advertised list, while zesDeviceEnumRasErrorSets returns count=0 and zesMemoryGetBandwidth returns UNSUPPORTED_FEATURE for every device.

Is this a regression?

  • Yes, this is a regression - functionality that previously worked is now broken

(We have only ever run this driver version on this hardware, so we cannot name a last-known-working version.)

Source Code / Reproducer

Build: gcc -O2 -o zes_ext_repro zes_ext_repro.c -lze_loader

zes_ext_repro.c (~90 lines, no dependencies beyond -lze_loader)
/*
 * zes_ext_repro.c — minimal reproducer for:
 *   "ZES extensions are advertised by zesDriverGetExtensionProperties but the
 *    corresponding calls return UNSUPPORTED or yield nothing"
 *
 * Build:  gcc -O2 -o zes_ext_repro zes_ext_repro.c -lze_loader
 * Run:    ./zes_ext_repro          (also try as root — result is identical)
 */
#include <level_zero/ze_api.h>
#include <level_zero/zes_api.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>

static const char *rs(ze_result_t r)
{
	switch (r) {
	case ZE_RESULT_SUCCESS: return "SUCCESS";
	case ZE_RESULT_ERROR_UNSUPPORTED_FEATURE: return "UNSUPPORTED_FEATURE";
	case ZE_RESULT_ERROR_UNINITIALIZED: return "UNINITIALIZED";
	case ZE_RESULT_ERROR_INVALID_ARGUMENT: return "INVALID_ARGUMENT";
	case ZE_RESULT_ERROR_INSUFFICIENT_PERMISSIONS: return "INSUFFICIENT_PERMISSIONS";
	default: return "OTHER";
	}
}

int main(void)
{
	ze_result_t r = zesInit(0);
	printf("zesInit: %s\n", rs(r));
	if (r != ZE_RESULT_SUCCESS) return 1;

	uint32_t ndrv = 0;
	zesDriverGet(&ndrv, NULL);
	zes_driver_handle_t *drv = calloc(ndrv, sizeof(*drv));
	zesDriverGet(&ndrv, drv);

	/* 1. What the driver advertises */
	uint32_t next = 0;
	zesDriverGetExtensionProperties(drv[0], &next, NULL);
	zes_driver_extension_properties_t *ext = calloc(next, sizeof(*ext));
	zesDriverGetExtensionProperties(drv[0], &next, ext);
	printf("\nAdvertised ZES extensions: %u\n", next);
	for (uint32_t i = 0; i < next; i++)
		if (strstr(ext[i].name, "ras_state") || strstr(ext[i].name, "page_offline") ||
		    strstr(ext[i].name, "mem_bandwidth"))
			printf("  ADVERTISED: %-56s v%u\n", ext[i].name, ext[i].version);

	uint32_t ndev = 0;
	zesDeviceGet(drv[0], &ndev, NULL);
	zes_device_handle_t *dev = calloc(ndev, sizeof(*dev));
	zesDeviceGet(drv[0], &ndev, dev);
	printf("\nDevices: %u\n", ndev);

	for (uint32_t d = 0; d < ndev; d++) {
		zes_pci_properties_t pci = {.stype = ZES_STRUCTURE_TYPE_PCI_PROPERTIES};
		zesDevicePciGetProperties(dev[d], &pci);
		printf("\n=== %04x:%02x:%02x.%x ===\n", pci.address.domain, pci.address.bus,
		       pci.address.device, pci.address.function);

		/* 2. ZES_extension_ras_state → no RAS sets at all */
		uint32_t nras = 0;
		r = zesDeviceEnumRasErrorSets(dev[d], &nras, NULL);
		printf("  zesDeviceEnumRasErrorSets   -> %-20s count=%u\n", rs(r), nras);

		/* 3. ZES_intel_memory_page_offline */
		uint32_t nmem = 0;
		zesDeviceEnumMemoryModules(dev[d], &nmem, NULL);
		zes_mem_handle_t *mem = calloc(nmem ? nmem : 1, sizeof(*mem));
		zesDeviceEnumMemoryModules(dev[d], &nmem, mem);
		if (nmem) {
			/* 4. ZES_extension_mem_bandwidth_counter_bits_properties */
			zes_mem_bandwidth_t bw = {0};
			r = zesMemoryGetBandwidth(mem[0], &bw);
			printf("  zesMemoryGetBandwidth       -> %-20s (0x%x)\n", rs(r), r);
		}
		free(mem);
	}
	printf("\nExpected: calls succeed, since the extensions are advertised.\n");
	printf("Actual:   see above — advertised but not usable.\n");
	free(ext); free(dev); free(drv);
	return 0;
}

Actual output on this system (8 devices, two shown for brevity):

zesInit: SUCCESS

Advertised ZES extensions: 17
  ADVERTISED: ZES_extension_ras_state                             v65537
  ADVERTISED: ZES_extension_mem_bandwidth_counter_bits_properties v65536
  ADVERTISED: ZES_intel_memory_page_offline                       v65536
  ADVERTISED: ZES_intel_memory_page_offline_property              v65536

Devices: 8

=== 0000:18:00.0 ===
  zesDeviceEnumRasErrorSets   -> SUCCESS              count=0
  zesMemoryGetBandwidth       -> UNSUPPORTED_FEATURE  (0x78000003)

=== 0000:1c:00.0 ===
  zesDeviceEnumRasErrorSets   -> SUCCESS              count=0
  zesMemoryGetBandwidth       -> UNSUPPORTED_FEATURE  (0x78000003)

   ... identical for all eight ...

Command Line / Application Details

./zes_ext_repro          # unprivileged
sudo ./zes_ext_repro     # identical output

Additional Notes

Why this matters in practice, in case it helps prioritise:

We run eight of these cards for inference with ECC enabled. With no RAS error sets we have no way to observe whether corrected memory errors are occurring. Disabling ECC frees exactly 1/8 of the memory (measured: 21414 MiB → 24480 MiB per chip), but doing so would remove both the protection and any means of noticing a problem — a silent bit flip would surface as nonsense model output.

Separately, two PCIe capability flags disagree with the call that reports them:

haveBandwidthCounters=1  havePacketCounters=1  haveReplayCounters=0
zesDevicePciGetStats -> 0x78000003 (UNSUPPORTED_FEATURE)

The third flag is honest — B60 has no PCIe replay counter. The first two are not.

Relation to #926 ("Even more counters in Sysman are broken on Arc Alchemist"): that report is about Arc A750 (Alchemist / DG2), where the counters return zeros. This one is Battlemage, where they return UNSUPPORTED_FEATURE, plus the RAS case which #926 does not cover. So this is not a duplicate — but two statements made there are worth revisiting:

In #926 (comment) (20 May 2026):
"VRAM bandwidth should be resolved"
"PCIe bandwidth reporting was added from BMG onwards"

On this hardware — which is BMG — with driver 26.31.39395.13 (newer than that comment), both are still unavailable:

zesMemoryGetBandwidth  -> UNSUPPORTED_FEATURE on all 8 devices
zesDevicePciGetStats   -> UNSUPPORTED_FEATURE on all 8 devices,
                          while haveBandwidthCounters / havePacketCounters report 1

We may of course be holding it wrong; if a specific configuration is required to enable these on BMG, we would be glad to test it and report back.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Component: Level Zero SysmanLevel Zero System Management API for hardware monitoringOS: LinuxIssue specific to Linux distributions (Ubuntu, Fedora, RHEL, etc.)Type: BugGeneral bug report, unexpected behavior or crash

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions