Acacia I2C, Serial and Timer + new Python tooling by omeh-a · Pull Request #751 · au-ts/sddf

omeh-a · 2026-07-08T00:56:21Z

This PR introduces Acacia to the sDDF and reworks all of our meta tooling around it.

Instead of storing our sDDF driver generation code in sdf_gen, we now store it locally in ./acacia_sddf/driver_class.py. Common logic is in ./acacia_sddf/sddf.py
tools/meta is removed and remaining Python like board.py is moved to ./acacia_sddf. This is to allow all of the Python in the sDDF to sit in one module that can be imported all in one blast. I moved acacia_sddf to the root of the repository to reflect its significance, as this code is in fact critical to assemble anything in the sDDF (as opposed to being a "two generic subdirectories deep tool")
board.py has been changed to get rid of some indirection, we now store compatible strings there for sanity.
All config.jsons are redundant and replaced with new functionality in driver_manifest.py. This file offers an API to let this same information be stored globally, and I intend for this to be used by each driver_class.py ... e.g. in i2c.py

# meson
add_driver_config(
    "meson",
    sDDFDriverConfig(
        compatible="amlogic,meson-axg-i2c",
        regions=[DTSRegion("regs", "rw", 4096, 0)],
        irqs=[DTSIRQ(0), DTSIRQ(1)],
    ),
)

# opentitan
add_driver_config(
    "opentitan",
    sDDFDriverConfig(
        compatible="eth,i2c",
        regions=[DTSRegion("regs", "rw", 4096, 0)],
        irqs=[DTSIRQ(4), DTSIRQ(0), DTSIRQ(1), DTSIRQ(7), DTSIRQ(9)],
    ),
)

... i.e. we store the old config.json in i2c.py.

There's not too much to see otherwise besides the new Acacia subsystem generation in i2c.py, serial.py and timer.py. Please let me know what you think about these.

I am leaving this PR as a draft until we have merged Acacia, just in case API changes occur.

I've implemented these three classes as an exemplar for Acacia usage in advance of porting the other classes by others. These three are also mutually dependent and it isn't possible to do i2c without the other two.

Signed-off-by: Lesley Rossouw <lesley.rossouw@unsw.edu.au> Timer tests out with acacia Signed-off-by: Lesley Rossouw <lesley.rossouw@unsw.edu.au> Serial tests out with Acacia Signed-off-by: Lesley Rossouw <lesley.rossouw@unsw.edu.au> Refactor: acacia_sddf is now a python module that inherits all subfiles, sDDF itself is a python module for import! Needed for SDK Signed-off-by: Lesley Rossouw <lesley.rossouw@unsw.edu.au> Fix issues with i2c.py - maps were swapped. Signed-off-by: Lesley Rossouw <lesley.rossouw@unsw.edu.au> Further fixes to keep up with Acacia PR request changes Signed-off-by: Lesley Rossouw <lesley.rossouw@unsw.edu.au>

Signed-off-by: Lesley Rossouw <lesley.rossouw@unsw.edu.au>

wom-bat · 2026-07-08T05:12:39Z

+    blk: Optional[DriverDouble] = DriverDouble(None, None)
+    partition: int = 0
+    baud_rate: Optional[int] = None
+


How does this work when you have multiple ethernet devices (of the same or different kinds) in the same board? I think these should be lists of DriverDouble

It didn't before these changes, from tools/meta/board.py it was reasonably similar. @Courtney3141 might need to comment on what the Firewall does, which I think uses a different board definition.

In general I think we mostly assume we have "single" instances of the driver subsystems/components. There are plenty of cases where it makes sense for systems to have multiple different block drivers, or i2c, or serial, and even multiple connections per client.

I think for now we should keep assuming that.

Maybe what we should do instead is that when you construct a system, by default what should happen is we iterate over the DTS file and instantiate a driver for every compatible string in the DTS we know about.

We can have a user API that has a stable naming/ordering scheme and then you ask for the Network subsystem "manager" to connect your client to e.g. ethernet device 0 or 1, or ethernet device at this PCI bus location, or identified by this PCI vendor+device, or at this DTS location. Most of our examples can then get away with asking for device 0 (because, maybe say, we use the DTS chosen node in the DTS we provide to force it to be the right one).

Then maybe users of our build system can construct systems in which you have connections, and if a particular block driver has no connections then we purge it from the system. (For clock drivers, maybe not, though you would assume there would be a connection between them and device drivers depending on them, so it could probably follow the same idea).

This makes sense in the LionsOS context too, really.

wom-bat · 2026-07-08T05:14:00Z

+
+# Keep this list in alphabetical order by board name
+# TODO: convert to Dictionary
+BOARDS: List[Board] = [


I personally would prefer splitting these out one per supported board, then just importing the one you want.

(I know the old sdfgen tooling did it this way)

Then examples need to import every board anyway?

wom-bat · 2026-07-08T05:16:16Z

+from .sddf import sDDFDriverClass, DeviceResourcesFactory, RegionResourceFactory
+from collections import defaultdict
+from typing import List, Dict, Type, Union, Optional
+


rearrange imports to put standard ones first

Also reframe from multiple imports on one line.

wom-bat · 2026-07-08T05:20:00Z

+
+
+# pulp
+add_driver_config(


I reckon these should be in per-driver files imported rather than enumerated here. So that the tooling can be used with third-party drivers without changing this file.

Well, you can already, see 'add_driver_config' you can import that and call it. I think there needs to be an global registry of drivers in some sense otherwise your meta program needs to duplicate driver selection logic.

Though it might be better to follow the "driver finder library" approach or something instead of doing what Linux calls the midlayer mistake. But then maybe that doesn't work with my proposed solution of creating drivers for every found DTS node where a compatible matches?

wom-bat · 2026-07-08T05:21:16Z


 SUPPORTED_BOARDS := \
 		odroidc4 \
+		maaxboard \


separate change, different PR

wom-bat · 2026-07-08T05:35:56Z

    board = next(filter(lambda b: b.name == args.board, BOARDS))
+    if board.arch != x86_64:
+        dtb = DeviceTreeBlob(args.dtb)
+    else:


An alternative would be to fudge up a DTS for the X86 systems we use. Linux X86 can use a DTB as an additional source of information (mainly to get the PCI and IOAPIC info)

Something like that isn't a bad idea... for what it's worth: this check is redundant. If there's no DTB given, this creates an empty dtb which is recognised by Acacia as being an x86 system

Signed-off-by: Lesley Rossouw <lesley.rossouw@unsw.edu.au>

midnightveil · 2026-08-04T04:59:37Z

@@ -0,0 +1,8 @@
+# Copyright 2026, UNSW


I don't think this should be under acacia_sddf. It's not really clear what acacia is for a user. I think the timer/serial classes should either live in the appropriate serial/ or timer/ directory and helpers under tools/meta or tools/acacia.

There is a blunt reason for this: it means that we only need one Python import and it means that each driver class doesn't need to abuse importlib to find the others. I really think it's fine to have end users learn what Acacia is, given that they have to install it to use the sDDF. I did it as you suggest initially and it made things horrible. I'm really not eager to change this.

I don't think having to set up PYTHONPATH (which you can do entirely in the Makefile) is that bad. If you want just one import you can still do that without separating the components from the rest of the components, you just have some path-mangling.

The other thing is to consider external users: LionsOS, for instance, could just do "import sddf.timer" if it's located in the timer folder, whereas with this way you need to either do "import sddf.acacia_sddf.timer" or have top level exports (not necessary if your project structure follows python module structure, no need for init.py etc). They'd have to put sDDF in their PythonPath but that's fine and reasonable.

tldr; no, I disagree.

Users of anything will be using acacia anyway though? How can a user of acacia who must be using it in their metaprogram not already know what acacia is? Making the entire sDDF tree a python module just complicates things. The only functional diff from what you're describing is whether the module is called sddf or acacia_sddf. We can rename acacia_sddf to just sddf if it matters ... I personally think it's inconsequential, but I also really think it's worse for code organisation to scatter the python through the source tree.

Sidenote: I don't think we will need to import sddf.acacia_sddf.blah either? We can do this exactly as it is done in the metaprogram and not make a top level sDDF module

midnightveil · 2026-08-04T05:00:15Z

+    arch: Arch
+    paddr_top: int
+    # Driver mappings -> (compatible, preferred_node) tuples
+    serial: Optional[DriverDouble] = DriverDouble(None, None)


Optional[] but then the default isn't None, instead it's DriveDouble(None, None). Why would it ever be None then?

Also, if it is DriverDouble(None, None), then this doesn't type-hint as DriverDouble is (str, str), not Optional[str].

midnightveil · 2026-08-04T05:12:07Z

+    blk: Optional[DriverDouble] = DriverDouble(None, None)
+    partition: int = 0
+    baud_rate: Optional[int] = None
+


It didn't before these changes, from tools/meta/board.py it was reasonably similar. @Courtney3141 might need to comment on what the Firewall does, which I think uses a different board definition.

In general I think we mostly assume we have "single" instances of the driver subsystems/components. There are plenty of cases where it makes sense for systems to have multiple different block drivers, or i2c, or serial, and even multiple connections per client.

midnightveil · 2026-08-04T05:13:14Z

+
+# Keep this list in alphabetical order by board name
+# TODO: convert to Dictionary
+BOARDS: List[Board] = [


Then examples need to import every board anyway?

midnightveil · 2026-08-04T05:23:20Z

+    blk: Optional[DriverDouble] = DriverDouble(None, None)
+    partition: int = 0
+    baud_rate: Optional[int] = None
+


I think for now we should keep assuming that.

Maybe what we should do instead is that when you construct a system, by default what should happen is we iterate over the DTS file and instantiate a driver for every compatible string in the DTS we know about.

We can have a user API that has a stable naming/ordering scheme and then you ask for the Network subsystem "manager" to connect your client to e.g. ethernet device 0 or 1, or ethernet device at this PCI bus location, or identified by this PCI vendor+device, or at this DTS location. Most of our examples can then get away with asking for device 0 (because, maybe say, we use the DTS chosen node in the DTS we provide to force it to be the right one).

Then maybe users of our build system can construct systems in which you have connections, and if a particular block driver has no connections then we purge it from the system. (For clock drivers, maybe not, though you would assume there would be a connection between them and device drivers depending on them, so it could probably follow the same idea).

This makes sense in the LionsOS context too, really.

midnightveil · 2026-08-04T05:57:29Z

+
+
+# pulp
+add_driver_config(


Well, you can already, see 'add_driver_config' you can import that and call it. I think there needs to be an global registry of drivers in some sense otherwise your meta program needs to duplicate driver selection logic.

Though it might be better to follow the "driver finder library" approach or something instead of doing what Linux calls the midlayer mistake. But then maybe that doesn't work with my proposed solution of creating drivers for every found DTS node where a compatible matches?

midnightveil · 2026-08-04T05:58:38Z

-        f.write(sdf.render())
+    out_file = f"{output_dir}/{sdf_file}"
+    sdf.make_config_structs()
+    print(f"Saving to {out_file}")


midnightveil · 2026-08-04T05:59:03Z

-        f.write(sdf.render())
+    out_file = f"{output_dir}/{sdf_file}"
+    sdf.make_config_structs()
+    print(f"Saving to {out_file}")


midnightveil · 2026-08-04T06:00:11Z

-BOARDS = board_module.BOARDS
-
-ProtectionDomain = SystemDescription.ProtectionDomain
+# board_module = importlib.import_module("board")


midnightveil · 2026-08-04T06:00:44Z

-        f.write(sdf.render())
+    sdf.make_config_structs()
+    out_file = f"{output_dir}/{sdf_file}"
+    print(f"Saving to {out_file}")


midnightveil · 2026-08-04T06:08:57Z

+        assert type(self.irqs) is list
+
+
+class __sDDFDriverManifest:


If anything I feel like this PR should just transliterate the exact same functionality from the zig sdfgen classes to Python, so that the difference is extremely minimal limited to a few type changes and import differences. Then we can discuss changing the API exposed in meta.py files at a later point in time to improve them wholistically.

These changes were necessary due to acacia working differently than sdf-gen. There's no real way around this without basically just putting the same functionality elsewhere in the python that is added, and indeed this is doing something similarly to what sdf_gen did by scanning the sddf for config.jsons and keeping a register of them

Not all of the changes were necessary. Perhaps moving away from config.json is better. Is it definitely? IDK. But it's a separate change that is logically distinct from a Zig -> Python rewrite.

We can't just do a zig to python rewrite when we need to move all the changes that were formerly in a different tool, in a different language, to here. Again, this is just doing something similar to what sdf_gen did.

See sddf.zig ... this is a replacement for these

var drivers: std.array_list.Managed(Config.Driver) = undefined; var classes: std.array_list.Managed(Config.DeviceClass) = undefined;

We can obviously change this in the future. But this bookkeeping isn't new.

(I can add searching for config.jsons if we want, just simpler calling direct for now..)

Not sure if this was considered and then decided against but can you avoid all the add_driver_config repetition and use the json by doing something like this:

Make the sDDFDriverClass class use the DT path to find the DT node like it does atm, then can it retrieve the compatible strings and provide this to the sDDFDriverManifest. Then the sDDFDriverManifest selects the json config from the compatible strings and parses the config into the form equivalent to what all the add_driver_config function calls are doing atm? I guess you would also provide the driver class name so the manifest knows which directory to search.

Make the sDDFDriverClass class use the DT path to find the DT node like it does atm, then can it retrieve the compatible strings and provide this to the sDDFDriverManifest. Then the sDDFDriverManifest selects the json config from the compatible strings and parses the config into the form equivalent to what all the add_driver_config function calls are doing atm? I guess you would also provide the driver class name so the manifest knows which directory to search.

The main thing is that we will need to rejig this all soon anyway. Fundamentally the sdf_gen model isn't really any good for handling dependencies or per-board deviations. The goal here for me was to basically do the same thing minimally for now

… update classes to respect this. Minor changes to api; got rid of (perilous) manual call to create resources Signed-off-by: Lesley Rossouw <lesley.rossouw@unsw.edu.au>

Signed-off-by: Lesley Rossouw <lesley.rossouw@unsw.edu.au>

cazb2 · 2026-08-08T02:21:36Z

+
+
+@dataclass
+class Board:


Can we remove the board.py indirection? This seems to duplicate information already present in the DTB which is never ideal since a human has to do this and is likely to make mistakes (such as the zcu paddr being wrong previously).

This is probably a consideration independent of this PR.

Per other comment I replied to, basically yes we can, and we probably should, but that's something that is beyond the scope of just adding Acacia

omeh-a added 3 commits July 3, 2026 09:20

Updated ina219 and bus scan i2c examples to use acacia

a55c47c

Signed-off-by: Lesley Rossouw <lesley.rossouw@unsw.edu.au>

Lint Python with Acacia additions

013449b

Signed-off-by: Lesley Rossouw <lesley.rossouw@unsw.edu.au>

omeh-a added drivers Issues pertaining to driver code for a device class driver-examples Issues related to examples for drivers for a target serial timer i2c labels Jul 8, 2026

wom-bat requested changes Jul 8, 2026

View reviewed changes

Update signatures to put SDF first for Acacia

6d06dfb

Signed-off-by: Lesley Rossouw <lesley.rossouw@unsw.edu.au>

omeh-a force-pushed the sdfgenpy branch from 370e7a7 to 6d06dfb Compare July 12, 2026 23:39

omeh-a added 2 commits July 13, 2026 10:09

Extract per-board / x86 metaprogram hacks to class.py

de0f706

Signed-off-by: Lesley Rossouw <lesley.rossouw@unsw.edu.au>

Add I2CAddress type and I2C address decoding for I2C class

55b8d7b

Signed-off-by: Lesley Rossouw <lesley.rossouw@unsw.edu.au>

Courtney3141 mentioned this pull request Jul 25, 2026

Update config structs module to use a parser to create a tree from dwarf dump output au-ts/microkit_acacia#22

Open

midnightveil requested changes Aug 4, 2026

View reviewed changes

midnightveil reviewed Aug 4, 2026

View reviewed changes

omeh-a added 2 commits August 6, 2026 15:56

Refactor sDDFDeviceClass to initialise configstructs at correct time,…

f8215fb

… update classes to respect this. Minor changes to api; got rid of (perilous) manual call to create resources Signed-off-by: Lesley Rossouw <lesley.rossouw@unsw.edu.au>

Added blk support using acacia

1d5b5c5

Signed-off-by: Lesley Rossouw <lesley.rossouw@unsw.edu.au>

omeh-a force-pushed the sdfgenpy branch from a95f14c to 1d5b5c5 Compare August 6, 2026 05:58

cazb2 reviewed Aug 8, 2026

View reviewed changes

Uh oh!

Conversation

omeh-a commented Jul 8, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

omeh-a Aug 5, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

omeh-a commented Jul 8, 2026 •

edited

Loading

omeh-a Aug 5, 2026 •

edited

Loading