A question about "Libc" example in Explainer.md

[yamt]

i'm reading https://github.com/WebAssembly/component-model/blob/main/design/mvp/Explainer.md
namely, the example cited below.
if i read it correctly, in the example libc instance exports its memory and main instance imports it.
my understanding is that in that case these modules (Main and Libc) need to have some negotiation about how
the shared memory is used. (eg. where to put their string literals)
i couldn't find any mechanism for such a negotiation. is there something i'm missing?

(component
  (import "wasi:logging" (instance $logging
    (export "log" (func (param string)))
  ))
  (import "libc" (core module $Libc
    (export "mem" (memory 1))
    (export "realloc" (func (param i32 i32) (result i32)))
  ))
  (core instance $libc (instantiate $Libc))
  (core func $log (canon lower
    (func $logging "log")
    (memory (core memory $libc "mem")) (realloc (func $libc "realloc"))
  ))
  (core module $Main
    (import "libc" "memory" (memory 1))
    (import "libc" "realloc" (func (param i32 i32) (result i32)))
    (import "wasi:logging" "log" (func $log (param i32 i32)))
    (func (export "run") (param i32 i32) (result i32 i32)
      ... (call $log) ...
    )
  )
  (core instance $main (instantiate $Main
    (with "libc" (instance $libc))
    (with "wasi:logging" (instance (export "log" (func $log))))
  ))
  (func $run (param string) (result string) (canon lift
    (core func $main "run")
    (memory $libc "mem") (realloc (func $libc "realloc"))
  ))
  (export "run" (func $run))
)

[lukewagner]

Hi! Good question. Yes, when two core modules share memory, there does need to be some negotiation and that's not shown in the example. Fundamentally, this is an custom ABI choice made independently by each language toolchain. For example, one option is to say that all global data has to be dynamically allocated by calling realloc and then copying in from a passive data segment (via memory.init) and then the address of string literals are shared via exported global i32s. This is roughly symmetric to the discussion about function pointer identity in the shared-everything dynamic linking example.

[yamt]

thank you for explanaton.
(i haven't noticed files in examples directory at all.)

Fundamentally, this is an custom ABI choice made independently by each language toolchain

are there any such conventions available right now? (for C, or any other languages)
or is it a "future work"?

[lukewagner]

In the canonical ABI, it's "future work". Predating the component model by many years, the toolchain-conventions repo contains a dynamic linking doc that currently depends on support from an external loader (often implemented by JS using the JS API in the browser). What I think we'd like to do with the Canonical ABI is to map module imports (which are unsupported in the current PR) to canonical core imports. Based on this, the lift-canonical-module mapping could prescribe a fixed linkage between all the imported core modules. It wouldn't be able to capture the full expressivity of the component model, but the goal would be to make it expressive enough to cover what's in examples/SharedEverythingDynamicLinking.md.

[yamt]

In the canonical ABI, it's "future work". Predating the component model by many years, the toolchain-conventions repo contains a dynamic linking doc that currently depends on support from an external loader (often implemented by JS using the JS API in the browser). What I think we'd like to do with the Canonical ABI is to map module imports (which are unsupported in the current PR) to canonical core imports. Based on this, the lift-canonical-module mapping could prescribe a fixed linkage between all the imported core modules. It wouldn't be able to capture the full expressivity of the component model, but the goal would be to make it expressive enough to cover what's in examples/SharedEverythingDynamicLinking.md.

do you mean it's in the scope of canonical abi to provide some kind of replacement of env.__memory_base and env.__table_base?

[lukewagner]

In theory, yes. This is a fairly new idea, so it may have fundamental problems that I'm not aware of yet and there are a lot more details to work out. @sunfishcode may also have more thoughts here.

[sunfishcode]

Yeah, something liike __memory_base and __table_base is possible. They might not be imports from env; I've been contemplating a scheme where modules load themselves, using malloc, memory.init, table.grow, and table.init, rather than having a separate dynamic loader, in which case they could compute their own __memory_base and __table_base rather than import them. But we can figure that out when we start implementing dynamic libraries.

[lukewagner]

Tentatively closing as the questions seem answered, but feel free to reopen with new questions.

[ayakoakasaka]

Hi, let me ask to re-open this topic because this feature(dynamic linking) will be essential for using Wasm for IoT to save the footprint size. (assume that there is a valuable wasm library that will be used from multiple wasm modules)

Question

1. Is this feature (dynamic linking with unified loader) is same as "parametric linking" described at This presentation P11 ?

2. proposal in this comment describes the same thing with "Data segment imports/exports" in This presentation P26 ?

Simply saying, I want to know the specification of "parametric linking."

[lukewagner]

"Parametric linking" is a term that we've invented to describe the collection of features in the component-model that allow more-flexible, more-composable linking of components and modules. Specifically, I consider "parametric linking" to include:

• the feature that allows a component to both import and nest modules and components (concretely, via component- and core module imports and definitions in the proposal)
• the feature that allows a component to instantiate a module or component, supplying all imports explicitly as arguments (concretely, via instance and core instance definitions in the proposal)

These features are described in Explainer.md/Binary.md and formalized in #101. They are also, iiuc, fully implemented in wasmtime and jco transpile. So all this covers the specification of "parametric linking".

What's still partially missing, but in-progress, is the per-language tooling support for emitting shareable core modules and emitting components that import shareable core modules. IIUC, @dicej has done some impressive work on this in the context of Python, to make dynamic linking of Python extensions work by reusing the low-level DynamicLinking toolchain support used by Emscripten. But more work is required to add proper dynamic linking support to, e.g., wasi-sdk so that it's easy for everyone to do.

I hope that helps, let me know if you have any more questions.

[ayakoakasaka]

Thank you for your comment.
It was constructive to understand the current status of "parametric linking." I must try wasmtime.

IIUC, @dicej has done some impressive work on this in the context of Python
Thank you for the information. I guess he is doing another impressive work which I must also try.
[3] https://github.com/dicej/wasi-sdk/tree/dynamic-linking

I'm excited to use this feature in our IoT environment.