add first prototype of texElementImage2D

[cabanier]

cc @kenrussell

[kenrussell]

One initial comment. Likely this PR will remain unmerged for a while until there are prototype implementations of this new API. In the meantime, progress can be made on the API's semantics, addressing the TBD mentioned here, etc.

[foolip]

cc @progers @schenney-chromium @szager @chrishtr

[foolip]

The answer to WICG/html-in-canvas#68 in this PR seems to be that we do keep level, and that's what we have in Chromium's implementation. Just flagging the point of discussion in case someone feels differently.

[foolip]

@cabanier what's still needed to make this ready for review? whatwg/html#11588 is making steady progress and I want to make sure the WebGL side is in sync.

Importantly, the HTML spec now has a concept of an "element image snapshot" which is what must be used in canvas APIs like this one. Can you update the spec text to use that concept? You can use a direct link to https://whatpr.org/html/11588/canvas.html#concept-canvas-element-image-snapshots until the HTML spec PR is merged.

[jakearchibald]

This seems pretty limiting compared to the 2d equivalent. Could this take optional width & height args? This seems especially important for high-dpr.

It also seems to be desirable to use a portion of an element in a texture, eg for extreme zoomed cases. This is also something you can do with the 2d API.

[szager-chromium]

I think the IDL changes in this PR are a bit behind; the current chromium implementation has four overrides for texElementImage2D which allow for specification of both source rect (for painting a section of an element, or for extending the painted area to include ink overflow) and also destination size (i.e. texture size).

@foolip can you update the PR to match the chromium implementation?

[cabanier]

The width/height was there originally but we were told that those APIs were going to be removed in the future. Has that changed?
What is the difference between passing width/height and setting it on the element?

[jakearchibald]

Setting it on the element means scaling everything else up within it, which is a pain. Trying to deal with the sub-rect case is even more painful.

[jakearchibald]

@szager-chromium that's good to hear. Although, it's a little frustrating that for some things, the explainer is more up to date, and for other things the PRs are more up to date, and for other things, like this, neither has the details.

[jakearchibald]

Where's the up to date proposal for the WebGPU API addition?

[szager-chromium]

I'm not sure there is one. I found an issue tracking this, but I'm not aware of any completed spec work. As with WebGL, I think the IDL in the chromium repository might be the most up-to-date reference.

[szager-chromium]

Now that I'm looking at the code (which ironically I wrote), I see there there is a variant that allows you specify destination width/height, and applies that as a scale factor to the element's paint record. There is also a variant that allows you to specify a source rect that behaves the same as for texElementImage2D. There is not however a variant that allows you specify both of those.

I can't remember what I was thinking there, but the WebGPU API shape has definitely not had a lot of scrutiny yet, and can still be easily changed.

[jakearchibald]

It does seem worth having an API that allows both. It'd seem odd if the WebGPU version was the least capable.

I guess this can get more review when it's properly proposed somewhere.

[szager-chromium]

The missing overload has been added to the WebGPU equivalent API.

[kenrussell]

Four overloads of this single entry point seem like a little much to me. Would it be possible to collapse this down to two by saying that passing 0 for GLsizei width/GLsizei height has default behavior of deriving the width/height from the CSS borderbox of the element at the time of rendering?

[kenrussell]

This is a duplicate of the entry on line 2240 and should be removed.

[kenrussell]

This overload - if it's necessary - should define how the width and height are computed, similarly to the overload at line 2215.

[szager-chromium]

Four overloads of this single entry point seem like a little much to me. Would it be possible to collapse this down to two by saying that passing 0 for GLsizei width/GLsizei height has default behavior of deriving the width/height from the CSS borderbox of the element at the time of rendering?

Functionally this would work, but it contrasts with texImage2D(), which does not interpret width/height of zero to mean intrinsic size. I'd be a bit concerned that this behavior might be magical and non-intuitive.

[lexaknyazev]

@szager-chromium

The existing texImage* commands support texture uploads from TexImageSource objects defined as:

typedef (ImageBitmap or
         ImageData or
         HTMLImageElement or
         HTMLCanvasElement or
         HTMLVideoElement or
         OffscreenCanvas or
         VideoFrame) TexImageSource;

Have you considered adding a new type to this list, e.g., RenderedElement, that would be returned by a brand new function like renderElement(only_relevant_opts,... ) and then could be used in all existing texture-related GL commands? Then this entire feature would be detached from multiple GL-specific semantics that otherwise must be carefully considered with the currently proposed design.

Just saying that rendering HTML to a 2D Canvas and using that Canvas as a WebGL texture source provides much more functional and convenient experience (modulo pixel redirection) and likely bypasses all issues listed below simply because they are already defined for uploads from TexImageSource objects.

---

Issue 1: Texture allocation vs texture upload

The texImage2D command that was clearly used as a base for texElementImage2D design is one of the most error-prone in GL because it does multiple things at once. Of course, implementations usually skip certain steps as appropriate but the semantics are still as listed:

1. It (re-)defines the texture state, including dimensions and format.
2. It (re-)allocates the texture memory.
3. It replaces the texture data, potentially with runtime type conversions.

Sub-region texture updates are possible with the texSubImage2D command that does only the third step from the list above but it looks like such functionality is not supported in the current proposal.

WebGL 2.0 supports a more efficient and safer workflow with immutable-format textures like this:

gl.texStorage2D(...) // forever defines all dimensions and the pixel format, zero-inits the texture
gl.texSubImage2D(...) // updates the specified texture region with provided data

The benefit here is that there won't be any implicit redefinitions and/or reallocations that are a common source of bugs both in apps and in implementations. This workflow would require setting the texture's dimensions before uploading the data, though.

Issue 2: Texture types

The texImage2D command that was clearly used as a base for texElementImage2D design supports two texture types (via the target parameter): TEXTURE_2D and six TEXTURE_CUBE_MAP_* values corresponding to cube map faces. Nothing technically prevents an application from rendering HTML elements to cube map faces because a rendered element is just an array of pixels at the end of the day. However, cube map faces must be squares and have equal dimensions, so the command variants that omit the target texture dimensions are likely less suitable for this texture type.

WebGL 2.0 also supports 2D texture arrays and sliced 3D textures via texImage3D and similar commands. As with cube maps, there's nothing wrong with uploading rendered elements to such texture types but GL supports per-slice data uploads only with the sub-region, i.e., texSubImage3D, command.

Issue 3: Texture formats

The internalformat, format, type tuple has a very specific meaning in GL that does not fully make sense for uploading rendered elements:

• internalformat is supposed to be the actual GPU format used by the texture. There're historical reasons for supporting so called "unsized" formats like RGBA for that parameter but that's obsolete and should not be used going forward. Looking at the current proposal, the RGB and RGBA values should be changed to RGB8 and RGBA8 respectively.
• format and type describe the application data supplied to the texture upload command. So when the texture data comes from some ArrayBufferView the format and type parameters are used to tell GL how to interpret raw data. In case of uploading rendered elements, there's no application-side byte data to describe so these parameters serve no purpose.

[szager-chromium]

@lexaknyazev -- Thank you for the very detailed analysis, this is very helpful.

We have discussed the possibility of adding another type to the TexImageSource union, but only briefly and without addressing the issues you've raised. The API proposal does feature an intermediate object called ElementImage, which is a static snapshot of an Element's painted output returned by a call to canvas.captureElementImage(element). However, I see two complications with adding it to TexImageSource.

• ElementImage was added to the API to facilitate canvas drawing on a worker thread, where references to Element are not permitted. For non-worker-thread usage, we allow the source argument to the API to be either Element or ElementImage. We lose some convenience if the call to captureElementImage() becomes mandatory. Not a huge deal, but we have been trying to keep the API as streamlined as possible in the common case.

• All of the existing variants of TexImageSource are bitmaps or trivially convertible to bitmap, but ElementImage is not. We rely on the other parameters to texElementImage2D (specifically sx,sy,swidth,sheight,width,height) when rasterizing the ElementImage to a bitmap. So we would actually need a different intermediate method call to produce the bitmap. So the invocation becomes something like:

const bitmap = canvas.captureElementBitmap(element_or_element_image, sx, sy, swidth, sheight, width, height);
gl.texImage2D(bitmap, ...);

I don't think this is an improvement over your suggestion of drawing the element into a 2D canvas and then calling gl.texImage2D(canvas_2d, ...), so I don't see the point in adding it.

Of the three issues you described, I think the first is the most important. There should be a way to re-upload the texture without re-allocating it. This is especially important for continuously-animating pages where the HTML content may update on every animation frame. I think this one should be addressed before the feature ships in Chrome.

Issue 2 seems a bit esoteric to me; and as you point out it's avoidable by specifying an explicit destination size. Maybe there's something we can do to make that nicer, but I wouldn't consider this a blocker.

The internalformat, format, type arguments were added just to conform to existing API's. I've observed that it's commonplace in OpenGL to have long lists of parameters that require specific static values, so I guess "when in Rome." I think we'd be happy to drop those arguments if there are no objections.

[lexaknyazev]

Issue 2 seems a bit esoteric to me; and as you point out it's avoidable by specifying an explicit destination size. Maybe there's something we can do to make that nicer, but I wouldn't consider this a blocker.

The first step with the issue 2 is to decide whether texture types other than TEXTURE_2D should be supported. The spec has to define valid target values anyway and in the worst case it can say that only TEXTURE_2D is valid. However, if cube map faces are also valid, there's no technical reason to not support TEXTURE_2D_ARRAY but that type would require a slightly different API shape to be created and/or uploaded, which would depend on the resolution of the sub-region updates.

The internalformat, format, type arguments were added just to conform to existing API's.

There are some subtleties to consider here. For example, the current proposal has this line:

Internal Format	Format	Type
RGBA16F	RGBA	HALF_FLOAT FLOAT

Setting the type to FLOAT and assuming literal GL semantics, it would mean that the browser is supposed to rasterize the element to float32 pixels and then pass them to the GPU for driver-internal conversion to float16. I don't think any implementation would intentionally support that path. Removing format and type would naturally eliminate such cases.

Another question is about all no-alpha formats. They are usually emulated in lower layers and in worst cases may involve costly per-pixel processing, e.g., to copy three-component RGB values to natively-supported RGBA with alpha set to 1.0. If the element rasterizer has an option to produce all-opaque alpha, that option could be passed explicitly to make sure that the declared texture format matches the internal representation.

[cabanier]

Of the three issues you described, I think the first is the most important. There should be a way to re-upload the texture without re-allocating it.

Some versions of this API take a width and height for the texture so if the author passes in the same values, there wouldn't be a reallocation.

This is especially important for continuously-animating pages where the HTML content may update on every animation frame. I think this one should be addressed before the feature ships in Chrome.

Do you know how this is handled inside Chromium? I suspect it has to optimized this internally to get good performance.

[szager-chromium]

Of the three issues you described, I think the first is the most important. There should be a way to re-upload the texture without re-allocating it.

Some versions of this API take a width and height for the texture so if the author passes in the same values, there wouldn't be a reallocation.

This is especially important for continuously-animating pages where the HTML content may update on every animation frame. I think this one should be addressed before the feature ships in Chrome.

Do you know how this is handled inside Chromium? I suspect it has to optimized this internally to get good performance.

This is frankly well outside my area of expertise; but in the chromium implementation we funnel into the same code path as texImage2D quite early so I would expect that any existing optimizations of that function to avoid reallocations will apply to texElementImage2D.

[szager-chromium]

Issue 2 seems a bit esoteric to me; and as you point out it's avoidable by specifying an explicit destination size. Maybe there's something we can do to make that nicer, but I wouldn't consider this a blocker.

The first step with the issue 2 is to decide whether texture types other than TEXTURE_2D should be supported. The spec has to define valid target values anyway and in the worst case it can say that only TEXTURE_2D is valid. However, if cube map faces are also valid, there's no technical reason to not support TEXTURE_2D_ARRAY but that type would require a slightly different API shape to be created and/or uploaded, which would depend on the resolution of the sub-region updates.

I would be inclined to restrict this to only TEXTURE_2D. Given the vagaries of converting from CSS pixels to the canvas grid, I think any kind of origami would be fraught, so I'm in favor of following the KISS principle.

The internalformat, format, type arguments were added just to conform to existing API's.

There are some subtleties to consider here. For example, the current proposal has this line:

Internal Format
Format
Type

RGBA16F
RGBA
HALF_FLOAT FLOAT

Setting the type to FLOAT and assuming literal GL semantics, it would mean that the browser is supposed to rasterize the element to float32 pixels and then pass them to the GPU for driver-internal conversion to float16. I don't think any implementation would intentionally support that path. Removing format and type would naturally eliminate such cases.

Removing those arguments sounds good to me!

Another question is about all no-alpha formats. They are usually emulated in lower layers and in worst cases may involve costly per-pixel processing, e.g., to copy three-component RGB values to natively-supported RGBA with alpha set to 1.0. If the element rasterizer has an option to produce all-opaque alpha, that option could be passed explicitly to make sure that the declared texture format matches the internal representation.

I'm pretty sure the web rendering system always produces RGBA (not sure if this is chromium-specific, but I doubt it). But any way you slice it, I can't think of any reason why we would want to allow users to pick a different internal format, even if it could be supported. So again, my KISS instinct is to just drop this parameter.

[lexaknyazev]

I'm pretty sure the web rendering system always produces RGBA (not sure if this is chromium-specific, but I doubt it). But any way you slice it, I can't think of any reason why we would want to allow users to pick a different internal format, even if it could be supported. So again, my KISS instinct is to just drop this parameter.

This means removing all RGB, RGB16F, and RGB32F lines from the format tables.

And while we're here, are 8-bit values supposed to be sRGB-encoded? If so, the internal format for them should be SRGB8_ALPHA8 because WebGL does not support format reinterpretation.

[szager-chromium]

On the question of excessive memory reallocations: even if we are able to optimize for the case where the size does not change with successive invocations, that still leaves a potential pitfall when animating the size or scale of an element. In that case, you would presumably want to allocate a texture at the maximum size of the animation and then draw sub-areas of it during other animation phases.

I think this could still be accomplished in a couple of ways with the API as written:

• Use constant swidth/sheight parameters that match the maximum size of the animation. This would likely require some fiddling with the sx/sy parameters to preserve alignment.

• Place the animating element inside a <div> that is statically sized to the maximum animation size, and draw the <div>. This is effectively equivalent equivalent to the first suggestion, but the alignment can be handled via CSS rather than sx/sy parameters.

Neither of those seems too onerous to me, but it does require the developer to be aware of the issue and take measures to avoid it. Requiring a call to texStorage2D() would force the issue, at the cost of a bit of extra complexity up front, even in simple cases.

[kenrussell]

Is the precise size of the Element or ElementImage known at the time the app intends to upload one or the other into the WebGL texture? My understanding from https://github.com/WICG/html-in-canvas is that if the app registers a ResizeObserver for the element's devicePixelContentBox, then it will get the most up-to-date pixel-accurate size at the point where the paint event is fired. And the ElementImage contains the width and height.

In that case it would be feasible to do a sub-image upload via a new texElementSubImage2D into a texture allocated via texStorage2D. If not - it's impractical.

It might be better to start simple regardless and only support texElementImage2D. Chrome's had longstanding performance issues where it was difficult to implement GPU-accelerated sub-image uploads for video textures given WebGL's semantics (rejecting the upload if the texture's internal format was incompatible with that of the existing WebGL texture). It was easier to redefine the texture every time with the source's internal format and size.

[szager-chromium]

I think I'd prefer to start as simple as possible and then return to the idea of texElementSubImage2D if and when there's demand for it. I'm open to other opinions though.

[cabanier]

@szager-chromium I tried to incorporate everyone's feedback in the new PR. Let me know if you'd like to see changes.

[cabanier]

@szager-chromium Any comments on the latest PR?

[ewewraw]

And while we're here, are 8-bit values supposed to be sRGB-encoded? If so, the internal format for them should be SRGB8_ALPHA8 because WebGL does not support format reinterpretation.

Looking at the canvas spec with the default set to srgb, the default internal format colorspace should probably be sRGB as well.

When placing html with RGB8 passed as internalFormat as the explainer suggests, I can see the difference in colors between regular HTML and HTML-in-canvas, so probably the default format should be a gamma corrected one?

[cabanier]

Looking at the canvas spec with the default set to srgb, the default internal format colorspace should probably be sRGB as well.

When placing html with RGB8 passed as internalFormat as the explainer suggests, I can see the difference in colors between regular HTML and HTML-in-canvas, so probably the default format should be a gamma corrected one?

I think there should be no difference when it's set to RGB8. @szager-chromium, is this a bug in Chromium's implementation?

[cabanier]

@lexaknyazev please let me know if you agree with my latest changes.

[lexaknyazev]

The spec should be very clear about the transfer functions used in each case.

I'd expect that 8-bit values are always sRGB-encoded and the choice between RGBA8 and SRGB8_ALPHA8 is only to allow apps to bypass hardware sRGB decoding during texture sampling if they wish so for some reason.

However, floating-point values are usually not sRGB encoded. If we aren't sure about the latter, maybe floating-point options could be omitted for now.

[szager-chromium]

Looking at the canvas spec with the default set to srgb, the default internal format colorspace should probably be sRGB as well.
When placing html with RGB8 passed as internalFormat as the explainer suggests, I can see the difference in colors between regular HTML and HTML-in-canvas, so probably the default format should be a gamma corrected one?

I think there should be no difference when it's set to RGB8. @szager-chromium, is this a bug in Chromium's implementation?

This it outside my expertise, but I think SRGB is already-gamma-corrected and RGBA is not, and the GL implementation treats them accordingly. Since we're dealing with web content, SRGB seems like the natural choice.

@lexaknyazev -- does this sound right to you?

[lexaknyazev]

This it outside my expertise, but I think SRGB is already-gamma-corrected and RGBA is not, and the GL implementation treats them accordingly. Since we're dealing with web content, SRGB seems like the natural choice.

The difference between RGBA8 and SRGB8_ALPHA8 is whether texture sampling performs sRGB-to-linear decode on read.

Consider the following sequence:

1. HTML rasterization produces a bitmap with a unorm8 value of 127/255.
2. That value is uploaded in an RGBA8 GL texture.
3. That texture is then sampled in a shader and the sampled value is 0.498 (because of the internal format used).
4. The shader simply copies that sampled value to the output.
5. If the canvas' drawing buffer storage is RGBA8 (default), then no further conversion happens and 127/255 is written to canvas.

You can argue that such workflow is neither semantically nor mathematically accurate but it should roundtrip rasterized values correctly.

A "proper" workflow would look like this:

1. HTML rasterization produces a bitmap with a unorm8 value of 127/255.
2. That value is uploaded in an SRGB8_ALPHA8 GL texture.
3. That texture is then sampled in a shader and the sampled value is 0.21223 (because of the internal format used).
4. The shader simply copies that sampled value to the output.
5. If the canvas' drawing buffer storage is SRGB8_ALPHA8 (set in advance by calling drawingBufferStorage), then that value is encoded back and 127/255 is written to canvas.

---

Important addition: unorm8 values produced by HTML rasterization must not depend on the internal format used for the GL texture.

[cabanier]

This it outside my expertise, but I think SRGB is already-gamma-corrected and RGBA is not, and the GL implementation treats them accordingly. Since we're dealing with web content, SRGB seems like the natural choice.

The difference between RGBA8 and SRGB8_ALPHA8 is whether texture sampling performs sRGB-to-linear decode on read.

I tried to incorporate your suggestion in 1f133db
Please let me know if it's correct and needed.

[lexaknyazev]

This looks good for the UNSIGNED_BYTE case but if the spec is going to include floating-point options, they'd need a separate clarification.

[cabanier]

Can browser render HTML to floats? @szager-chromium, is it specified how chromium renders HTML to an HDR 2D canvas?

[szager-chromium]

@ccameron-chromium, I think you know this area, can you answer the question about wide color gamuts and canvas?

[cabanier]

@lexaknyazev FYI I added the level parameter back in.

[szager-chromium]

@cabanier -- based on conversation with WebGL veterans, we'd like to change the API to behave more like texSubImage2D rather than texImage2D. This has two main implications:

• xoffset and yoffset arguments to be added to specify an offset into the destination texture
• No automatic texture storage allocation; like texSubImage2D, the receiving texture must already have allocated storage via a preceding call to texImage2D with a null argument for srcData (WebGL 1 or 2), or a preceding call to texStorage2D (WebGL 2 only).

I've got a CL up to implement this behavior in chromium, and I'm going to add inline comments to this PR where it needs to be updated.

[szager-chromium]

Add GLint xoffset and GLint yoffset parameters after sheight.

[lexaknyazev]

The current Element API is

texElementImage2D(target, level, internalformat, element, config)

The texSubImage2D API is

texSubImage2D(target, level, xoffset, yoffset, ...)

Given that the proposed xoffset and yoffset parameters refer to the GL API, why add them to the config object (that is Element-specific) instead of the command (that is GL-specific)? I'd expect them to be in the same group as the level and the WebGL WG decided that level should be among the command parameters, not the config object.

[lexaknyazev]

Also since the new design no longer allocates a GL texture, the internalformat parameter needs to be moved to the config.

[lexaknyazev]

In other words:

• parameters that drive Element rasterization should be in the config;
• parameters that directly interact with GL should be on their usual places.

[szager-chromium]

The motivation for moving parameters to WebGLCopyElementImageConfig was just to avoid the need for overloads of the method just to accommodate various permutations of geometry arguments. The existing WebGLCopyElementImageConfig.(width|height) parameters already specify a destination size, and any scaling they imply is performed by GL, not by web content rendering. So I don't think adding xoffset and yoffset here contradicts any existing design principle or separation of web-specific from GL-specific arguments. The purpose of WebGLCopyElementImageConfig is really to encapsulate all the geometry.

[szager-chromium]

This is not what I have observed.

GL (on its own) does not have an ability to scale texture data during upload. You could have seen the Chromium behavior related to texture uploads from DOM images (which is not fully documented).

OK fair enough; but this doesn't really change my point about the purpose of WebGLCopyElementImageConfig being to contain all the geometry parameters, source and destination alike. From the perspective of a web developer, width and height are destination parameters, and I don't think a distinction between "part of OpenGL" and "part of the WebGL layer" is meaningful for them.

[lexaknyazev]

width and height are destination parameters

Not quite if the command follows the texSubImage design.

There are several widths and heights here that exist in separate domains:

1. Dimensions of the texture. These are set in advance by the developer (by calling regular texStorage or texImage commands) and not related to the Element API in any way.
2. Dimensions and the offset of the Element's source rectangle. These affect Element rasterization and are not related to the texture.
3. Target rasterization dimensions. These also only affect the rasterization and are not related to the texture (as it exists already).
4. Texture's sub-region offset (xoffset and yoffset). These have nothing to do with the rasterization geometry and only exist within the texture dimensions set in 1.

[szager-chromium]

I have in mind two reasons to prefer adding xoffset and yoffset to the config and leaving internalFormat where it is:

• It places all geometry parameters in config, which I think is good ergonomics.

• It doesn't break existing calling conventions if we allow xoffset and yoffset to default to zero when not specified.

In my mind, that's more compelling than keeping the GL-specific bits outside the config. The very fact that this function uses a dictionary of parameters already sets it apart from existing WebGL API's.

Maybe we should get this on the WebGL working group agenda and discuss it there, what do you think?

[cabanier]

Maybe we should get this on the WebGL working group agenda and discuss it there, what do you think?

+1. I'll hold off on making changes until it's been discussed.

[szager-chromium]

@lexaknyazev -- thinking more about this, I see your point about aligning with the function signature of texSubImage2D and keeping the GL-specific parameters separate. Until you pointed it out, I didn't actually know that the automatic scaling behavior of the width and height parameters was specific to the WebGL layer.

I'm OK with putting xoffset and yoffset in the main method signature rather than in the config. I also think that we can probably get rid of the internalFormat parameter. So the new IDL would be:

  undefined texElementImage2D(GLenum target, GLint level, GLint xoffset, GLint yoffset,
                              (Element or ElementImage) element,
                              optional WebGLCopyElementImageConfig config = {}); // May throw DOMException

Does this look good to you?

[szager-chromium]

Replace "they define the generated texture size" with "they specify the area of the texture to be modified; this will result in scaling of the source graphic if the specified size does not match the source element's intrinsic size."

[cabanier]

• No automatic texture storage allocation; like texSubImage2D, the receiving texture must already have allocated storage via a preceding call to texImage2D with a null argument for srcData (WebGL 1 or 2), or a preceding call to texStorage2D (WebGL 2 only).

This is a big change. How will the author know the dimension of the texture?

[szager-chromium]

Add GLint xoffset and GLint yoffset

[szager-chromium]

• No automatic texture storage allocation; like texSubImage2D, the receiving texture must already have allocated storage via a preceding call to texImage2D with a null argument for srcData (WebGL 1 or 2), or a preceding call to texStorage2D (WebGL 2 only).

This is a big change. How will the author know the dimension of the texture?

Here is some overly-verbose boilerplate for drawing an element at its intrinsic/natural size; I expect developers can use a slimmed-down version of this (or maybe we can add a convenience API to provide it):

    // Pre-allocate texture storage big enough for draw_element.
    const canvasCSSWidth =
      Number.parseFloat(getComputedStyle(canvas).width);
    const canvasCSSHeight =
      Number.parseFloat(getComputedStyle(canvas).height);
    // Scaling factor from CSS pixels to canvas grid.
    const canvasScaleX = canvas.width / canvasCSSWidth;
    const canvasScaleY = canvas.height / canvasCSSHeight;
    const targetCSSWidth =
      Number.parseFloat(getComputedStyle(target).width);
    const targetCSSHeight =
      Number.parseFloat(getComputedStyle(target).height);
    // Allocate buffer to accomodate target in canvas grid units. Include
    // one-pixel fudge factor to allow for differences between single-precision
    // floating-point arithmetic used by GL implementation and double-precision
    // arithmetic used by JavaScript.
    const texWidth = Math.ceil(targetCSSWidth * canvasScaleX + 1);
    const texHeight = Math.ceil(targetCSSHeight * canvasScaleY + 1);
    gl.texImage2D(gl.TEXTURE_2D, 0, internalFormat, texWidth, texHeight, 0,
                  srcFormat, destType, null);

The feedback I got from WebGL veterans is that the automatic allocation behavior of texImage2D is widely considered to be a regrettable mistake and common performance footgun, and modern best practice is to pre-allocate texture storage and use texSubImage2D. Another issue was the common technique of using sprite sheets to store many graphics in a single buffer. The existing API offered no way to accomplish that.

Consider, for example, a simple layout animation:

<canvas id="canvas" width=100 height=100 layoutsubtree>
  <div id="target" style="width:50px; transition: width 1s;">Hello, world!</div>
</canvas>

<script>
const gl = canvas.getContext("webgl2");
canvas.onpaint = evt => {
  gl.texElementImage2D(gl.TEXTURE_2D, 0, gl.RGBA8, target);
  drawScene();
};
target.style.width = "100px";

Under the current proposal, this would re-allocate the texture buffer on every paint. The recommended best practice for developers will be to pre-allocate a texture big enough to hold the maximum size of the target under the effects of the animation.