-
Notifications
You must be signed in to change notification settings - Fork 3.1k
Refactor: separate camera state from renderer-owned rasterization (cairo) + consolidate renderer related modules #4989
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from 5 commits
71a4c8c
288ce99
4a2f751
bb803be
d19d5c4
53bf048
6fbe2d4
873c046
70761d5
c6cd373
96ae608
94ad6c2
5623b1f
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -326,7 +326,7 @@ renderer reference or read mutable global configuration. Directories are created | |
| lazily when their owning operation first writes. The writer remains Manim's | ||
| interface to ``libav`` for media assembly. The Cairo renderer (see the | ||
| implementation `here | ||
| <https://github.com/ManimCommunity/manim/blob/main/manim/renderer/cairo_renderer.py>`__) | ||
| <https://github.com/ManimCommunity/manim/blob/main/manim/renderer/cairo/renderer.py>`__) | ||
| does not require further renderer-specific initialization. OpenGL creates a | ||
| window only when the resolved presentation specification requests a live preview. | ||
| The ``-p`` / ``--preview`` option does not create this window; it opens the | ||
|
|
@@ -998,44 +998,35 @@ the *static mobjects* are assumed to have already been painted statically to | |
| the background of the scene). All of the hard work then happens when the renderer | ||
| updates its current frame via a call to :meth:`.CairoRenderer.update_frame`: | ||
|
|
||
| First, the renderer prepares its :class:`.Camera` by checking whether the renderer | ||
| has a ``static_image`` different from ``None`` stored already. If so, it sets the | ||
| image as the *background image* of the camera via :meth:`.Camera.set_frame_to_background`, | ||
| and otherwise it just resets the camera via :meth:`.Camera.reset`. The camera is then | ||
| asked to capture the scene with a call to :meth:`.Camera.capture_mobjects`. | ||
|
|
||
| Things get a bit technical here, and at some point it is more efficient to | ||
| delve into the implementation -- but here is a summary of what happens once the | ||
| camera is asked to capture the scene: | ||
|
|
||
| - First, a flat list of mobjects is created (so submobjects get extracted from | ||
| their parents). This list is then processed in groups of the same type of | ||
| mobjects (e.g., a batch of vectorized mobjects, followed by a batch of image mobjects, | ||
| followed by more vectorized mobjects, etc. -- in many cases there will just be | ||
| one batch of vectorized mobjects). | ||
| - Depending on the type of the currently processed batch, the camera uses dedicated | ||
| *display functions* to convert the :class:`.Mobject` Python object to | ||
| a NumPy array stored in the camera's ``pixel_array`` attribute. | ||
| The most important example in that context is the display function for | ||
| vectorized mobjects, :meth:`.Camera.display_multiple_vectorized_mobjects`, | ||
| or the more particular (in case you did not add a background image to your | ||
| :class:`.VMobject`), :meth:`.Camera.display_multiple_non_background_colored_vmobjects`. | ||
| This method first gets the current Cairo context, and then, for every (vectorized) | ||
| mobject in the batch, calls :meth:`.Camera.display_vectorized`. There, | ||
| the actual background stroke, fill, and then stroke of the mobject is | ||
| drawn onto the context. See :meth:`.Camera.apply_stroke` and | ||
| :meth:`.Camera.set_cairo_context_color` for more details -- but it does not get | ||
| much deeper than that, in the latter method the actual Bézier curves | ||
| determined by the points of the mobject are drawn; this is where the low-level | ||
| interaction with Cairo happens. | ||
|
|
||
| After all batches have been processed, the camera has an image representation | ||
| of the Scene at the current time stamp in form of a NumPy array stored in its | ||
| ``pixel_array`` attribute. The renderer passes a top-left-origin, | ||
| C-contiguous ``uint8`` RGBA array to its :class:`.SceneFileWriter`. OpenGL uses | ||
| the same array contract and performs GPU readback at this renderer boundary only | ||
| when file output needs a frame. This concludes one iteration of the render loop, | ||
| and once the time progression has been processed completely, a final bit | ||
| First, the renderer prepares its own Cairo raster target. If a reusable | ||
| ``static_image`` is available, the renderer copies it into that target; otherwise it | ||
| resets the target from the semantic background settings of :class:`.Camera`. | ||
| Background images whose dimensions differ from the target are resized to the target | ||
| dimensions. The camera itself does not own pixels or a Cairo context; its constructor | ||
| accepts semantic view settings rather than pixel dimensions or frame-rate options. | ||
|
|
||
| Things get a bit technical here, and at some point it is more efficient to delve into | ||
| the implementation -- but the renderer-owned drawing process can be summarized as | ||
| follows: | ||
|
|
||
| - The camera supplies a flat, ordered list of visible mobjects and applies pure | ||
| view/projection and shading transformations. Its animatable ``frame`` describes the | ||
| logical region being viewed. | ||
| - Private Cairo renderer helpers process consecutive batches of vectorized, point | ||
| cloud, and image mobjects without changing their draw order. | ||
| - Vectorized mobjects are converted to Cairo paths and drawn with their background | ||
| stroke, fill, and foreground stroke. Point clouds and image mobjects are converted | ||
| to target pixel coordinates and composited by renderer helpers. | ||
| - A :class:`.MultiCamera` describes nested camera-backed views. Their | ||
| :class:`.ImageMobjectFromCamera` display mobjects contain geometry and sampling | ||
| settings but no placeholder or live pixels. The renderer creates secondary targets | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Not clear what it means for a camera to (not) contain "placeholder or live pixels". |
||
| lazily, excludes each view's own display from its source camera, and composites the | ||
| result into the primary target. | ||
|
|
||
| After all batches have been processed, :class:`.CairoRenderer` owns the image | ||
| representation of the Scene. It passes a fresh top-left-origin, C-contiguous ``uint8`` | ||
| RGBA array to its :class:`.SceneFileWriter`. This concludes one iteration of the | ||
| render loop, and once the time progression has been processed completely, a final bit | ||
| of cleanup is performed before the :meth:`.Scene.play_internal` call is completed. | ||
|
|
||
| A TL;DR for the render loop, in the context of our toy example, reads as follows: | ||
|
|
@@ -1049,11 +1040,11 @@ A TL;DR for the render loop, in the context of our toy example, reads as follows | |
| state of the transformation animation to the desired time stamp (for example, | ||
| at time stamp ``t = 45/30``, the animation is completed to a rate of | ||
| ``alpha = 0.5``). | ||
| - Then the scene asks the renderer to do its job. The renderer asks its camera | ||
| to capture the scene, the only mobject that needs to be processed at this point | ||
| is the main mobject attached to the transformation; the camera converts the | ||
| current state of the mobject to entries in a NumPy array. The renderer passes | ||
| this array to the file writer. | ||
| - Then the scene asks the renderer to do its job. The only mobject that needs to | ||
| be processed at this point is the main mobject attached to the transformation. | ||
| The camera supplies its semantic view transform, while renderer-owned Cairo | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Perhaps just "supplies its view transform"? |
||
| helpers draw the current mobject state into the renderer's raster target. The | ||
| renderer reads that target and passes an owned array to the file writer. | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
I think it should be more clear exactly which array is being passed to the file writer; it tells the user nothing that it is "owned". |
||
| - At the end of the loop, 90 frames have been passed to the file writer. | ||
|
|
||
| Completing the render loop | ||
|
|
||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
"The camera itself does not own pixels or a Cairo context; its constructor accepts semantic view settings rather than pixel dimensions or frame-rate options." should probably be removed; the user doesn't care what the camera doesn't do.