-
Notifications
You must be signed in to change notification settings - Fork 3.1k
Refactor: separate camera state from renderer-owned rasterization (cairo) + consolidate renderer related modules #4989
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from 10 commits
71a4c8c
288ce99
4a2f751
bb803be
d19d5c4
53bf048
6fbe2d4
873c046
70761d5
c6cd373
96ae608
94ad6c2
5623b1f
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,190 @@ | ||
| Working with cameras and scene images | ||
| ===================================== | ||
|
|
||
| A camera describes the logical view of a scene: its position, visible extent, and | ||
| projection. A renderer turns that view into pixels. You normally work with the camera | ||
| through ``self.camera`` and request images through the scene, without managing a renderer. | ||
|
|
||
| Moving the Cairo camera | ||
| ----------------------- | ||
|
|
||
| The ordinary Cairo :class:`.Camera` has an animatable ``frame``. You do not need a | ||
| special scene subclass to pan or zoom:: | ||
|
|
||
| class CameraExample(Scene): | ||
| def construct(self): | ||
| square = Square().shift(2 * RIGHT) | ||
| self.add(square) | ||
| self.play(self.camera.frame.animate.move_to(square)) | ||
| self.play(self.camera.frame.animate.scale(0.5)) | ||
|
|
||
| A smaller frame zooms in; a larger frame shows more of the scene. Save and restore the | ||
| frame with the usual mobject operations:: | ||
|
|
||
| self.camera.frame.save_state() | ||
| self.play(self.camera.auto_zoom([square])) | ||
| self.play(Restore(self.camera.frame)) | ||
|
|
||
| :class:`.MovingCameraScene` remains available as a descriptive name for this behavior. | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I would either remove this entirely or make explicit that |
||
| These frame examples describe the Cairo camera; OpenGL uses its own camera controls. | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Perhaps make this more clear (or include a section for OpenGL as well?) |
||
|
|
||
| Logical view and image resolution | ||
| --------------------------------- | ||
|
|
||
| Frame dimensions are in scene units; pixel dimensions specify the raster resolution. | ||
| Configure pixel dimensions before constructing the scene or renderer:: | ||
|
behackl marked this conversation as resolved.
Outdated
|
||
|
|
||
| with tempconfig({"pixel_width": 640, "pixel_height": 360}): | ||
| scene = Scene() | ||
| scene.add(Square()) | ||
| image = scene.get_image() | ||
|
|
||
| A default Cairo camera preserves ``config.frame_width`` and derives height from the | ||
| configured pixel aspect ratio. Square and portrait output therefore preserve ordinary | ||
| geometry. One explicit camera dimension determines the other using that aspect ratio; | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. What does |
||
| two dimensions or a custom frame preserve the geometry you specify:: | ||
|
|
||
| camera = Camera(frame_width=8, frame_height=4) | ||
| camera.frame.move_to([2, 1, 0]) | ||
|
|
||
| With both dimensions explicit, choose the same logical and raster aspect ratio when | ||
| undistorted output is required. Drawing does not resize your semantic frame. Scene | ||
| image requests use the existing renderer dimensions, not later pixel-config edits. | ||
|
behackl marked this conversation as resolved.
Outdated
|
||
|
|
||
| Inspecting the current scene | ||
| ---------------------------- | ||
|
|
||
| :meth:`.Scene.get_image` freshly draws the current scene and returns a PIL image. | ||
| It includes manual changes since the last animation and the current camera view:: | ||
|
|
||
| class InspectExample(Scene): | ||
| def construct(self): | ||
| square = Square() | ||
| self.add(square) | ||
| self.get_image().save("before.png") | ||
| self.play(square.animate.shift(RIGHT)) | ||
| self.get_image().save("after.png") | ||
|
|
||
| Use ``scene.show()`` to open a fresh image in PIL's external image viewer. In a notebook, | ||
| display the returned PIL image directly. Saving and opening an image are explicit; | ||
| ``get_image()`` itself does not write a media artifact or open a viewer. | ||
|
behackl marked this conversation as resolved.
Outdated
|
||
|
|
||
| An image request does not execute construction, run updaters, advance scene time, or | ||
| append a movie frame. It photographs the graph as it stands, even if updater-derived | ||
| geometry has not yet been refreshed. The post-animation graph may differ from the last | ||
| encoded sample because animation finish/cleanup has already run. This is inspection, | ||
| not seeking or replaying an earlier animation position. | ||
|
|
||
| Request images between plays or at an idle prompt. OpenGL capture must run on the thread | ||
| that owns the rendering context; arbitrary worker-thread calls, including background | ||
| embedded-shell calls, are not dispatched automatically. Both Cairo and OpenGL draw into | ||
| independent temporary targets rather than replacing the active frame. Returned images | ||
| remain usable after those temporary targets are released. Explicit image requests also | ||
| work in dry-run mode; they are intentional raster work requested by your Python code. | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This part is very AI-sloppy and should be rewritten. In particular, it spends too much time on what the code doesn't do. I also wanna call out that the "graph" is mentioned nowhere else in Manim docs. I like the mention of a scene graph, but if we want it, it has to be introduced properly somewhere. |
||
|
|
||
| Inspecting individual mobjects | ||
| ------------------------------ | ||
|
|
||
| For ordinary Cairo mobjects, use :meth:`.Mobject.get_image` or :meth:`.Mobject.show`:: | ||
|
|
||
| Square().show() | ||
| Group(Square().shift(LEFT), Circle().shift(RIGHT)).get_image().save("objects.png") | ||
| image = square.get_image(camera=self.camera) | ||
|
|
||
| The optional camera selects the view, not the scene contents: only the supplied mobject | ||
| and its family are drawn. Without it, a default camera is used. These standalone helpers | ||
| are Cairo-specific; use ``scene.get_image()`` for an OpenGL scene, including its meshes. | ||
|
behackl marked this conversation as resolved.
Outdated
|
||
|
|
||
| Three-dimensional and nested views | ||
| ---------------------------------- | ||
|
|
||
| Use :class:`.ThreeDScene` and its camera orientation methods for three-dimensional | ||
| scenes. Image inspection uses the current projection and fixed-object declarations, | ||
| just like ordinary drawing. | ||
|
|
||
| For an inset magnified view, :class:`.ZoomedScene` provides the camera and display | ||
| relationship:: | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Not crazy about "the camera and display relationship" but I'm unsure what a better phrasing would look like. |
||
|
|
||
| class DetailExample(ZoomedScene): | ||
| def construct(self): | ||
| self.add(Square()) | ||
| self.activate_zooming(animate=False) | ||
| self.get_image().save("detail.png") | ||
|
|
||
| Multiple camera views | ||
| --------------------- | ||
|
|
||
| The Cairo backend supports several camera views within one scene through | ||
| :class:`.MultiCamera`. The primary camera draws the overall scene; each secondary | ||
| camera supplies an image displayed by an :class:`.ImageMobjectFromCamera` mobject. | ||
| This is live composition of the same scene, not separate Scene executions or separate | ||
| video outputs. This API is not supported by the OpenGL backend. | ||
|
behackl marked this conversation as resolved.
Outdated
|
||
|
|
||
| There are two independent controls: | ||
|
|
||
| * The secondary camera's ``frame`` selects the region to look at. Move it to pan, | ||
| or shrink it to zoom in. | ||
| * The display mobject selects where that view appears in the primary scene. Move or | ||
| scale it like another mobject, without changing the secondary camera's view. | ||
|
behackl marked this conversation as resolved.
Outdated
|
||
|
|
||
| For example, this scene places two detail views above the original objects:: | ||
|
|
||
| class TwoCameraViews(Scene): | ||
| def __init__(self, **kwargs): | ||
| super().__init__(camera_class=MultiCamera, **kwargs) | ||
|
|
||
| def construct(self): | ||
| circle = Circle(color=YELLOW).shift(2 * LEFT + DOWN) | ||
| square = Square(color=BLUE).shift(2 * RIGHT + DOWN) | ||
| self.add(circle, square) | ||
|
|
||
| left_camera = Camera(frame_width=4, frame_height=3) | ||
| right_camera = Camera(frame_width=4, frame_height=3) | ||
| left_camera.frame.move_to(circle) | ||
| right_camera.frame.move_to(square) | ||
|
|
||
| left_view = ImageMobjectFromCamera(left_camera) | ||
| right_view = ImageMobjectFromCamera(right_camera) | ||
| left_view.scale_to_fit_width(3).to_corner(UL) | ||
| right_view.scale_to_fit_width(3).to_corner(UR) | ||
|
|
||
| for view in (left_view, right_view): | ||
| view.add_display_frame() | ||
| self.camera.add_image_mobject_from_camera(view) | ||
| self.add(view) | ||
|
|
||
| # Zoom the left view without resizing its display. | ||
| self.play(left_camera.frame.animate.scale(0.5)) | ||
| # Pan the right view from the square to the circle. | ||
| self.play(right_camera.frame.animate.move_to(circle)) | ||
| self.wait() | ||
|
|
||
| Run this example with ``--renderer=cairo``. Selecting ``MultiCamera`` in the constructor | ||
| ensures it is installed before the scene's renderer is initialized. | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I suggest a separate "changing camera class" section which explains this, since this is the current intended approach for all custom cameras. Also "installed" is weird phrasing here, is there a more pythonesque word you could use instead? |
||
|
|
||
| Both registration and scene membership matter: registering a display tells MultiCamera | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. "Both registration and scene membership matter" is mega-slopspeak :D |
||
| to produce its view; ``self.add(view)`` places the display in the scene's draw order. | ||
| ``add_display_frame()`` adds an optional visible border. The secondary camera's own | ||
| ``frame`` is a view control and is not automatically shown as an outline in the scene. | ||
|
|
||
| A display initially matches its source camera's aspect ratio. Scale it uniformly to | ||
| preserve that ratio; stretching only its width or height can distort the image. | ||
| The renderer chooses the secondary raster size from the display's size relative to | ||
| the primary view and manages resizing and pixel transfer automatically. | ||
|
|
||
| Secondary cameras share the scene's contents rather than having separate object lists. | ||
| Each display and its border are excluded from their own source view. In the example, | ||
| the detail cameras look below the insets so neither inset appears in the other. | ||
| Sibling views are processed in registration order; do not rely on them recursively | ||
| containing each other. For deeper nesting, a secondary camera may itself be a | ||
| MultiCamera with its own registered displays. Cyclic camera registrations are rejected | ||
| rather than rendered recursively forever. | ||
|
|
||
| To remove a view entirely, remove both its visible mobject and its registration:: | ||
|
|
||
| self.remove(left_view) | ||
| self.camera.image_mobjects_from_cameras.remove(left_view) | ||
|
|
||
| The renderer retires unused secondary targets on the next draw. A scene image requested | ||
| with ``self.get_image()`` includes all currently registered and visible views; there is | ||
| no need to copy camera pixels or refresh each inset yourself. | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Another instance of the doc specifying what the user doesn't need to do. |
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -326,7 +326,7 @@ renderer reference or read mutable global configuration. Directories are created | |
| lazily when their owning operation first writes. The writer remains Manim's | ||
| interface to ``libav`` for media assembly. The Cairo renderer (see the | ||
| implementation `here | ||
| <https://github.com/ManimCommunity/manim/blob/main/manim/renderer/cairo_renderer.py>`__) | ||
| <https://github.com/ManimCommunity/manim/blob/main/manim/renderer/cairo/renderer.py>`__) | ||
| does not require further renderer-specific initialization. OpenGL creates a | ||
| window only when the resolved presentation specification requests a live preview. | ||
| The ``-p`` / ``--preview`` option does not create this window; it opens the | ||
|
|
@@ -998,44 +998,35 @@ the *static mobjects* are assumed to have already been painted statically to | |
| the background of the scene). All of the hard work then happens when the renderer | ||
| updates its current frame via a call to :meth:`.CairoRenderer.update_frame`: | ||
|
|
||
| First, the renderer prepares its :class:`.Camera` by checking whether the renderer | ||
| has a ``static_image`` different from ``None`` stored already. If so, it sets the | ||
| image as the *background image* of the camera via :meth:`.Camera.set_frame_to_background`, | ||
| and otherwise it just resets the camera via :meth:`.Camera.reset`. The camera is then | ||
| asked to capture the scene with a call to :meth:`.Camera.capture_mobjects`. | ||
|
|
||
| Things get a bit technical here, and at some point it is more efficient to | ||
| delve into the implementation -- but here is a summary of what happens once the | ||
| camera is asked to capture the scene: | ||
|
|
||
| - First, a flat list of mobjects is created (so submobjects get extracted from | ||
| their parents). This list is then processed in groups of the same type of | ||
| mobjects (e.g., a batch of vectorized mobjects, followed by a batch of image mobjects, | ||
| followed by more vectorized mobjects, etc. -- in many cases there will just be | ||
| one batch of vectorized mobjects). | ||
| - Depending on the type of the currently processed batch, the camera uses dedicated | ||
| *display functions* to convert the :class:`.Mobject` Python object to | ||
| a NumPy array stored in the camera's ``pixel_array`` attribute. | ||
| The most important example in that context is the display function for | ||
| vectorized mobjects, :meth:`.Camera.display_multiple_vectorized_mobjects`, | ||
| or the more particular (in case you did not add a background image to your | ||
| :class:`.VMobject`), :meth:`.Camera.display_multiple_non_background_colored_vmobjects`. | ||
| This method first gets the current Cairo context, and then, for every (vectorized) | ||
| mobject in the batch, calls :meth:`.Camera.display_vectorized`. There, | ||
| the actual background stroke, fill, and then stroke of the mobject is | ||
| drawn onto the context. See :meth:`.Camera.apply_stroke` and | ||
| :meth:`.Camera.set_cairo_context_color` for more details -- but it does not get | ||
| much deeper than that, in the latter method the actual Bézier curves | ||
| determined by the points of the mobject are drawn; this is where the low-level | ||
| interaction with Cairo happens. | ||
|
|
||
| After all batches have been processed, the camera has an image representation | ||
| of the Scene at the current time stamp in form of a NumPy array stored in its | ||
| ``pixel_array`` attribute. The renderer passes a top-left-origin, | ||
| C-contiguous ``uint8`` RGBA array to its :class:`.SceneFileWriter`. OpenGL uses | ||
| the same array contract and performs GPU readback at this renderer boundary only | ||
| when file output needs a frame. This concludes one iteration of the render loop, | ||
| and once the time progression has been processed completely, a final bit | ||
| First, the renderer prepares its own Cairo raster target. If a reusable | ||
| ``static_image`` is available, the renderer copies it into that target; otherwise it | ||
| resets the target from the semantic background settings of :class:`.Camera`. | ||
| Background images whose dimensions differ from the target are resized to the target | ||
| dimensions. The camera itself does not own pixels or a Cairo context; its constructor | ||
| accepts semantic view settings rather than pixel dimensions or frame-rate options. | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. "The camera itself does not own pixels or a Cairo context; its constructor accepts semantic view settings rather than pixel dimensions or frame-rate options." should probably be removed; the user doesn't care what the camera doesn't do. |
||
|
|
||
| Things get a bit technical here, and at some point it is more efficient to delve into | ||
| the implementation -- but the renderer-owned drawing process can be summarized as | ||
| follows: | ||
|
|
||
| - The camera supplies a flat, ordered list of visible mobjects and applies pure | ||
| view/projection and shading transformations. Its animatable ``frame`` describes the | ||
| logical region being viewed. | ||
| - Private Cairo renderer helpers process consecutive batches of vectorized, point | ||
| cloud, and image mobjects without changing their draw order. | ||
| - Vectorized mobjects are converted to Cairo paths and drawn with their background | ||
| stroke, fill, and foreground stroke. Point clouds and image mobjects are converted | ||
| to target pixel coordinates and composited by renderer helpers. | ||
| - A :class:`.MultiCamera` describes nested camera-backed views. Their | ||
| :class:`.ImageMobjectFromCamera` display mobjects contain geometry and sampling | ||
| settings but no placeholder or live pixels. The renderer creates secondary targets | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Not clear what it means for a camera to (not) contain "placeholder or live pixels". |
||
| lazily, excludes each view's own display from its source camera, and composites the | ||
| result into the primary target. | ||
|
|
||
| After all batches have been processed, :class:`.CairoRenderer` owns the image | ||
| representation of the Scene. It passes a fresh top-left-origin, C-contiguous ``uint8`` | ||
| RGBA array to its :class:`.SceneFileWriter`. This concludes one iteration of the | ||
| render loop, and once the time progression has been processed completely, a final bit | ||
| of cleanup is performed before the :meth:`.Scene.play_internal` call is completed. | ||
|
|
||
| A TL;DR for the render loop, in the context of our toy example, reads as follows: | ||
|
|
@@ -1049,11 +1040,11 @@ A TL;DR for the render loop, in the context of our toy example, reads as follows | |
| state of the transformation animation to the desired time stamp (for example, | ||
| at time stamp ``t = 45/30``, the animation is completed to a rate of | ||
| ``alpha = 0.5``). | ||
| - Then the scene asks the renderer to do its job. The renderer asks its camera | ||
| to capture the scene, the only mobject that needs to be processed at this point | ||
| is the main mobject attached to the transformation; the camera converts the | ||
| current state of the mobject to entries in a NumPy array. The renderer passes | ||
| this array to the file writer. | ||
| - Then the scene asks the renderer to do its job. The only mobject that needs to | ||
| be processed at this point is the main mobject attached to the transformation. | ||
| The camera supplies its semantic view transform, while renderer-owned Cairo | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Perhaps just "supplies its view transform"? |
||
| helpers draw the current mobject state into the renderer's raster target. The | ||
| renderer reads that target and passes an owned array to the file writer. | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
I think it should be more clear exactly which array is being passed to the file writer; it tells the user nothing that it is "owned". |
||
| - At the end of the loop, 90 frames have been passed to the file writer. | ||
|
|
||
| Completing the render loop | ||
|
|
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -7,6 +7,7 @@ Thematic Guides | |
| :glob: | ||
|
|
||
| configuration | ||
| cameras | ||
| deep_dive | ||
| using_text | ||
| add_voiceovers | ||
Uh oh!
There was an error while loading. Please reload this page.