Repository navigation
Lower CPU usage by avoiding pixel format transformation #102
Description
Activity
Yah this plugin does a CPU transformation of the pixel format to UYVY (422) since that's what we advertise to the host software (see also apple docs).
It's been in the back of my mind to investigate this more -- it may be possible that one of the supported native pixel formats in OBS may also work as a webcam pixel format in the DAL plugin. :
Here is the list of constants I can use for that value in the code. Maybe
kCMVideoCodecType_AppleProRes4444would work with OBS's native I444 format.I'd love help trying this out. Originally it was just a trial and error process until I figured out that
VIDEO_FORMAT_UYVYin OBS worked askCMVideoCodecType_422YpCbCr8in the DAL plugin.I can definitely test this for you. Let me know how. Thank you.
If you can build this from source (see DEVELOPING.md) then I would remove the lines in plugin-main.mm that do the output conversion and try different codec constants in the DAL plugin to see if they work.
Working on this but I seem to be missing something.
The OBS plugin has a reference to VIDEO_FORMAT_UYVY
The DAL plugin has a reference to kCMVideoCodecType_422YpCbCr8But I cannot figure out the reference to that in OBS.
I see this VIDEO_FORMAT_UYVY and it is related to kCVPixelFormatType_422YpCbCr8Do I also need to change the line above in the DAL plugin as well?
I have tested a few different VIDEO_FORMATS_* and they are below.
VIDEO_FORMAT_I444:
VIDEO_FORMAT_YUY2:
VIDEO_FORMAT_NV12:
Yeah you probably would need to change the
kCVPixelFormatType_422YpCbCr8reference in the DAL plugin as well for this to work.If you push your changes to a branch on your fork and make a draft PR, happy to follow along with your code changes and chime in if I have ideas.
- changed the title
[-]Potential high CPU usage[/-][+]Potential high CPU usage due to pixel format transformation.[/+]on May 28, 2020 I tried this out (in this branch) with also changing the references to
422in the DAL plugin to444and it crashed 🤷Next maybe I'd try to match RGB formats maybe.
diff --git a/src/dal-plugin/CMSampleBufferUtils.mm b/src/dal-plugin/CMSampleBufferUtils.mm index b13abc4d..cccc4795 100644 --- a/src/dal-plugin/CMSampleBufferUtils.mm +++ b/src/dal-plugin/CMSampleBufferUtils.mm @@ -19,7 +19,7 @@ OSStatus CMSampleBufferCreateFromData(NSSize size, CMSampleTimingInfo timingInfo // Create an empty pixel buffer CVPixelBufferRef pixelBuffer; - err = CVPixelBufferCreate(kCFAllocatorDefault, size.width, size.height, kCVPixelFormatType_422YpCbCr8, nil, &pixelBuffer); + err = CVPixelBufferCreate(kCFAllocatorDefault, size.width, size.height, kCMVideoCodecType_AppleProRes4444, nil, &pixelBuffer); if (err != noErr) { DLog(@"CVPixelBufferCreate err %d", err); return err; @@ -118,7 +118,7 @@ OSStatus CMSampleBufferCreateFromDataNoCopy(NSSize size, CMSampleTimingInfo timi }; CMFormatDescriptionRef format; - err = CMVideoFormatDescriptionCreate(NULL, kCMVideoCodecType_422YpCbCr8, size.width, size.height, (__bridge CFDictionaryRef)extensions, &format); + err = CMVideoFormatDescriptionCreate(NULL, kCMVideoCodecType_AppleProRes4444, size.width, size.height, (__bridge CFDictionaryRef)extensions, &format); if (err != noErr) { DLog(@"CMVideoFormatDescriptionCreate err %d", err); return err; diff --git a/src/dal-plugin/Stream.mm b/src/dal-plugin/Stream.mm index 7f1c5504..2ce13cee 100644 --- a/src/dal-plugin/Stream.mm +++ b/src/dal-plugin/Stream.mm @@ -268,7 +268,7 @@ - (CMVideoFormatDescriptionRef)getFormatDescription { CMVideoFormatDescriptionRef formatDescription; - OSStatus err = CMVideoFormatDescriptionCreate(kCFAllocatorDefault, kCMVideoCodecType_422YpCbCr8, self.testCardSize.width, self.testCardSize.height, NULL, &formatDescription); + OSStatus err = CMVideoFormatDescriptionCreate(kCFAllocatorDefault, kCMVideoCodecType_AppleProRes4444, self.testCardSize.width, self.testCardSize.height, NULL, &formatDescription); if (err != noErr) { DLog(@"Error %d from CMVideoFormatDescriptionCreate", err); } diff --git a/src/obs-plugin/plugin-main.mm b/src/obs-plugin/plugin-main.mm index 7e211adc..0158f9e0 100644 --- a/src/obs-plugin/plugin-main.mm +++ b/src/obs-plugin/plugin-main.mm @@ -48,11 +48,11 @@ static bool virtualcam_output_start(void *data) obs_get_video_info(&videoInfo); - struct video_scale_info conversion = {}; - conversion.format = VIDEO_FORMAT_UYVY; - conversion.width = videoInfo.output_width; - conversion.height = videoInfo.output_height; - obs_output_set_video_conversion(output, &conversion); +// struct video_scale_info conversion = {}; +// conversion.format = VIDEO_FORMAT_UYVY; +// conversion.width = videoInfo.output_width; +// conversion.height = videoInfo.output_height; +// obs_output_set_video_conversion(output, &conversion); if (!obs_output_begin_data_capture(output, 0)) { return false; }
I'm also getting extremely high CPU usage with this, (in the neighbourhoods of 80%) I'll give these settings a go.
- changed the title
[-]Potential high CPU usage due to pixel format transformation.[/-][+]Lower CPU usage by avoiding pixel format transformation[/+]on Dec 14, 2020
With the virtual cam disabled, my CPU usage in OBS hovers around 2.9%-3.3%. After starting virtual cam it jumps and hovers between 5.5%-6.0%.
Does this plugin respect the encoder settings for either Streaming/Recording?
Other items changed from default.
OUTPUT:
Encoder Apple VT H264 Hardware Encoder
VIDEO:
Base and Output: 1920x1080
Downscale Filter: Bilinear