summaryrefslogtreecommitdiff
path: root/src/plugins/multimedia/ffmpeg
Commit message (Collapse)AuthorAgeFilesLines
* Improve getting ffmpeg frame durationArtem Dyomin2023-05-162-5/+19
| | | | | | | | | | | | | FFmpeg doc says that AVFrame.duration represents the duration except cases if it's 0 (unknown). It's appeared in only 6.0 so we use a helper to get it. So let's apply the rule to our AVFrame wrapper. Pick-to: 6.5 Change-Id: I9596364561f88929796c0097381bfc26bf1d4565 Reviewed-by: Qt CI Bot <qt_ci_bot@qt-project.org> Reviewed-by: Lars Knoll <lars@knoll.priv.no>
* Rename file qwindowsiuptr_p.h to qcomptr_p.hJøger Hansegård2023-05-153-3/+3
| | | | | | | | | | | | For consistency, rename the file that now contains the QComPtr class. We also rename the include guard accordingly. This is a follow-up change after renaming the QWindowsIUPtr class to QComPtr to ensure consistent naming. Task-number: QTBUG-113460 Pick-to: 6.5 Change-Id: I5f234a20faabbe65840a32e93fa039ad48cf06ef Reviewed-by: Lars Knoll <lars@knoll.priv.no>
* Rename QWindowsIUPointer template class to QComPtrJøger Hansegård2023-05-153-47/+47
| | | | | | | | | | | | | | The name QWindowsIUPointer feels clunky and does not add clarity. On the Windows platform, the term ComPtr, CComPtr or com_ptr is well established and understood. By renaming the QWindowsIUPointer to QComPtr we get code that is easier to read, particularly when the pointer is used in template classes such as std::vector<QComPtr> or QMaybe<QComPtr>. Task-number: QTBUG-113460 Pick-to: 6.5 Change-Id: Ia3ef06c4a1c0e45503ba5204a4d2fd6197114b15 Reviewed-by: Lars Knoll <lars@knoll.priv.no>
* Remove not actual resampler's settingArtem Dyomin2023-05-111-2/+0
| | | | | | | | | | | The "async" option was relevant for the old implementation, but now it just makes problems, namely, by making internal buffering too high, and the actual output size could be very different with the expected one, what can cause a weird behavior. Pick-to: 6.5 Change-Id: I4637d02e7f5e321fb3f674a386ef43667ed9cda9 Reviewed-by: Lars Knoll <lars@knoll.priv.no>
* Implement QFFmpegScreenCaptureDxgi in terms of common FFMPeg base classJøger Hansegård2023-05-113-125/+67
| | | | | | | | | | | | | | | | | | | | | | | | The QtMultimedia FFMpeg screen capture plugins all share a common base class 'QFFMpegScreenCaptureBase' that implements the bulk of the screen capture functionality, including the grabber thread and its screen capture loop. The Windows screen capture class 'QFFMpegScreenCaptureDXGI' class is, on the other hand, having its own implementation which makes it look and behave differently from the other implementations. This change refactors the Windows DXGI screen capture class to share base class with the other platform implementations. This reduces code duplication, and makes the class easier to maintain because it follows the same pattern as the other screen capture implementations. No functionality is changed through this refactoring. Task-number: QTBUG-113460 Pick-to: 6.5 Change-Id: Idd157c96e1705a9fe6eb591920177f6afb88cd2a Reviewed-by: Lars Knoll <lars@knoll.priv.no> Reviewed-by: Artem Dyomin <artem.dyomin@qt.io>
* Remove screen capture dependency on DirectX debug layerJøger Hansegård2023-05-092-2/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | When creating a Direct3D 11 device, it is possible to enable additional debug logging from the Direct3D subsystem by passing the D3D11_CREATE_DEVICE_DEBUG flag to the D3D11CreateDevice function. This is useful for debugging, but requires the D3D11SDKLayers.dll debug layer to be installed on the target machine. If the debug layer is not installed, calls to D3D11CreateDevice fails with the DXGI_ERROR_SDK_COMPONENT_MISSING status code. The debug layer is not installed by default on Windows 10 or Windows 11, and we can therefore not expect that this layer is available on any Windows computer. We also see that some QtMultimedia tests fail on CI nodes because the debug layer is not present. This change fixes the problem by removing the D3D11_CREATE_DEVICE_DEBUG flag from calls to D3D11CreateDevice. This gives less diagnostics for developers, but full debug logging can still be enabled through the DirectX control panel, which can be used to forcefully enable the debug layer even if the flag is not present when creating the device. An alternative solution would be to first try to create the device with the debug layer enabled by passing the D3D11_CREATE_DEVICE_DEBUG flag, and then retry without this flag if creating the device failed. The drawback of this approach is that the performance of the screen capture feature may depend on if the debug layer is installed or not. Pick-to: 6.5 Change-Id: Icad2965b61de953e3f93c8aeb902f37e5b31c10a Reviewed-by: Lars Knoll <lars@knoll.priv.no>
* Reduce choppy sound on audio playback if audio sink is almost emptyArtem Dyomin2023-05-056-7/+84
| | | | | | | | | | | | | | If audio sink buffer is almost empty, that happens on pause/play, the sound pretty often has little gaps. The solution is using some little samples compensation (slows down the playback on 1% for 1-2sec what allows to increase buffer loadig from 0-10% to 20%). In other words, the patch implements slight "soft" audio samples compensation in order to avoid having empty adio sink buffer. Pick-to: 6.5 Change-Id: I6b963996eab8f8b8f610fcd4a566405aec3d13cf Reviewed-by: Lars Knoll <lars@knoll.priv.no>
* Exclude files including EGL/egl.h from Unity BuildAmir Masoud Abdol2023-04-251-0/+3
| | | | | | | | | | It seems that when <EGL/egl.h> is included, we get conflicting symbols between QUrl, and X11 symbols. Pick-to: 6.5 Task-number: QTBUG-109394 Change-Id: Ided106a6b07f4a2d8c85419f86a5220f4cb77a23 Reviewed-by: Joerg Bornemann <joerg.bornemann@qt.io>
* Fix playing of video files with Chinese namesPavel Dubsky2023-04-201-1/+1
| | | | | | | | | | | | | | | The old implementation was using QUrl::toEncoded in order to retrieve a filename from a URL, but it uses toLatin1 internally which substitutes all non-ASCII characters with question marks that leads to ill formed path. The new implementation uses QUrl::toString with a subsequent call to QString::toUtf8 and it basically leads to the same results while preserving non-ASCII characters. Task-number: QTBUG-111951 Pick-to: 6.5 Change-Id: I0ad32143bc4afbdaec34511d7910ff106cb3cf45 Reviewed-by: Artem Dyomin <artem.dyomin@qt.io> Reviewed-by: Lars Knoll <lars@knoll.priv.no>
* Fix a few memory leaksArtem Dyomin2023-04-134-10/+24
| | | | | | | | | | - avcodec_open2 doesn't take ownership of options - fix not deleted surface in tests Pick-to: 6.5 Change-Id: Ia410792b27af4ef5ccaf1c6774344d3b6571952a Reviewed-by: Qt CI Bot <qt_ci_bot@qt-project.org> Reviewed-by: Lars Knoll <lars@knoll.priv.no>
* Temporary remove not stable fixArtem Dyomin2023-04-131-2/+0
| | | | | | | | | | | The fix was recently merged and now it's the reason for CI fails. Actually, the proper fix should be done in another place, in VideoRenderer, but it needs additional investigation. Pick-to: 6.5 Task-number: QTBUG-112173 Change-Id: I0d5a93666db2dbac7576ef5af994e31141192101 Reviewed-by: Tor Arne Vestbø <tor.arne.vestbo@qt.io>
* Implement seamless ffmpeg playback loopingArtem Dyomin2023-04-1214-107/+339
| | | | | | | | | | | | | | | | | | | | | | | Users need seamless looping, in other words, looping without little delays on jumping from the media end to the start. The only way to make it seamles and smooth is to intrude into the playback engine. As result, we have just a regular delay between frames on shifting. Also, a bunch of adjuscent small improvements have been introduced: - simplify criteria of the demuxer's buffer size, it was more complex and a bit not relevant before. Actually, we need to check only max buffering time. - Improve handling of media with streams of different length. - Fix setting of the playback rate before opening sources. - Add new auto tests for looping. - Add debug logs. Pick-to: 6.5 Task-number: QTBUG-112305 Change-Id: Ic9073d77535f5aae19f9ea48d4e745c9f2fa9ea3 Reviewed-by: Lars Knoll <lars@knoll.priv.no>
* Use lower-case header/library names for with mingw-w64Marius Kittler2023-04-121-2/+2
| | | | | | | | | | When compiling under GNU/Linux with mingw-w64 this is required as the filesystem is usually case-sensitive and mingw-w64 headers and libraries all have lower-case names. This change shouldn't hurt when compiling under Windows. Change-Id: I46c64a8f2923cbe21ce3e6b922ce8b099681e528 Reviewed-by: Oliver Wolff <oliver.wolff@qt.io>
* ffmpeg: Clear the last frame in videoSink after stopping the playerBartlomiej Moskal2023-04-061-0/+2
| | | | | | | | | | | | | | | The frame buffer was not properly cleared after stopping video. The last frame was still kept by QPlatformVideoSink (even if video was stopped). That caused the issue when trying to configure the MediaCodec for the next video file. This commit contains clearing the last frame in videoSink, after stopping video on the qffmpegmediaplayer. Task-number: QTBUG-112173 Pick-to: 6.5 Change-Id: I0d5a93666db2dbac7577ef5af994e34141192105 Reviewed-by: Lars Knoll <lars@knoll.priv.no>
* Android-ffmpeg: Do not activate the camera when setting camera deviceBartlomiej Moskal2023-03-271-2/+5
| | | | | | | | | | This commit removes default camera activation when setting camera device. Instead, the camera is only activated if it was previously active. Task-number: QTBUG-112175 Change-Id: I7df5777d5a8c8f6c80cdb7e4b66d6fb7f4cf8b9a Reviewed-by: Lars Knoll <lars@knoll.priv.no>
* Implement streaming audio input to output through ffmpeg capture sessionArtem Dyomin2023-03-275-51/+176
| | | | | | | | | | | | | | The use case: a user sets QAudioInput and QAudioOutput to QMediaCaptureSession, the audio stream from the input is directed to the output. Probably, the implementation will need some improvements, but, in general, it has been done in the commit. Pick-to: 6.5 Task-number: QTBUG-111213 Change-Id: I70502fb38a297a56f030e7722f0a8d58c870903f Reviewed-by: Lars Knoll <lars@knoll.priv.no>
* Prevent the race condition on encoders finalizationArtem Dyomin2023-03-242-8/+9
| | | | | | | | | | | | | It's not safe to call Finalizer::deleteLater at the end of the QThread::run since the deleting may occur earlier than the actual thread finishing. The doc of QThread recommends connection of deleteLater with QThread::finished for such cases. Pick-to: 6.5 Change-Id: I05aef603001f80b44e96c568d34aa77b693a2224 Reviewed-by: Lars Knoll <lars@knoll.priv.no>
* Implement the base video source class for camera and screen capturingArtem Dyomin2023-03-2213-55/+39
| | | | | | | | | | | | QPlatformVideoSource becomes the base of QPlatformCamera and QPlatformScreenCapture The advantage is using the single video source interface in the encoder instead of defferent ones. Pick-to: 6.5 Change-Id: Ic8bab3258dbe3efe851330d9a0e0461266dc64c9 Reviewed-by: Lars Knoll <lars@knoll.priv.no> Reviewed-by: Qt CI Bot <qt_ci_bot@qt-project.org>
* Add the base class for screen capture threadsArtem Dyomin2023-03-229-253/+314
| | | | | | | | | | | | Advantages: - get rid of the code duplication - generalize errors handling - added an opportunity for simple profiling of screen capturing - formats handling generalization will be added in next commits Pick-to: 6.5 Change-Id: I79ef73b01f39372c8ec7946bc737a476e6239193 Reviewed-by: Lars Knoll <lars@knoll.priv.no>
* Filter codecs if hw acceleration is not availableArtem Dyomin2023-03-221-5/+42
| | | | | | | | | | | | | | | The main reason of the patch is the trickiness of hw acceleration customization on Linux. In some cases hw accelerated ffmpeg encoders doesn't work even though they are declared as available, and the user is able to turn off the encoding vaapi hw acceleration. In this case, on the initialization of QFFmpegMediaFormatInfo, the function findAVEncoder shouldn't consider codecs with disabled hw accel as available. Pick-to: 6.5 Change-Id: Iafe2a9b1bb5e635882a974f752fe0311539223b2 Reviewed-by: Lars Knoll <lars@knoll.priv.no>
* Improve ffmpeg camera formats selection on darwin OSArtem Dyomin2023-03-206-71/+236
| | | | | | | | | | | | | | | | | | Handled a bunch of corner cases, briefly they are: - handle the case when ffmpeg videotoolbox doesn't support format. From now, it's optionally available to use sw frame or choose another format. Both approaches are working, but for now, the first one is prefferable. - handle the rare case when device output formats don't contain camera format. - improve the best camera format selection, it'll take into account an opoortunity to use hw acceleration for the format. - improve logging. Task-number: QTBUG-109009 Pick-to: 6.5 Change-Id: I82773f73a47bc1d22730f1e5ec2eaacbfa4ec1b1 Reviewed-by: Lars Knoll <lars@knoll.priv.no>
* Handle videotoolbox -> opengl texture conversion failArtem Dyomin2023-03-201-6/+13
| | | | | | | | | | If the conversion failed, we should return nullptr and fallback to memory conversion since the texture set is invalid in this case. Pick-to: 6.5 Change-Id: Iea3fdc032841e3d4ea7996c28f3faba2fa3800ad Reviewed-by: Lars Knoll <lars@knoll.priv.no>
* Remove extra buffer allocation in mapped encoder framesArtem Dyomin2023-03-201-1/+0
| | | | | | | | | | In encoder we perform mapping that doesn't need any additional memory. The memory holder is the original frame which is stored in opaque_ref. Pick-to: 6.5 Change-Id: I76737106eca33e75d758de23e764ff359d90ac3f Reviewed-by: Lars Knoll <lars@knoll.priv.no>
* Fix encoding on specific macOS hardwareArtem Dyomin2023-03-201-1/+8
| | | | | | | | | | | | | On some old harwares, videotoolbox hw encoding fails due to some internal reasons, ffmpeg recommends setting option allow_sw=1 to handle the case. Under the hood, ffmpeg sets videotoolbox hw acceleration enabled instead of required, that means that encoders *_videotoolbox might proceed with encoding even if hardware failed. Pick-to: 6.5 Change-Id: Ic4f3124e33fe38e1075b5a167338f487623a47c2 Reviewed-by: Lars Knoll <lars@knoll.priv.no>
* Add specific env var for testing without hw textures conversionArtem Dyomin2023-03-151-0/+9
| | | | | | | | | | | | The reason of this is that textures conversion may be is not stable on the user's side, and we need an opportunity to check if this problem affects the user's issue. For now, we already have such a problem to be checked. Pick-to: 6.5 Task-number: QTBUG-111459 Change-Id: I7f432fb7fcd0c6e1dc8b9b1b8e6d100d966a4076 Reviewed-by: Lars Knoll <lars@knoll.priv.no>
* Temporary disable QFFmpegScreenCaptureUwpArtem Dyomin2023-03-151-2/+3
| | | | | | | | | | | | | | | | The windows implementation QFFmpegScreenCaptureUwp is not stable, on Windows 11 it works, on Windows 10 it throws asserts, crashes, or produces side effects. The other implementation, QFFmpegScreenCaptureDxgi, works better. The suggestion is to turn off QFFmpegScreenCaptureUwp in the release so that QFFmpegScreenCaptureDxgi can be used instead, and we provide working functionality. Task-number: QTBUG-108892 Pick-to: 6.5 6.5.0 Change-Id: I027194572ae141ec154c566f88371a1aa8adc7de Reviewed-by: Lars Knoll <lars@knoll.priv.no>
* Add more logs to encoder initializationPavel Dubsky2023-03-141-2/+22
| | | | | | | | | | | Additional logs are added to the initialization stage when camera or screen capture are added to the encoder. Also a log was added to a thread init function of a muxer. Pick-to: 6.5 Change-Id: Ic656029a5a45886d04363201a39a07d77f5ec6ed Reviewed-by: Artem Dyomin <artem.dyomin@qt.io> Reviewed-by: Lars Knoll <lars@knoll.priv.no>
* Fix compilation with old ffmpeg versionsArtem Dyomin2023-03-145-32/+47
| | | | | | | | | Old ffmpeg versions are still used on some Yocto versions. Pick-to: 6.5 Fixes: QTBUG-111900 Change-Id: Ifccdbbc19c03e5fe127cbeaed21cc671de96cca0 Reviewed-by: Samuli Piippo <samuli.piippo@qt.io>
* Refactor VideoFrameEncoder construction and imrove codecs findingArtem Dyomin2023-03-1111-225/+526
| | | | | | | | | | | | | | | | | | | | | This is the next stage of refactoring encoding/decoding. Current patch includes: - decompose huge constructor of VideoFrameEncoder onto utility funcs - implement more utilities for working with av formats. - apply scores approach for flexible codecs and formats selection. The patch allows to consider sw format scores when choosing a codec. - fix some gaps from the previous refactoring, namely a. consider that hw encoder may have no hwconfig due to ffmpeg issues b. corner cases in selection of target formsts c. fix corner case in get_format for decoder - add additional corner cases checking to avoid crashes in case codecs initialization fails. - implement optional dump of advanced ffmpeg codecs info, that will be useful for debug, including on user side. Pick-to: 6.5 Change-Id: I1647ff73b4acbaac25112a64b9592f352e0f2078 Reviewed-by: Lars Knoll <lars@knoll.priv.no>
* Android: Postpone FFmpeg camera activationBartlomiej Moskal2023-03-102-3/+8
| | | | | | | | | | | | | Activation of the Android FFmpeg camera should be postponed until the first frame is read. Handling the first frame allows the implementation to get information about the pixel format of the frame and prepare the encoder. Until this happnes, we are unable to use the camera effectively (e.g. for recording). Task-number: QTBUG-99098 Pick-to: 6.5 Change-Id: Id0574b25318cb8cb09c70cca12e21f8970e31dac Reviewed-by: Lars Knoll <lars@knoll.priv.no>
* Fix encoding time_base factorArtem Dyomin2023-03-104-12/+44
| | | | | | | | | | | | | A few fixes regarding the topic: - time_base should be set to codec - consider the fact that time_base of streams can be changed in ffmpeg - set time_base to frames (some codecs want it to be set explicitly) The patch fixes the crash on recording WMV video. Pick-to: 6.5 Change-Id: Ibb40a0fef96eadafdf8723d9f11e9eabacc1325f Reviewed-by: Lars Knoll <lars@knoll.priv.no>
* Android: Integrate HW encoding for FFMpeg backend in AndroidSamuel Mira2023-03-0910-137/+500
| | | | | | | | | | | | | | | Video encoding was done using software that gave below acceptable results. This patch adds the necessary to use the Mediacodec encoders recently available in ffmpeg. For that to happen it was necessary to change the image reader of the QtCamera2 to use YUV420p frame instead of JPEG. That also improves performance since decoding is no longer necessary. This pipeline only has one copy done on FFMpeg side to upload to Mediacodec input buffer. Fixes: QTBUG-102235 Pick-to: 6.5 6.5.0 Change-Id: Ief5421c700dd0a77763a7713f9b452dd01828241 Reviewed-by: Lars Knoll <lars@knoll.priv.no>
* Add Security framework on macOS for linking with ffmpeg pluginArtem Dyomin2023-03-071-5/+7
| | | | | | | | | | | | | We should include network to the ffmpeg build. If building ffmpeg with network, Security framework is needed for linking with QtMM. The related commit in qt5 with enabling network is coming soon. Task-number: QTBUG-110708 Pick-to: 6.5 6.5.0 Change-Id: I7294896e7635f67e5208848fe37da173df5587b7 Reviewed-by: Doris Verria <doris.verria@qt.io>
* Improve ffmpeg hw accel and codecs findingArtem Dyomin2023-03-0611-242/+485
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | A bunch of the improvements mostly related to linux. The most problem with linux hw acceleration is that there're no exact working acceleration for all hardwares. On linux we had only hw acceleration 'vaapi' that requires installation of vaapi drivers, also on some hardwares it's not supported or not stable. Pretty often ffmpeg returns vaapi as supported hw device but it failes on creation vaapi hw context. The same might happen with other accelerations as well. The commit makes selection and creation of hw devices with codec for them more reliable and flexible, also, it extends the list of possible hw accel devices for linux, and makes customization of codecs choosing easier in the future. The functionality has been checked with cuda and vdpau hw accelerations on PC with nvidia rtx a2000. Both decoding and encoding work fine, handling of vdpau textures is to be done in the future. The hope and prospective is that will work on embedded platforms with nvidia gpu, it is to be tested afterwards. Since hw accel types are very platform specific thing, I propose runtime env vars QT_FFMPEG_ENCODING_HW_DEVICE_TYPES and QT_FFMPEG_DECODING_HW_DEVICE_TYPES to allow advanced users configuring priorities of used acceleration for the exact target platform. The changes have been tested on linux, macos, windows. Should be done afterwards: - conversion vdpau textures to opengl, QTBUG-111593 - add vaapi, vdpau, cuda to linux CI builds. I've checked that their adding don't affect ffmpeg lib size to much, less than 1%. - describe hw acceleration details and some setup guide in the qt documentation. - handle codec open fail during codecs search. Pick-to: 6.5 6.5.0 Change-Id: I8f120f53573f928a0673ed4ad94607a0ac57822d Reviewed-by: Lars Knoll <lars@knoll.priv.no>
* Implement looping for ffmpeg mediaplayer backendArtem Dyomin2023-03-057-74/+98
| | | | | | | | | | | - Looping of ffmpeg mediaplayer has been added with auto tests - Minor code cleanup (that left from previous implmntations) Pick-to: 6.5 Task-number: QTBUG-111209 Change-Id: I3839ee866bc2dc571919e41a18b093fb15165293 Reviewed-by: Qt CI Bot <qt_ci_bot@qt-project.org> Reviewed-by: Lars Knoll <lars@knoll.priv.no>
* Fix audiosink issues on darwinArtem Dyomin2023-03-031-1/+2
| | | | | | | | | | | | | | | | | Fixed problems: - Fix sound stucks on multiple resets/starts of QAudioSink. It was possible to reproduce on playback position change in mediaplayer. - Improve audiosink stop (reduce waiting time). The optimization is based on the fact that it's possible to call AudioOutputUnitStop from the thread where it was started. - add some auto test + imrove errors logging in tests. Tests work fine locally but still need some tune on CI Pick-to: 6.5 6.4 6.5.0 Task-number: QTBUG-111567 Change-Id: I0eb5c32af4c12dfc0694ee8f5967b4960a0b4ab2 Reviewed-by: Doris Verria <doris.verria@qt.io>
* Android: Prevent possible crashSamuel Mira2023-03-011-2/+4
| | | | | | | | | | Should not happen but it is possible to get an AVFrame without data to release on Android. If already released the frame->data[3] will be NULL. Pick-to: 6.5 Change-Id: I7d8af55f9c3ed9a03cd1dd4b3a38c332b8f06dc0 Reviewed-by: Lars Knoll <lars@knoll.priv.no>
* Fix access to multi front/back cameras on Android devices for ffmpegSamuel Mira2023-03-011-2/+2
| | | | | | | | | | | Fix 633597b00423adf8359763bc14d185e3ca91efbe applied to ffmpeg backend on android. Task-number: QTBUG-59726 Pick-to: 6.5 Change-Id: Id6db8154861d5ad7fcc1501db2a33cdd59d13a1c Reviewed-by: Lars Knoll <lars@knoll.priv.no> Reviewed-by: Artem Dyomin <artem.dyomin@qt.io>
* Fix C++20 builds of QtMultimediaAllan Sandfeld Jensen2023-02-241-3/+3
| | | | | | | Avoid deprecated implicit capture Change-Id: I650cf695a0eb528b1c14c33f8041608c58eb2f8e Reviewed-by: Lars Knoll <lars@knoll.priv.no>
* Add missing QT_BEGIN_NAMESPACEAllan Sandfeld Jensen2023-02-201-0/+2
| | | | | | | | | Fixes namespaced builds with enabling VA-API Pick-to: 6.5 Fixes: QTBUG-111269 Change-Id: I032b7c25c8395b8ab54d42bb77a55af1766a2992 Reviewed-by: Lars Knoll <lars@knoll.priv.no>
* Minor cleanup of duplicated codeArtem Dyomin2023-02-173-37/+21
| | | | | | | | | Remove duplicated deleters; use a template instead Pick-to: 6.5 6.4 Change-Id: I79d4ae2866199583fec0d3dd357274a81f40b7fa Reviewed-by: Qt CI Bot <qt_ci_bot@qt-project.org> Reviewed-by: Lars Knoll <lars@knoll.priv.no>
* Fix ffmpeg encoding issuesArtem Dyomin2023-02-178-21/+70
| | | | | | | | | | | | | | Fixed a bunch of problems: - wrong frame intervals in macos camera and screen capturing - missing frame rate in screen capturing format - wrong scaling in encoder Also, some recording tests have been added Task-number: QTBUG-103226 Pick-to: 6.5 Change-Id: Id775f31e01d75d5c9f3c4ec20c33074acab1ab20 Reviewed-by: Lars Knoll <lars@knoll.priv.no>
* Add some exclusions for CMake Unity (Jumbo) buildsFriedemann Kleint2023-02-151-0/+3
| | | | | | | | Pick-to: 6.5 Task-number: QTBUG-109394 Change-Id: I60a6ae76db8e9d2a217075af26cc4de7de908f57 Reviewed-by: Doris Verria <doris.verria@qt.io> Reviewed-by: Joerg Bornemann <joerg.bornemann@qt.io>
* Implement custom X11 screen capturingArtem Dyomin2023-02-124-0/+455
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The implementation works on linux with run X11. It's disabled with wayland. Achivements - Fixed problem with performance. The standard implementation affected CPU too much. On my local release build the speedup in capturing of a single frame is 4-5 times. The explanation of such a huge difference is that opening of capturing contexts on X11 is an expensive action, whereas implemented single context allocation and transferring images through shared memory (Xshm) is pretty fast. Also, QScreen::grabWindow permorms additional pixels processings under the hood what is not needed for videoframes. - Improved errors handling - The patch allows to run screen capture backend UTs on linux CI with X11, namely, ubuntu, opensuse. Test results: https://testresults.qt.io/coin/integration/qt/qtmultimedia/tasks/web_qt_qtmultimedia_1675977558004 - Probably, fixed problem with windows overlaping, the matching tests became pass, it should be checked on different platforms. Also, the approach cannot capture windows out of the screen borders, we throw an error signal. Both windows issues are to be investigated deeper in 6.6. With the next commits I might move some common threading code for SC to a separate class. Task-number: QTBUG-103226 Pick-to: 6.5 Change-Id: Ibd3600fdbc9714f14dc80c2a330aea5e0d064791 Reviewed-by: Lars Knoll <lars@knoll.priv.no>
* Use new permissions API on macOS/iOSIvan Solovev2023-02-082-49/+12
| | | | | | | | | | | [ChangeLog][Important Behavior Changes][macOS][iOS] The platform implementation no longer requests Camera and Microphone permissions, only checks if they are granted or not. The client application must request the permissions using the QPermission API. Task-number: QTBUG-109965 Change-Id: Ibb9f78b8ec363a9c625d81d984b15f5c30548c7e Reviewed-by: Timur Pocheptsov <timur.pocheptsov@qt.io>
* Use new permissions API on AndroidIvan Solovev2023-02-081-11/+8
| | | | | | | | | | | | | | | [ChangeLog][Important Behavior Changes][Android] The platform implementation no longer requests Camera and Microphone permissions, only checks if they are granted or not. The client application must request the permissions using the QPermission API. Note: The WRITE_EXTERNAL_STORAGE permission is not handled by this patch, because it's not supported by the new QPermission API for now. This permission is still requested from the library code. Task-number: QTBUG-109965 Change-Id: Ia51d324db3893940fe1ef18ced197cfd7b724a99 Reviewed-by: Tor Arne Vestbø <tor.arne.vestbo@qt.io>
* Rewrite screen capturing on macosArtem Dyomin2023-02-077-131/+636
| | | | | | | | | | | | | | | | | | | | | | | | The previous generic implementation uses QScreen::grabWindow that is not stable for both cases, screens and windows capturing. In order to fix it, I've reimplemented it via native macos functionality. Some part of the functionality is shared with avfcamera. As a result, we got the following: - fix not capturing of some windows on the screen - enable hw acceleration for screens capturing (shared with camera functionality) - enable foreign app windows capturing (by CGWindowID as WId). We might need some additional api for getting the list of ids. Probably, the code is reusable for ios, to be investigated. Screen capturing tests work locally if permissions are granted; on CI some problems with giving the permissions. Task-number: QTBUG-103226 Pick-to: 6.5 Change-Id: Iada34467e4b9074eb43a1fbbdc4409e663ea668a Reviewed-by: Lars Knoll <lars@knoll.priv.no>
* Proposition to be able to turn on ffmpeg debug logsArtem Dyomin2023-02-061-0/+3
| | | | | | | | | | | It might be useful for developers and for asking users to provide more logs. If it's OK, we might document it. Pick-to: 6.5 Change-Id: Ibe0937d1837d3cfd81f2ddb9998cf58168570844 Reviewed-by: Lars Knoll <lars@knoll.priv.no>
* Fix encoder UB if no audio inputsArtem Dyomin2023-02-031-2/+6
| | | | | | | | | | | The case is reproduced if QAudioInput is set to QMediaCaptureSession but the platform doesn't have audio inputs. The case is to be unit tested after implementation of comprehensive mock of MediaDevices Pick-to: 6.5 6.4 Change-Id: I372c8c1260cb3fa0d70ea48a31c315bd9580cb06 Reviewed-by: Lars Knoll <lars@knoll.priv.no>
* Android: don't change Camera State for each frameBartlomiej Moskal2023-02-032-23/+16
| | | | | | | | | | | | | | | | | | | | onCaptureStarted, onCaptureCompleted and onCaptureFailed methods are called for each frame. Camera state (camera activity relies on it) should not be updated with such frequency. Remove onCaptureSessionCompleted and onCaptureSessionStarted handling, as the only purpose of them were to set a camera state. Remove camera status change for onCaptureFailed, as other requests are unaffected with this failure Instead of CaptureCallback start to use StateCallback for Camera state updates (onActive and onClosed methods) Fixes: QTBUG-110317 Change-Id: I9af1399033e24dbbf395f55ae23e1696aa4d42ce Reviewed-by: Samuel Mira <samuel.mira@qt.io> Reviewed-by: Assam Boudjelthia <assam.boudjelthia@qt.io>