opencv+ffmpeg+QOpenGLWidget开发的音视频播放器demo

news/2024/7/10 20:38:12 标签: opencv, ffmpeg, 音视频

前言

    本篇文档的demo包含了 1.使用OpenCV对图像进行处理，对图像进行置灰，旋转，抠图，高斯模糊，中值滤波，部分区域清除置黑，背景移除，边缘检测等操作；2.单纯使用opencv播放显示视频；3.使用opencv和openGL播放显示视频；4.在ffmpeg解码后，使用opencv显示视频，并支持对视频的旋转翻转、裁剪、添加文字、添加logo、亮度调节、置灰、录像截图，音频开关等功能。视频播放器同时支持本地文件与网络码流地址的播放。本篇博客的最后有提供工程代码的下载。

一、OpenCV简单介绍：

    OpenCV是一个基于Apache2.0许可（开源）发行的跨平台计算机视觉和机器学习软件库，可以运行在Linux、Windows、Android和Mac OS操作系统上。它由一系列 C 函数和少量 C++ 类构成，同时提供了Python、Ruby、MATLAB等语言的接口，实现了图像处理和计算机视觉方面的很多通用算法。应用领域包含：人机互动、物体识别、图像分割、人脸识别、动作识别、运动跟踪、机器人、运动分析、机器视觉、结构分析、汽车安全驾驶等。

二、demo调用OpenCV：

    本篇demo使用的是OpenCV-4.5.1, 是直接下载windows版本，没有额外去编译OpenCV的库，安装exe后，提取头文件和相关的库文件，用于工程调用即可。OpenCV下载路径：https://sourceforge.net/projects/opencvlibrary/

    OpenCV常用函数介绍：
    1）Mat Image = imread(filename); //从指定文件加载图像并返回
    2）namedWindow(“IMG-WIN”, cv::WINDOW_NORMAL);//创建图像显示窗口
    3）destroyWindow(“IMG-WIN”);//销毁图像显示窗口
    4）imshow(“IMG-WIN”, Image);//将图片显示在窗口中
    5）imwrite将图像保存到指定文件
    6）其他

三、demo功能介绍：

3.1.图像操作

在这里插入图片描述
    图像操作包含了显示原图、置灰、旋转、抠图、高斯模糊、中值滤波，部分区域清除置黑，背景移除，边缘检测等操作。点击显示原图后，通过点击其他按钮，可以实现图像的不同效果，例如置灰：
在这里插入图片描述
    其他功能可下载demo进行操作，点击显示原图，可恢复到之前图片显示，点击保存图片按钮可将处理后的图片进行保存。

3.2.图像操作相关代码

//图像操作
/
void MainWindow::on_pushButton_showImage_clicked()
{
    // load image using opencv
    m_srcImage = imread("./Image/test1.jpg");

    std::string winName = "Display window";
    namedWindow(winName, WINDOW_AUTOSIZE);
    imshow(winName, m_srcImage);

    if(m_bShowImage == false)
    {
        HWND hwnd = static_cast<HWND>(cvGetWindowHandle(winName.c_str()));
        HWND parent = GetParent(hwnd);//得到nameWindow窗口的父句柄
        SetParent(hwnd, (HWND)ui->widget_image->winId());//设置ui控件的句柄是父句柄
        ShowWindow(parent, SW_HIDE);//隐藏掉nameWindow窗口

        resizeWindow(winName, cv::Size(ui->widget_image->width(), ui->widget_image->height()));
    }

    m_destImage = m_srcImage;
    m_bShowImage = true;
}

void MainWindow::on_pushButton_saveImage_clicked()
{
    if(m_bShowImage)
    {
        std::string saveFile = "./Image/test-new.jpg";
        if(imwrite(saveFile, m_destImage))
        {
            QMessageBox::information(this, "myOpencv", "Save Image Success.");
        }
        else
        {
            QMessageBox::critical(this, "myOpencv", "Save Image Failed.");
        }
    }
    else
        QMessageBox::critical(this, "myOpencv", "Please click on the Show Original Image button first.");
}

void MainWindow::on_pushButton_gray_clicked()
{
    if(m_bShowImage)
    {
        cvtColor(m_srcImage, m_destImage, COLOR_BGR2GRAY);

        std::string winName = "Display window";
        namedWindow(winName, WINDOW_AUTOSIZE);
        imshow(winName, m_destImage);
    }
    else
        QMessageBox::critical(this, "myOpencv", "Please click on the Show Original Image button first.");
}

void MainWindow::on_pushButton_rotate_clicked()
{
    if(m_bShowImage)
    {
        warpAffine(m_srcImage, m_destImage, getRotationMatrix2D({m_srcImage.cols/2.0f, m_srcImage.rows/2.0f},
                                                            45,0.5),m_srcImage.size(),cv::INTER_LINEAR, cv::BORDER_CONSTANT, {255,0,0});
        std::string winName = "Display window";
        namedWindow(winName, WINDOW_AUTOSIZE);
        imshow(winName, m_destImage);
    }
    else
        QMessageBox::critical(this, "myOpencv", "Please click on the Show Original Image button first.");
}

void MainWindow::on_pushButton_matting_clicked()
{
    if(m_bShowImage)
    {
        m_destImage = m_srcImage({20, 20, 60, 60}).clone();

        std::string winName = "Display window";
        namedWindow(winName, WINDOW_AUTOSIZE);
        imshow(winName, m_destImage);
    }
    else
        QMessageBox::critical(this, "myOpencv", "Please click on the Show Original Image button first.");
}

void MainWindow::on_pushButton_gauss_clicked()
{
    if(m_bShowImage)
    {
        GaussianBlur(m_srcImage, m_destImage, cv::Size(5,5), 0);

        std::string winName = "Display window";
        namedWindow(winName, WINDOW_AUTOSIZE);
        imshow(winName, m_destImage);
    }
    else
        QMessageBox::critical(this, "myOpencv", "Please click on the Show Original Image button first.");
}

void MainWindow::on_pushButton_median_clicked()
{
    if(m_bShowImage)
    {
        medianBlur(m_srcImage, m_destImage, 5);

        std::string winName = "Display window";
        namedWindow(winName, WINDOW_AUTOSIZE);
        imshow(winName, m_destImage);
    }
    else
        QMessageBox::critical(this, "myOpencv", "Please click on the Show Original Image button first.");
}

void MainWindow::on_pushButton_clear_clicked()
{
    if(m_bShowImage)
    {
        cv::Rect roi(40, 40, 80, 80);
        Mat roiMat = m_destImage(roi);
        roiMat.setTo(cv::Scalar(0,0,0));

        std::string winName = "Display window";
        namedWindow(winName, WINDOW_AUTOSIZE);
        imshow(winName, m_destImage);
    }
    else
        QMessageBox::critical(this, "myOpencv", "Please click on the Show Original Image button first.");
}

void MainWindow::on_pushButton_clearBackground_clicked()
{
    //原图
    Mat image = imread("./Image/test1.jpg");

    // 创建一个掩码（mask），用于指定哪些区域是前景，哪些区域是背景
    cv::Mat mask(image.size(), CV_8UC1, cv::Scalar(cv::GC_BGD));
    cv::Rect rectangle(50, 50, image.cols - 100, image.rows - 100);
    cv::grabCut(image, mask, rectangle, cv::Mat(), cv::Mat(), 5, cv::GC_INIT_WITH_RECT);

    // 将掩码中被标记为前景的像素设为白色，背景设为黑色
    cv::Mat foregroundMask = (mask == cv::GC_PR_FGD) | (mask == cv::GC_FGD);
    cv::Mat foregroundImage(image.size(), CV_8UC3, cv::Scalar(0, 0, 0));
    image.copyTo(foregroundImage, foregroundMask);

    // 显示结果图像
    cv::imshow("Foreground", foregroundImage);
    cv::waitKey(0);
}

void MainWindow::on_pushButton_edgeDetection_clicked()
{
    // 读取图片
    cv::Mat image = cv::imread("./Image/test1.jpg", cv::IMREAD_GRAYSCALE);

    // 边缘检测
    cv::Mat edges;
    cv::Canny(image, edges, 100, 200);

    // 显示结果
    cv::imshow("Edges", edges);
    cv::waitKey(0);
}

3.3.视频操作

在这里插入图片描述

    代码中通过宏定义可以旋转不同的视频处理方式（默认使用ffmpeg解码后，opencv播放显示视频），如下：

#define OPENCV_PLAY_VIDEO                   0   //单纯使用opencv播放显示视频
#define OPENCV_OPENGL_PLAY_VIDEO            0   //使用opencv和openGL播放显示视频
#define OPENCV_FFMPEG_PLAY_VIDEO            1   //使用ffmpeg解码后，opencv显示视频

    播放器支持对视频的旋转翻转、裁剪、添加文字、添加logo、亮度调节、置灰、录像截图，音频开关等功能，同时支持本地文件与网络码流地址的播放。

    简单演示如下：
在这里插入图片描述

    对原视频进行相关处理后，可通过点击录像截图按钮进行视频与图片的保存。通过开关音频按钮，支持对音频的播放处理。

    demo工程中有提供测试视频和测试logo，可以进行添加测试。
在这里插入图片描述

3.4.视频操作部分代码

void MainWindow::on_pushButton_Open_clicked()
{
    QString sFileName = QFileDialog::getOpenFileName(this, QString::fromLocal8Bit("Select Video File"));
    if(sFileName.isEmpty())
    {
        QMessageBox::critical(this, "myOpencv", QObject::tr("错误:文件不能为空."));
        return;
    }

    ui->lineEdit_url->setText(sFileName);
}

void MainWindow::on_pushButton_Play_clicked()
{
    QString sStreamAddr = ui->lineEdit_url->text();
    if(sStreamAddr.isEmpty())
    {
        QMessageBox::critical(this, "myOpencv", QObject::tr("错误:码流地址不能为空."));
        return;
    }

#if OPENCV_FFMPEG_PLAY_VIDEO
    if(nullptr == m_pMediaThread)
    {
        MY_DEBUG << "new MediaThread";
        m_pMediaThread = new MediaThread(ui->openGLWidget);
    }
    else
    {
        if(m_pMediaThread->isRunning())
        {
            QMessageBox::critical(this, "myOpencv", "错误:请先点击停止按钮关闭视频.");
            return;
        }
    }

    bool bMediaInit = m_pMediaThread->Init(sStreamAddr);
    if(bMediaInit)
    {
        m_pMediaThread->startThread();
    }

#elif OPENCV_PLAY_VIDEO
    if(m_pOpencvPlayThread1 == nullptr)
    {
        m_pOpencvPlayThread1 = new COpencvPlayThread1(ui->openGLWidget, sStreamAddr);
    }
    m_pOpencvPlayThread1->startThread();

#elif OPENCV_OPENGL_PLAY_VIDEO
    if(m_pOpencvPlayThread2 == nullptr)
    {
        m_pOpencvPlayThread2 = new COpencvPlayThread2(ui->openGLWidget, sStreamAddr);
    }
    m_pOpencvPlayThread2->startThread();

#endif

}

void MainWindow::on_pushButton_Stop_clicked()
{
#if OPENCV_PLAY_VIDEO
    if(m_pOpencvPlayThread1)
        m_pOpencvPlayThread1->stopThread();
#endif

#if OPENCV_OPENGL_PLAY_VIDEO
    if(m_pOpencvPlayThread2)
        m_pOpencvPlayThread2->stopThread();
#endif

#if OPENCV_FFMPEG_PLAY_VIDEO
    if(m_pMediaThread)
    {
        disconnect(m_pMediaThread, SIGNAL(sig_emitImage(const QImage&)),
                ui->openGLWidget, SLOT(slot_showImage(const QImage&)));

        m_pMediaThread->stopThread();
        m_pMediaThread->quit();
        m_pMediaThread->wait();
        m_pMediaThread->DeInit();
    }
#endif

}

void MainWindow::on_pushButton_startRecord_clicked()
{
    if(m_pMediaThread)
        m_pMediaThread->startRecord();
}

void MainWindow::on_pushButton_stopRecord_clicked()
{
    if(m_pMediaThread)
    {
        m_pMediaThread->stopRecord();
        QDesktopServices::openUrl(QUrl::fromLocalFile(RECORD_DEFAULT_PATH));
    }
}

void MainWindow::on_pushButton_snapshot_clicked()
{
    if(m_pMediaThread)
    {
        m_pMediaThread->Snapshot();
        QDesktopServices::openUrl(QUrl::fromLocalFile(SNAPSHOT_DEFAULT_PATH));
    }
}

void MainWindow::on_pushButton_video_rotate_clicked()
{
    if(m_pMediaThread)
    {
        int nRotate = ui->comboBox_rotate->currentIndex() - 1;
        if(nRotate < 0)
            m_pMediaThread->setRotate(false, nRotate);
        else
            m_pMediaThread->setRotate(true, nRotate);
    }
}

void MainWindow::on_pushButton_video_overturn_clicked()
{
    if(m_pMediaThread)
    {
        int nOverturn = ui->comboBox_overturn->currentIndex() - 1;
        if(nOverturn < 0)
            m_pMediaThread->setFlip(false, nOverturn);
        else
            m_pMediaThread->setFlip(true, nOverturn);
    }
}

bool bGrey = false;
void MainWindow::on_pushButton_video_grey_clicked()
{
    if(m_pMediaThread)
    {
        if(bGrey)
        {
            m_pMediaThread->setGray(false);
            bGrey = false;
        }
        else
        {
            m_pMediaThread->setGray(true);
            bGrey = true;
        }
    }
}

void MainWindow::on_pushButton_video_cropping_clicked()
{
    if(m_pMediaThread)
    {
        int x = ui->lineEdit_crop_X->text().toInt();
        int y = ui->lineEdit_crop_Y->text().toInt();
        int w = ui->lineEdit_crop_W->text().toInt();
        int h = ui->lineEdit_crop_H->text().toInt();

        m_pMediaThread->setCrop(true, x, y, w, h);
    }
}

void MainWindow::on_pushButton_video_addText_clicked()
{
    if(m_pMediaThread)
    {
        int x = ui->lineEdit_text_X->text().toInt();
        int y = ui->lineEdit_text_Y->text().toInt();
        int nColor = ui->comboBox_text_rgb->currentIndex();
        double size = ui->comboBox_text_fonsize->currentText().toDouble();
        QString sText = ui->lineEdit_videoText->text();

        MY_DEBUG << "size:" << size;
        m_pMediaThread->setAddText(true, sText, x, y, nColor, size);
    }
}

void MainWindow::on_pushButton_logoPath_clicked()
{
    QString sFileName = QFileDialog::getOpenFileName(this, QString::fromLocal8Bit("Select Logo File"));
    if(sFileName.isEmpty())
    {
        QMessageBox::critical(this, "myOpenCV", QObject::tr("错误:文件不能为空."));
        return;
    }

    ui->lineEdit_logo_path->setText(sFileName);
}

void MainWindow::on_pushButton_addLogo_clicked()
{
    if(m_pMediaThread)
    {
        int x = ui->lineEdit_logo_X->text().toInt();
        int y = ui->lineEdit_logo_Y->text().toInt();
        QString sAddLogoPath = ui->lineEdit_logo_path->text();

        m_pMediaThread->setAddLogo(true, x, y, sAddLogoPath);
    }
}

void MainWindow::on_pushButton_setBrightness_clicked()
{
    if(m_pMediaThread)
    {
        double brightness = ui->comboBox_brightnessVal->currentText().toDouble();
        m_pMediaThread->setImagePara(brightness);
    }
}

void MainWindow::on_pushButton_restore_clicked()
{
    if(m_pMediaThread)
    {
        int x = ui->lineEdit_crop_X->text().toInt();
        int y = ui->lineEdit_crop_Y->text().toInt();
        int w = ui->lineEdit_crop_W->text().toInt();
        int h = ui->lineEdit_crop_H->text().toInt();

        m_pMediaThread->setCrop(false, x, y, w, h);
    }
}

void MainWindow::on_pushButto_video_deleteText_clicked()
{
    if(m_pMediaThread)
    {
        int x = ui->lineEdit_text_X->text().toInt();
        int y = ui->lineEdit_text_Y->text().toInt();
        int nColor = ui->comboBox_text_rgb->currentIndex();
        double size = ui->comboBox_text_rgb->currentText().toDouble();
        QString sText = ui->lineEdit_videoText->text();

        m_pMediaThread->setAddText(false, sText, x, y, nColor, size);
    }
}

void MainWindow::on_pushButton_deleteLogo_clicked()
{
    if(m_pMediaThread)
    {
        int x = ui->lineEdit_logo_X->text().toInt();
        int y = ui->lineEdit_logo_Y->text().toInt();
        QString sAddLogoPath = ui->lineEdit_logo_path->text();

        m_pMediaThread->setAddLogo(false, x, y, sAddLogoPath);
    }
}

void MainWindow::on_pushButton_audioOpen_clicked()
{
    ctAudioPlayer::getInstance().isPlay(true);
}

void MainWindow::on_pushButton_audioClose_clicked()
{
    ctAudioPlayer::getInstance().isPlay(false);
}

void ctFFmpeg::dealWithOpenCV(AVFrame *pFrame/*, int time*/)
{
    if(!m_bQuit)
    {
        cv::Mat destCvMat(cv::Size(m_nVideoW, m_nVideoH), CV_8UC3, cv::Scalar(1, 2, 3));
        destCvMat.data = (uchar*)pFrame->data[0];

        //旋转
        if(m_bRotate)
        {
            MY_DEBUG << "Set Rotate m_nRotate:" << m_nRotate;
            rotate(destCvMat, destCvMat, m_nRotate);
        }

        //翻转
        if(m_bFlip)
        {
            MY_DEBUG << "Set Flip";
            flip(destCvMat, destCvMat, m_nFlip);
        }

        //裁剪
        if(m_bCrop)
        {
            destCvMat = destCvMat(Rect(m_nCropX, m_nCropY, m_nCropW, m_nCropH));
        }

        //灰度
        if(m_bGray)
        {
            MY_DEBUG << "Set Gray";
            cvtColor(destCvMat, destCvMat, COLOR_BGR2GRAY);
        }

        //添加文字
        if(m_bAddText)
        {
            MY_DEBUG << "m_nAddTextX:" << m_nAddTextX << ", m_nAddTextY：" << m_nAddTextY << ",m_nAddTextSize:" << m_nAddTextSize;
            cv::Point ptPos(m_nAddTextX, m_nAddTextY);
            int fontFace = cv::FONT_HERSHEY_COMPLEX;
            double fontScale = m_nAddTextSize;
            int thickness = 2;
            cv::Scalar color;

            switch(m_nAddTextColor)
            {
                case ADD_TEXT_COLOR_TYPE_BLUE:
                    color[0]=255; color[1]=0; color[2]=0;
                    cv::putText(destCvMat, m_sAddText.toStdString(), ptPos, fontFace, fontScale, color, thickness);
                break;
                case ADD_TEXT_COLOR_TYPE_GREEN:
                    color[0]=0; color[1]=255; color[2]=0;
                    cv::putText(destCvMat, m_sAddText.toStdString(), ptPos, fontFace, fontScale, color, thickness);
                break;
                case ADD_TEXT_COLOR_TYPE_RED:
                default:
                    color[0]=0; color[1]=0; color[2]=255;//红色
                    cv::putText(destCvMat, m_sAddText.toStdString(), ptPos, fontFace, fontScale, color, thickness);
                break;
            }
        }

        //添加logo
        if(m_bAddLogo)
        {
            cv::Mat logoImage = cv::imread(m_sAddLogoPath.toStdString());
            cv::Mat roi = destCvMat(cv::Rect(m_nAddLogoX, m_nAddLogoY, logoImage.cols, logoImage.rows));
            cv::addWeighted(roi, 0.5, logoImage, 0.5, 0, roi);
        }

        //亮度\暗度
        cv::addWeighted(destCvMat, m_nBrightness, cv::Scalar(0, 0, 0), 0, 0, destCvMat);

#if OPENCV_NAME_WINDOW_SHOW
        cv::imshow(m_sWinName.toStdString(), destCvMat);
#endif

        //效率太慢, 容易导致崩溃, 不适合转QImage再用openGL播放
        //...
        //emit sig_getImage(image);

        //截图
        if(m_bSnapshot)
        {
            cv::cvtColor(destCvMat, destCvMat, cv::COLOR_BGR2RGB);

            QImage image(destCvMat.data, destCvMat.cols, destCvMat.rows, static_cast<int>(destCvMat.step), QImage::Format_RGB888);

            QPixmap pixPicture = QPixmap::fromImage(image);

            QString sPath = "./snapshot/";
            QDate date = QDate::currentDate();
            QTime time = QTime::currentTime();
            m_sSnapPath = QString("%1%2-%3-%4-%5%6%7.jpg").arg(sPath).arg(date.year()). \
                    arg(date.month()).arg(date.day()).arg(time.hour()).arg(time.minute()). \
                    arg(time.second());
            MY_DEBUG << "Snapshot... m_sSnapPath:" << m_sSnapPath;
            pixPicture.save(m_sSnapPath, "jpg");
            m_bSnapshot = false;
        }
    }
}

四、工程源码下载：

demo播放器下载：https://download.csdn.net/download/linyibin_123/88217460

demo播放器工程源码下载：https://download.csdn.net/download/linyibin_123/88217472


http://www.niftyadmin.cn/n/4940103.html

相关文章

【日常积累】Linux下文件乱码问题

【日常积累】Linux下文件乱码问题

linux下删除乱码文件、目录 由于编码原因&#xff0c;在linux服务器上上传、创建中文文件或目录时&#xff0c;会产生乱码&#xff0c;如果想删除它&#xff0c;有时候发现用rm命令是删除不了的 这种情况下&#xff0c;用find命令可以删除乱码的文件或目录。 首先进入乱码文件…
阅读更多...
解决flutter showDialog下拉框,复选框等无法及时响应的问题

解决flutter showDialog下拉框,复选框等无法及时响应的问题

使用StatefulBuilder _showDialogr() {showDialog(context: context,builder: (BuildContext ctx) {return StatefulBuilder(builder: (BuildContext context, StateSetter setState) {return Scaffold(body: Column(children: <Widget>[Container(height: 400,padding: …
阅读更多...
RabbitMQ之TTL+死信队列实现延迟队列

RabbitMQ之TTL+死信队列实现延迟队列

RabbitMQ是一个流行的消息队列系统&#xff0c;它提供了许多有用的功能&#xff0c;其中之一是TTL&#xff08;Time To Live&#xff09;和死信队列。这些功能可以用来实现延迟队列&#xff0c;让我们来看看如何使用它们。 首先&#xff0c;什么是TTL&#xff1f;TTL是消息的存…
阅读更多...
接入网概述

接入网概述

接入网概述 接入网基本概念接入网“最后一公里”解决方案数字用户线xDSL技术的发展与特点xDSL的局限性PON网络架构小结 接入网基本概念 在家里终端设备连接ONT然后进入接入网&#xff0c;这个接入网大概在2-20km左右&#xff0c;中间是通过光纤进行连接&#xff0c;是无源的&a…
阅读更多...
微信小程序云开发快速入门（2/4）

微信小程序云开发快速入门（2/4）

前言 我们对《微信小程序云开发快速入门&#xff08;1/4&#xff09;》的知识进行回顾一下。在上章节我们知道了云开发的优势以及能力&#xff0c;并且我们还完成了码仔备忘录的本地版到网络版的改造&#xff0c;主要学习了云数据库同时还通过在小程序使用云API直接操作了云数…
阅读更多...
Salesforce特别元数据部署技巧

Salesforce特别元数据部署技巧

标准的picklist字段部署 <?xml version"1.0" encoding"UTF-8" standalone"yes"?> <Package xmlns"http://soap.sforce.com/2006/04/metadata"><types><members>Opportunity.StageName</members><…
阅读更多...
gitlabrunner集成maven\sonar\node\nexus实现自动打包、代码扫描、发布到本地仓库

gitlabrunner集成maven\sonar\node\nexus实现自动打包、代码扫描、发布到本地仓库

一 摘要 本文主要介绍gitlabrunner 与sonar 集成&#xff0c;实现代码扫描 二 环境信息 2.1 软件版本信息 软件版本安装方式操作系统centos7.9gitlab14.10dockergitlabrunner14.10dockerjdk1.8制作镜像里maven3.6.3制作镜像里sonar7.9.3dockersonar-scanner4.8.0制作镜像里…
阅读更多...
C、C++项目中 configure、makefile.am、makefile.in、makefile 之间的关系

C、C++项目中 configure、makefile.am、makefile.in、makefile 之间的关系

一、configure、makefile.am、makefile.in、makefile 之间的关系 这四个文件都是与 GNU Make&#xff08;一个用于管理程序的编译和安装过程的工具&#xff09;有关的文件&#xff0c;它们的关系如下&#xff1a; configure&#xff1a;是一个脚本文件&#xff0c;用于根据系统…
阅读更多...
最新文章

Copyright @ 2022~2023