[media] s5p-fimc: Prevent AB-BA deadlock during links reconfiguration

This patch patch eliminates potential AB-BA deadlock when one process calls
open(), or VIDIOC_S/TRY_FMT ioctl  on the FIMC capture video node, while
other thread is reconfiguring media links via media device node:
/dev/video? open()		  /dev/media? MEDIA_IOC_SETUP_LINK ioctl
mutex_lock(video_lock)            mutex_lock(graph_lock)
    fimc_pipeline_open()               fimc_md_link_notify()
        mutex_lock(graph_lock)	          mutex_lock(video_lock)
          ...                               ...
The deadlock is avoided by always taking the graph mutex first in video
node open() or an ioctl, before the video lock is acquired. Reversed
order seems impossible, since media device driver's link_notify callback
is called with media graph mutex already held.
To ensure proper locking order VIDIOC_S_FMT and VIDIOC_TRY_FMT ioctls are
not serialized in the v4l2-core and the driver takes care of it itself.

Signed-off-by: Sylwester Nawrocki <s.nawrocki@samsung.com>
Signed-off-by: Kyungmin Park <kyungmin.park@samsung.com>
Signed-off-by: Mauro Carvalho Chehab <mchehab@redhat.com>

1 files changed, 4 insertions, 2 deletions

diff --git a/drivers/media/platform/s5p-fimc/fimc-lite.c b/drivers/media/platform/s5p-fimc/fimc-lite.c
index 765b8e4..ef31c39 100644
--- a/drivers/media/platform/s5p-fimc/fimc-lite.c
+++ b/drivers/media/platform/s5p-fimc/fimc-lite.c
@@ -459,11 +459,12 @@ static void fimc_lite_clear_event_counters(struct fimc_lite *fimc)
 static int fimc_lite_open(struct file *file)
 {
 	struct fimc_lite *fimc = video_drvdata(file);
+	struct media_entity *me = &fimc->vfd.entity;
 	int ret;
 
-	if (mutex_lock_interruptible(&fimc->lock))
-		return -ERESTARTSYS;
+	mutex_lock(&me->parent->graph_mutex);
 
+	mutex_lock(&fimc->lock);
 	if (fimc->out_path != FIMC_IO_DMA) {
 		ret = -EBUSY;
 		goto done;
@@ -492,6 +493,7 @@ static int fimc_lite_open(struct file *file)
 	}
 done:
 	mutex_unlock(&fimc->lock);
+	mutex_unlock(&me->parent->graph_mutex);
 	return ret;
 }