qeth: fix deadlock between recovery and bonding driver

The recovery thread, when failing, tears down the respective interface. To do
so, it needs to obtain the rtnl lock first, as the interface configuration is
changed.
If another process tries to modify an interface setting at the same time, that
process can obtain the rtnl lock first, but the respective callback in the qeth
driver will block until recovery has completed - which cannot happen since the
calling process already obtained it.
In one particular case, the bonding driver acquired the rtnl lock to modify the
card's MAC address, while the recovery failed at the same time due to the card
being removed. Hence qeth_l2_set_mac_address (implicitly holding the rtnl lock)
was waiting on qeth_l2_recover, which deadlocked when waiting on the rtnl lock.
This patch uses rtnl_trylock instead of rtnl_lock in the recovery thread. If the
lock cannot be obtained, the interface will be left up, but the card state
remains in CARD_STATE_RECOVER, which will prevent any further activities on the
card.

Signed-off-by: Stefan Raspl <raspl@linux.vnet.ibm.com>
Signed-off-by: Frank Blaschka <frank.blaschka@de.ibm.com>
Reviewed-by: Ursula Braun <ursula.braun@de.ibm.com>
Signed-off-by: David S. Miller <davem@davemloft.net>

2 files changed, 12 insertions, 10 deletions

diff --git a/drivers/s390/net/qeth_l2_main.c b/drivers/s390/net/qeth_l2_main.c
index 2db4093..e67e025 100644
--- a/drivers/s390/net/qeth_l2_main.c
+++ b/drivers/s390/net/qeth_l2_main.c
@@ -1141,11 +1141,12 @@ static int qeth_l2_recover(void *ptr)
 		dev_info(&card->gdev->dev,
 			"Device successfully recovered!\n");
 	else {
-		rtnl_lock();
-		dev_close(card->dev);
-		rtnl_unlock();
-		dev_warn(&card->gdev->dev, "The qeth device driver "
-			"failed to recover an error on the device\n");
+		if (rtnl_trylock()) {
+			dev_close(card->dev);
+			rtnl_unlock();
+			dev_warn(&card->gdev->dev, "The qeth device driver "
+				"failed to recover an error on the device\n");
+		}
 	}
 	qeth_clear_thread_start_bit(card, QETH_RECOVER_THREAD);
 	qeth_clear_thread_running_bit(card, QETH_RECOVER_THREAD);
diff --git a/drivers/s390/net/qeth_l3_main.c b/drivers/s390/net/qeth_l3_main.c
index 4cd310c..5ba3906 100644
--- a/drivers/s390/net/qeth_l3_main.c
+++ b/drivers/s390/net/qeth_l3_main.c
@@ -3510,11 +3510,12 @@ static int qeth_l3_recover(void *ptr)
 		dev_info(&card->gdev->dev,
 			"Device successfully recovered!\n");
 	else {
-		rtnl_lock();
-		dev_close(card->dev);
-		rtnl_unlock();
-		dev_warn(&card->gdev->dev, "The qeth device driver "
-			"failed to recover an error on the device\n");
+		if (rtnl_trylock()) {
+			dev_close(card->dev);
+			rtnl_unlock();
+			dev_warn(&card->gdev->dev, "The qeth device driver "
+				"failed to recover an error on the device\n");
+		}
 	}
 	qeth_clear_thread_start_bit(card, QETH_RECOVER_THREAD);
 	qeth_clear_thread_running_bit(card, QETH_RECOVER_THREAD);