djitellopy crashes randomly when trying to join the frame capture thread

[tomkark]

Hello,

I've been using this library for a project of mine, i've been having a lot of problems of djitellopy crashing, so i was trying to test the library all by itself without my code, and i've written this code:

from djitellopy import Tello
from threading import Thread
from time import sleep, time
import cv2 as cv

thread_command_flag = True


def threaded_hover(drone: Tello):
    global thread_command_flag
    while thread_command_flag:
        sleep(10)
        drone.send_command_with_return("command")


def rotation(tello):
    first_time = time()
    tello.send_command_without_return("rc 0 0 0 8")
    while True:
        if time() - first_time >= 140:
            break
        try:
            frame = tello.get_frame_read().frame
            frame = cv.resize(frame, (640, 480))
            frame_gray = cv.cvtColor(frame, cv.COLOR_BGR2GRAY)
        except Exception as e:
            print("FRAME EXCEPTION:", e)
            print("FRAME:", frame)
            sleep(0.5)
            continue
        cv.imshow('frame', frame_gray)
        k = cv.waitKey(25)
        if k == 27:
            break
    tello.send_command_without_return("rc 0 0 0 0")
    sleep(1)


def init():
    tello = Tello()
    tello.connect()
    tello.streamon()
    thread_hover = Thread(target=threaded_hover, args=(tello,))
    thread_hover.start()
    sleep(5)
    tello.takeoff()
    sleep(3)
    rotation(tello)
    tello.move_up(40)
    sleep(3)
    rotation(tello)
    tello.streamoff()
    sleep(180)
    tello.streamon()
    sleep(3)
    tello.rotate_clockwise(180)
    sleep(6)
    tello.streamoff()
    sleep(1)
    tello.land()
    global thread_command_flag
    thread_command_flag = False
    sleep(0.2)
    thread_hover.join()
    tello.end()


init()

This code is supposed to to takeoff and stream a video of two rotations which last 140 seconds, using rc 0 0 0 8,
and then wait in the air for 3 minutes simulating my system which is irrelevant for this issue.

But for some reasons, it crashes at random parts of the code, and not all the time, but most times.

the thread i'm creating just keeps the drone in the air so it wont land.

I've tried multiple different Tello drones (the basic white one) and it happend at least once in all of them.

The error is as follows:

Traceback (most recent call last):
  File "/usr/lib/python3.9/threading.py", line 954, in _bootstrap_inner
    self.run()
  File "/usr/lib/python3.9/threading.py", line 892, in run
    self._target(*self._args, **self._kwargs)
  File "/home/---/.local/lib/python3.9/site-packages/djitellopy/tello.py", line 1065, in update_frame
    self.stop()
  File "/home/---/.local/lib/python3.9/site-packages/djitellopy/tello.py", line 1074, in stop
    self.worker.join()
  File "/usr/lib/python3.9/threading.py", line 1030, in join
    raise RuntimeError("cannot join current thread")

Happens randomly, could be while doing a rotation, after the rotations, while doing the rotation after the 3 minute wait, so i can't really estimate even where it originates, if the problem is in my way of working with Tello i will be glad to get some help.

Tried Python 3.8/3.8.13/3.9.

Also happens on my regular ZBook and also on a raspberry pi.

This happend a minute or two after streamoff was called as well, if this helps in any way.

Will also list versions of packages in pip (using Python3.8.13):

Package         Version
--------------- ---------
certifi         2021.10.8
cycler          0.11.0
djitellopy      2.4.0
fonttools       4.33.3
imageio         2.19.2
joblib          1.1.0
kiwisolver      1.4.2
matplotlib      3.5.2
networkx        2.8.1
numpy           1.22.3
opencv-python   4.5.5.64
packaging       21.3
Pillow          9.1.1
pip             21.2.4
psutil          5.9.0
pyparsing       3.0.9
python-dateutil 2.8.2
PyWavelets      1.3.0
scikit-image    0.19.2
scikit-learn    1.1.0
scipy           1.8.1
setuptools      61.2.0
six             1.16.0
sklearn         0.0
threadpoolctl   3.1.0
tifffile        2022.5.4
wheel           0.37.1

[M4GNV5]

Hey, in fact it seems like a bug in DJITelloPy. When the library does not retrieve frames anymore (i.e. the video stream stops) the worker thread tries to stop and join itself.

However this also means for some reason your tello is stopping to send images, i.e. this if condition becomes True. Maybe this is caused by you sending command commands? Maybe this lets the tello stop the video stream because it thinks a new computer is controlling it now? Your could try other commands for hovering, maybe the query commands work? or just rc 0 0 0 0?

[tomkark]

Hey, in fact it seems like a bug in DJITelloPy. When the library does not retrieve frames anymore (i.e. the video stream stops) the worker thread tries to stop and join itself.

However this also means for some reason your tello is stopping to send images, i.e. this if condition becomes True. Maybe this is caused by you sending command commands? Maybe this lets the tello stop the video stream because it thinks a new computer is controlling it now? Your could try other commands for hovering, maybe the query commands work? or just rc 0 0 0 0?

Actually if I send an rc 0 0 0 0 it will stop the rotation, so it there a way to keep it hovering?

i tried using battery? instead of "command" and it still crashed, i think this is a pure bug in the library.

[tomkark]

Actually, forgot to mention in the above reply:

It works well sometimes which is weird, but also it works when i read the raw video using opencv which implies the video capture does not stop?

[dyxwxjdy]

I try to re-open cap stream instead of stop the thread, so change to this code solve my problem:

def update_frame(self):
    """Thread worker function to retrieve frames from a VideoCapture
    Internal method, you normally wouldn't call this yourself.
    """
    while not self.stopped:
        if not self.grabbed or not self.cap.isOpened():
            # self.stop()
            print('release and reopen cap')
            self.cap.release()
            if not self.cap.isOpened():
                self.cap.open(self.address)
            start = time.time()
            while time.time() - start < Tello.FRAME_GRAB_TIMEOUT:
                Tello.LOGGER.debug('trying to grab a frame...')
                self.grabbed, self.frame = self.cap.read()
                if self.frame is not None:
                    break
                time.sleep(0.05)

            if not self.grabbed or self.frame is None:
                # raise Exception('Failed to grab first frame from video stream')
                continue  #try again until get frame

        else:
            self.grabbed, self.frame = self.cap.read()

[M4GNV5]

The crash problem should be fixed since #186
I do not think calling streamon/streamoff multiple times wasnt supported before, but should work since 96c0dd7, just make sure to also call get_frame_read after each streamon
Thus I assume we can close this issue.