Multithreading PyQt applications with QThreadPool


#1

A common problem when building GUI applications is "locking up" of the interface when attempting to perform long-running background tasks. In this tutorial I'll cover one of the simplest way to achieve concurrent execution in PyQt.


This is a companion discussion topic for the original entry at https://www.pymadethis.com/article/multithreading-pyqt-applications-with-qthreadpool/

#2

Thank you so much for sharing this with us, its really helpful and understandable considering I couldn't understand threading in QT after 3-4 hours of research!


#3

Thanks Shobhit, great to hear it's helpful! A lot of what you'll find online about Qt multithreading is way out of date, gets very confusing — I know, I've been there ;)


#4

Your article is amazing dude. Could you please consider contributing some threading examples to https://stackoverflow.com/d... ? I know it's a lot to ask, but it's very hard to find good examples.


#5

You are a god. This is the best Threading tutorial I've come across.


#6

Very nice article. One question. Do you know a way to stop all threads currently executed in threads of the threadpool? The typical example e.g. you have a list of 100 files to process (let it be videos) and I'm processing 4 at the same time using QThreadPool. Whenever the processing of one video is finished, I'm starting a new thread processing the next video. Now there is a button "Cancle" and whenever the button is pressed I would like to stop immediately all running threads and further threads from starting. Stop starting new threads isn't a problem, but I didn't found a way to stop all running threads. I added a "kill" function to the worker class, which sets a flag to False, when called and adapted my function loop to stop when this flag is changed. But I'm only able to connect the button to the last worker started, not to all running. You know of any neat way to stop all running threads?


#7

Thanks for this article. Been beating my head against the wall the last few hours, trying to understand PyQt Threading. This helped me a TON.


#8

Hi Martin, brilliant article. I'm implementing this myself, but can not figure out how to communicate with a running thread in a QThreadPool using slots / signals. Could you give me some guidance on this? I look forward to buying your eBook when I have the knowledge to fully appreciate it!


#9

Hi Cameron — are you asking about sending data "into" the running thread? You can actually use the same mechanism as described above for getting data out. Just create another signal on WorkerSignals object e.g. "input" and connect a method to that signal in the __init__ block of your worker. Inside the method you can than chance any object properties you like.


#10

Hey fkratzert, sorry for the late response. You will need to maintain a list of "handles" to the different workers. A simple way to do this is to keep a list of currently active workers outside. You can then just iterate the active workers and set your flag.

Remember to connect your "finished" signal to clean up and remove the worker from the list.


#11

Hey Martin,
thanks for your late reply but I already figured it out ;) But hopefully if someone has the same question he will find your answer.


#12

Thought that might be the case ;) great to hear it's working for you though.


#13

Has anyone tried to get this to work with GPIO and having PyQt work with button inputs? It would be greatly appreciated if someone could show an example of this working.


#14

Hey, can you explain a bit more how you want it to work with GPIO inputs? Using gpiozero you get signal-like behaviour built in. But you should certainly be able to interact with GPIO directly from within a running Qt thread + e.g. send the incoming data out through a signal.


#15

IT WORKS!

After ages of trying to get a QProgressBar to update along with the output from rsync, I finally have it working. Thanks for this tutorial. I also bought your book as a way of saying "thanks".


#16

Very very awesome and well written! I'm going buy your book now after this post.

Just one question though. You mentioned that in an upcoming tutorial that you are going to be covering concurrent futures. I'm looking to do this with multiprocessing(not threaded). Is there any way to do non-blocking progress updates to a QProgressDialog with concurrent.futures ProcessPoolExecuter?

In the application I'm writing, I have a series of jobs that can might take some time to complete which I pass to a worker process. I'm using processes instead of threads because I want to keep the ability to kill a worker process if absolutely needed.

edit: Now purchased!


#17

Sorry I missed this message earlier. What you're asking will be possible, but you'll hit issues trying to update the GUI from a separate process + you can't use Qt signals in this setup. If I remember correctly the way I've done this in the past is to set up a message Queue to send back this status. On the GUI site you need to check this queue (using a QTimer) for incoming data and use this to update. Not quite as elegant... so if you find a better solution would be great to hear it


#18

How would you add dynamic call backs? Is that even possible? If I add more callbacks does that mean that each of those callbacks has to be used in my methods that will emit? Not sure if I am making sense. Let me know if you need me to clarify


#19

Also how would you make sure the threads stop when you close the gui. Right now I have an issue where I close my application and the terminal is still locked even though the GUI is gone. I have to kill the process running.


#20

Thank you for article. By the way, is this correct?
self.kwargs = kwargs
self.signals = WorkerSignals()

# Add the callback to our kwargs
kwargs['progress_callback'] = self.signals.progress

Why kwargs['progress_callback'], not self.kwargs['progress_callback']?