python多线程示例- 给多任务处理增加监视保护线程

继上篇文章python多线程示例 -- 有限调度器处理队列数据做一个拓展功能，给多任务处理增加一个时间限制保护

  • 新增一个计时线程，这里示例为计时12s，增加了一个信号量timeoutSignal
  • 当计时线程休眠18s后，释放timeoutSignal，主程序处理中获取到时，会退出while循环，利用线程join函数来阻塞主线程，等到所有线程结束后，主进程会结束
from queue import Queue, Empty
from threading import Lock, Thread, BoundedSemaphore, current_thread, activeCount
import logging
import time


class AppLogger:
    def __init__(self, moduleName, logfile):
        self._logger = logging.getLogger(moduleName)
        handler = logging.FileHandler(logfile)
        fmt = "%(asctime)-15s %(levelname)s %(filename)s %(lineno)d %(message)s"
        formatter = logging.Formatter(fmt)
        handler.setFormatter(formatter)
        self._logger.addHandler(handler)
        self._logger.setLevel(logging.INFO)

        console = logging.StreamHandler()
        console.setLevel(logging.INFO)
        console.setFormatter(formatter)
        self._logger.addHandler(console)

        self.warnning = self._logger.warning
        self.error = self._logger.error
        self.info = self._logger.info
        self.debug = self._logger.debug


dataQueueMaxLen = 100
schedulerMaxCount = 4


lock = Lock()

scheduleSources = BoundedSemaphore(schedulerMaxCount)

dataQueue = Queue(dataQueueMaxLen)

myLogger = AppLogger("myapp","test.log")

class Scheduler:
    def __init__(self, func, data):
        self._thread = Thread(target=func, args=data)
        self.record = "Scheduler(%s, %s)" % (func.__name__, *data)

    def __str__(self):
        return self.record

    def start(self):
        # myLogger.info(("START ", self.record))
        self._thread.start()

    def join(self):
        self._thread.join()

dataList = range(dataQueueMaxLen)

for data in dataList:
    dataQueue.put(data)

def testFunc(data):
    thd = current_thread()
    myLogger.info("Thread %s start testFunc %s sleep 5 seconds" % (thd , data))

    time.sleep(5)
    scheduleSources.release()
    myLogger.info("Thread %s exit testFunc %s" % (thd, data))

runningProcList = []

def startScheduler(func, data):
    if scheduleSources.acquire():
        proc = Scheduler(func, [data])
        proc.start()
        runningProcList.append(proc)


timeoutSignal = BoundedSemaphore(1)

def monitor():
    timeoutSignal.acquire()
    time.sleep(12)
    timeoutSignal.release()

if __name__ == "__main__":
    monitorThd = Thread(target=monitor)
    monitorThd.start()

    while not dataQueue.empty():
        if timeoutSignal.acquire(timeout=0):
            myLogger.warnning("timeout wait other thread end and Exit main process")
            break
        data = dataQueue.get()
        startScheduler(testFunc, data)
        myLogger.info("current running thread count %d " % activeCount())

    for proc in runningProcList:
        proc.join()

可以从最终日志输出结果中可以看到 程序启动12后，主线程收到了超时信号2020-04-06 09:22:05,005 WARNING multiProcTest.py 95 timeout wait other thread end and Exit main process，退出while循环，主线程等待还未结束的线程退出后再结束

C:\Python36\python.exe D:/MyWorkSpace/CCMtask/multiProcTest.py
2020-04-06 09:21:48,721 INFO multiProcTest.py 67 Thread <Thread(Thread-2, started 76540)> start testFunc 0 sleep 5 seconds
2020-04-06 09:21:48,721 INFO multiProcTest.py 99 current running thread count 3 
2020-04-06 09:21:49,955 INFO multiProcTest.py 67 Thread <Thread(Thread-3, started 87780)> start testFunc 1 sleep 5 seconds
2020-04-06 09:21:49,955 INFO multiProcTest.py 99 current running thread count 4 
2020-04-06 09:21:49,971 INFO multiProcTest.py 99 current running thread count 5 
2020-04-06 09:21:49,955 INFO multiProcTest.py 67 Thread <Thread(Thread-4, started 88128)> start testFunc 2 sleep 5 seconds
2020-04-06 09:21:49,971 INFO multiProcTest.py 67 Thread <Thread(Thread-5, started 2312)> start testFunc 3 sleep 5 seconds
2020-04-06 09:21:49,971 INFO multiProcTest.py 99 current running thread count 6 
2020-04-06 09:21:54,976 INFO multiProcTest.py 71 Thread <Thread(Thread-2, started 76540)> exit testFunc 0
2020-04-06 09:21:54,976 INFO multiProcTest.py 71 Thread <Thread(Thread-3, started 87780)> exit testFunc 1
2020-04-06 09:21:54,976 INFO multiProcTest.py 71 Thread <Thread(Thread-4, started 88128)> exit testFunc 2
2020-04-06 09:21:54,976 INFO multiProcTest.py 67 Thread <Thread(Thread-6, started 82564)> start testFunc 4 sleep 5 seconds
2020-04-06 09:21:54,976 INFO multiProcTest.py 99 current running thread count 6 
2020-04-06 09:21:54,976 INFO multiProcTest.py 67 Thread <Thread(Thread-7, started 86944)> start testFunc 5 sleep 5 seconds
2020-04-06 09:21:54,976 INFO multiProcTest.py 99 current running thread count 5 
2020-04-06 09:21:54,976 INFO multiProcTest.py 67 Thread <Thread(Thread-8, started 87232)> start testFunc 6 sleep 5 seconds
2020-04-06 09:21:54,976 INFO multiProcTest.py 99 current running thread count 6 
2020-04-06 09:21:54,991 INFO multiProcTest.py 71 Thread <Thread(Thread-5, started 2312)> exit testFunc 3
2020-04-06 09:21:54,991 INFO multiProcTest.py 67 Thread <Thread(Thread-9, started 72020)> start testFunc 7 sleep 5 seconds
2020-04-06 09:21:54,991 INFO multiProcTest.py 99 current running thread count 6 
2020-04-06 09:21:59,991 INFO multiProcTest.py 71 Thread <Thread(Thread-6, started 82564)> exit testFunc 4
2020-04-06 09:21:59,991 INFO multiProcTest.py 71 Thread <Thread(Thread-7, started 86944)> exit testFunc 5
2020-04-06 09:21:59,991 INFO multiProcTest.py 71 Thread <Thread(Thread-8, started 87232)> exit testFunc 6
2020-04-06 09:21:59,991 INFO multiProcTest.py 67 Thread <Thread(Thread-10, started 86476)> start testFunc 8 sleep 5 seconds
2020-04-06 09:21:59,991 INFO multiProcTest.py 99 current running thread count 5 
2020-04-06 09:21:59,991 INFO multiProcTest.py 67 Thread <Thread(Thread-11, started 86644)> start testFunc 9 sleep 5 seconds
2020-04-06 09:21:59,991 INFO multiProcTest.py 99 current running thread count 5 
2020-04-06 09:21:59,991 INFO multiProcTest.py 67 Thread <Thread(Thread-12, started 86228)> start testFunc 10 sleep 5 seconds
2020-04-06 09:21:59,991 INFO multiProcTest.py 99 current running thread count 6 
2020-04-06 09:22:00,006 INFO multiProcTest.py 71 Thread <Thread(Thread-9, started 72020)> exit testFunc 7
2020-04-06 09:22:00,006 INFO multiProcTest.py 67 Thread <Thread(Thread-13, started 89180)> start testFunc 11 sleep 5 seconds
2020-04-06 09:22:00,006 INFO multiProcTest.py 99 current running thread count 6 
2020-04-06 09:22:05,005 INFO multiProcTest.py 71 Thread <Thread(Thread-10, started 86476)> exit testFunc 8
2020-04-06 09:22:05,005 INFO multiProcTest.py 71 Thread <Thread(Thread-12, started 86228)> exit testFunc 10
2020-04-06 09:22:05,005 INFO multiProcTest.py 71 Thread <Thread(Thread-11, started 86644)> exit testFunc 9
2020-04-06 09:22:05,005 INFO multiProcTest.py 67 Thread <Thread(Thread-14, started 86376)> start testFunc 12 sleep 5 seconds
2020-04-06 09:22:05,005 INFO multiProcTest.py 99 current running thread count 3 
2020-04-06 09:22:05,005 WARNING multiProcTest.py 95 timeout wait other thread end and Exit main process
2020-04-06 09:22:05,020 INFO multiProcTest.py 71 Thread <Thread(Thread-13, started 89180)> exit testFunc 11
2020-04-06 09:22:10,020 INFO multiProcTest.py 71 Thread <Thread(Thread-14, started 86376)> exit testFunc 12
2020-04-06 09:22:10,020 INFO multiProcTest.py 104 main proc exit

Process finished with exit code 0
©著作权归作者所有,转载或内容合作请联系作者
【社区内容提示】社区部分内容疑似由AI辅助生成，浏览时请结合常识与多方信息审慎甄别。
平台声明：文章内容（如有图片或视频亦包括在内）由作者上传并发布，文章内容仅代表作者本人观点，简书系信息发布平台，仅提供信息存储服务。

相关阅读更多精彩内容

友情链接更多精彩内容