线程池ThreadPoolExecutor

使用futures模块:

  • 线程池可以用来限制同时运行的最大线程数;
  • 在主进程中获取某一个线程或者某一个任务的状态或者返回值;
  • 当一个线程运行完成时主线程可以立即知道;
  • 让多线程和多进程的编码接口一致。

一. 函数体:

def get_html(sleep_time):
    time.sleep(sleep_time)
    # print('get_page success', sleep_time)
    return 'success,{}'.format(sleep_time)

二. 任务操作

submit提交

使用submit提交函数到线程池, 打开一个线程, submit是立即返回, 非阻塞方法

from concurrent.futures import ThreadPoolExecutor

if __name__ == '__main__':
    # 参数表示可以同时执行的嘴阀线程数量
    excu = ThreadPoolExecutor(3)
     # 使用submit提交函数到线程池, 打开一个线程, submit是立即返回, 非阻塞
    task1 = excu.submit(get_html, (5))
cancel取消

取消进程, 如果cancel失败会返回一个false, 如果任务处在执行中或者执行完成状态, 则会取消失败

    task2 = excu.submit(get_html, (5))
    print('查看取消', task2.cancel())   
done 判断完成.
 task1 = excu.submit(get_html, (5))
print('查看task1是否完成', task1.done())
result查看进程返回结果
 task2 = excu.submit(get_html, (5))
print('查看task2的运行返回结果', task2.result())
使用excutor.map
from concurrent.futures import ThreadPoolExecutor
if __name__ == '__main__':
    excu = ThreadPoolExecutor(3)
    urls = [2, 3, 4]
    # 通过excu.map()获取已经完成的task返回值
    for data in excu.map(get_html, urls):
         print('the result is ', data)
as_completed获取已经完成的task
from concurrent.futures import ThreadPoolExecutor, as_completed,
if __name__ == '__main__':
    excu = ThreadPoolExecutor(3)
    urls = [2, 3, 4]
     # 通过as_completed获取已经完成的task1
    all_task = [excu.submit(get_html, (url)) for url in urls]
    for future in as_completed(all_task):
        data = future.result()
        print(data)
wait阻塞以等待线程完成
from concurrent.futures import ThreadPoolExecutor, as_completed, wait
if __name__ == '__main__':
    excu = ThreadPoolExecutor(3)
    urls = [2, 3, 4]
     # 通过as_completed获取已经完成的task1
    all_task = [excu.submit(get_html, (url)) for url in urls]
    for future in as_completed(all_task):
        data = future.result()
        print(data)
    wait(all_task)
    print('end')

推荐: https://manjusaka.itscoder.com/2017/11/28/something-about-concurrent-future/

Leave a Reply

Your email address will not be published.