线程池ThreadPoolExecutor
使用futures模块:
- 线程池可以用来限制同时运行的最大线程数;
- 在主进程中获取某一个线程或者某一个任务的状态或者返回值;
- 当一个线程运行完成时主线程可以立即知道;
- 让多线程和多进程的编码接口一致。
一. 函数体:
def get_html(sleep_time):
time.sleep(sleep_time)
# print('get_page success', sleep_time)
return 'success,{}'.format(sleep_time)
二. 任务操作
submit提交
使用submit提交函数到线程池, 打开一个线程, submit是立即返回, 非阻塞方法
from concurrent.futures import ThreadPoolExecutor
if __name__ == '__main__':
# 参数表示可以同时执行的嘴阀线程数量
excu = ThreadPoolExecutor(3)
# 使用submit提交函数到线程池, 打开一个线程, submit是立即返回, 非阻塞
task1 = excu.submit(get_html, (5))
cancel取消
取消进程, 如果cancel失败会返回一个false, 如果任务处在执行中或者执行完成状态, 则会取消失败
task2 = excu.submit(get_html, (5))
print('查看取消', task2.cancel())
done 判断完成.
task1 = excu.submit(get_html, (5))
print('查看task1是否完成', task1.done())
result查看进程返回结果
task2 = excu.submit(get_html, (5))
print('查看task2的运行返回结果', task2.result())
使用excutor.map
from concurrent.futures import ThreadPoolExecutor
if __name__ == '__main__':
excu = ThreadPoolExecutor(3)
urls = [2, 3, 4]
# 通过excu.map()获取已经完成的task返回值
for data in excu.map(get_html, urls):
print('the result is ', data)
as_completed获取已经完成的task
from concurrent.futures import ThreadPoolExecutor, as_completed,
if __name__ == '__main__':
excu = ThreadPoolExecutor(3)
urls = [2, 3, 4]
# 通过as_completed获取已经完成的task1
all_task = [excu.submit(get_html, (url)) for url in urls]
for future in as_completed(all_task):
data = future.result()
print(data)
wait阻塞以等待线程完成
from concurrent.futures import ThreadPoolExecutor, as_completed, wait
if __name__ == '__main__':
excu = ThreadPoolExecutor(3)
urls = [2, 3, 4]
# 通过as_completed获取已经完成的task1
all_task = [excu.submit(get_html, (url)) for url in urls]
for future in as_completed(all_task):
data = future.result()
print(data)
wait(all_task)
print('end')
推荐: https://manjusaka.itscoder.com/2017/11/28/something-about-concurrent-future/