RuntimeError: An attempt has been made to start a new process before the current pr

时间:2019-02-22
本文章向大家介绍RuntimeError: An attempt has been made to start a new process before the current pr,主要包括RuntimeError: An attempt has been made to start a new process before the current pr使用实例、应用技巧、基本知识点总结和需要注意事项,具有一定的参考价值,需要的朋友可以参考一下。

pytorch 加载数据时报错:

RuntimeError: 
        An attempt has been made to start a new process before the
        current process has finished its bootstrapping phase.

        This probably means that you are not using fork to start your
        child processes and you have forgotten to use the proper idiom
        in the main module:

            if __name__ == '__main__':
                freeze_support()
                ...

        The "freeze_support()" line can be omitted if the program
        is not going to be frozen to produce an executable.
 

from torch.utils.data import DataLoader
from text_loader import TextDataset

print("hello")

if __name__=="__main__": #必须加name ==maiin才不会报错,多线程问题
    dataset = TextDataset()
    train_loader = DataLoader(dataset=dataset,
                              batch_size=3,
                              shuffle=True,
                              num_workers=2)

    for i, (src, target) in enumerate(train_loader):
        print(i, "data:", src)
        print("target:", target)
        break



输出:
hello
hello
hello
这里输出了三个hello,因为两个进程加自身一个进程,共三个,

原因:多进程需要在main函数中运行,

解决方法1:

加main函数,在main中调用

解决方法2:

num_workers改为0,单进程加载

 

from torch.utils.data import DataLoader
from text_loader import TextDataset

print("hello")

if __name__=="__main__": #必须加name ==maiin才不会报错,多线程问题
    dataset = TextDataset()
    train_loader = DataLoader(dataset=dataset,
                              batch_size=3,
                              shuffle=True,
                              num_workers=0)

    for i, (src, target) in enumerate(train_loader):
        print(i, "data:", src)
        print("target:", target)
        break

改为num_workers=0后输出一个hello,进程为1个