NAS_Evolution

神經架構搜尋 NAS 與演化計算

2002 -- Evolving Neural Networks through Augmenting Topologies

NeuroEvolution of Augmenting Topologies (NEAT)

• http://nn.cs.utexas.edu/downloads/papers/stanley.ec02.pdf

1. 提出 crossover 的方法
2. fitness 考慮網路大小

2018 -- Regularized Evolution for Image Classifier Architecture Search

• https://arxiv.org/pdf/1802.01548.pdf

承襲 NEAT，改現代架構 (以層連接)

Algorithm 1 Aging Evolution
    population ← empty queue . The population.
    history ← ∅ . Will contain all models.
    while |population| < P do . Initialize population.
        model.arch ← RANDOMARCHITECTURE()
        model.accuracy ← TRAINANDEVAL(model.arch)
        add model to right of population
        add model to history
    end while
    while |history| < C do . Evolve for C cycles.
        sample ← ∅ . Parent candidates.
        while |sample| < S do
        candidate ← random element from population
        . The element stays in the population.
        add candidate to sample
    end while
    parent ← highest-accuracy model in sample
    child.arch ← MUTATE(parent.arch)
    child.accuracy ← TRAINANDEVAL(child.arch)
    add child to right of population
    add child to history
    remove dead from left of population . Oldest.
    discard dead
end while
return highest-accuracy model in history