Reptile: A scalable meta-learning algorithm
OpenAI publishes Reptile, a meta-learning algorithm that updates initial parameters toward task-optimised parameters using standard optimisers like SGD or Adam, with performance comparable to first-order MAML.
We’ve developed a simple meta-learning algorithm called Reptile which works by repeatedly sampling a task, performing stochastic gradient descent on it, and updating the initial parameters towards the final parameters learned on that task. Reptile is the application of the Shortest Descent algorithm to the meta-learning setting, and is mathematically similar to first-order MAML (which is a version of the well-known MAML algorithm) that only needs black-box access to an optimizer such as SGD or Adam, with similar computational efficiency and performance.