Bert - How is ELMo different from BERT?

In contrast to OpenAI GPT's left-to-right Transformer, BERT's is bidirectional. To create features for future tasks, ELMo integrates left-to-right and right-to-left LSTM that were trained separately.