We live in the golden age of open source data for NLP. For free, no strings attached, you have access to millions of Wikipedia articles devoted to every conceivable entity, written in hundreds of languages, all interwoven with Wikidata’s structured knowledge base. But it is surprisingly hard to harness this public treasure for the purpose of training entity recognition models.

In this talk I will describe work at Primer on the “Data Factory”, a layer of code on top of Wikipedia and Wikidata that we are using to generate high-quality entity recognition data on demand. I will also share progress on an ambitious use of that data: a true zero-shot entity recognition model. Andfinally, I will introduce BabyBear, the open source framework we created to make entity recognition models run faster and cheaper.