Image for post
Image for post
Photo by @mana5280

Tiny and Powerful NLP for Text With pQRNN

Natural language processing with projection-based modelling and Quasi-Recurrent Neural Networks (QRNN) from Google AI

The Google AI blog is one to follow. I will do my best to cover an article written the 21st of September 2020 by Prabhu Kaliamoorthi, Software Engineer, at Google Research.

Image for post
Image for post
Image from Google AI blog retrieved the 23rd of September 2020

“The novelty of pQRNN is in how it combines a simple projection operation with a quasi-RNN encoder for fast, parallel processing.”

According to a paper on the topic that the author has linked:

  1. The neural network then uniquely identifies each segment using a trainable parameter vector, which comprises the embedding table.
    → in which text is segmented has a significant impact on the model performance, size, and latency.
Image for post
Image for post
Image from Google AI blog retrieved the 23rd of September 2020
Image for post
Image for post
Image from Google AI blog retrieved the 23rd of September 2020
  1. A dense bottleneck layer.
  2. A stack of Quasi-Recurrent Neural Networks (QRNN) encoders.
Image for post
Image for post
Image from Google AI blog retrieved the 23rd of September 2020

Written by

AI Policy and Ethics at www.nora.ai. Student at University of Copenhagen MSc in Social Data Science. All views are my own. twitter.com/AlexMoltzau

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store