Exploring and Exploiting Structure and Self-supervision in Sequence Learning