Efficient Inference In Open Retrieval Question Answering Systems