The selection, analysis and implications of rare AGN in large surveys