Identity Drift in Facial Recognition Explained