The way unfavinator works is the same as deleting tweets in Twitter. Why don't you make it possible to go back to the first favorite? the old favs are the one that need to be deleted and is a feature that can't be found on twitter.