I think Tamas’ point is exactly right, compare will be your issue here - I asked a similar question a while ago and the advice (rightly as it turned out) was a similar one.
What that means I guess is that it can pay to think a bit more about how you can minimize the number of comparisons you have to do, e.g. normalise strings (lowercase and remove spaces) and remove exact duplicates first, maybe remove duplicates immediately so that they don’t get compared again etc. and any other tricks which might be problem specific!