Wood and metal move, especially over time and under the tension of playing and strings. You'll never have absolutely perfectly levelled frets as a result, but you can isolate trouble spots, file them ...