Do I need a perfect credit score to refinance?

No. High credit score will definitely give you an advantage in interest rate, however, having a lower credit score will not necessary disqualify you.