No. Is a complete sentence.
We seem to believe that if we don't explain ourselves, we aren't being polite. "I'm sorry, I can't do that because. . . . . . . .. (fill in some 'excuse'). It isn't a requirement to share your reasons for refusing anything. In fact, it weakens your refusal. Now, if you prefer the inquirer to understand something about you more fully, esp perhaps for a future similar request, by all means explain, but don't over-explain. Share, but no need, usually, to apologize. You are the only one who must believe, agree with, prefer, your refusal. A simple, "No thank you." or, "I'm not available", or "No." can suffice. Watch your tone however, if you do not wish to be thought unkind, uninterested or mistaken. Smile, add a "thank you for the opportunity", or "perhaps another time' if you feel the need to soften a negative response. Just be certain. What about spam caller...