Will the team leader or any other team member know if I decline an invitation to a team?

No. You can either accept or decline an invitation to join a team, but your response remains private to you. If you accept the invitation, you will be listed as a team member.