Servant leader sounds soft, and I can see why someone might believe that subordinates will take advantage of servant leader.
However, as this blogger notes, being a servant leader does not mean that the leader will be a doormat. Leaders willing to be accountable and holding others to the same level of accountability will always be respected.
People have to produce, and those who are not willing to produce need to be dealt with. A servant leader who is consistent and holds all people to high standards won’t be used as a doormat.