If you allow SSH access to your server, there are some simple steps you can take to restrict access and protect yourself from brute force attacks. Two of my favorite scripts to do this are Advanced Policy Firewall coupled with Brute Force Detection, both by R-FX Networks.
Locking Down Your Linux Server with APF + BFD
L