In the case that it may not be managed by your ISP (check first!) you can find the IP address of the 800 router by using either the command "ipconfig" (windows) or "ifconfig" (*nix). It will list a default gateway address, this will be the IP address of the router. Now, unless you know the password to the router, trying to telnet to it will be useless. Also, if a password isn't set on the router, you will not be able to log in through telnet. So, try to console into the router. There is a way to bypass the password on Cisco devices, but before you go that route, check with your provider to absolutely make sure it's not their's.
Also, be a little more specific about how you want to access this server. Do you need ssh, terminal services, vpn, etc.?