I propose organizing hosts into three groups: edge hosts, hub hosts, and protected hosts.
Edge hosts accept connections from the internet, but contain no sensitive data and are locked down to the highest degree possible. Many organizations already have hosts like this - public shell servers or similar. Two or more machines are preferable in this role for the sake of redundancy.
Hub hosts accept connections only from exposed hosts, and serve as intermediaries in connecting to protected hosts. Even though they should contain no special data, hub hosts are sensitive because they are the only places from which protected hosts allow connections. Management machines are ok for this role, but dedicated machines would be ideal. Again, having two hubs is good for redundancy.
Protected hosts contain sensitive data and accept connections only from hub hosts. Almost all machines fall into this category.
To access sensitive information, an attacker must have a reasonable knowledge of your network topography, and now compromise at least three machines. Mixing the architecture and operating system of the hosts may slow an attacker further, giving you time to respond before the attack is complete.
Legitimate users also pass through three systems to access protected hosts. With password access, though, the extra layers of protection can be of little use. If the edge machine is compromised, then an attacker can capture an administrator's password as he logs in, and potentially use it to access other resources.
SSH keys provide a more secure alternative to passwords, because a user logging into a machine with a key does not expose his credentials to that machine. If, however, the user then uses a another key to connect to a second machine from the first, his credentials are still at risk. An attacker controlling the machine has access to the key, and can sniff any password the user may need to enter to decrypt the key. With the decrypted key, the attacker has full control of the user's credentials.
To extend the advantages of SSH keys across several machines, it is possible to use port forwarding to establish an encrypted tunnel between machines. For example, a user on an untrusted network logs into to an edge host:
sh$ ssh user@edge -N -L 2222:hub:22Instead of opening a shell on the edge host, this command redirects port 22 on the hub host to port 2222 on the user's local machine. Now the user can log into the hub host:
sh$ ssh user@localhost -p 2222 -N -L 2223:protected:22Again, instead of a shell, SSH opens a port forwarding tunnel. Now port 22 on the protected host is redirected to port 2223 on the user's local machine. One final command and the user logs into the protected machine:
sh$ ssh user@localhost -p 2223All three machines that the user connects to can be compromised by an attacker, and yet his credentials remain safe. Each machine only observes the challenge and response exchange of data signed with the user's private key. No private keys or passwords ever cross the wire.
It's important to note that the one weak link here is the user's local machine. If an attacker has compromised that machine, she can capture whatever private keys and passwords the user uses to connect to remote machines. Even if the user keeps his private key on an external device with a crypto processor, the attacker can still modify the SSH client on his machine to install backdoors on every machine he logs into. A personal laptop with entensive physical and software security is therefore an ideal platform for remote access.
Typing three arcane commands to connect to a protected host can be a pain, though, especially if you want to connect to more than one at a time and have to keep the port numbers straight. For users whose local machines are running a UNIX-like operating system (including Cygwin), I have written tssh, a simple shell script that automates the process. Combined with ssh-agent, it can provide easy and relatively secure access to remote systems:
sh$ tssh root@protected Hop 1 Hop 2 Hop 3 Last login: Mon Sep 19 12:00:00 2005 from hub sh#Note that the scheme I describe is not equivalent to SSH agent forwarding (the -A command line option to ssh). As the ssh man page states,
"Agent forwarding should be enabled with caution. Users with the ability to bypass file permissions on the remote host (for the agent’s Unix-domain socket) can access the local agent through the forwarded connection. An attacker cannot obtain key material from the agent, however they can perform operations on the keys that enable them to authenticate using the identities loaded into the agent."
In other words, if any of the intermediate systems have been compromised and you forward your agent to them, your credentials can be used by the attacker to log into other systems, if not stolen outright.