Practical IPsec VPN Information

IPsec (common for IKEv1 and IKEv2):

  • H.A.G.L.E:
    • Hashing algorithm (HMAC) (e.g. SHA256, MD5)
      • Authentication (e.g. PSK or certificates to match ID)
        • Group (DiffieHellman) (e.g. DH2=1024bit, DH14=2048bit)
          • Lifetime (session = 3600 = 60 mins)
    • Encryption Algorithm (e.g. 3DES, AES)
  • Ports:
    • IPsec: UDP 500 <–> 500
    • NAT-T IKEv1: UDP 4500 <–> 4500
    • NAT-T IKEv2: UDP 4500 <–> high port (TBC)


  • Phase 1 transform set: ID + Proposals (HMAC, Encryption & Phase1 DH) + authenticate each way (PSK, certs) -> ISAKMP SA (management), bidirectional
  • Phase 2: IP subnets + Phase2 DH (if PFS enabled) -> IPSEC SA (data), unidirectional
  • Quick mode” occurs in Phase 2, refers to the exchange of 3 messages to negotiate the new IPSEC SA
  • NAT-T is an option that must be explicitly enabled if required

AH vs ESP:

  • AH authenticates the entire packet. Because it encompasses the IP header it will be invalidated by NAT. Do not use AH if either side will be behind a NAT router.
  • ESP provides encryption and authentication, but the authentication is only applied to the enclosed local datagram.
  • Do not enable AH for dial-in users, because of the NAT issue.

Perfect Forward Security (PFS):

  • Setting must match on both sides
  • Enhances security by generating new dissimilar keys with random data provided by both sides
  • Creation of new keys takes longer with PFS enabled, but security is enhanced.
  • Should be enabled except where latency is critical

Dead-Peer Detection (DPD):

  • Sends R_U_THERE message every 15 seconds to check if the peer is responding and tears down the Security Associations if there is no response.
  • This may help keep the tunnel connected
  • Without DPD the VPN will only establish for “interesting traffic”, i.e. when the destination IP matches the remote subnet

Aggressive vs. Main:

  • Aggressive mode decreases the number of packets required to establish the VPN by including the ID and public key in the initial packet.
  • Aggressive mode sends the initiator ID in the clear
  • Aggressive mode may be beneficial for mobile or wireless clients where low latency to establish the VPN is required.
  • Aggressive mode should not be used with PSK, due to vulnerabilities
  • Aggressive mode is susceptible to replay attacks

IKEv2 enhancements:

  • Uses 4 messages (instead of 6 in IKEv1 Main mode)
  • Includes built-in NAT-Traversal
  • Supports Extended Authentication Protocols (EAPs) – certificates, Xauth, etc.
  • Keep Alive enabled by default
  • Supports mobility (MOBIKE), this allows VPN connection to be retained when IP address changes. MOBIKE is transparent and does not require user configuration.
  • Usually used with certificates, may require the CA server certificate from the responder to be installed as a trusted root authority so that the client knows to trust the responder


  • Adds an additional username and password authentication to IKEv1 or IKEv2 IPsec connections.
  • Allows a common VPN “profile” to be shared between users
  • Mostly used when Proprietary or 3rd Party VPN client software is utilized


  • Is a tunnelling protocol that encapsulate a private tunnel within IPsec, as such it has NAT limitations in the same way as IKEv1 and NAT-Traversal must explicitly be enabled.
  • L2TP will assign an IP address from the responders network
  • L2TP allows layer 2 (broadcast) communications from the client