IPSec protocols. IPsec - increasing security The protocols of the ipsec family provide

In the late sixties, the American Defense Advanced Research Projects Agency DARPA decided to create an experimental network called ARPANet. In the seventies, ARPANet began to be considered a functioning US network, and through this network it was possible to gain access to leading US university and research centers. In the early eighties, standardization of programming languages ​​began, and then network interaction protocols. The result of this work was the development of a seven-layer ISO/OSI network interaction model and the TCP/IP protocol family, which became the basis for the construction of both local and global networks.

Basic mechanisms information exchange in TCP/IP networks were generally formed in the early eighties, and were aimed primarily at ensuring the delivery of data packets between different operating systems using heterogeneous communication channels. Despite the fact that the idea of ​​​​creating the ARPANet network (later turned into modern Internet) belonged to a government defense organization, in fact the network originated in the research world, and inherited the tradition of openness of the academic community. Even before the commercialization of the Internet (which occurred in the mid-nineties), many reputable researchers noted problems associated with the security of the TCP/IP protocol stack. The basic concepts of the TCP/IP protocols do not fully satisfy (and in some cases contradict) modern ideas about computer security.

Until recently, the Internet was used mainly for processing information using relatively simple protocols: email, file transfer, remote access. Today, thanks to the widespread use of WWW technologies, distributed multimedia information processing tools are increasingly being used. At the same time, the volume of data processed in client/server environments and intended for simultaneous collective access by a large number of subscribers is growing. Several application level protocols have been developed to provide information security applications such as email (PEM, PGP, etc.), WWW (Secure HTTP, SSL, etc.), network management (SNMPv2, etc.). However, the presence of security features in the basic protocols of the TCP/IP family will allow information exchange between a wide range of different applications and services.

Brief historical background of the appearance of the protocol

In 1994, the Internet Architecture Board (IAB) released the report "Security of the Internet Architecture." This document described the main areas of application of additional security tools on the Internet, namely protection against unauthorized monitoring, packet spoofing, and data flow control. Among the first and most important protective measures was the need to develop a concept and basic mechanisms to ensure the integrity and confidentiality of data flows. Since changing the basic protocols of the TCP/IP family would cause a complete restructuring of the Internet, the task was set to ensure the security of information exchange in open telecommunication networks based on existing protocols. Thus, the Secure IP specification began to be created, complementary to the IPv4 and IPv6 protocols.

IPSec architecture

IP Security is a set of protocols dealing with issues of encryption, authentication and security during the transport of IP packets; it now includes nearly 20 standards proposals and 18 RFCs.

The IP Security specification (known today as IPsec) is developed by the IETF IP Security Protocol Working Group. IPsec originally included 3 algorithm-independent core specifications, published as RFCs: IP Security Architecture, Authentication Header (AH), Encapsulation of Encrypted Data (ESP) (RFC1825, 1826, and 1827). It should be noted that in November 1998, the IP Security Protocol Working Group proposed new versions of these specifications, which currently have the status of preliminary standards, these are RFC2401 - RFC2412. Note that RFC1825-27 has been considered obsolete for several years and is not really used. In addition, there are several algorithm-dependent specifications using the MD5, SHA, and DES protocols.

Rice. 1 – IPSec architecture.

The IP Security Protocol Working Group is also developing management protocols key information. The task of this group is Internet development Key Management Protocol (IKMP), an application-level key management protocol that is independent of the security protocols used. Key management concepts using the specification are currently being explored Internet Security Association and Key Management Protocol (ISAKMP) and Oakley Key Determination Protocol. The ISAKMP specification describes mechanisms for negotiating the attributes of the protocols used, while the Oakley protocol allows you to install session keys on computers on the Internet. Previously, the possibilities of using key management mechanisms of the SKIP protocol were also considered, but now such possibilities are practically not used anywhere. Emerging key information management standards may support Key Distribution Centers similar to those used in Kerberos. Key management protocols for IPSec based on Kerberos are now being developed by a relatively new working group KINK (Kerberized Internet Negotiation of Keys).

Data integrity and confidentiality guarantees in the IPsec specification are provided through the use of authentication and encryption mechanisms, respectively. The latter, in turn, are based on the preliminary agreement of the parties to the so-called information exchange. “security context” – applied cryptographic algorithms, key information management algorithms and their parameters. The IPsec specification provides for the ability for parties to exchange information to support various protocols and parameters for authentication and encryption of data packets, as well as various key distribution schemes. In this case, the result of agreeing on the security context is the establishment of a security parameter index (SPI), which is a pointer to a certain element of the internal structure of the information exchange party, describing possible sets of security parameters.

Essentially, IPSec, which will become an integral part of IPv6, operates at the third layer, i.e. at the network layer. As a result, transmitted IP packets will be protected in a manner transparent to network applications and infrastructure. Unlike SSL (Secure Socket Layer), which operates at layer 4 (i.e., transport) and is more closely associated with higher layers of the OSI model, IPSec is designed to provide low-level security.


Rice. 2 - OSI/ISO model.

To IP data ready to be transmitted over virtual private network,IPSec adds a header to identify protected packets. Before being transmitted over the Internet, these packets are encapsulated within other IP packets. IPSec supports several encryption types, including Data Encryption Standard (DES) and Message Digest 5 (MD5).

To establish a secure connection, both participants in the session must be able to quickly agree on security parameters, such as authentication algorithms and keys. IPSec supports two types of key management schemes through which participants can negotiate session parameters. This dual support at one time caused some friction in the IETF Working Group.

WITH current version IP, IPv4, either Internet Secure Association Key Management Protocol (ISAKMP) or Simple Key Management for Internet Protocol can be used. WITH new version IP, IPv6, will have to use ISAKMP, now known as IKE, although the possibility of using SKIP is not excluded. However, it should be kept in mind that SKIP has not been considered as a key management candidate for a long time, and was removed from the list of possible candidates back in 1997.

Header AH

The Authentication Header (AH) is a common optional header and is typically located between the IP packet's main header and the data field. The presence of AH does not in any way affect the process of transmitting information from transport and higher levels. The main and only purpose of AH is to provide protection against attacks related to unauthorized changes in the contents of a packet, including spoofing the source address of the network layer. Higher level protocols must be modified to verify the authenticity of the received data.

The AH format is quite simple and consists of a 96-bit header and variable length data consisting of 32-bit words. The field names reflect their contents fairly clearly: Next Header indicates the next header, Payload Len represents the packet length, SPI is a pointer to the security context, and Sequence Number Field contains the packet's sequence number.


Rice. 3 - AH header format.

The packet sequence number was introduced into AH in 1997 as part of the IPsec specification revision process. The value of this field is generated by the sender and serves to protect against attacks related to reuse of authentication process data. Because the Internet does not guarantee the order in which packets will be delivered, the recipient must store information about the maximum sequence number of a successfully authenticated packet and whether a number of packets containing previous sequence numbers have been received (usually 64).

Unlike checksum calculation algorithms used in protocols for transmitting information over switched communication lines or over local network channels and aimed at correcting random errors in the transmission medium, mechanisms for ensuring data integrity in open telecommunication networks must have means of protection against targeted changes. One such mechanism is a special use of the MD5 algorithm: during the formation of the AH, a hash function is sequentially calculated from the union of the packet itself and some pre-agreed key, and then from the union of the resulting result and the transformed key. This is the default mechanism to ensure that all IPv6 implementations have at least one common algorithm that is not subject to export restrictions.

ESP header

When encrypted data encapsulation is used, the ESP header is the last of the optional headers "visible" in the packet. Since the main purpose of ESP is to ensure data privacy, different types information may require the use of significantly different encryption algorithms. Consequently, the ESP format can undergo significant changes depending on the cryptographic algorithms used. However, the following mandatory fields can be distinguished: SPI, indicating the security context, and Sequence Number Field, containing the sequence number of the packet. Field "ESP Authentication Data" ( check sum), is optional in the ESP header. The recipient of the ESP packet decrypts the ESP header and uses the parameters and data of the applied encryption algorithm to decode the transport layer information.


Rice. 4 - ESP header format.

There are two modes of using ESP and AH (as well as their combinations) - transport and tunnel.

Transport mode

Transport mode is used to encrypt the data field of an IP packet containing transport layer protocols (TCP, UDP, ICMP), which, in turn, contains application service information. An example of a transport mode application is the transmission Email. All intermediate nodes along a packet's route from sender to receiver use only the public network layer information and possibly some optional packet headers (in IPv6). The disadvantage of transport mode is the lack of mechanisms for hiding the specific sender and recipient of a packet, as well as the ability to analyze traffic. The result of such an analysis can be information about the volumes and directions of information transfer, areas of interest of subscribers, and the location of managers.

Tunnel mode

Tunnel mode encrypts the entire packet, including the network layer header. The tunnel mode is used if it is necessary to hide the organization’s information exchange with the outside world. In this case, the address fields of the network layer header of a packet using tunnel mode are filled in firewall organizations and do not contain information about the specific sender of the package. When transmitting information from the outside world to local network organization uses the network address of the firewall as the destination address. After the firewall decrypts the initial network layer header, the packet is forwarded to the recipient.

Security Associations

A Security Association (SA) is a connection that provides security services for traffic that passes through it. Two computers on each side of the SA store the mode, protocol, algorithms, and keys used in the SA. Each SA is used in only one direction. Bidirectional communication requires two SAs. Each SA implements one mode and protocol; thus, if two protocols need to be used for one packet (such as AH and ESP), then two SAs are required.

Security policy

The security policy is stored in the SPD (Security Policy Database). SPD can specify one of three actions for a data packet: discard the packet, do not process the packet using IPSec, or process the packet using IPSec. In the latter case, the SPD also indicates which SA should be used (if, of course, a suitable SA has already been created) or indicates with what parameters a new SA should be created.

SPD is a very flexible control mechanism that allows very good control over the processing of each packet. Packages are classified by a large number fields, and SPD may examine some or all of the fields to determine the appropriate action. This could result in all traffic between two machines being carried using a single SA, or separate SAs being used for each application, or even for each TCP connection.

ISAKMP/Oakley

ISAKMP defines the general structure of protocols that are used to establish SAs and perform other key management functions. ISAKMP supports several Domains of Interpretation (DOI), one of which is IPSec-DOI. ISAKMP does not define a complete protocol, but rather provides "building blocks" for various DOIs and key exchange protocols.

The Oakley protocol is a key discovery protocol that uses the Diffie-Hellman key replacement algorithm. The Oakley protocol supports Perfect Forward Secrecy (PFS). The presence of PFS means that it is impossible to decrypt all traffic if any key in the system is compromised.

IKE

IKE is the default key exchange protocol for ISAKMP. this moment being the only one. IKE sits on top of ISAKMP and does the actual establishment of both the ISAKMP SA and the IPSec SA. IKE supports a set of various primitive functions for use in protocols. Among them are the hash function and the pseudo-random function (PRF).

A hash function is a collision-resistant function. Collision resistance refers to the fact that it is impossible to find two different messages m 1 And m 2, such that H(m 1)=H(m2), Where H— hash function.

As for pseudo-random functions, a hash function is currently used instead of special PRFs in the HMAC design (HMAC is a message authentication mechanism using hash functions). To define HMAC, we need a cryptographic hash function (let's call it H) and a secret key K. We assume that H is a hash function where data is hashed using a compression procedure applied sequentially to a sequence of data blocks. We denote by B the length of such blocks in bytes, and the length of blocks obtained as a result of hashing by L (L

Ipad = byte 0x36, repeated B times;
opad = byte 0x5C repeated B times.

To calculate HMAC from "text" data, you need to perform the following operation:

H(K XOR opad, H(K XOR ipad, text))

From the description it follows that IKE uses HASH values ​​to authenticate parties. Note that HASH in this case refers solely to the Payload name in ISAKMP, and this name has nothing to do with its contents.

Attacks on AH, ESP and IKE.

All types of attacks on IPSec components can be divided into the following groups: attacks that exploit the finite resources of the system (a typical example is a Denial of Service attack, Denial-of-service or DOS attack), attacks that exploit the features and errors of a specific IPSec implementation and and finally, attacks based on the weaknesses of the protocols themselves. AH and ESP. Purely cryptographic attacks can be ignored - both protocols define the concept of “transform”, where all cryptography is hidden. If the crypto-algorithm used is stable, and the transform defined with it does not introduce additional weaknesses (this is not always the case, therefore it is more correct to consider the strength of the entire system - Protocol-Transform-Algorithm), then from this side everything is fine. What remains? Replay Attack - leveled out by using Sequence Number (in one single case this does not work - when using ESP without authentication and without AH). Further, the order of actions (first encryption, then authentication) guarantees quick rejection of “bad” packets (moreover, according to recent research in the world of cryptography, this order of actions is the most secure; the reverse order in some, albeit very special cases, can lead to potential security holes; fortunately, neither SSL, nor IKE, nor other common “authenticate first, encrypt later” protocols apply to these special cases, and therefore do not have these holes). What remains is the Denial-Of-Service attack. As you know, this is an attack from which there is no complete defense. However, the quick rejection of bad packets and the absence of any external reaction to them (according to the RFC) make it possible to deal with this attack more or less well. In principle, most (if not all) known network attacks (sniffing, spoofing, hijacking, etc.) are successfully resisted by AH and ESP when used correctly. With IKE it's a little more complicated. The protocol is very complex and difficult to analyze. In addition, due to typos (in the formula for calculating HASH_R) when writing it and not entirely successful solutions (the same HASH_R and HASH_I), it contains several potential “holes” (in particular, in the first phase, not all Payloads in the message are authenticated), however , they are not very serious and lead, at most, to a refusal to establish a connection. IKE more or less successfully protects itself from attacks such as replay, spoofing, sniffing, hijacking. Cryptography is somewhat more complicated - it is not carried out separately, as in AH and ESP, but is implemented in the protocol itself. However, if you use persistent algorithms and primitives (PRF), there should be no problems. To some extent, it can be considered as a weakness of IPsec that DES is indicated as the only mandatory cryptographic algorithm in the current specifications (this is true for both ESP and IKE), 56 bits of the key of which are no longer considered sufficient. However, this is a purely formal weakness - the specifications themselves are algorithm-independent, and almost all well-known vendors have long implemented 3DES (and some have already implemented AES). Thus, if implemented correctly, the most “dangerous” attack remains Denial-Of-Service .

Protocol evaluation

The IPSec protocol has received mixed reviews from experts. On the one hand, it is noted that the IPSec protocol is the best among all other previously developed protocols for protecting data transmitted over the network (including PPTP developed by Microsoft). According to the other side, there is excessive complexity and redundancy of the protocol. Thus, Niels Ferguson and Bruce Schneier in their work "A Cryptographic Evaluation of IPsec" note that they found serious security problems in almost all major components of IPsec. These authors also note that the protocol suite requires significant improvements in order for it to provide a good level of security. The paper describes a number of attacks that exploit both the weaknesses of the general data processing scheme and the weaknesses of cryptographic algorithms.

Conclusion

In this article, we covered some basic points regarding the IPsec network security protocol. It is worth noting that the IPsec protocol dominates most virtual private network implementations. Currently, the market offers both software implementations (for example, the protocol is implemented in the Microsoft Windows 2000 operating system) and hardware and software implementations of IPsec - these are solutions from Cisco, Nokia. Despite the large number of different solutions, they are all quite compatible with each other. The article concludes with a table comparing IPSec and the now widely used SSL.

Peculiarities IPSec SSL
Hardware independence Yes Yes
Code No application changes required. May require access to the TCP/IP stack source code. Application changes required. New DLLs or access to application source code may be required.
Protection Entire IP packet. Enables protection for higher-level protocols. Application level only.
Packet filtering Based on authenticated headers, sender and recipient addresses, etc. Simple and cheap. Suitable for routers. Based on high-level content and semantics. More intelligent and more complex.
Performance Fewer context switches and data movement. More context switches and data movement. Large blocks of data can speed up cryptographic operations and provide better compression.
Platforms Any systems, including routers Mainly end systems (clients/servers), also firewalls.
Firewall/VPN All traffic is protected. Only application level traffic is protected. ICMP, RSVP, QoS, etc. may be unprotected.
Transparency For users and applications. Only for users.
Current status Emerging standard. Widely used by WWW browsers, also used by several other products.

Links

  • www.ietf.org/html.charters/ipsec-charter.html - Home page of the IETF working group. There are also links to RFCs and proposals for standards.
  • www.microsoft.com/rus/windows2000/library/security/w2k_IPSecurity.asp – Information about the implementation of the IPSec protocol in Windows2000 Server.

Acknowledgments

In contact with

Classmates

IPsec is not one protocol, but a system of protocols designed to protect data at the network level of IP networks. This article will describe the theory of using IPsec to create a VPN tunnel.

Introduction

VPN based on IPsec technology can be divided into two parts:

  • Internet Key Exchange (IKE) Protocol
  • IPsec protocols (AH/ESP/both)

The first part (IKE) is the negotiation phase, during which the two VPN points choose which methods will be used to protect the IP traffic sent between them. In addition, IKE is also used to manage connections by introducing the concept of Security Associations (SA) for each connection. SAs only point in one direction, so a typical IPsec connection uses two SAs.

The second part is those IP data that must be encrypted and authenticated before being transmitted using the methods agreed upon in the first part (IKE). There are different IPsec protocols that can be used: AH, ESP or both.

The sequence for establishing a VPN over IPsec can be briefly described as:

  • IKE negotiates IKE layer security
  • IKE negotiates IPsec layer security
  • protected data is transmitted via VPN IPsec

IKE, Internet Key Exchange

To encrypt and authenticate data, you need to select the encryption/authentication method (algorithm) and the keys used in them. The task of the Internet Key Exchange protocol, IKE, in this case comes down to distributing “session key” data and agreeing on algorithms that will protect data between VPN points.

The main tasks of IKE:

  • Authentication of VPN points to each other
  • Establishing new IPsec connections (through the creation of SA pairs)
  • Managing current connections

IKE keeps track of connections by assigning each of them a certain Security Associations, SA. The SA describes the parameters of a particular connection, including the IPsec protocol (AH/ESP or both), session keys used for encryption/decryption and/or authentication of data. SA is unidirectional, so multiple SAs are used per connection. In most cases, when only ESP or AH is used, only two SAs are created for each of the connections, one for incoming traffic and one for outgoing traffic. When ESP and AH are used together, SA requires four.

The IKE negotiation process goes through several stages (phases). These phases include:

  1. IKE Phase-1:
    — The protection of IKE itself (ISAKMP tunnel) is negotiated
  2. IKE phase two (IKE Phase-2):
    — IPsec protection is negotiated
    — Receiving data from the first phase to generate session keys

IKE and IPsec connections are limited in duration (in seconds) and in the amount of data transferred (in kilobytes). This is done to increase security.
The duration of an IPsec connection is generally shorter than IKE. Therefore, when an IPsec connection expires, a new IPsec connection is recreated through the second negotiation phase. The first negotiation phase is used only when re-creating the IKE connection.

To negotiate IKE, the concept of IKE Proposal is introduced - this is a proposal on how to protect data. The VPN point that initiates the IPsec connection sends a list (sentence) indicating different methods for securing the connection.
Negotiations can be conducted both about establishing a new IPsec connection and about establishing a new IKE connection. In the case of IPsec, the protected data is the traffic sent through the VPN tunnel, and in the case of IKE, the protected data is the data from the IKE negotiations themselves.
The VPN point that receives the list (suggestion) selects the most suitable one from it and indicates it in the response. If none of the offers can be selected, the VPN gateway refuses.
The proposal contains all the necessary information for choosing an encryption algorithm and authentication, etc.

Phase 1 IKE - IKE Security Negotiation (ISAKMP Tunnel)
In the first phase of negotiation, VPN points authenticate each other based on a common key (Pre-Shared Key). For authentication, hash algorithms are used: MD5, SHA-1, SHA-2.
However, before authenticating each other, so as not to transmit information in clear text, VPN points exchange lists of proposals (Proposals), described earlier. Only after an offer that suits both VPN points is selected does the VPN point authenticate each other.
Authentication can be accomplished in different ways: through Pre-Shared Keys, certificates, or . Shared keys are the most common authentication method.
Phase 1 IKE negotiation can occur in one of two modes: main and aggressive. The main mode takes longer, but is also more secure. In its process, six messages are exchanged. Aggressive mode is faster, limiting itself to three messages.
The main work of the first phase of IKE lies in the exchange of Diffie-Hellman keys. It is based on public key encryption, each party encrypts the authentication parameter (Pre-Shared Key) with the public key of its neighbor, who, having received this message, decrypts it with his private key. Another way to authenticate each other is through the use of certificates.

Phase 2 IKE – IPsec Security Negotiation
In the second phase, the IPsec connection protection method is selected.
The second phase uses keying material extracted from the Diffie-Hellman key exchange that occurred in the first phase. Based on this material, session keys are created, which are used to protect data in the VPN tunnel.

If the mechanism is used Perfect Forwarding Secrecy (PFS), then a new Diffie-Hellman key exchange will be used for each phase two negotiation. Slightly reducing the speed of operation, this procedure ensures that the session keys are independent of each other, which increases security, since even if one of the keys is compromised, it cannot be used to select the rest.

There is only one operating mode for the second phase of IKE negotiation, it is called quick mode. During the second phase negotiation process, three messages are exchanged.

At the end of the second phase, a VPN connection is established.

IKE options.
During connection establishment, several parameters are used, without negotiation of which it is impossible to establish a VPN connection.

  • End Node Identification
    How nodes authenticate each other. The most commonly used key is the shared key. Shared key authentication uses the Diffie-Hellman algorithm.
  • Local and remote network/host
    Defines the traffic that will be allowed through the VPN tunnel.
  • Tunnel or transport mode.
    IPsec can operate in two modes: tunnel and transport. The choice of mode depends on the objects being protected.
    Tunnel mode used for protection between remote objects, i.e. The IP packet is completely encapsulated in a new one and only the connection between the two VPN points will be visible to an outside observer. The real source and destination IP addresses will only be visible after the packet is decapsulated and received at the VPN receiving point. Thus, tunnel mode is most often used for VPN connections.
    Transport mode protects the data of the IP packet (TCP, UDP and upper-layer protocols), and the header of the original IP packet itself will be preserved. This way, the observer will see the original source and destination, but not the data being transmitted. This mode is most often used when protecting a local network connection between hosts.
  • Remote Gateway
    The VPN is the recipient of the secure connection, which will decrypt/authenticate the data from the other side and send it to the final destination.
  • IKE operating mode
    IKE negotiation can operate in two modes: basic And aggressive.
    The difference between them is that in aggressive mode, fewer packets are used, which allows for faster connection establishment. On the other hand, the aggressive mode does not transmit some negotiation parameters, such as Diffie-Hellman groups and PFS, which requires preliminary identical configuration of them on the participating connection points.
  • IPsec protocols
    There are two IPsec protocols: Authentication Header (AH) and Encapsulating Security Payload (ESP), which perform encryption and authentication functions.
    ESP allows encryption, authentication separately or simultaneously.
    AH only allows authentication. The difference with ESP authentication is that AH also authenticates the outer IP header, allowing you to confirm that the packet actually arrived from the source specified in it.
  • IKE encryption
    Specifies the IKE encryption algorithm to use and its keys. Various symmetric encryption algorithms are supported, for example: DES, 3DES, AES.
  • IKE authentication
    The authentication algorithm used in IKE negotiation. May be: SHA, MD5.
  • IKE Diffie-Hellman (DH) groups
    The DF group used for key exchange in IKE. The larger the group, the larger the size of the exchange keys.
  • IKE Connection Lifespan
    It is indicated both by time (seconds) and by the size of the transferred data (kilobytes). As soon as one of the counters reaches the threshold value, a new first phase starts. If no data has been transferred since the IKE connection was created, no new connections will be created until one of the parties wants to create a VPN connection.
  • PFS
    With PFS disabled, the key creation material will be retrieved in the first phase of IKE negotiation at the time of key exchange. In the second phase of IKE negotiation, session keys will be created based on the received material. When PFS is enabled, when creating new session keys, a new material will be used for them each time. Thus, if a key is compromised, it is not possible to create new keys based on it.
    PFS can be used in two modes: the first PFS on keys will start a new key exchange in the first IKE phase each time a negotiation is started
    second phase. The second PFS on identities mode will remove the first phase SA each time a second phase negotiation has passed, ensuring that no second phase negotiation will be encrypted with an identical key to the previous one.
  • IPsec DH groups
    DF group data is similar to those used in IKE, only used for PFS.
  • IPsec encryption
    Algorithm used to encrypt data. Used when using ESP in encryption mode. Example algorithms: DES, 3DES, AES.
  • IPsec authentication
    The algorithm used to authenticate transmitted data. Used in case of AH or ESP in authentication mode. Example algorithms: SHA, MD5.
  • IPsec lifetime
    The lifetime of a VPN connection is indicated both by time (seconds) and by the size of the transferred data (kilobytes). The first counter to reach the limit will trigger the re-creation of session keys. If no data has been transferred since the IKE connection was created, no new connections will be created until one of the parties wants to create a VPN connection.

IKE Authentication Methods

  • Manual mode
    The simplest of the methods, in which IKE is not used, and authentication and encryption keys, as well as some other parameters, are set manually at both points of the VPN connection.
  • Through shared keys (Pre-Shared Keys, PSK)
    A pre-entered shared key on both points of the VPN connection. The difference from the previous method is that it uses IKE, which allows endpoints to be authenticated and use rotating session keys instead of fixed encryption keys.
  • Certificates
    Each VPN point uses: its own private key, its own public key, its own certificate including its own public key and signed by a trusted certification authority. Unlike the previous method, it allows you to avoid entering one common key at all points of the VPN connection, replacing it with personal certificates signed by a trusted authority.

IPsec protocols

IPsec protocols are used to protect transmitted data. The choice of protocol and its keys occurs during IKE negotiation.

AH (Authentication Header)

AH provides the ability to authenticate transmitted data. To do this, a cryptographic hash function is used in relation to the data contained in the IP packet. The output of this function (the hash) is sent along with the packet and allows the remote VPN point to confirm the integrity of the original IP packet, confirming that it has not been modified along the way. In addition to the data of the IP packet, AH also authenticates part of its header.

In transport mode, AH embeds its header after the original IP packet.
In tunnel mode, AH embeds its header after the outer (new) IP header and before the inner (original) IP header.

ESP (Encapsulating Security Payload)

The ESP protocol is used for encryption, authentication, or both with respect to an IP packet.

In transport mode, the ESP protocol inserts its header after the original IP header.
In tunnel mode, the ESP header is located after the outer (new) IP header and before the inner (original).

Two main differences between ESP and AH:

  • In addition to authentication, ESP also provides encryption capabilities (AH does not provide this)
  • ESP in tunnel mode only authenticates the original IP header (AH also authenticates the external one).

Working behind NAT (NAT Traversal)
To support work behind NAT, a separate specification was implemented. If the VPN point supports this specification, IPsec supports operation behind NAT, but there are certain requirements.
NAT support consists of two parts:

  • At the IKE layer, end devices exchange information with each other about support, NAT Traversal, and the version of the supported specification
  • At the ESP level, the generated packet is encapsulated in UDP.

NAT Traversal is only used if both endpoints support it.
NAT Definition: Both VPN endpoints send hashes of their IP addresses along with the UDP source port of the IKE negotiation. This information is used by the recipient to determine whether the source IP address and/or port has changed. If these parameters have not been changed, then traffic does not pass through NAT and the NAT Traversal mechanism is not needed. If the address or port has been changed, then there is NAT between the devices.

Once the endpoints determine that NAT Traversal is needed, the IKE negotiation moves from UDP port 500 to port 4500. This is done because some devices do not correctly handle the IKE session on port 500 when using NAT.
Another problem arises from the fact that the ESP protocol is a transport layer protocol and sits directly on top of IP. Because of this, the concepts of a TCP/UDP port do not apply to it, which makes it impossible to connect more than one client to one gateway via NAT. To solve this problem, ESP is packed into a UDP datagram and sent to port 4500, the same one that IKE uses when NAT Traversal is enabled.
NAT Traversal is built into the protocols that support it and works without prior configuration.

IPSec relies on a number of technologies and encryption methods, but IPSec can generally be thought of as the following main steps:

    Step 1. Starting the IPSec process.

    Traffic that requires encryption according to the IPSec security policy agreed upon by the IPSec parties begins the IKE process. Step 2. IKE phase one

    . The IKE process authenticates IPSec parties and negotiates IKE security association parameters, resulting in a secure channel for negotiating IPSec security association parameters during the second phase of IKE. Step 3.

    IKE phase two . The IKE process negotiates IPSec security association parameters and establishes appropriate IPSec security associations for the communicating parties' devices.

    Step 4. Data transfer. Communication occurs between communicating IPSec parties based on IPSec parameters and keys stored in the security association database.

Step 5.

Terminating an IPSec tunnel

In transport mode, only the information part of the IP packet is encrypted. Routing is not affected since the IP packet header is not changed. Transport mode is typically used to establish connections between hosts.

In tunnel mode, the entire IP packet is encrypted. In order for it to be transmitted over the network, it is placed in another IP packet. This creates a secure IP tunnel. Tunnel mode can be used to connect remote computers to a virtual private network or to organize secure data transfer over open communication channels (Internet) between gateways to connect different parts of the virtual private network.

IPSec Transform Negotiation

The IKE protocol negotiates IPSec transformations (IPSec security algorithms). IPSec transformations and their associated encryption algorithms are as follows:

    AH protocol (Authentication Header - authentication header). A secure protocol that provides authentication and (optional) replay detection service.

    The AH protocol acts as a digital signature and ensures that the data in the IP packet is not tampered with. The AH protocol does not provide data encryption and decryption service. This protocol can be used either independently or in conjunction with the ESP protocol. ESP (Encapsulating Security Payload) protocol.

    A security protocol that provides confidentiality and data protection, as well as (optional) authentication and replay detection services. Cisco IPSec-enabled products use ESP to encrypt the payload of IP packets. The ESP protocol can be used independently or in conjunction with AH.

    DES standard (Data Encription Standard - data encryption standard). A variant of DES that uses three iterations of standard DES with three different keys, essentially tripling the strength of DES. The 3DES algorithm is used within IPSec to encrypt and decrypt the data stream. This algorithm uses a 168-bit key, which guarantees high encryption reliability. IKE and IPSec use the 3DES algorithm to encrypt messages.

    AES(advanced encryption standard).

The AES protocol uses the Rine Dale4 encryption algorithm, which provides significantly stronger encryption.

    Many cryptographers believe that AES is generally unbreakable. AES is now a federal information processing standard. It is defined as an encryption algorithm for use by US government organizations to protect sensitive but unclassified information. The problem with AES is that it requires more computing power to implement than similar protocols. IPSec conversion also uses two standard hashing algorithms to provide data authentication.

    MD5 algorithm (Message Digest 5). A hashing algorithm used to authenticate data packets. Cisco products use an MD5-calculated HMAC (Hashed Message Authentication Code), a variant of the message authentication code that is further protected by hashing. Hashing is a one-way (i.e., irreversible) encryption process that produces a fixed-length output for an input message of arbitrary length. IKE, AH and ESP use MD5 for data authentication.

Under the IKE protocol, symmetric keys are created using the Diffie-Hellman algorithm using DES, 3DES, MD5 and SHA. The Diffie-Hellman protocol is a cryptographic protocol based on the use of public keys. It allows two parties to agree on a shared secret key without having a sufficiently secure communication channel. Shared secret keys are required for the DES and NMAC algorithms. The Diffie-Hellman algorithm is used within IKE to generate session keys. Diffie-Hellman (DH) groups – define the “strength” of the encryption key that is used in the key exchange procedure. The higher the group number, the “stronger” and safer the key. However, one should take into account the fact that as the DH group number increases, the “strength” and security level of the key increases, but at the same time the load on the central processor increases, since generating a “stronger” key requires more time and resources.

WatchGuard devices support DH groups 1, 2 and 5:

    DH group 1: 768-bit key

    DH group 2: 1024-bit key

    DH group 5: 1536-bit key

Both devices that communicate via VPN must use the same DH group. The DH group that will be used by the devices is selected during the IPSec Phase 1 procedure.

Views: 8033

0 Let's look at the details of the technologies that make up IPSec. The standards used within IPSec are quite complex to understand, so in this section we will look at each of the components of IPSec in detail. To understand what IPSEC is, use the document “IPSEC as a network traffic security protocol” published earlier on this site. This article is a continuation of the above document.

IPSec uses the following technologies:

  • NA protocol;
  • ESP protocol;
  • DES encryption standard;
  • 3DES encryption standard;
  • IKE protocol;
  • Diffie-Hellman key agreement method;
  • hashed message authenticity codes (HMAC);
  • RSA protection;
  • certification centers.

NA protocol

This protocol provides authentication and data integrity for IP packets sent between two systems. The NA protocol is not
provides confidentiality (i.e. encryption) of packets. Authentication is accomplished by applying a one-way, key-dependent hashing function to the packet, generating a message "profile". A change to any part of the packet along the transmission path will be detected by the recipient by applying a similar one-way hashing function to the received data and comparing the calculated message profile value with the one specified by the sender. The authenticity of the information received is guaranteed by the fact that both systems use the same secret key for one-way hashing. The operation diagram of the AN protocol is shown below. The following actions are performed.

  1. The IP header and payload of the packet are hashed.
  2. The resulting hash code is used to construct a new AH header, which is attached to the original packet between the header and the payload block.
  3. The new packet is sent to the second IPSec party.
  4. The receiving side calculates the hash code value for the IP header and payload, extracts the transmitted hash code value from the AH header, and compares the two values. The corresponding hash code values ​​must match exactly. If even one bit of the packet changes along the way, the packet hash code calculated by the recipient will not match the value specified in the AH header.
The AH protocol provides authentication for as many IP header fields as possible, as well as for the data fields of higher-layer protocols. However, some IP header fields may change in transit. The values ​​of mutable fields (for example, the TTL field indicating the lifetime of a packet) are changed by intermediate network devices through which the packet passes, and such changes cannot be predicted by the sender. The values ​​of mutable fields should not be protected by the AH protocol. Thus, the protection that AH provides to the IP header is somewhat limited. The AH protocol can also additionally provide packet replay protection by specifying a packet sequence number in the IP header. A complete description of the AH protocol is contained in RFC 2402.

ESP protocol

ESP is a security protocol that provides confidentiality (i.e. encryption), source authentication and data integrity, as well as (optional) anti-replay service and limited traffic confidentiality by resisting attempts to analyze the data stream.

The ESP protocol provides privacy through encryption at the IP packet level. At the same time, many symmetric encryption scheme algorithms are supported. The default algorithm for IPSec is DES with a 56-bit key. This cipher must be present to ensure compatibility between all IPSec-capable products. Cisco products also support the 3DES algorithm, which provides stronger encryption. Privacy can be selected independently of other services.

Data origin authentication and connectionless integrity support are used together and are optional (i.e., optional). These features can also be combined with a privacy service.
The replay protection service can only be selected if data origin authentication is selected, and the choice of this service is the sole prerogative of the recipient. Although by default the sender is required to automatically increment the sequence number used for replay protection, this service is only effective if the recipient checks the sequence number. Traffic confidentiality requires choosing a tunnel mode. This is most effective in a security gateway, where source-destination masquerading can be performed on all traffic at once. It should be noted here that although both privacy and authentication are options, at least one of these services must be selected.
The services provided by ESP depend on the parameters that are specified in the IPSec configuration and selected when creating the IPSec security association. However, choosing confidentiality without integrity/authentication (either within ESP or separately using NA) leaves the adversary open to certain types of attacks that may limit the usefulness of the confidentiality service used in this way.
The ESP header is inserted into the packet after the IP header, before the higher-layer protocol header (in transport mode) or before the encapsulated IP header (in tunnel mode). A complete description of the ESP protocol is contained in RFC 2406.

ESP encryption using NMAC

ESP may also provide packet authentication using an optional authentication field. In Cisco IOS software and PIX Firewalls, this service is called ESP HMAC. Authentication values ​​are calculated after encryption is performed. The IPSec standard in use today describes the SHA1 and MD5 algorithms as mandatory for NMAC.
The main difference between ESP authentication and NA authentication is their scope. ESP does not protect any IP header fields unless ESP encapsulation is intended (tunnel mode). The figure shows which fields are protected when using ESP NMAC.


Note that encryption only covers the payload data, while ESP with NMAC ESP hashing covers the ESP header and payload data. The IP header is not protected. The ESP NMAC service cannot be used independently, but must be combined with the ESP encryption protocol.

IPSec tunnel and transport modes

IPSec operates in either tunnel or transport mode. Figure shows the implementation diagram of the tunnel mode. In this mode, the entire original IP datagram is encrypted and becomes the payload in a new IP packet with a new IP header and an additional IPSec header (abbreviated HDR in the figure). Tunnel mode allows a network device (such as the PIX Firewall) to act as an IPSec gateway or proxy server that performs encryption for hosts behind the firewall. The source router encrypts the packet and forwards it over the IPSec tunnel. The destination PIX Firewall decrypts the received IPSec packet, extracts the original IP datagram, and forwards it to the destination system. The main advantage of tunnel mode is that end systems do not need to be modified to enable them to use IPSec. Tunnel mode also prevents an adversary from analyzing the data stream. When exchanging in tunnel mode, an adversary is able to determine only the endpoints of the tunnel, but not the true source and destination of packets passing through the tunnel, even if the endpoints of the tunnel are located on the source and destination systems.


The diagram in Fig. below illustrates the transport mode. Here, only the IP payload is encrypted and the original IP header remains intact.
The IPSec header is added. The advantage of this mode is that it only adds a few bytes to each packet. In addition, open network devices can see the true source and destination addresses of a packet.


This allows special capabilities of intermediate networks (such as guaranteed quality of service) to be exploited based on information in the IP header. However, the Layer 4 header is encrypted, which limits the ability to analyze the packet. Unfortunately, transmitting the IP header in clear text in transport mode allows an attacker to perform some degree of data flow analysis. For example, an attacker can find out how many packets were transmitted by IPSec parties operating in transport mode. But the attacker can only know that IP packets were forwarded. It will not be able to determine whether they were an email message or some other attachment if the ESP protocol was used.

Using tunnel and transport modes

Let's look at a few examples illustrating the rules for choosing a tunnel or transport mode. The figure below shows situations in which tunnel mode is used. This mode is most often used to encrypt data flow between IPSec security gateways - for example, between a Cisco router and a PIX Firewall. IPSec gateways perform IPSec functions for devices located behind such gateways (in the above figure, this is Alice’s personal computer and HR servers). In this example, Alice gains secure access to the HR servers through an IPSec tunnel established between the gateways.

Tunnel mode is also used to communicate between endpoints running IPSec software, such as between a CiscoSecure VPN client and an IPSec gateway.
In this example, tunnel mode is used to create an IPSec tunnel between a Cisco router and a server running IPSec software. Note that in Cisco IOS software and the PIX Firewall, tunnel mode is the default mode for IPSec communications.
Transport mode is used between endpoints that support IPSec, or between an endpoint and a gateway if the gateway is interpreted as a host. In Fig. Below is Example D, which illustrates the use of transport mode to create an encrypted IPSec tunnel from Alice's computer running Microsoft Windows 2000 client software to a Cisco VPN 3000 concentrator, allowing Alice to use an L2TP tunnel over IPSec.

Using AH and ESP

In certain situations, the problem of choosing between AN and ESP may seem difficult to solve, but it can be simplified by following a few rules. If you need to know that data from an identified source is transmitted without violating its integrity and does not need to ensure its confidentiality, use the AH protocol, which protects higher-layer protocols and IP header fields that do not change in transit. Security means that the corresponding values ​​cannot be changed because this will be detected by the second IPSec party and any modified IP datagram will be rejected. The AH protocol does not provide protection against eavesdropping on the channel and viewing the header and data by an intruder. But since the header and data cannot be changed undetectably, the modified packets are rejected.

If you need to keep data secret (ensure confidentiality), use ESP. This protocol encrypts higher-layer protocols in transport mode and the entire original IP datagram in tunnel mode, so that it is impossible to extract information about packets by sniffing the transmission channel. The ESP protocol can also provide an authentication service for packets. However, when using ESP in transport mode, the outer original IP header is not protected, and in tunnel mode, the new IP header is not protected. When using IPSec, users are more likely to use tunnel mode than transport mode.

Let's look at the architecture of the IPSec protocol family. The purpose of this family of protocols is to provide various IP-level security services for IPv4 and IPv6 protocols. Let's look at the security services provided by IPSec protocols and the use of these protocols in TCP/IP networks.

When these services are installed correctly, they do not interfere with users, hosts, or other Internet components that do not use these security services to protect their traffic. These services are algorithm-independent. This means that new cryptographic algorithms can be added without changing the protocols themselves. For example, different user groups may use different sets of algorithms.

A standard set of default algorithms has been defined to ensure interoperability across the Internet. Using these algorithms in conjunction with the traffic protection provided by IPSec and key management protocols will allow the system and application developer to provide a high degree of cryptographic security.

IPSec can be implemented either in the OS or in the router or firewall.

IPSec provides confidentiality, data integrity, access control and data source authentication for IP datagrams. These services are provided by maintaining state between the source and destination of IP datagrams. This state defines the specific datagram-level security services, the cryptographic algorithms used for the services provided, and the keys for those algorithms.

We list the main tasks of the IPSec protocols:

  1. Providing cryptographic protection at the IP level for the IPv4 and IPv6 protocols, namely ensuring confidentiality and integrity of data and the integrity of a certain sequence of datagrams.
  2. Provides transparency for IP traffic that does not require IPSec protocols.
  3. Ensuring extensibility, i.e. the ability to add new sets of algorithms without changing the protocol itself.

IPSec is designed for secure communication using cryptography for the IPv4 and IPv6 protocols. Security services include access control, integrity and confidentiality data and protection against replay attacks, which is ensured by guaranteeing the integrity of a certain sequence of datagrams. These services are provided at the IP layer, providing security for the IP protocol and higher-level protocols.

IPSec supports two forms of integrity: data integrity and the integrity of a particular sequence of datagrams. Data integrity detects modification of a specific IP datagram, regardless of the sequence of datagrams in the traffic stream. Datagram sequence integrity is an anti-reply service that detects the receipt of duplicate IP datagrams. This is different from connection integrity, which has more stringent traffic integrity requirements, namely the ability to detect lost or reordered messages.

Let's consider the implementation of IPSec protocols, the main components of the system and their interaction to provide security services.

IPSec runs on the host (Host – H) or security gateway (Security Gateway – SG), providing protection for IP traffic. The term security gateway is used to refer to a router that implements IPsec protocols.

Protection is based on requirements defined in the Security Policy Database (SPD), set and maintained by the administrator. In general, packets are processed in one of three ways, based on IP header and transport layer information as specified in the SPD. Each packet is either discarded, passed without processing, or processed according to the SPD entry for that packet.

Possible ways to implement IPSec

There are several ways to implement IPSec on the host or in conjunction with a router or firewall (to create a security gateway).

  1. Integration of IPSec into a specific implementation of the IP protocol. This requires access to the IP source code and is done on both hosts and security gateways.
  2. "Bump-in-the-stack" (BITS) implementations are where IPSec is implemented "at the bottom" of an existing IP protocol stack implementation, nesting its implementation between the standard IP protocol implementation and local network drivers. In this case, access to the IP stack source code is not required. This approach is usually implemented on hosts when IPSec is implemented as a plug-in library.
  3. Using an external crypto processor. This is usually called a "Bump-in-the-wire" (BITW) implementation. Such implementations can be used on both hosts and gateways. Typically BITW devices are IP addressable.

Traffic security protocols and the concept of secure association

The traffic protection services provided by IPSec are implemented using two secure traffic protocols: Authentication Header (AH) and Encapsulating Security Payload (ESP).

The following protocols are defined to protect traffic in IPSec:

  1. The Encapsulating Security Payload (ESP) protocol ensures the confidentiality and integrity of protocols located higher in the protocol stack and can additionally provide anti-replay service, i.e. the integrity of a certain sequence of datagrams.
  2. The Authentication Header (AH) protocol ensures the integrity of protocols located higher in the protocol stack and the integrity of individual IP header fields, which do not change when sent from sender to recipient, and can additionally provide anti-replay service, i.e. the integrity of a certain sequence of datagrams. In IPSec v2, implementation of this protocol is optional.
  3. The parameters of these protocols are defined in the Internet Key Exchange (IKE) key distribution protocol.

Associated with traffic that is secured by IPSec is the concept of a Security Association (SA). The SA contains all the information needed to perform various network security services.

SA is a simplex (unidirectional) logical connection, created between two endpoints that use one of the IPSec protocols for security. ESP and AH transmit traffic over SA. All traffic carried over SA is processed in accordance with the security policy specified at the ends of the connection.

We will describe various aspects of SA management, determine possible ways to manage security policy, methods of processing traffic and managing SA.

The SA defines the security service parameters that are applied to the traffic. Typically, a bidirectional connection between two hosts or between two Security Gateways requires two SAs (one for each direction).

We will consider SA only for unicast connections.

Two SA modes are defined: transport mode and tunneling mode. Transport mode used to create a VPN between two hosts. In IPv4, the Transport Mode Security Protocol header appears immediately after the IP header. In the ESP protocol, the SA transport mode provides security services only for higher-level protocols, but not for the IP header. In the case of AH, protection also extends to individual parts of the IP header.

Another SA mode is tunneling mode. If one end of the connection is a security gateway, then IPSec SA standards must be performed in tunnel mode, but many manufacturers allow both tunnel and transport modes in this case. Note that when traffic is destined for a Security Gateway, such as in the case of ping or SNMP commands, the Security Gateway is treated as a host and is typically used transport mode. Two hosts can install if necessary tunnel mode.

In tunnel mode, an external IP header is added, the addresses of which are security gateways. The internal IP header points to the end hosts. The security protocol header is located after the outer IP header and before the inner IP header. If AH is used in tunnel mode, parts of the outer IP header are protected, as is the entire tunneled IP packet, i.e. all internal headers are protected, as are all higher-level protocols. If ESP is used, protection is provided only for the tunneled packet and not for the outer header.

Let's briefly summarize:

  1. Host can support both modes, both transport and tunnel.
  2. Security Gateway As a rule, it uses only tunnel mode. If it supports transport mode, then this mode is typically used only when the non-threat gateway is the recipient of traffic, for example, for network management.

Set of marketable