IPSec relies on a number of technologies and encryption methods, but IPSec can generally be thought of as the following main steps:

    Step 1. Starting the IPSec process. Traffic that requires encryption according to the IPSec security policy agreed upon by the IPSec parties begins the IKE process.

    Step 2. IKE phase one. The IKE process authenticates IPSec parties and negotiates IKE security association parameters, resulting in a secure channel for negotiating IPSec security association parameters during the second phase of IKE.

    Step 3. IKE phase two. The IKE process negotiates IPSec security association parameters and establishes appropriate IPSec security associations for communicating party devices.

    Step 4. Data transfer. Communication occurs between communicating IPSec parties based on IPSec parameters and keys stored in the security association database.

    Step 5. Terminating an IPSec tunnel. IPSec security associations terminate either because they are deleted or because their lifetime limit has been exceeded.

IPSec operating modes

There are two modes of IPSec operation: transport and tunnel.

In transport mode, only the information part of the IP packet is encrypted. Routing is not affected since the IP packet header is not changed. Transport mode is typically used to establish connections between hosts.

In tunnel mode, the entire IP packet is encrypted. In order for it to be transmitted over the network, it is placed in another IP packet. This creates a secure IP tunnel. Tunnel mode can be used to connect remote computers to a virtual private network or to organize secure data transfer through open channels communications (Internet) between gateways to connect different parts of the virtual private network.

IPSec Transform Negotiation

The IKE protocol negotiates IPSec transformations (IPSec security algorithms). IPSec transformations and their associated encryption algorithms are as follows:

    AH protocol (Authentication Header - authentication header). A secure protocol that provides authentication and (optional) replay detection service. The AH protocol acts as a digital signature and ensures that the data in the IP packet is not tampered with. The AH protocol does not provide data encryption and decryption service. This protocol can be used either independently or in conjunction with the ESP protocol.

    ESP (Encapsulating Security Payload) protocol. A security protocol that provides confidentiality and data protection, as well as (optional) authentication and replay detection services. Cisco IPSec-enabled products use ESP to encrypt the payload of IP packets. The ESP protocol can be used independently or in conjunction with AH.

    DES standard (Data Encription Standard - data encryption standard). Algorithm for encrypting and decrypting packet data. The DES algorithm is used within both IPSec and IKE. The DES algorithm uses a 56-bit key, which means not only higher consumption of computing resources, but also stronger encryption. The DES algorithm is symmetric algorithm encryption, which requires identical encryption secret keys in the devices of each communicating IPSec party. The Diffie-Hellman algorithm is used to create symmetric keys. IKE and IPSec use the DES algorithm to encrypt messages.

    "Triple" DES (3DES). A variant of DES that uses three iterations of standard DES with three different keys, essentially tripling the strength of DES. The 3DES algorithm is used within IPSec to encrypt and decrypt the data stream. This algorithm uses a 168-bit key, which guarantees high encryption reliability. IKE and IPSec use the 3DES algorithm to encrypt messages.

    AES(advanced encryption standard). The AES protocol uses the Rine Dale4 encryption algorithm, which provides significantly stronger encryption. Many cryptographers believe that AES is generally unbreakable. AES is now a federal information processing standard. It is defined as an encryption algorithm for use by US government organizations to protect sensitive but unclassified information. The problem with AES is that it requires more computing power to implement than similar protocols.

IPSec conversion also uses two standard hashing algorithms to provide data authentication.

    MD5 algorithm (Message Digest 5). A hashing algorithm used to authenticate data packets. Cisco products use an MD5-calculated HMAC (Hashed Message Authentication Code), a variant of the message authentication code that is further protected by hashing. Hashing is a one-way (i.e., irreversible) encryption process that produces a fixed-length output for an input message of arbitrary length. IKE, AH and ESP use MD5 for data authentication.

    Algorithm SHA-1 (Secure Hash Algorithm-1 - secure hashing algorithm 1). A hashing algorithm used to authenticate data packets. Cisco products use a variant of the HMAC code that is calculated using SHA-1. IKE, AH and ESP use SHA-1 for data authentication.

Under the IKE protocol, symmetric keys are created using the Diffie-Hellman algorithm using DES, 3DES, MD5 and SHA. The Diffie-Hellman protocol is a cryptographic protocol based on the application public keys. It allows two parties to agree on a shared secret key without having a sufficiently secure communication channel. Shared secret keys are required for the DES and NMAC algorithms. The Diffie-Hellman algorithm is used within IKE to generate session keys. Diffie-Hellman (DH) groups – define the “strength” of the encryption key that is used in the key exchange procedure. The higher the group number, the “stronger” and safer the key. However, one should take into account the fact that as the DH group number increases, the “strength” and security level of the key increases, but at the same time the load on the central processor increases, since generating a “stronger” key requires more time and resources.

WatchGuard devices support DH groups 1, 2 and 5:

    DH group 1: 768-bit key

    DH group 2: 1024-bit key

    DH group 5: 1536-bit key

Both devices that communicate via VPN must use the same DH group. The DH group that will be used by the devices is selected during the IPSec Phase 1 procedure.

In the late sixties, the American Defense Advanced Research Projects Agency DARPA decided to create an experimental network called ARPANet. In the seventies, ARPANet began to be considered a functioning US network, and through this network it was possible to gain access to leading US university and research centers. In the early eighties, standardization of programming languages ​​began, and then network interaction protocols. The result of this work was the development of a seven-layer ISO/OSI network interaction model and the TCP/IP protocol family, which became the basis for the construction of both local and global networks.

Basic mechanisms information exchange in TCP/IP networks were generally formed in the early eighties, and were aimed primarily at ensuring the delivery of data packets between different operating systems using heterogeneous communication channels. Despite the fact that the idea of ​​​​creating the ARPANet network (later turned into modern Internet) belonged to a government defense organization, in fact the network originated in the research world, and inherited the tradition of openness of the academic community. Even before the commercialization of the Internet (which occurred in the mid-nineties), many reputable researchers noted problems associated with the security of the TCP/IP protocol stack. The basic concepts of the TCP/IP protocols do not fully satisfy (and in some cases contradict) modern ideas about computer security.

Until recently, the Internet was used mainly for processing information using relatively simple protocols: email, file transfer, remote access. Today, thanks to the widespread use of WWW technologies, distributed multimedia information processing tools are increasingly being used. At the same time, the volume of data processed in client/server environments and intended for simultaneous collective access by a large number of subscribers is growing. Several application level protocols have been developed to provide information security applications such as email (PEM, PGP, etc.), WWW (Secure HTTP, SSL, etc.), network management (SNMPv2, etc.). However, the presence of security features in the basic protocols of the TCP/IP family will allow information exchange between a wide range of different applications and services.

Brief historical background of the appearance of the protocol

In 1994, the Internet Architecture Board (IAB) released the report "Security of the Internet Architecture." This document described the main applications additional funds security on the Internet, namely protection against unauthorized monitoring, packet spoofing and data flow control. Among the first and most important protective measures was the need to develop a concept and basic mechanisms to ensure the integrity and confidentiality of data flows. Since changing the basic protocols of the TCP/IP family would cause a complete restructuring of the Internet, the task was set to ensure the security of information exchange in open telecommunication networks based on existing protocols. Thus, the Secure IP specification began to be created, complementary to the IPv4 and IPv6 protocols.

IPSec architecture

IP Security is a set of protocols dealing with issues of encryption, authentication and security during the transport of IP packets; it now includes nearly 20 standards proposals and 18 RFCs.

The IP Security specification (known today as IPsec) is developed by the IETF IP Security Protocol Working Group. IPsec originally included 3 algorithm-independent core specifications, published as RFCs: IP Security Architecture, Authentication Header (AH), Encapsulation of Encrypted Data (ESP) (RFC1825, 1826, and 1827). It should be noted that in November 1998, the IP Security Protocol Working Group proposed new versions of these specifications, which currently have the status of preliminary standards, these are RFC2401 - RFC2412. Note that RFC1825-27 has been considered obsolete for several years and is not really used. In addition, there are several algorithm-dependent specifications using the MD5, SHA, and DES protocols.

Rice. 1 – IPSec architecture.

The IP Security Protocol Working Group is also developing key information management protocols. This group's mission is to develop the Internet Key Management Protocol (IKMP), an application-level key management protocol that is independent of the security protocols used. Key management concepts using the specification are currently being explored Internet Security Association and Key Management Protocol (ISAKMP) and Oakley Key Determination Protocol. The ISAKMP specification describes mechanisms for negotiating the attributes of the protocols used, while the Oakley protocol allows you to install session keys on computers on the Internet. Previously, the possibilities of using key management mechanisms of the SKIP protocol were also considered, but now such possibilities are practically not used anywhere. Emerging key information management standards may support Key Distribution Centers similar to those used in Kerberos. Kerberos-based key management protocols for IPSec are currently being addressed by the relatively new KINK (Kerberized Internet Negotiation of Keys) working group.

Data integrity and confidentiality guarantees in the IPsec specification are provided through the use of authentication and encryption mechanisms, respectively. The latter, in turn, are based on the preliminary agreement of the parties to the so-called information exchange. “security context” – applied cryptographic algorithms, key information management algorithms and their parameters. The IPsec specification provides for the ability for parties to exchange information to support various protocols and parameters for authentication and encryption of data packets, as well as various key distribution schemes. In this case, the result of negotiating the security context is the establishment of a security parameter index (SPI), which is a pointer to a specific element internal structure side of the information exchange describing possible sets of security parameters.

Essentially, IPSec, which will become integral part IPv6 operates at the third level, i.e. at the network level. As a result, transmitted IP packets will be protected in a manner transparent to network applications and infrastructure. Unlike SSL (Secure Socket Layer), which operates at layer 4 (i.e., transport) and is more closely associated with higher layers of the OSI model, IPSec is designed to provide low-level security.


Rice. 2 - OSI/ISO model.

IPSec adds a header to IP data ready to be sent over a virtual private network to identify the protected packets. Before being transmitted over the Internet, these packets are encapsulated within other IP packets. IPSec supports several encryption types, including Data Encryption Standard (DES) and Message Digest 5 (MD5).

To establish a secure connection, both participants in the session must be able to quickly agree on security parameters, such as authentication algorithms and keys. IPSec supports two types of key management schemes through which participants can negotiate session parameters. This dual support at one time caused some friction in the IETF Working Group.

With the current version of IP, IPv4, either Internet Secure Association Key Management Protocol (ISAKMP) or Simple Key Management for Internet Protocol can be used. WITH new version IP, IPv6, will have to use ISAKMP, now known as IKE, although the possibility of using SKIP is not excluded. However, it should be kept in mind that SKIP has not been considered as a key management candidate for a long time, and was removed from the list of possible candidates back in 1997.

Header AH

The Authentication Header (AH) is a common optional header and is typically located between the IP packet's main header and the data field. The presence of AH does not in any way affect the process of transmitting information from transport and higher levels. The main and only purpose of AH is to provide protection against attacks related to unauthorized changes in the contents of the packet, including spoofing the source address network layer. Protocols more high level must be modified in order to verify the authenticity of the received data.

The AH format is quite simple and consists of a 96-bit header and variable length data consisting of 32-bit words. The field names reflect their contents fairly clearly: Next Header indicates the next header, Payload Len represents the packet length, SPI is a pointer to the security context, and Sequence Number Field contains the packet's sequence number.


Rice. 3 - AH header format.

The packet sequence number was introduced into AH in 1997 as part of the IPsec specification revision process. The value of this field is generated by the sender and serves to protect against attacks related to reuse authentication process data. Because the Internet does not guarantee the order in which packets will be delivered, the recipient must store information about the maximum sequence number of a successfully authenticated packet and whether a number of packets containing previous sequence numbers have been received (usually 64).

Unlike checksum calculation algorithms used in protocols for transmitting information over switched communication lines or over local network channels and aimed at correcting random errors in the transmission medium, mechanisms for ensuring data integrity in open telecommunication networks must have means of protection against targeted changes. One such mechanism is special application MD5 algorithm: in the process of forming AH, the hash function is sequentially calculated from the combination of the packet itself and some pre-agreed key, and then from the combination of the obtained result and the transformed key. This mechanism is the default to ensure that all IPv6 implementations have at least one common algorithm that is not subject to export restrictions.

ESP header

When encrypted data encapsulation is used, the ESP header is the last of the optional headers "visible" in the packet. Because the primary purpose of ESP is to ensure data confidentiality, different types of information may require significantly different encryption algorithms. Consequently, the ESP format can undergo significant changes depending on the cryptographic algorithms used. However, the following mandatory fields can be distinguished: SPI, indicating the security context, and Sequence Number Field, containing the sequence number of the packet. Field "ESP Authentication Data" ( check sum), is optional in the ESP header. The recipient of the ESP packet decrypts the ESP header and uses the parameters and data of the applied encryption algorithm to decode the transport layer information.


Rice. 4 - ESP header format.

There are two modes of using ESP and AH (as well as their combinations) - transport and tunnel.

Transport mode

Transport mode is used to encrypt the data field of an IP packet containing transport layer protocols (TCP, UDP, ICMP), which, in turn, contains application service information. An example of a transport mode application is the transmission Email. All intermediate nodes along a packet's route from sender to receiver use only the public network layer information and possibly some optional packet headers (in IPv6). The disadvantage of transport mode is the lack of mechanisms for hiding the specific sender and recipient of a packet, as well as the ability to analyze traffic. The result of such an analysis can be information about the volumes and directions of information transfer, areas of interest of subscribers, and the location of managers.

Tunnel mode

Tunnel mode encrypts the entire packet, including the network layer header. The tunnel mode is used if it is necessary to hide the organization’s information exchange with the outside world. In this case, the address fields of the network layer header of a packet using tunnel mode, are populated by the organization's firewall and do not contain information about the specific sender of the packet. When transmitting information from outside world V local network organization uses the network address of the firewall as the destination address. After the firewall decrypts the initial network layer header, the packet is forwarded to the recipient.

Security Associations

A Security Association (SA) is a connection that provides security services for traffic that passes through it. Two computers on each side of the SA store the mode, protocol, algorithms, and keys used in the SA. Each SA is used in only one direction. Bidirectional communication requires two SAs. Each SA implements one mode and protocol; thus, if two protocols need to be used for one packet (such as AH and ESP), then two SAs are required.

Security policy

The security policy is stored in the SPD (Security Policy Database). SPD can specify one of three actions for a data packet: discard the packet, do not process the packet using IPSec, or process the packet using IPSec. In the latter case, the SPD also indicates which SA should be used (if, of course, a suitable SA has already been created) or indicates with what parameters a new SA should be created.

SPD is a very flexible control mechanism that allows very good management processing each packet. Packets are classified by a large number of fields, and the SPD may examine some or all of the fields to determine the appropriate action. This could result in all traffic between two machines being carried using a single SA, or separate SAs being used for each application, or even for each TCP connection.

ISAKMP/Oakley

ISAKMP defines the general structure of protocols that are used to establish SAs and perform other key management functions. ISAKMP supports several Domains of Interpretation (DOI), one of which is IPSec-DOI. ISAKMP does not define a complete protocol, but rather provides "building blocks" for various DOIs and key exchange protocols.

The Oakley protocol is a key discovery protocol that uses the Diffie-Hellman key replacement algorithm. The Oakley protocol supports Perfect Forward Secrecy (PFS). The presence of PFS means that it is impossible to decrypt all traffic if any key in the system is compromised.

IKE

IKE is the default key exchange protocol for ISAKMP. this moment being the only one. IKE sits on top of ISAKMP and does the actual establishment of both the ISAKMP SA and the IPSec SA. IKE supports a set of various primitive functions for use in protocols. Among them are the hash function and the pseudo-random function (PRF).

A hash function is a collision-resistant function. Collision resistance refers to the fact that it is impossible to find two different messages m 1 And m 2, such that H(m 1)=H(m2), Where H— hash function.

As for pseudo-random functions, a hash function is currently used instead of special PRFs in the HMAC design (HMAC is a message authentication mechanism using hash functions). To define HMAC, we need a cryptographic hash function (let's call it H) and a secret key K. We assume that H is a hash function where data is hashed using a compression procedure applied sequentially to a sequence of data blocks. We denote by B the length of such blocks in bytes, and the length of blocks obtained as a result of hashing by L (L

Ipad = byte 0x36, repeated B times;
opad = byte 0x5C repeated B times.

To calculate HMAC from "text" data, you need to perform the following operation:

H(K XOR opad, H(K XOR ipad, text))

From the description it follows that IKE uses HASH values ​​to authenticate parties. Note that HASH in this case refers solely to the Payload name in ISAKMP, and this name has nothing to do with its contents.

Attacks on AH, ESP and IKE.

All types of attacks on IPSec components can be divided into the following groups: attacks that exploit the finite resources of the system (a typical example is a Denial of Service attack, Denial-of-service or DOS attack), attacks that exploit the features and errors of a specific IPSec implementation and and finally, attacks based on the weaknesses of the protocols themselves. AH and ESP. Purely cryptographic attacks can be ignored - both protocols define the concept of “transform”, where all cryptography is hidden. If the crypto-algorithm used is stable, and the transform defined with it does not introduce additional weaknesses (this is not always the case, therefore it is more correct to consider the strength of the entire system - Protocol-Transform-Algorithm), then from this side everything is fine. What remains? Replay Attack - leveled out by using Sequence Number (in one single case this does not work - when using ESP without authentication and without AH). Further, the order of actions (first encryption, then authentication) guarantees quick rejection of “bad” packets (moreover, according to recent research in the world of cryptography, this order of actions is the most secure; the reverse order in some, albeit very special cases, can lead to potential security holes; fortunately, neither SSL, nor IKE, nor other common “authenticate first, encrypt later” protocols apply to these special cases, and therefore do not have these holes). What remains is the Denial-Of-Service attack. As you know, this is an attack from which there is no complete defense. However, the quick rejection of bad packets and the absence of any external reaction to them (according to the RFC) make it possible to deal with this attack more or less well. In principle, most (if not all) known network attacks (sniffing, spoofing, hijacking, etc.) are successfully resisted by AH and ESP when used correctly. With IKE it's a little more complicated. The protocol is very complex and difficult to analyze. In addition, due to typos (in the formula for calculating HASH_R) when writing it and not entirely successful solutions (the same HASH_R and HASH_I), it contains several potential “holes” (in particular, in the first phase, not all Payloads in the message are authenticated), however , they are not very serious and lead, at most, to a refusal to establish a connection. IKE more or less successfully protects itself from attacks such as replay, spoofing, sniffing, hijacking. Cryptography is somewhat more complicated - it is not carried out separately, as in AH and ESP, but is implemented in the protocol itself. However, if you use persistent algorithms and primitives (PRF), there should be no problems. To some extent, it can be considered as a weakness of IPsec that DES is indicated as the only mandatory cryptographic algorithm in the current specifications (this is true for both ESP and IKE), 56 bits of the key of which are no longer considered sufficient. However, this is a purely formal weakness - the specifications themselves are algorithm-independent, and almost all well-known vendors have long implemented 3DES (and some have already implemented AES). Thus, if implemented correctly, the most “dangerous” attack remains Denial-Of-Service .

Protocol evaluation

The IPSec protocol has received mixed reviews from experts. On the one hand, it is noted that the IPSec protocol is the best among all other previously developed protocols for protecting data transmitted over the network (including PPTP developed by Microsoft). According to the other side, there is excessive complexity and redundancy of the protocol. Thus, Niels Ferguson and Bruce Schneier in their work "A Cryptographic Evaluation of IPsec" note that they found serious security problems in almost all major components of IPsec. These authors also note that the protocol suite requires significant improvements in order for it to provide a good level of security. The paper describes a number of attacks that exploit both the weaknesses of the general data processing scheme and the weaknesses of cryptographic algorithms.

Conclusion

In this article, we covered some basic points regarding the IPsec network security protocol. It is worth noting that the IPsec protocol dominates most virtual private network implementations. Currently, the market offers both software implementations (for example, the protocol is implemented in the Microsoft Windows 2000 operating system) and hardware and software implementations of IPsec - these are solutions from Cisco, Nokia. Despite the large number of different solutions, they are all quite compatible with each other. The article concludes with a table comparing IPSec and the now widely used SSL.

Peculiarities IPSec SSL
Hardware independence Yes Yes
Code No application changes required. May require access to the TCP/IP stack source code. Application changes required. New DLLs or access to application source code may be required.
Protection Entire IP packet. Enables protection for higher-level protocols. Application level only.
Packet filtering Based on authenticated headers, sender and recipient addresses, etc. Simple and cheap. Suitable for routers. Based on high-level content and semantics. More intelligent and more complex.
Performance Fewer context switches and data movement. More context switches and data movement. Large blocks of data can speed up cryptographic operations and provide better compression.
Platforms Any systems, including routers Mainly end systems (clients/servers), also firewalls.
Firewall/VPN All traffic is protected. Only application level traffic is protected. ICMP, RSVP, QoS, etc. may be unprotected.
Transparency For users and applications. Only for users.
Current status Emerging standard. Widely used by WWW browsers, also used by several other products.

Links

  • www.ietf.org/html.charters/ipsec-charter.html - Home page of the IETF working group. There are also links to RFCs and proposals for standards.
  • www.microsoft.com/rus/windows2000/library/security/w2k_IPSecurity.asp – Information about the implementation of the IPSec protocol in Windows2000 Server.

Acknowledgments

In contact with

Classmates

In the modern world, various VPN technologies are used everywhere. Some (for example, PPTP) are recognized as unsafe over time and gradually die out, others (OpenVPN), on the contrary, are increasing their momentum every year. But the undisputed leader and most recognizable technology for creating and maintaining secure private channels is still IPsec VPN. Sometimes during a pentest you can find a seriously protected network with only the five hundredth UDP port sticking out. Everything else can be closed, patched and reliably filtered. In such a situation, the thought may arise that there is nothing special to do here. But it is not always the case. In addition, there is a widespread idea that IPsec, even in default configurations, is impregnable and provides the proper level of security. This is exactly the situation we will look at in practice today. But first, in order to combat IPsec as effectively as possible, you need to understand what it is and how it works. This is what we'll do!

IPsec from the inside

Before moving directly to IPsec itself, it would be nice to remember what types of VPNs there are. There are a great many classifications of VPNs, but we will not dive deeply into network technologies and will take the simplest one. Therefore, we will divide VPN into two main types - site-to-site VPN connections (they can also be called permanent) and remote access VPN (RA, also temporary).
The first type serves for constant communication of various network islands, for example, a central office with many scattered branches. Well, RA VPN is a scenario when a client connects for a short period of time, gains access to certain network resources, and then safely disconnects after completing the work.

We will be interested in the second option, since in the event of a successful attack, it is possible to immediately gain access to the internal network of the enterprise, which is quite a serious achievement for a pentester. IPsec, in turn, allows you to implement both site-to-site and remote access VPN. What kind of technology is this and what components does it consist of?

It is worth noting that IPsec is not one, but a whole set of different protocols that provide transparent and secure data protection. The specificity of IPsec is that it is implemented at the network layer, complementing it in such a way that everything happens unnoticed for subsequent layers. The main difficulty is that in the process of establishing a connection, two participants in a secure channel need to agree on a fairly large number of different parameters. Namely, they must authenticate each other, generate and exchange keys (and through an untrusted environment), and also agree on which protocols to encrypt data.

It is for this reason that IPsec consists of a stack of protocols whose responsibility is to ensure the establishment, operation and management of a secure connection. The entire connection establishment process consists of two phases: the first phase is used to ensure the secure exchange of ISAKMP messages in the second phase. ISAKMP (Internet Security Association and Key Management Protocol) is a protocol that serves to negotiate and update security policies (SA) between participants in a VPN connection. These policies specifically indicate which protocol to use to encrypt (AES or 3DES) and what to authenticate with (SHA or MD5).

Two Main Phases of IPsec

So, we found out that the participants first need to agree on what mechanisms will be used to create a secure connection, so now the IKE protocol comes into play. IKE (Internet Key Exchange) is used to form an IPsec SA (Security Association, those same security policies), in other words, to coordinate the work of participants in a secure connection. Through this protocol, participants agree on what encryption algorithm will be used, what algorithm will be used to check integrity, and how to authenticate each other. It should be noted that today there are two versions of the protocol: IKEv1 and IKEv2. We will only be interested in IKEv1: despite the fact that the IETF (The Internet Engineering Task Force) first introduced it in 1998, it is still very often used, especially for RA VPNs (see Figure 1).

As for IKEv2, its first drafts were made in 2005, it was fully described in RFC 5996 (2010), and only at the end of last year it was announced as an Internet standard (RFC 7296). You can read more about the differences between IKEv1 and IKEv2 in the sidebar. Having dealt with IKE, we return to the IPsec phases. During the first phase, participants authenticate each other and agree on the parameters for establishing a special connection intended only for exchanging information about the desired encryption algorithms and other details of the future IPsec tunnel. The parameters of this first tunnel (also called the ISAKMP tunnel) are determined by the ISAKMP policy. The first step is to agree on hashes and encryption algorithms, then exchange Diffie-Hellman (DH) keys, and only then figure out who is who. That is, the last step is the authentication process, either using a PSK or RSA key. And if the parties come to an agreement, then an ISAKMP tunnel is established, through which the second phase of IKE already passes.

In the second phase, the participants who already trust each other agree on how to build the main tunnel for directly transferring data. They offer each other the options specified in the transform-set parameter, and if they agree, they raise the main tunnel. It is important to emphasize that once it is established, the auxiliary ISAKMP tunnel does not go anywhere - it is used to periodically update the SA of the main tunnel. As a result, IPsec in some way establishes not one, but two tunnels.

How to process data

Now a few words about transform-set. After all, you need to somehow encrypt the data going through the tunnel. Therefore, in a typical configuration, transform-set is a set of parameters that explicitly indicate how the packet should be processed. Accordingly, there are two options for such data processing - the ESP and AH protocols. ESP (Encapsulating Security Payload) deals directly with data encryption and can also provide data integrity verification. AH (Authentication Header), in turn, is only responsible for authenticating the source and checking data integrity.

For example, the crypto ipsec transform-set SET10 esp-aes command will tell the router that the transform-set named SET10 should only work using the ESP protocol and with AES encryption. Looking ahead, I will say that hereinafter we will use Cisco routers and firewalls as targets. Actually, with ESP everything is more or less clear, its job is to encrypt and thereby ensure confidentiality, but why then is AH needed? AH provides data authentication, that is, it confirms that this data came exactly from the one with whom we established the connection, and was not changed along the way. It provides what is sometimes called anti-replay protection. In modern networks, AH is practically not used; only ESP can be found everywhere.

The parameters (aka SA) selected to encrypt information in an IPsec tunnel have a lifetime, after which they must be replaced. The default lifetime IPsec SA setting is 86,400 seconds, or 24 hours.
As a result, the participants received an encrypted tunnel with parameters that suited them all, and sent data streams to be encrypted there. Periodically, in accordance with lifetime, the encryption keys for the main tunnel are updated: the participants again communicate via the ISAKMP tunnel, go through the second phase and re-establish the SA.

IKEv1 modes

We've taken a quick look at the basic mechanics of how IPsec works, but there are a few more things we need to focus on. The first phase, among other things, can operate in two modes: main mode or aggressive mode. We have already discussed the first option above, but we are interested in the aggressive mode. This mode uses three messages (instead of six in main mode). In this case, the one who initiates the connection gives all his data at once - what he wants and what he can do, as well as his part of the DH exchange. The responder then immediately completes its portion of the DH generation. As a result, there are essentially only two stages in this mode. That is, the first two stages from main mode (hash agreement and DH exchange) are, as it were, compressed into one. As a result, this mode is much more dangerous due to the fact that the response comes with a lot of technical information in plaintext. And most importantly, the VPN gateway can send a password hash, which is used for authentication in the first phase (this password is often called a pre-shared key or PSK).

Well, all subsequent encryption occurs without changes, as usual. Why then is this mode still used? The fact is that it is much faster, about twice as fast. Of particular interest to the pentester is the fact that the aggressive mode is very often used in RA IPsec VPN. Another small feature of RA IPsec VPN when using aggressive mode: when the client contacts the server, it sends it an identifier (group name). Tunnel group name (see Figure 2) is the name of an entry that contains a set of policies for a given IPsec connection. This is already one of the features specific to Cisco equipment.


Two phases were not enough

It would seem that this is not a very simple scheme of work, but in reality it is still a little more complicated. Over time, it became clear that PSK alone was not enough to ensure security. For example, if an employee's workstation was compromised, the attacker would be able to immediately gain access to the entire internal network of the enterprise. Therefore, phase 1.5 was developed right between the first and second classical phases. By the way, this phase is usually not used in a standard site-to-site VPN connection, but is used when organizing remote VPN connections (our case). This phase contains two new extensions - Extended Authentication (XAUTH) and Mode-Configuration (MODECFG).

XAUTH is an additional user authentication within the IKE protocol. This authentication is also sometimes called IPsec second factor. Well, MODECFG serves to transmit additional information to the client, this could be an IP address, mask, DNS server, etc. It can be seen that this phase simply complements those previously discussed, but its usefulness is undoubted.

IKEv2 vs IKEv1

Both protocols operate on UDP port number 500, but are incompatible with each other; a situation where there is IKEv1 at one end of the tunnel and IKEv2 at the other is not allowed. Here are the main differences between the second version and the first:

  • In IKEv2 there are no longer such concepts as aggressive or main modes.
  • In IKEv2, the term first phase is replaced by IKE_SA_INIT (an exchange of two messages that ensures the negotiation of encryption/hashing protocols and the generation of DH keys), and the second phase is replaced by IKE_AUTH (also two messages that implement authentication itself).
  • Mode Config (what was called phase 1.5 in IKEv1) is now described directly in the protocol specification and is an integral part of it.
  • IKEv2 added an additional mechanism to protect against DoS attacks. Its essence is that before responding to each request in establishing a secure connection (IKE_SA_INIT) IKEv2, the VPN gateway sends a certain cookie to the source of such a request and waits for a response. If the source responded - everything is in order, you can start generating DH with it. If the source does not respond (this is what happens in the case of a DoS attack; this technique is reminiscent of TCP SYN flood), then the VPN gateway simply forgets about it. Without this mechanism, with every request from anyone, the VPN gateway would try to generate a DH key (which is a fairly resource-intensive process) and would soon run into problems. As a result, due to the fact that all operations now require confirmation from the other side of the connection, it is impossible to create a large number of half-open sessions on the attacked device.

We're reaching the milestone

Having finally understood the operating features of IPsec and its components, you can move on to the main thing - practical attacks. The topology will be quite simple and at the same time close to reality (see Fig. 3).


The first step is to determine the presence of an IPsec VPN gateway. This can be done by performing a port scan, but there is a small feature here. ISAKMP uses the UDP protocol, port 500, while default scanning with Nmap affects only TCP ports. And as a result there will be a message: All 1000 scanned ports on 37.59.0.253 are filtered.

It seems that all ports are filtered and there are no open ports. But after executing the command

Nmap -sU --top-ports=20 37.59.0.253 Starting Nmap 6.47 (http://nmap.org) at 2015-03-21 12:29 GMT Nmap scan report for 37.59.0.253 Host is up (0.066s latency) . PORT STATE SERVICE 500/udp open isakmp

We make sure that this is not the case and that this is indeed a VPN device.

Let's attack the first phase

Now we will be interested in the first phase, aggressive mode and authentication using pre-shared key (PSK). In this scenario, as we remember, the VPN device or responder sends a hashed PSK to the initiator. One of the most famous utilities for testing the IKE protocol is ike-scan, it is included in the Kali Linux distribution. Ike-scan allows you to send IKE messages with various parameters and, accordingly, decode and parse response packets. Let's try to probe the target device:

Root@kali:~# ike-scan -M -A 37.59.0.253 0 returned handshake; 0 returned notify

The -A switch indicates that aggressive mode should be used, and -M indicates that the results should be displayed line by line (multiline), for more convenient reading. It is clear that no result was obtained. The reason is that you need to specify the same identifier, the name of the VPN group. Of course, the ike-scan utility allows you to set this identifier as one of its parameters. But since it is still unknown to us, let’s take an arbitrary value, for example 0000.

Root@kali:~# ike-scan -M -A --id=0000 37.59.0.253 37.59.0.253 Aggressive Mode Handshake returned

This time we see that the answer was received (see Fig. 5) and we were provided with quite a lot of useful information. A fairly important part of the information received is the transform-set. In our case, it states that “Enc=3DES Hash=SHA1 Group=2:modp1024 Auth=PSK”.

All these parameters can be specified for the ike-scan utility using the --trans switch. For example, --trans=5,2,1,2 will indicate that the encryption algorithm is 3DES, hashing HMAC-SHA, authentication method PSK and the second type of DH group (1024-bit MODP). You can view the value correspondence tables at this address. Let's add another key (-P) in order to display the payload of the package directly, or rather the PSK hash.

Root@kali:~# ike-scan -M -A --id=0000 37.59.0.253 -P

Overcoming the first difficulties

It would seem that the hash has been obtained and you can try to brute it, but everything is not so simple. Once upon a time, in 2005, some Cisco hardware had a vulnerability: these devices gave a hash only if the attacker passed the correct ID value. Now, of course, it is almost impossible to find such equipment and a hashed value is always sent, regardless of whether the attacker sent the correct ID value or not. Obviously, brute force hashing is pointless. Therefore, the first task is to determine the correct ID value in order to obtain the correct hash. And a recently discovered vulnerability will help us with this. The point is that there is a slight difference between the responses during the initial exchange of messages. In short, if the correct group name is used, there are four attempts to continue establishing the VPN connection plus two encrypted phase two packets. While in the case of an incorrect ID, only two packets arrive in response. As you can see, the difference is quite significant, so SpiderLabs (the author of the equally interesting Responder tool) first developed a PoC and then the IKEForce utility to exploit this vulnerability.

What is the strength of IKE

You can install IKEForce in an arbitrary directory by running the command

Git clone https://github.com/SpiderLabs/ikeforce

It works in two main modes - calculation mode -e (enumeration) and brute force mode -b (bruteforce). We'll get to the second one when we look at attacks on the second factor, but we'll deal with the first one now. Before you begin the actual process of determining the ID, you need to set the exact value of transform-set. We have already defined it earlier, so we will specify it with the -t 5 2 1 2 option. As a result, the process of finding the ID will look like this:

Python ikeforce.py 37.59.0.253 -e -w wordlists/group.txt -t 5 2 1 2

As a result, it was possible to obtain the correct ID value quite quickly (Fig. 7). The first step has been completed, you can move on.

We receive PSK

Now you need to save the PSK hash to a file using the correct group name; this can be done using ike-scan:

Ike-scan -M -A --id=vpn 37.59.0.253 -Pkey.psk

And now that the correct ID value has been found and the correct PSK hash has been obtained, we can finally start offline brute force. There are quite a lot of options for such brute force - this is the classic psk-crack utility, and John the Ripper (with a jumbo patch), and even oclHashcat, which, as is known, allows you to use the power of the GPU. For simplicity, we will use psk-crack, which supports both direct brute force and dictionary attack:

Psk-crack -d /usr/share/ike-scan/psk-crack-dictionary key.psk

But even successfully restoring PSK (see Fig. 8) is only half the battle. At this stage, we need to remember that what awaits us next is XAUTH and the second factor of IPsec VPN.

Dealing with the second IPsec factor

So, let me remind you that XAUTH is an additional security, a second authentication factor, and it is in phase 1.5. There can be several options for XAUTH - this includes verification using the RADIUS protocol, one-time passwords (OTP), and a regular local user base. We will focus on the standard situation when a local user base is used to check the second factor. Until recently, there was no publicly available tool for brute force XAUTH. But with the advent of IKEForce, this problem received a worthy solution. Launching the XAUTH brute force is quite simple:

Python ikeforce.py 37.59.0.253 -b -i vpn -k cisco123 -u admin -w wordlists/passwd.txt -t 5 2 1 2 [+]Program started in XAUTH Brute Force Mode [+]Single user provided - brute forcing passwords for user: admin [*]XAUTH Authentication Successful! Username: admin Password: cisco

In this case, all previously found values ​​are indicated: ID (switch -i), restored PSK (switch -k) and expected login (switch -u). IKEForce supports both brute force search of a login and search through a list of logins, which can be specified with the -U parameter. In case of possible selection blocking, there is the -s option, which allows you to reduce the brute force speed. By the way, the utility comes with several good dictionaries, especially useful for setting the value of the ID parameter.

Login to the internal network

Now that we have all the data, the last step remains - actually penetrating the local network. To do this, we will need some kind of VPN client, of which there are a great many. But in the case of Kali, you can easily use the already pre-installed VPNC. In order for it to work, you need to adjust one configuration file - /etc/vpnc/vpn.conf. If it doesn’t exist, then you need to create and fill in a number of obvious parameters:

IPSec gateway 37.59.0.253 IPSec ID vpn IPSec secret cisco123 IKE Authmode psk Xauth Username admin Xauth password cisco

Here we see that absolutely all the data found in the previous steps was used - the ID value, PSK, login and password of the second factor. After which the connection itself occurs with one command:

Root@kali:~# vpnc vpn

Disabling is also quite simple:

Root@kali:~# vpnc-disconnect

You can check if the connection is working using the ifconfig tun0 command.

How to build reliable protection

Protection against the attacks discussed today must be comprehensive: you need to install patches on time, use strong pre-shared keys, which, if possible, should be replaced with digital certificates. Password policy and other obvious information security elements also play an important role in ensuring security. It should also be noted that the situation is gradually changing, and over time only IKEv2 will remain.

What's the result?

We've covered the RA IPsec VPN audit process in detail. Yes, of course, this task is far from trivial. You need to take many steps, and difficulties may await you at each of them, but if successful, the result is more than impressive. Gaining access to internal network resources opens up the widest scope for further actions. Therefore, those who are responsible for protecting the network perimeter should not rely on ready-made default templates, but carefully think through each security layer. Well, for those who conduct pentests, the detected UDP port 500 is a reason to conduct a deep analysis of IPsec VPN security and, possibly, get good results.

IPSec protocols Organization of a secure channel https://www.site/lan/protokoly-ipsec https://www.site/@@site-logo/logo.png

IPSec protocols

Organization of a secure channel

IPSec protocols

Establishing a secure channel using AH, ESP and IKE.

Internet Protocol Security (IPSec) is called a system in Internet standards. Indeed, IPSec is a consistent set of open standards that today has a well-defined core, and at the same time it can be quite easily expanded with new protocols, algorithms and functions.

The main purpose of IPSec protocols is to ensure secure data transmission over IP networks. The use of IPSec guarantees:

  • integrity, i.e. that the data during transmission was not distorted, lost or duplicated;
  • authenticity, i.e. that the data was transmitted by the sender who proved that he is who he claims to be;
  • confidentiality, i.e. that the data is transmitted in a form that prevents unauthorized viewing.

(Note that, in accordance with the classical definition, the concept of data security includes one more requirement - the availability of data, which in the context considered can be interpreted as a guarantee of their delivery. IPSec protocols do not solve this problem, leaving it to the TCP transport layer protocol.)

SECURED CHANNELS AT DIFFERENT LEVELS

IPSec is only one of many, although the most popular today, technology for secure data transmission over a public (unsecured) network. For technologies of this purpose, a general name is used - secure channel. The term "channel" emphasizes the fact that data protection is provided between two network nodes (hosts or gateways) along a virtual path laid in a packet-switched network.

A secure channel can be built using system tools implemented at different layers of the OSI model (see Figure 1). If a protocol of one of the upper levels (application, presentation or session) is used to protect data, then this method of protection does not depend on which networks (IP or IPX, Ethernet or ATM) are used to transport data, which can be considered an undoubted advantage. On the other hand, the application becomes dependent on a specific security protocol, i.e. for applications such a protocol is not transparent.

A secure channel at the highest, application level has one more drawback - a limited scope. The protocol protects only a very specific network service - file, hypertext or email. For example, the S/MIME protocol exclusively protects email messages. Therefore, a corresponding secure version of the protocol must be developed for each service.

The most well-known secure channel protocol operating at the next presentation level is the Secure Socket Layer (SSL) protocol and its new open implementation Transport Layer Security (TLS). Lowering the protocol level makes it a much more versatile security tool. Now any application and any application-level protocol can use a single security protocol. However, applications still need to be rewritten to include explicit calls to secure channel protocol functions.

The lower down in the stack secure channel facilities are implemented, the easier it is to make them transparent to applications and application protocols. At the network and data link levels, applications' dependence on security protocols disappears completely. However, here we are faced with another problem - the dependence of the security protocol on a specific network technology. Indeed, different parts of a large composite network generally use different link protocols, so it is impossible to lay a secure channel through this heterogeneous environment using a single link-layer protocol.

Consider, for example, the Point-to-Point Tunneling Protocol (PPTP), which operates at the data link layer. It is based on the PPP protocol, which is widely used in point-to-point connections, such as leased lines. The PPTP protocol not only provides security transparency for applications and application-level services, but is also independent of the network layer protocol used: in particular, the PPTP protocol can transport packets both in IP networks and in networks based on the IPX, DECnet protocols or NetBEUI. However, since the PPP protocol is not used in all networks (in most local networks the Ethernet protocol operates at the data link level, and in global networks - ATM and frame relay protocols), PPTP cannot be considered a universal tool.

IPSec, which operates at the network layer, is a compromise option. On the one hand, it is transparent to applications, and on the other hand, it can work on almost all networks, since it is based on the widely used IP protocol: currently in the world only 1% of computers do not support IP at all, the remaining 99% use it either as a single protocol, or as one of several protocols.

DISTRIBUTION OF FUNCTIONS BETWEEN IPSEC PROTOCOLS

The core of IPSec consists of three protocols: the authentication protocol (Authenti-cation Header, AH), the encryption protocol (Encapsulation Security Payload, ESP) and the key exchange protocol (Internet Key Exchange, IKE). The functions of maintaining a secure channel are distributed among these protocols as follows:

  • AH protocol guarantees data integrity and authenticity;
  • The ESP protocol encrypts transmitted data, guaranteeing confidentiality, but it can also support authentication and data integrity;
  • The IKE protocol solves the auxiliary task of automatically providing channel endpoints with the secret keys necessary for the operation of authentication and data encryption protocols.

As can be seen from the brief description of the functions, the capabilities of the AH and ESP protocols partially overlap. The AH protocol is responsible only for ensuring data integrity and authentication, while the ESP protocol is more powerful, since it can encrypt data, and in addition, perform the functions of the AH protocol (although, as we will see later, it provides authentication and integrity in a slightly reduced form ). The ESP protocol can support encryption and authentication/integrity functions in any combination, i.e., either both groups of functions, or only authentication/integrity, or only encryption.

To encrypt data in IPSec, any symmetric encryption algorithm that uses secret keys can be used. Ensuring data integrity and authentication is also based on one of the encryption techniques - encryption using a one-way function, also called a hash function or digest function.

This function, applied to encrypted data, results in a digest value consisting of a fixed small number of bytes. The digest is sent in an IP packet along with the original message. The recipient, knowing which one-way encryption function was used to compose the digest, recalculates it using the original message. If the values ​​of the received and calculated digests are the same, this means that the contents of the packet were not subject to any changes during transmission. Knowing the digest does not make it possible to reconstruct the original message and therefore cannot be used for protection, but it does allow you to check the integrity of the data.

The digest is a kind of checksum for the original message. However, there is also a significant difference. The use of a checksum is a means of verifying the integrity of messages transmitted over unreliable communication lines and is not intended to combat malicious activity. In fact, the presence of a checksum in the transmitted packet will not prevent an attacker from replacing the original message by adding a new checksum value to it. Unlike a checksum, a secret key is used when calculating the digest. If a one-way function with a parameter (the secret key) known only to the sender and recipient was used to obtain the digest, any modification to the original message would be immediately detected.

The separation of security functions between the two protocols AH and ESP is caused by the practice used in many countries to restrict the export and/or import of tools that ensure data confidentiality through encryption. Each of these two protocols can be used either independently or simultaneously with the other, so that in cases where encryption cannot be used due to current restrictions, the system can only be supplied with the AH protocol. Naturally, protecting data only using the AH protocol will in many cases be insufficient, since the receiving side in this case will only be sure that the data was sent by exactly the node from which it was expected and arrived in the form in which it was received. sent. The AH protocol cannot protect against unauthorized viewing along the data path, since it does not encrypt it. To encrypt data, it is necessary to use the ESP protocol, which can also verify its integrity and authenticity.

SAFE ASSOCIATION

In order for the AH and ESP protocols to do their job of protecting the transmitted data, the IKE protocol establishes a logical connection between the two endpoints, which in IPSec standards is called a “Security Association” (SA). Establishing an SA begins with mutual authentication of the parties, because all security measures become meaningless if data is transmitted or received by the wrong person or from the wrong person. The SA parameters you select next determine which of the two protocols, AH or ESP, is used to protect data, what functions the security protocol performs: for example, only authentication and integrity checking or, in addition, protection against false replay. A very important parameter of a secure association is the so-called cryptographic material, i.e. the secret keys used in the operation of the AH and ESP protocols.

The IPSec system also allows for a manual method of establishing a secure association, in which the administrator configures each end node so that they support agreed upon association parameters, including secret keys.

The AH or ESP protocol already operates within the established logical connection SA, with its help the required protection of the transmitted data is carried out using the selected parameters.

The secure association settings must be acceptable to both endpoints of the secure channel. Therefore, when using the automatic SA establishment procedure, IKE protocols operating on opposite sides of the channel select parameters during the negotiation process, just as two modems determine the maximum acceptable exchange rate for both parties. For each task solved by the AH and ESP protocols, several authentication and encryption schemes are offered - this makes IPSec a very flexible tool. (Note that the choice of the digest function to solve the authentication problem does not in any way affect the choice of algorithm for data encryption.)

To ensure compatibility, the standard version of IPsec defines a certain mandatory “tool” set: in particular, one of the one-way encryption functions MD5 or SHA-1 can always be used for data authentication, and the encryption algorithms certainly include DES. At the same time, manufacturers of products that include IPSec are free to expand the protocol with other authentication and encryption algorithms, which they successfully do. For example, many IPSec implementations support the popular Triple DES encryption algorithm, as well as relatively new algorithms - Blowfish, Cast, CDMF, Idea, RC5.

IPSec standards allow gateways to use either one SA to carry traffic from all hosts interacting over the Internet, or to create an arbitrary number of SAs for this purpose, for example, one for each TCP connection. A secure SA is a unidirectional (simplex) logical connection in IPSec, so two SAs must be established for bidirectional communications.

TRANSPORT AND TUNNEL MODES

The AH and ESP protocols can protect data in two modes: transport and tunnel. In transport mode, the transmission of an IP packet through the network is carried out using the original header of this packet, and in tunnel mode, the original packet is placed in a new IP packet and data transmission across the network is carried out based on the header of the new IP packet. The use of one mode or another depends on the requirements for data protection, as well as on the role played in the network by the node that terminates the secure channel. Thus, a node can be a host (end node) or a gateway (intermediate node). Accordingly, there are three schemes for using IPSec: host-to-host, gateway-to-gateway, and host-to-gateway.

In the first scheme, a secure channel, or what is the same thing in this context, a secure association, is established between two end nodes of the network (see Figure 2). The IPSec protocol in this case runs on the end node and protects the data arriving at it. For the host-to-host scheme, the transport mode of protection is most often used, although the tunnel mode is also allowed.

In accordance with the second scheme, a secure channel is established between two intermediate nodes, the so-called security gateways (SG), each of which runs the IPSec protocol. Secure communications can occur between any two endpoints connected to networks that are located behind security gateways. Endpoints are not required to support IPSec and transmit their traffic unprotected through trusted enterprise Intranets. Traffic destined for the public network passes through the security gateway, which provides IPSec protection on its behalf. Gateways can only use tunnel mode.

The host-gateway scheme is often used for remote access. Here, a secure channel is established between a remote host running IPSec and a gateway that protects traffic for all hosts on the enterprise Intranet network. The remote host can use both transport and tunnel mode when sending packets to the gateway, but the gateway sends the packet to the host only in tunnel mode. This scheme can be complicated by creating another secure channel in parallel - between the remote host and any host belonging to the internal network protected by the gateway. This combined use of two SAs allows you to reliably protect traffic on the internal network.

Natalia Olifer

Operations with a document

a brief historical background of the appearance of the protocol

In 1994, the Internet Architecture Board (IAB) released the report "Security of the Internet Architecture." This document described the main areas of application of additional security tools on the Internet, namely protection against unauthorized monitoring, packet spoofing, and data flow control. Among the first and most important protective measures was the need to develop a concept and basic mechanisms to ensure the integrity and confidentiality of data flows. Since changing the basic protocols of the TCP/IP family would cause a complete restructuring of the Internet, the task was set to ensure the security of information exchange in open telecommunication networks based on existing protocols. Thus, the Secure IP specification began to be created, complementary to the IPv4 and IPv6 protocols.

IPSec architecture

IP Security is a set of protocols dealing with issues of encryption, authentication and security during the transport of IP packets; it now includes nearly 20 standards proposals and 18 RFCs.
The IP Security specification (known today as IPsec) is developed by the IETF IP Security Protocol Working Group. IPsec originally included 3 algorithm-independent core specifications, published as RFCs: IP Security Architecture, Authentication Header (AH), Encapsulation of Encrypted Data (ESP) (RFC1825, 1826, and 1827). It should be noted that in November 1998, the IP Security Protocol Working Group proposed new versions of these specifications, which currently have the status of preliminary standards, these are RFC2401 - RFC2412. Note that RFC1825-27 has been considered obsolete for several years and is not really used. In addition, there are several algorithm-dependent specifications using the MD5, SHA, and DES protocols.

Fig.1. IPSec architecture

The IP Security Protocol Working Group is also developing key information management protocols. This group's mission is to develop the Internet Key Management Protocol (IKMP), an application-level key management protocol that is independent of the security protocols used. Key management concepts are currently being explored using the Internet Security Association and Key Management Protocol (ISAKMP) specification and the Oakley Key Determination Protocol. The ISAKMP specification describes mechanisms for negotiating the attributes of the protocols used, while the Oakley protocol allows you to install session keys on computers on the Internet. Previously, the possibilities of using key management mechanisms of the SKIP protocol were also considered, but now such possibilities are practically not used anywhere. Emerging key information management standards may support Key Distribution Centers similar to those used in Kerberos. Kerberos-based key management protocols for IPSec are currently being addressed by the relatively new KINK (Kerberized Internet Negotiation of Keys) working group.
Data integrity and confidentiality guarantees in the IPsec specification are provided through the use of authentication and encryption mechanisms, respectively. The latter, in turn, are based on the preliminary agreement of the parties to the so-called information exchange. "security context" - applied cryptographic algorithms, key information management algorithms and their parameters. The IPsec specification provides for the ability for parties to exchange information to support various protocols and parameters for authentication and encryption of data packets, as well as various key distribution schemes. In this case, the result of agreeing on the security context is the establishment of a security parameter index (SPI), which is a pointer to a certain element of the internal structure of the information exchange party, describing possible sets of security parameters.
Essentially, IPSec, which will become an integral part of IPv6, operates at the third layer, i.e. at the network layer. As a result, transmitted IP packets will be protected in a manner transparent to network applications and infrastructure. Unlike SSL (Secure Socket Layer), which operates at layer 4 (i.e., transport) and is more closely associated with higher layers of the OSI model, IPSec is designed to provide low-level security.

Fig.2. OSI/ISO model

IPSec adds a header to IP data ready to be sent over a virtual private network to identify the protected packets. Before being transmitted over the Internet, these packets are encapsulated within other IP packets. IPSec supports several encryption types, including Data Encryption Standard (DES) and Message Digest 5 (MD5).
To establish a secure connection, both participants in the session must be able to quickly agree on security parameters, such as authentication algorithms and keys. IPSec supports two types of key management schemes through which participants can negotiate session parameters. This dual support at one time caused some friction in the IETF Working Group.
With the current version of IP, IPv4, either Internet Secure Association Key Management Protocol (ISAKMP) or Simple Key Management for Internet Protocol can be used. With the new version of IP, IPv6, you will have to use ISAKMP, now known as IKE, although the possibility of using SKIP is not excluded. However, it should be kept in mind that SKIP has not been considered as a key management candidate for a long time, and was removed from the list of possible candidates back in 1997.

AH and ESP headers

authentication header AH

The Authentication Header (AH) is a common optional header and is typically located between the IP packet's main header and the data field. The presence of AH does not in any way affect the process of transmitting information from transport and higher levels. The main and only purpose of AH is to provide protection against attacks related to unauthorized changes in the contents of a packet, including spoofing the source address of the network layer. Higher level protocols must be modified to verify the authenticity of the received data.
The AH format is quite simple and consists of a 96-bit header and variable length data consisting of 32-bit words. The field names reflect their contents fairly clearly: Next Header indicates the next header, Payload Len represents the packet length, SPI is a pointer to the security context, and Sequence Number Field contains the packet's sequence number.

Fig.3. AH Header Format

The packet sequence number was introduced into AH in 1997 as part of the IPsec specification revision process. The value of this field is generated by the sender and serves to protect against attacks related to reuse of authentication process data. Because the Internet does not guarantee the order in which packets will be delivered, the recipient must store information about the maximum sequence number of a successfully authenticated packet and whether a number of packets containing previous sequence numbers have been received (usually 64).
Unlike checksum calculation algorithms used in protocols for transmitting information over switched communication lines or over local network channels and aimed at correcting random errors in the transmission medium, mechanisms for ensuring data integrity in open telecommunication networks must have means of protection against targeted changes. One such mechanism is a special use of the MD5 algorithm: during the formation of the AH, a hash function is sequentially calculated from the union of the packet itself and some pre-agreed key, and then from the union of the resulting result and the transformed key. This is the default mechanism to ensure that all IPv6 implementations have at least one common algorithm that is not subject to export restrictions.

encapsulation of encrypted ESP data

When encrypted data encapsulation is used, the ESP header is the last of the optional headers "visible" in the packet. Because the primary purpose of ESP is to ensure data confidentiality, different types of information may require significantly different encryption algorithms. Consequently, the ESP format can undergo significant changes depending on the cryptographic algorithms used. However, the following mandatory fields can be distinguished: SPI, indicating the security context, and Sequence Number Field, containing the sequence number of the packet. The "ESP Authentication Data" field (checksum) is optional in the ESP header. The recipient of the ESP packet decrypts the ESP header and uses the parameters and data of the applied encryption algorithm to decode the transport layer information.

Fig.4. ESP Header Format

There are two modes of using ESP and AH (as well as their combinations) - transport and tunnel:
Transport mode is used to encrypt the data field of an IP packet containing transport layer protocols (TCP, UDP, ICMP), which, in turn, contains application service information. An example of the use of transport mode is the transmission of electronic mail. All intermediate nodes along a packet's route from sender to receiver use only the public network layer information and possibly some optional packet headers (in IPv6). The disadvantage of transport mode is the lack of mechanisms for hiding the specific sender and recipient of a packet, as well as the ability to analyze traffic. The result of such an analysis can be information about the volumes and directions of information transfer, areas of interest of subscribers, and the location of managers.
Tunnel mode encrypts the entire packet, including the network layer header. The tunnel mode is used if it is necessary to hide the organization’s information exchange with the outside world. In this case, the address fields of the network layer header of a packet using tunnel mode are filled in by the organization's firewall and do not contain information about the specific sender of the packet. When transmitting information from the outside world to the organization's local network, the network address of the firewall is used as the destination address. After the firewall decrypts the initial network layer header, the packet is forwarded to the recipient.

Security Associations

A Security Association (SA) is a connection that provides security services for traffic that passes through it. Two computers on each side of the SA store the mode, protocol, algorithms, and keys used in the SA. Each SA is used in only one direction. Bidirectional communication requires two SAs. Each SA implements one mode and protocol; thus, if two protocols need to be used for one packet (such as AH and ESP), then two SAs are required.

Security policy

The security policy is stored in the SPD (Security Policy Database). SPD can specify one of three actions for a data packet: discard the packet, do not process the packet using IPSec, or process the packet using IPSec. In the latter case, the SPD also indicates which SA should be used (if, of course, a suitable SA has already been created) or indicates with what parameters a new SA should be created.
SPD is a very flexible control mechanism that allows very good control over the processing of each packet. Packets are classified by a large number of fields, and the SPD may examine some or all of the fields to determine the appropriate action. This could result in all traffic between two machines being carried using a single SA, or separate SAs being used for each application, or even for each TCP connection.

ISAKMP/Oakley protocol

ISAKMP defines the general structure of protocols that are used to establish SAs and perform other key management functions. ISAKMP supports several Domains of Interpretation (DOI), one of which is IPSec-DOI. ISAKMP does not define a complete protocol, but rather provides "building blocks" for various DOIs and key exchange protocols.
The Oakley protocol is a key discovery protocol that uses the Diffie-Hellman key replacement algorithm. The Oakley protocol supports Perfect Forward Secrecy (PFS). The presence of PFS means that it is impossible to decrypt all traffic if any key in the system is compromised.

IKE protocol

IKE is the default key exchange protocol for ISAKMP, and is currently the only one. IKE sits on top of ISAKMP and does the actual establishment of both the ISAKMP SA and the IPSec SA. IKE supports a set of various primitive functions for use in protocols. Among them are the hash function and the pseudo-random function (PRF).
A hash function is a collision-resistant function. Collision resistance means the fact that it is impossible to find two different messages m1 and m2 such that H(m1)=H(m2), where H is a hash function.
As for pseudo-random functions, a hash function is currently used instead of special PRFs in the HMAC design (HMAC is a message authentication mechanism using hash functions). To define HMAC, we need a cryptographic hash function (let's call it H) and a secret key K. We assume that H is a hash function where data is hashed using a compression procedure applied sequentially to a sequence of data blocks. We denote by B the length of such blocks in bytes, and the length of blocks obtained as a result of hashing by L (L

ipad = byte 0x36, repeated B times;
opad = byte 0x5C repeated B times.

To calculate HMAC from "text" data, you need to perform the following operation:

H(K XOR opad, H(K XOR ipad, text))

From the description it follows that IKE uses HASH values ​​to authenticate parties. Note that HASH in this case refers solely to the Payload name in ISAKMP, and this name has nothing to do with its contents.

attacks on AH, ESP and IKE

All types of attacks on IPSec components can be divided into the following groups: attacks that exploit the finite resources of the system (a typical example is a Denial of Service attack, Denial-of-service or DOS attack), attacks that exploit the features and errors of a specific IPSec implementation and , finally, attacks based on the weaknesses of the AH and ESP protocols themselves. Purely cryptographic attacks can be ignored - both protocols define the concept of “transform”, where all cryptography is hidden. If the crypto-algorithm used is stable, and the transform defined with it does not introduce additional weaknesses (this is not always the case, therefore it is more correct to consider the strength of the entire system - Protocol-Transform-Algorithm), then from this side everything is fine. What remains? Replay Attack - leveled by using Sequence Number (in one single case this does not work - when using ESP without authentication and without AH). Further, the order of actions (first encryption, then authentication) guarantees quick rejection of “bad” packets (moreover, according to recent research in the world of cryptography, this order of actions is the most secure; the reverse order in some, albeit very special cases, can lead to potential security holes; fortunately, neither SSL, nor IKE, nor other common “authenticate first, encrypt later” protocols apply to these special cases, and therefore do not have these holes). What remains is the Denial-Of-Service attack.

As you know, this is an attack from which there is no complete defense. However, the quick rejection of bad packets and the absence of any external reaction to them (according to the RFC) make it possible to deal with this attack more or less well. In principle, most (if not all) known network attacks (sniffing, spoofing, hijacking, etc.) are successfully resisted by AH and ESP when used correctly. With IKE it's a little more complicated. The protocol is very complex and difficult to analyze. In addition, due to typos (in the formula for calculating HASH_R) when writing it and not entirely successful solutions (the same HASH_R and HASH_I), it contains several potential “holes” (in particular, in the first phase, not all Payloads in the message are authenticated), however , they are not very serious and lead, at most, to a refusal to establish a connection. IKE more or less successfully protects itself from attacks such as replay, spoofing, sniffing, hijacking. With cryptography it is somewhat more complicated - it is not carried out separately, as in AH and ESP, but is implemented in the protocol itself. However, if you use persistent algorithms and primitives (PRF), there should be no problems. To some extent, it can be considered as a weakness of IPsec that DES is indicated as the only mandatory cryptographic algorithm in the current specifications (this is true for both ESP and IKE), 56 bits of the key of which are no longer considered sufficient. However, this is a purely formal weakness - the specifications themselves are algorithm-independent, and almost all well-known vendors have long implemented 3DES (and some have already implemented AES). Thus, if implemented correctly, the most “dangerous” attack remains Denial-Of-Service .

IPSec protocol evaluation

The IPSec protocol has received mixed reviews from experts. On the one hand, it is noted that the IPSec protocol is the best among all other previously developed protocols for protecting data transmitted over the network (including PPTP developed by Microsoft). According to the other side, there is excessive complexity and redundancy of the protocol. Thus, Niels Ferguson and Bruce Schneier in their work "A Cryptographic Evaluation of IPsec" note that they found serious security problems in almost all major components of IPsec. These authors also note that the protocol suite requires significant improvement in order for it to provide a good level of security.