Introduction to Datagram Transport Layer Security (DTLS)

Introduction to Datagram Transport Layer Security (DTLS)

Among the many protocols that protect web traffic and email, Transport Layer Security (TLS) is the most commonly used. Transmission Control Protocol (TCP) provides a reliable transport channel that operates in a transparent connection-oriented manner.

Over the last few years, UDP-based application protocols have become more common. Since TLS relies on sequence numbers to determine integrity, each record cannot be decrypted independently. A record integrity check on N+1 will fail if the record N is not received.

Handshake messages are assumed to be delivered reliably by the TLS handshake layer, and a break is triggered if they aren't. Datagram based applications need DTLS for security to prevent eavesdropping, tampering, and message forgery.

Fig 1.1- DTLS Handshake 

Using UDP as a transport layer protocol, it secures communication between resource-constrained devices and applications like online gaming, live streaming, and Internet telephony. The sequence numbers are explicitly added to DTLS using IPsec ESP. Losses and reordered data are not compensated.

A defined order must be followed when transmitting and receiving messages in TLS. It will produce an error if the order is not correct. There is an incompatibility between this feature and UDP's packet reordering and message loss. IP fragmentation is caused by the TLS handshake messages, which are larger than the datagram size

Packet loss recovery in DTLS
DTLS employs retransmission timers to prevent packet loss. During the initial handshake phase of DTLS, clients send hello messages to servers and wait for a server-send hello verify request. When the timer expires, the client knows that either the client hello or the hello verify request was unsuccessful. 

Clients retransmit messages, and servers retransmit when they receive retransmissions. The server also has retransmission timers that retransmit when they expire. The hello verify request is intended to be small enough that it will not be fragmented, alleviating concerns about multiple hello verify requests.

Packet Re-ordering in DTLS
In DTLS, the handshake message is given a particular sequence number within that handshake. The peer that receives this handshake message decides whether or not this is the next message it anticipates. It processes the message if it matches the expected message in the received message. If not, after all previous messages have been received, the message is placed in a queue for future handling.

TLS/DTLS datagram Sizes
When compared to the size of the UDP datagrams, the DTLS and TLS handshake messages are quite large. Each DTLS handshake message is split up into a number of DTLS records, each of which is designed to fit in a single IP datagram, in order to address this problem.

Both the fragment offset and fragment length are included in every handshake message. As a result, the recipient who has access to every byte of a handshake message can piece together the original, undamaged message.