Networking example with DPDK

Deploy

1. Create the AMI with Packer

1. export AWS_ACCESS_KEY_ID=<your access key id> and export AWS_SECRET_ACCESS_KEY=<your access key> in your shell.
2. packer init . only necessary once, but it also doesn't hurt to execute it again.
3. packer build . to actually create the AMI.

• Other useful commands are:
• packer fmt . to format the packer configuration files.
• packer validate . to validate the packer configuration files.
• Note that the . is necessary with packer but not with terraform.
• packer only creates the AMI for you, so you have to delete it later yourself with the AWS Management Console.
• go to EC2 > Images > AMIs
• select the AMI > Actions > Deregister AMI > select Delete associated snapshots and confirm Derigister AMI
• Storing AMIs after they been created is quite cheap, as you only have to pay for storage. At the moment of the writing (October 2025), EBS Snapshots Storage Pricing was $0.05/GB-month.

2. Create the EC2 and ENI instances with Terraform

1. export AWS_ACCESS_KEY_ID=<your access key id> and export AWS_SECRET_ACCESS_KEY=<your access key> in your shell.
2. terraform init only necessary once, but it also doesn't hurt to execute it again.
3. terraform apply builds the infrastructure described int the configuration files.
• Affirm with yes, otherwise no infrastructure is provisioned.
4. terraform destroy destroys the infrastructure that was created with terraform apply.

• Other useful commands are:
• terraform fmt
• terraform validate
• terraform output prints the output again that was printed after terraform apply exited successfully.
• terraform state list list resources in the state
• terraform state

For more information, please consult the great tutorials for Packer and Terraform (both are specifically for AWS) and consult the documentation for Packer and Terraform.

Connect to AWS and copy files

terraform already copies the src directory to the instances.

It is planned that the code is built on the EC2 instances by terraform soon.

In order to connect to the AWS instances, you can use utils/awsconnect tx or utils/awsconnect tx, instead of manually copying the URL from the terraform output.

Similarly, you can use utils/awscp tx and utils/awscp tx to copy the src directory to the instances if you made any changes to it.

Build DPDK locally

On the cloud instance, you might not care if DPDK is installed globally or locally. But on your local machine, you probably want DPDK not to be installed globally. In this case, we follow this step-by-step guide :

• cd dpdk-25.03/ && mkdir build && cd build
• meson --prefix $(pwd)/../install .. .
• ninja install
• ninja install is necessary to create the pkgconfig files.
• This will install DPDK into the directory install on the same level as the build directory, and correctly set the prefix in the pkgconfig file. We only need to append the path to it in the PKG_CONFIG_PATH environment variable with export PKG_CONFIG_PATH=../dpdk-25.03/install/lib/x86_64-linux-gnu/pkgconfig/:$PKG_CONFIG_PATH
• If you run pkg-config --modversion libdpdk before performing the above export, it will probably print something like Package libdpdk was not found in the pkg-config search path. But after adding the location of the libdpdk.pc file, it will print the DPDK version.
• The DPDK application like TX or RX is then compiled with the usual meson setup build and ninja -C build. This will also stop clangd from complaining that it doesn't find the DPDK headers.

Build

1. meson setup build
• If you changed the meson.build file, but already have a build directory, use meson configure build
• If compiling fails because modern system header like <print> (introduced in C++23) are not found, pass the desired compiler as environment variable to meson with CXX=g++-14 meson setup build --wipe.
• ln -s build/compile_commands.json helps the language server to find include paths and use the correct C++ version.
2. ninja -C build

Run

Execute sudo ./build/rx. This will print the MAC address of rx, which you can then use in sudo ./build/tx -- <tx IP address> <rx IP address> <rx MAC address>.

MAC address

Sending Ethernet frames requires the MAC address of the sender and the recipient. It could be acquired

• after binding the NIC to the vfio-pci driver with int rte_eth_macaddr_get(uint16_t port_id, struct rte_ether_addr *mac_addr); and manually implementing a way to answer with the MAC address, or
• before binding the NIC to the vfio-pci driver by sending an ARP request to the IP address and relying on the OS kernel to answer with the MAC address.

Code Explanation

rte_cpu_to_be_16(x) is a macro that changes the byte order of x.

• Big-endian (BE) has the most significant byte is at the smallest memory address,
• Little-endian (LE) has the least significant byte at the smallest memory address.
• Inside each byte, the bit order is the same in both cases.
• On the wire, networks always use BE, whereas many CPUs use LE.
• Since in hex dumps, the memory is usually displayed with the smallest address on the top left and the largest address on the bottom right, BE is usually more intuitive to read from a hex dump. For example the value 256 would be displayed as 0x0100 in BE and as 0x0001 in LE.
• x86 architecture offers the bswap instruction to reverse the byte order of a 32-bit or 64-bit (destination) register.
• The x86 instruction movbe is equivalent to mov, followed by bswap, but more efficient.

End-of-options marker --

DPDK uses everything before the -- as options for the initialization of the EAL and everything after the -- for the DPDK application. The Keep Alive sample application for example uses ./<build_dir>/examples/dpdk-l2fwd-keepalive [EAL options] -- -p PORTMASK [-q NQ] [-K PERIOD] [-T PERIOD]. The initialization function rte_eal_init(argc,argv) is passed the command line parameter array, parses it and returns how many of them it parsed before encountering --. Then argv is incremented by this amount and parsed by l2fd_parse_args.

This is similar to the convention for unix command line tools, where -- often (but not to always) acts as end-of-options marker. Options are passed with a flag. An example could be touch -r <other-file> <new-file>, that uses the option -r <other-file> to use the creation time of other-file as creation time for new-file. Everything that follows after the -- is treated as positional argument, i.e. arguments identified by their position. In cp <src> <dst>, src is the source and dst the destination, based on their position. With touch -- -r, a file can be created that has the name -r, which would not be possible without --. With rm -- -r, it can be deleted again.

Ethernet Frames

To understand DPDK, we start with a simple example which sends packets from one host to another via DPDK. The received Ethernet frames are explained in more detail here.