NS3 Ethernet PCP Buffer Decision DQN

This project implements a Deep Q-Network (DQN) based approach for managing Ethernet Priority Code Point (PCP, 802.1p) in the NS-3 network simulator. It leverages reinforcement learning to optimize traffic prioritization and scheduling in Ethernet networks. The repository includes scripts and setup instructions for integrating DQN with NS-3 using the ns3-ai framework, enabling advanced AI-driven networking experiments.

Development Environment

1. OS: WSL2 (or Ubuntu 24, recommend use Linux)
2. Python: 3.11
3. ns3: 3.41
4. ns3-ai
5. (an GPU is recommended. This exmaple uses RTX 2070 super 8GB)

Setup

Recommend run in Linux

1. Install required libs

   sudo apt install libboost-all-dev
   sudo apt install libprotobuf-dev protobuf-compiler
   sudo apt install pybind11-dev

2. Install ns-3.41

   wget https://www.nsnam.org/releases/ns-allinone-3.41.tar.bz2
   tar xf ns-allinone-3.41.tar.bz2
   cd ./ns-allinone-3.41/ns-3.41

3. Create an python venv (most use python 3.11!!!)

   Make sure you are at ./ns-allinone-3.41/ns-3.41

   python3.11 -m venv ns3ai_env
   source ns3ai_env/bin/activate

   # in (ns3ai_env)
   pip install -r requirements.txt

4. Install ns3-ai and configure ns3.41

   git clone https://github.com/hust-diangroup/ns3-ai.git contrib/ai

    ./ns3 configure --enable-examples -- -DPython_EXECUTABLE=ns3ai_env/bin/python -DPython3_EXECUTABLE=ns3ai_env/bin/python
    ./ns3 build ai

   pip install -e contrib/ai/python_utils
   pip install -e contrib/ai/model/gym-interface/py

Run Script

1. Copy project files to your ns-3.41

   copy all files in /scratch (this repo) to your ./ns-allinone-3.41/ns-3.41

2. Run the script

   • Round robin

     cd ns3.41
     ./ns3 run point2point-tcp-pcp-dqn

     Print example:

     PcpServerApp started, open port 5000
     PcpClientApp connected to 10.1.1.2:5000
     PcpClientApp connected to 10.1.2.2:5000
     PcpClientApp connected to 10.1.3.2:5000
     PcpClientApp connected to 10.1.4.2:5000
     Time: 1.06, Avg Latency: 1059 ms, Throughput: 35.9527 Mbps, Total Bytes: 4999500
     Buffer lengths: [20, 22, 19, 19, 24, 25, 17, 20]
     
     ...
     
     Time: 1.99, Avg Latency: 1989 ms, Throughput: 478.2 Mbps, Total Bytes: 124734000
     Buffer lengths: [8, 24, 7, 14, 20, 34, 17, 28]
     PcpClientApp stopped, sent packets: 83334
     PcpClientApp stopped, sent packets: 83334
     PcpClientApp stopped, sent packets: 83334
     PcpClientApp stopped, sent packets: 83334

   • DQN(training)

     cd ns3.41/scratch/point2point-tcp-pcp-dqn
     python run_train.py

     Print example:

     # TP=throughput(Mbps), L=latency(ms), Eps.=epsilon, A=action, R=reward, O=observation
     Episode 1/1:   0%| | 21000/124734000 [00:01<1:45:35, 19685.14step/s, TP=1.000, L=96.25, Eps.=1.000, A=6, R=-96.250, O=[  5   3   3   2   1   2   0   2 110 110 110 110 110 110 110 0]

3. Use NetAnim to see the topology

   sudo apt-get install mercurial qt4-dev-tools
   cd netanim-3.109
   make clean
   qmake NetAnim.pro
   make

   ./NetAnim

Export results

All file will generate under /ns-3.41

• With round robin (ns3)

  # pcap
  
  pcap_point2point-tcp-pcp-dqn-0-0.pcap
  ...
  pcap_point2point-tcp-pcp-dqn-4-3.pcap
  
  # server statics
  
  server-stats_point2point-pcp-dqn.json

• With DQN training (ns3 + ns3-ai)

  # dqn model
  
  p2p-tcp-pcp-dqn-model.keras
  p2p-tcp-pcp-dqn-model.tflite
  
  # pcap
  
  pcap_point2point-tcp-pcp-dqn-0-0.pcap
  ...
  pcap_point2point-tcp-pcp-dqn-4-3.pcap
  
  # server statics
  
  server-stats_point2point-pcp-dqn.json
  
  # training log (epsilon, obs, act, reward)
  
  training_log.json