root/Sejong_University_logo.svg.

png

Sejong University

Beamforming Design for Large-Scale

Antenna Arrays Using Deep Learning

AUTHORS

LE XUAN HOANG - 24114545

2025-04-17
Contents
List of Figures I

List of Tables II

1 Abstract 1

2 Introduction 2
2.1 The Topic of the Project . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2
2.2 Objective of the Project . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
2.3 Timeline . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3

3 Literature Review 4

4 System Design 6
4.1 System Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
4.2 BFNN Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
4.2.1 Input . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
4.2.2 Architecture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
4.2.3 Loss function . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
4.2.4 Performance metric . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
4.3 Assumption . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
4.4 Simulation parameters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11

5 Code Implementation 13
5.1 Device specification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
5.2 Environmental Setup . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
5.3 Structure of project folder . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
5.4 Replicating the Experiments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15

6 Performance Analysis 16
6.1 The Impact of Training Parameters . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
6.1.1 Parameters Setup . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
6.1.2 Performance Metric: Summed Spectral Efficiency . . . . . . . . . . . . . . . 16
6.1.3 Results and Discussion: Activation Functions . . . . . . . . . . . . . . . . . 17
6.1.4 Results and Discussion: Learning Rate and Epochs . . . . . . . . . . . . . . 18
6.1.5 Proposed Parameters for Subsequent Experience . . . . . . . . . . . . . . . 19
6.2 The Impact of Channel Parameters . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
6.2.1 Impact of CSI Quality PNR . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
6.2.2 Impact of Estimated Number of Paths Lest . . . . . . . . . . . . . . . . . . . 20

i
Convergence Project A for Unmanned Vehicles (417306-001) Contents

7 Conclusion 22

References 23

ii
Convergence Project A for Unmanned Vehicles (417306-001) List of Figures

List of Figures
1 Diagram of an MISO mmWave system with on RF chain [1]. . . . . . . . . . . . . . 6
2 Illustration of the two-stage DL-based HBF design approach: offline training and on-
line deployment [1]. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
3 The structure of project folder . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
4 Spectral Efficiency vs. SNR for different PNR levels . . . . . . . . . . . . . . . . . 20
5 Spectral Efficiency vs. SNR for different Lest levels . . . . . . . . . . . . . . . . . . 20

Page I of 24
Convergence Project A for Unmanned Vehicles (417306-001) List of Tables

List of Tables
1 Simulation parameters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
2 Experimental results with various Activations, Learning Rates, and Epochs . . . . . . 17
3 The summed spectral efficiency of different activation functions . . . . . . . . . . . 18
4 The summed spectral efficiency on different learning rates and epochs . . . . . . . . 18

Page II of 24
Convergence Project A for Unmanned Vehicles (417306-001) Abstract

1 Abstract
[1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15]
For the full code implementation, please refer to the GitHub repository: https://github.com/
lexuanhoang120/Beam-forming-with-deep-learning

Page 1 of 22
Convergence Project A for Unmanned Vehicles (417306-001) Introduction

2 Introduction
2.1 The Topic of the Project
Millimeter wave (mmWave) communication is essential for future high-data-rate wireless sys-
tems but suffers from high path loss, necessitating large antenna arrays and effective beamforming.
Practical hardware limitations often lead to hybrid analog-digital architectures, where optimizing the
phase-shifter-based analog beamformer under its constant modulus constraint presents a significant
challenge. Furthermore, real-world systems must contend with imperfect Channel State Information
(CSI), which can severely degrade the performance of traditional beamforming algorithms.
Deep Learning (DL) offers a promising data-driven alternative. Specifically, the Beamforming
Neural Network (BFNN) proposed by Lin and Zhu [1] was designed to address these challenges. It
directly generates the analog beamformer from estimated (imperfect) CSI, inherently satisfies hard-
ware constraints, uses a performance-oriented loss function (maximizing Spectral Efficiency - SE),
and demonstrated significant robustness and performance gains compared to traditional methods.
In this paper [1], the author proposed a deep learning (DL)-based beamforming (BF) design ap-
proach for large-scale antenna arrays. They developed a Beamforming Neural Network (BFNN) that
can be trained to optimize the beamformer for maximizing spectral efficiency while considering hard-
ware limitations and imperfect channel state information (CSI). The proposed BFNN demonstrates
significant performance improvements and strong robustness to imperfect CSI compared to traditional
BF algorithms. Key contributions of paper:

• A novel DL-based design approach that directly generates the optimized beamformer using
estimated CSI as input.

• A newly designed loss function that aligns closely with spectral efficiency performance.

• A two-stage design approach to enhance BFNN robustness against imperfect CSI.

While the BFNN framework [1] shows considerable potential, its sensitivity to specific config-
uration choices and channel conditions requires further investigation, as the original work relied on
empirically chosen parameters. This project aims to systematically analyze the performance of the
BFNN, focusing on how its achievable Spectral Efficiency (SE) is impacted by variations in its archi-
tecture, training parameters, and key channel characteristics like CSI quality. The primary objectives
are to:

• Investigate BFNN Architecture Sensitivity: Analyze the impact of network depth, width, and
activation functions on SE.

• Analyze BFNN Training Parameter Impact: Evaluate the effect of learning rate and the number
of training epochs on convergence and SE.

Page 2 of 22
Convergence Project A for Unmanned Vehicles (417306-001) Introduction

• Evaluate Robustness to Channel Conditions: Assess performance variations due to differing

numbers of channel paths (L) and quantify robustness to CSI quality by varying the Pilot-to-
Noise Ratio (PNR).

2.2 Objective of the Project

The primary goal is to analyze the impact of various parameters on the performance of the pro-
posed Beamforming Neural Network (BFNN). The key performance metric will be spectral efficiency
(SE), as used in the original paper [1].

• Validate the BFNN implementation by reproducing the baseline results.

• Provide comparative performance data and insights into parameter sensitivity and trade-offs.

• Gain a deeper understanding of the practical strengths and limitations of BFNN.

• Potentially offer guidelines for configuring the BFNN effectively in different scenarios.

After investigating these points, we will have some knowledge related to wireless communication.

2.3 Timeline
The project is conducted in a timeline of 6 weeks:

• Week 1: Paper selection and Literature review.

• Week 2: Writing proposal as Professor’s assignment .

• Week 3: Implementation of the method.

• Week 4: Continue the implementation of the method.

• Week 5: Writing the report and implementation of related methods.

• Week 6: Finalization of the report and implementation of related methods.

Page 3 of 22
Convergence Project A for Unmanned Vehicles (417306-001) Literature Review

3 Literature Review
Millimeter wave (mmWave) communication offers vast bandwidth for 5G and beyond but suffers
from high path loss, requiring large antenna arrays and beamforming [1, 3, 4]. Due to the high cost
and power consumption of fully digital beamforming, Hybrid Analog-Digital Beamforming (HBF) is a
practical alternative [1, 3, 4, 5]. HBF uses fewer RF chains connected via an analog network (typically
phase shifters), but optimizing this analog stage, especially under its constant modulus constraint, is
challenging [3, 4, 5].
Traditional model-based optimization methods for the analog beamformer include codebook-
based approaches like OMP [1], which are limited by codebook design, and continuous optimization
techniques like manifold optimization [3, 4] or iterative algorithms [5], which can be computationally
complex and may struggle with local optima. A major drawback shared by most traditional methods
[1, 3, 4, 5] is their reliance on perfect Channel State Information (CSI), an unrealistic assumption
leading to performance degradation in practice where only imperfect CSI is available [1, 2, 10].
Recognizing these limitations, researchers explored Deep Learning (DL) for beamforming [7,
12], leveraging its success in other wireless tasks [6, 8, 9, 11]. Early DL attempts for HBF [13, 14],
however, often still assumed perfect CSI, struggled to enforce hardware constraints directly, or relied
on codebooks.
A significant advancement was the Beamforming Neural Network (BFNN) proposed by Lin and
Zhu [1]. This project builds directly on their work. The BFNN uniquely addresses prior limitations
through several key contributions:
• It directly generates the analog beamformer using estimated (imperfect) CSI as input.
• It incorporates a custom "Lambda Layer" to inherently satisfy the constant modulus constraint.
• It uses a loss function based on maximizing Spectral Efficiency (SE), aligning training with the
communication goal.
• It employs a specific training strategy designed to enhance robustness to imperfect CSI.
The BFNN demonstrated superior SE and robustness compared to traditional methods [1, 4, 5],
especially under imperfect CSI. However, the original work [1] used empirically chosen network
hyperparameters. A systematic analysis of how the BFNN’s performance varies with its configuration
and operating conditions is needed.
This project aims to fill that gap by conducting a comprehensive sensitivity analysis of the
BFNN [1]. We will investigate how its spectral efficiency is affected by variations in network archi-
tecture (depth, width, activation functions), training parameters (learning rate, epochs), and channel
conditions (number of paths, CSI quality via PNR), providing crucial insights for its practical opti-
mization and deployment.
The detail of factors will be analyze:
• Model parameters

Page 4 of 22
Convergence Project A for Unmanned Vehicles (417306-001) Literature Review

– Activation Functions: Experiment with different activation functions in the dense layers
(Sigmoid, Tanh, ReLU variants).
– Learning Rate: Test different learning rates (0.001, 0.0001, 0.00001).
– Number of Training Epochs: Test different epochs (100, 500, 1000).

• Channel parameters

– Number of Estimated Channel Paths: Evaluate how different numbers of estimated channel
paths impact performance (1, 2).
– Pilot to Noise Ratio (PNR): Vary the PNR to simulate different levels of CSI quality (-20,
0, 20).

Page 5 of 22
Convergence Project A for Unmanned Vehicles (417306-001) System Design

4 System Design
This section details the simulated communication system, channel model, beamforming architec-
ture, and the specific configuration of the Beamforming Neural Network (BFNN) under investigation,
along with the parameters used for performance evaluation.

4.1 System Model

We simulate a downlink millimeter wave (mmWave) communication scenario consisting of a
single Base Station (BS) transmitting to a single User Equipment (UE), forming a Multiple-Input
Single-Output (MISO) channel configuration.

• Base Station (BS): Equipped with a large-scale Uniform Linear Array (ULA) comprising Nt =
64 antennas with half-wavelength spacing. Critically, the BS operates with limited hardware
complexity, featuring only a single Radio Frequency (RF) chain (NRF = 1).

• User Equipment (UE): Equipped with a single receive antenna.

figures/system_model.png

Figure 1: Diagram of an MISO mmWave system with on RF chain [1].

Consider the downlink of a narrowband multiple-input single-output (MISO) millimeter-wave

(mmWave) system with an analog beamforming (BF) architecture, as illustrated in Fig. 1. A base
station (BS) equipped with Nt antennas and a single RF chain transmits one data stream to a user
terminal with a single receive antenna.
Let s denote the transmitted symbol with normalized average symbol energy, i.e., E {|s|2 } = 1.
The symbol s is first multiplied by a scalar digital precoder vD (since there is only one RF chain, vD
is a scalar), and then by an Nt × 1 analog beamforming vector vRF , which is implemented using phase
shifters. The resulting transmitted signal is:

x = vRF vD s (1)

Page 6 of 22
Convergence Project A for Unmanned Vehicles (417306-001) System Design

The received signal r at the user, passing through the MISO mmWave channel h, is given by:

r = hH vRF vD s + n (2)
where n is additive white Gaussian noise (AWGN) modeled as a circularly symmetric complex
Gaussian random variable with zero mean and variance σ 2 , and hH is the channel vector between the
BS and the user.
We adopt the Saleh-Valenzuela channel model [10, 11], in which the channel vector hH includes
one line-of-sight (LoS) path and (L − 1) non-line-of-sight (NLoS) paths:
L
r
H Nt X
h = α l aH t
t (ϕl ) (3)
L l=1
where αl denotes the complex gain of the l-th path, and at (ϕtl ) represents the transmit antenna
array response vector associated with the azimuth angle of departure ϕtl . The term with l = 1 corre-
sponds to the LoS component.
Although this work focuses on narrowband analog beamforming aided by deep learning (DL), the
proposed design can be extended to broadband MIMO mmWave systems with hybrid beamforming
(HBF).
We use spectral efficiency (SE) as the optimization objective, a widely used metric in beamform-
ing literature [4, 5]. The SE for the studied system is given by:
 
1 H 2
R = log2 1 + 2 h vRF vD . (4)
σ
Under the constant modulus constraint |[vRF ]i |2 = 1 for i = 1, . . . , Np
t , and the transmit power
2
constraint ∥vRF vD ∥ ≤ P , it can be shown that the optimal value of vD is P/Nt . Substituting this
value simplifies the optimization problem for vRF as follows:
 
γ H 2
maximizevRF log2 1 + h vRF
Nt (5)
2
subject to |[vRF ]i | = 1, i = 1, . . . , Nt ,
where γ = P/σ 2 is the signal-to-noise ratio (SNR). Since SNR can typically be estimated more
accurately than the channel state information (CSI), we assume the estimated SNR γest = γ, and focus
our attention on handling imperfect CSI.
A crucial aspect of this simulation is the use of imperfect channel state information (CSI), which
reflects practical deployment conditions.

• Estimated CSI (hest ): This Nt × 1 vector represents the imperfect channel knowledge available
at the base station (BS). It is obtained using a practical mmWave channel estimation method
specifically, the hierarchical codebook-based estimator described in [2], as applied in [1]. The

Page 7 of 22
Convergence Project A for Unmanned Vehicles (417306-001) System Design

quality of hest depends on the Pilot-to-Noise Ratio (PNR) during the estimation phase. This
estimated CSI serves as the primary input to the BFNN during both training and testing.

• Perfect CSI (h): The true Nt × 1 channel vector is assumed to be available only during the
training phase of the BFNN. It is used exclusively to compute the loss function, enabling the
network to learn a mapping from imperfect channel estimates to effective beamformers targeting
the true channel.

• Estimated SNR (γest ): It is assumed that the signal-to-noise ratio (SNR) is estimated with high
accuracy, such that γest = γ.

4.2 BFNN Design

The core of the beamforming design is the BFNN proposed in [1]. This deep neural network
learns to map the estimated CSI (hest ) and estimated SNR (γest ) to the optimal analog beamforming
vector (vHF ).

figures/model.jpg

Figure 2: Illustration of the two-stage DL-based HBF design approach: offline training and online
deployment [1].

The proposed beamforming design consists of two stages: offline training and online deployment,
as depicted in Fig. 2.

• Offline Training Stage

– Simulated channel samples, pilot symbols, and noise are generated based on the system
model.
– A practical mmWave channel estimator (from [2]) is used at the base station (BS) to ob-
tain the estimated channel hest , using pilot transmissions and a hierarchical beamforming
codebook.

Page 8 of 22
Convergence Project A for Unmanned Vehicles (417306-001) System Design

– The estimated channel hest and estimated SNR γest (assumed to be equal to the true γ) are
used as inputs to the BFNN.
– The BFNN is trained to output the analog beamformer vRF by minimizing a loss function
based on spectral efficiency (SE), using the perfect CSI and true SNR only available during
training.
– This training enables the BFNN to learn how to approximate the ideal SE with perfect CSI
and to be robust against CSI estimation errors.
• Online Deployment Stage
– The same mmWave channel estimator is used to obtain hest at the BS.
– The trained BFNN takes hest as input and directly outputs the optimized analog beamform-
ing vector vRF .
– No perfect CSI is needed in this stage; the BFNN operates using only imperfect CSI.

4.2.1 Input
The input to the BFNN is a real-valued vector of size 2Nt , formed by concatenating the real and
imaginary parts of the estimated channel vector hest . Specifically, when Nt = 64, the input dimension
becomes 2 × 64 = 128.

4.2.2 Architecture
4.2.3 Loss function
The BFNN is trained offline using the Adam optimizer to minimize a loss function that is de-
signed to maximize the average spectral efficiency (SE) across the training dataset. The loss function
is defined as:
N  
1 X γn H 2
L=− log 1 + h vRF,n (6)
N n=1 2 Nt n
where:
• N is the total number of training samples,
• γn is the signal-to-noise ratio (SNR) of the n-th training sample,
• hn is the channel vector of the n-th sample,
• vRF,n is the analog beamforming vector output by the BFNN for the n-th sample,
• Nt is the number of transmit antennas.
Note that minimizing this loss function directly corresponds to maximizing the average SE over
the training dataset, making it well-aligned with the performance objective of the system.

Page 9 of 22
Convergence Project A for Unmanned Vehicles (417306-001) System Design

4.2.4 Performance metric

The primary metric used to evaluate the performance of the proposed BFNN is the Spectral
Efficiency (SE), which is measured in bits per second per Hertz (bits/s/Hz). For a given channel
realization h and the corresponding analog beamformer vRF , the instantaneous SE is calculated as:
N  
1 X γn H 2
SE = log h vRF,n (7)
N n=1 2 Nt n
where:

• N is the total number of training samples,

• γn is the signal-to-noise ratio (SNR) for the n-th sample,

• hH
n is the Hermitian transpose of the channel vector for the n-th sample,

• vRF,n is the analog beamforming vector generated by the BFNN for the n-th sample,

• Nt is the number of transmit antennas at the base station.

The final performance is typically evaluated by averaging the SE over multiple channel realiza-
tions and plotting it as a function of the SNR γ.

4.3 Assumption
System Model Validity: The project assumes the MISO downlink mmWave system model de-
scribed in [1] is a representative and relevant scenario for studying analog beamforming. This in-
cludes:

• Single-user MISO configuration.

• Narrowband channel.

• Uniform Linear Array (ULA) at the BS (Nt=64).

• Single RF chain architecture (NRF=1), leading to pure analog beamforming.

• Phase shifter implementation, imposing the constant modulus constraint.

Channel Model Accuracy: The Saleh-Valenzuela channel model with the specific parameters
used in [1] (Section IV: L=3 paths, specific power profiles, uniform AoD distribution) is assumed to
adequately represent typical mmWave propagation characteristics for the purpose of this comparative
analysis.

Page 10 of 22
Convergence Project A for Unmanned Vehicles (417306-001) System Design

CSI Estimation Method: It’s assumed that the method for obtaining imperfect CSI (hest) is
consistent with that used in [1] (i.e., based on the hierarchical estimator from [2]), and that its quality
is primarily determined by the Pilot-to-Noise Ratio (PNR). The simplification γest = γ from [1] is also
adopted.
Training Methodology: The two-stage training approach (offline training using hest as input and
h for loss calculation) and the use of the SE-based loss function (Eq.6) with the Adam optimizer are
assumed to be effective, as shown in [1].
Spectral Efficiency (SE) as Key Metric: SE is assumed to be the primary and sufficient perfor-
mance metric for evaluating the beamformer’s effectiveness, aligning with the objective function and
the analysis in [1].

4.4 Simulation parameters

Table 1 summarizes the key parameters used in the simulation, including system, channel, CSI,
BFNN architecture, and training parameters.

Page 11 of 22
Convergence Project A for Unmanned Vehicles (417306-001) System Design

Table 1: Simulation parameters

Category Parameter Description Values

Nt Number of BS antennas (ULA) 64
NRF Number of RF chains at BS 1
System & Hardware
– Number of UE antennas 1
γ (SNR) Signal-to-Noise Ratio (dB) Varied (−20 to 20
dB)
L True number of channel paths 3 (1 LoS, 2 NLoS)
Channel Model
Var(αl ) Path gain variance 1 (LoS), 10−0.5
(NLoS)
PNR Pilot-to-Noise Ratio for ĥest −20, 0, 20
CSI & Estimation (dB)
Lest Estimated number of paths (for 1, 2
context, see [1])
γest Estimated SNR input to BFNN γest = γ
Input Dim Size of BFNN input layer 128
Depth Number of Dense Hidden 3
BFNN Architecture Layers
Width Neurons per Hidden Layer 256 → 128 → 64
Activations Activation functions (Hidden → ReLU, Sigmoid,
Output Dense) Tanh
Output Dim Output dimension before 64
Lambda layer
Optimizer Algorithm for weight updates Adam
Learning Rate Step size for optimizer 0.001, 0.0001,
0.00001
BFNN Training Epochs Number of training passes 100, 500, 1000
Ntrain Number of training samples 9 × 104
Nval Number of validation samples 104
Ntest Number of testing samples 103
Batch Size Samples per weight update 256

Page 12 of 22
 Convergence Project A for Unmanned Vehicles (417306-001) Code Implementation

5 Code Implementation
The implementation of this project is based on the work presented in [1], but has been rewritten
using the PyTorch library1 . All of the code was written in Python. The code for creating the dataset,
as described in [1], was originally written in MATLAB. All the work from original paper is here2 .

5.1 Device specification

• CPU: Intel® Core™ i9-10900K CPU @ 3.70GHz×20

• GPU: 2×RTX 3090 (24GB)

• Memory: 128 GB

• Storage: 2.5 TB

5.2 Environmental Setup

These libraries must be installed prior to running the experiment. The version of Python used in
this project is 3.10.16. Below are some essential libraries required for the project:

• torch==2.6.0

• pandas==2.2.3

• matplotlib==3.10.1

• seaborn==0.13.2

• scipy==1.15.2

• tensorboard

We can install the required libraries using the following command in the terminal:
1 pip install −r requirements.txt

1
https://pytorch.org/
2
https://github.com/TianLin0509/BF-design-with-DL

Page 13 of 22
Convergence Project A for Unmanned Vehicles (417306-001) Code Implementation

5.3 Structure of project folder

Figure 3 describe the structure of project folder. Below is the description for each element in the
project.

figures/structure_folder.png

Figure 3: The structure of project folder

• make_dataset/: contains scripts or functions for creating datasets. It was writen in MATLAB.
Please kindly refer to the gen_samples.m for details. The codes are based on the work [1, 2].

• models_channel/: stores trained model files results related to the channel scenario.

• models_parameter/: stores tranined model files related to the parameter scenario.

• runs/: store experiment logs and training metadata in Tensorboard.

• train_set/: includes training and testing data.

• evaluation_results_channel.csv: CSV file containing evaluation results for the model in chan-
nel scenarios.

• evaluation_results_parameter.csv: CSV file containing evaluation results for the model in

parameter scenarios.

• README.md: providing an overview, setup instructions, and usage guide for the project.

• requirements.txt: lists all Python dependencies required for the project.

• result.ipynb: Jupyter Notebook, used for visualizing final results.

Page 14 of 22
Convergence Project A for Unmanned Vehicles (417306-001) Code Implementation

• test_channel.py: script for evaluating the trained models in channel scenarios.

• test_parameter.py: script for evaluating the trained models in parameter scenarios.

• train_channel.py: script for training the model in channel scenarios.

• train_parameter.py: script for training the model in parameter scenarios.

• utils.py: utility functions used across different scripts (data loading, computing rate, e.g.).

5.4 Replicating the Experiments

All files are organized to align with the experimental setup described in this report. To reproduce
the results, simply follow the steps below in order. For a detailed understanding of each script’s
functionality, refer directly to the source code and in-line comments.

• Step 1: Train the models for each scenario

Run the following commands in the terminal to train the models:

– Parameter-based scenario see Section 6.1 for more details:

1 python train_parameter.py

– Channel-based scenario see Section 6.2 for more details:

1 python train_channel.py

• Step 2: Evaluate the trained models

After training, use the following commands to evaluate the models:

– Parameter-based scenario see Section 6.1 for more details:

1 python test_parameter.py

– Channel-based scenario see Section 6.2 for more details:

1 python test_channel.py

• Step 3: Analyze the results

For result visualization and additional analysis, open and inspect the following Jupyter note-
book:
1 result.ipynb

Page 15 of 22
Convergence Project A for Unmanned Vehicles (417306-001) Performance Analysis

6 Performance Analysis
This section presents the simulation results evaluating the performance of the Beamforming Neu-
ral Network (BFNN). The analysis focuses on understanding the impact of key training parameters and
channel characteristics on the network’s ability to maximize spectral efficiency. For each experimental
setup during the training process, the model achieving the best performance on a validation set was
saved and subsequently evaluated on the testing dataset.

6.1 The Impact of Training Parameters

This subsection investigates the sensitivity of the BFNN’s performance to fundamental training
hyperparameters: activation function, learning rate, and the number of training epochs. The objective
is to identify settings that lead to effective learning and high spectral efficiency.

6.1.1 Parameters Setup

For this analysis, the core BFNN architecture remained fixed, consistent with the configuration
detailed in 4.2. The parameters systematically varied were:

• Activation Function (Hidden Layers): ReLU, Sigmoid, Tanh.

• Learning Rate: 0.00001, 0.0001, 0.001.

• Training Epochs: 100, 500, 1000.

All other system and channel parameters were held constant at their baseline values (e.g., Nt=64,
true L=3, specific PNR = 0).

6.1.2 Performance Metric: Summed Spectral Efficiency

Performance is primarily evaluated using the Summed Spectral Efficiency. This metric is cal-
culated by averaging the Spectral Efficiency (SE) over the test dataset at each simulated SNR point
(ranging from -20 dB to 20 dB) and then summing these average SE values across the entire SNR
range. A higher sum indicates better overall performance. While useful for comparing aggregate
trends, this metric averages potential performance variations at specific SNR regimes. Detailed results
for all 27 combinations are provided in Table. 2.

Page 16 of 22
Convergence Project A for Unmanned Vehicles (417306-001) Performance Analysis

Table 2: Experimental results with various Activations, Learning Rates, and Epochs

Experiment Activation Learning Rate Epochs Summed Spectral Efficiency

0 relu 0.00001 100 35.637077
1 relu 0.00001 500 37.192214
2 relu 0.00001 1000 37.248853
3 relu 0.0001 100 37.384750
4 relu 0.0001 500 37.651309
5 relu 0.0001 1000 37.434600
6 relu 0.001 100 37.986214
7 relu 0.001 500 37.826352
8 relu 0.001 1000 37.957547
9 sigmoid 0.00001 100 31.439638
10 sigmoid 0.00001 500 36.408837
11 sigmoid 0.00001 1000 37.435428
12 sigmoid 0.0001 100 37.500266
13 sigmoid 0.0001 500 38.421607
14 sigmoid 0.0001 1000 38.508183
15 sigmoid 0.001 100 38.935293
16 sigmoid 0.001 500 38.674803
17 sigmoid 0.001 1000 38.934986
18 tanh 0.00001 100 33.129587
19 tanh 0.00001 500 36.513843
20 tanh 0.00001 1000 37.390074
21 tanh 0.0001 100 37.720636
22 tanh 0.0001 500 38.136992
23 tanh 0.0001 1000 37.631534
24 tanh 0.001 100 38.088008
25 tanh 0.001 500 38.285456
26 tanh 0.001 1000 38.332740

6.1.3 Results and Discussion: Activation Functions

Table 3 aggregates the average Summed Spectral Efficiency achieved by each activation function,
averaged across all tested learning rates and epoch counts. On average, all three activation functions
(ReLU, Sigmoid, Tanh) yield remarkably similar overall performance. ReLU and Sigmoid are virtu-
ally tied, with Tanh performing only slightly lower on average. This suggests that, for this specific
BFNN architecture and task, the choice of activation function (among these common options) might
not be the most critical factor influencing the average outcome across various training settings.

Page 17 of 22
Convergence Project A for Unmanned Vehicles (417306-001) Performance Analysis

Table 3: The summed spectral efficiency of different activation functions

Activation Summed Spectral Efficiency

relu 37.368769
sigmoid 37.362116
tanh 37.247652

Table 4: The summed spectral efficiency on different learning rates and epochs

Epochs Learning Rate Summed Spectral Efficiency

100 0.00001 33.402101
100 0.0001 37.535217
100 0.001 38.336505
500 0.00001 36.704965
500 0.0001 38.069970
500 0.001 38.262203
1000 0.00001 37.358119
1000 0.0001 37.858106
1000 0.001 38.408424

6.1.4 Results and Discussion: Learning Rate and Epochs

Table 4 presents the average Summed Spectral Efficiency, illustrating the combined impact of
learning rate and training duration. The results highlight the strong interplay between learning rate
and training epochs, aligning with theoretical expectations:

• The lowest learning rate (0.00001) clearly leads to slow convergence, requiring substantially
more epochs (500-1000) to approach the performance achieved much faster by higher rates.
• The highest tested learning rate (0.001) demonstrates rapid convergence, achieving the best
average performance after only 100 epochs and maintaining high performance (reaching the
overall peak average) up to 1000 epochs. This suggests 0.001 is close to an optimal rate for this
setup, enabling efficient training.
• The intermediate learning rate (0.0001) offers a viable alternative, showing significant improve-
ment between 100 and 500 epochs, after which performance tends to plateau, indicating conver-
gence within that timeframe.
• Regarding epochs, 100 epochs is generally insufficient, especially with low learning rates. Per-
formance significantly benefits from increasing training to 500 epochs. Further training to 1000

Page 18 of 22
Convergence Project A for Unmanned Vehicles (417306-001) Performance Analysis

epochs yields diminishing returns for the faster learning rates (0.001, 0.0001), suggesting con-
vergence is largely achieved around 500-1000 epochs.

6.1.5 Proposed Parameters for Subsequent Experience

Based on this analysis, aiming for efficient training and high performance, the following BFNN
parameters were selected for the subsequent channel parameter investigations:

• Activation Function: ReLU (chosen as a robust, commonly used baseline that performed well
on average).

• Learning Rate: 0.001 (demonstrated fastest convergence and highest overall performance).

• Epochs: 1000 (selected to ensure full convergence, capturing potentially the best performance
achievable with the chosen learning rate).

6.2 The Impact of Channel Parameters

This subsection evaluates the BFNN’s performance sensitivity to variations in the channel envi-
ronment, specifically the quality of the Channel State Information (CSI) and the accuracy of channel
sparsity estimation. The BFNN model used here employs the parameters determined in 6.1.5 (Activa-
tion=ReLU, LR=0.001, Epochs=1000).

6.2.1 Impact of CSI Quality PNR

Figure 4 plots the Spectral Efficiency (SE) versus SNR for three different Pilot-to-Noise Ratio
(PNR) levels used during the channel estimation phase, which generates the hest input for the BFNN.
Higher PNR corresponds to higher quality, more accurate CSI.

• The graph clearly shows that SE increases with SNR for all PNR levels, as theoretically ex-
pected. Crucially, it demonstrates a strong dependence on CSI quality: higher PNR consistently
leads to higher SE across the entire SNR range. For instance, at an SNR of 20 dB, the SE
achieved with PNR = 20dB (11.15 bits/s/Hz) is substantially higher than with PNR = 0dB
(9.79 bits/s/Hz) and PNR = −20dB (7.33 bits/s/Hz).

• This result confirms the importance of accurate channel estimation for effective beamforming.
While the BFNN exhibits robustness, functioning even with poor CSI (PNR = −20dB ), its
performance ceiling is directly limited by the quality of the input hest. The performance gap
widens at higher SNRs, indicating that accurate beam steering enabled by good CSI is increas-
ingly critical when noise is less dominant. This underscores the practical need for efficient and
accurate channel estimation mechanisms to fully leverage DL-based beamforming techniques
like the BFNN.

Page 19 of 22
Convergence Project A for Unmanned Vehicles (417306-001) Performance Analysis

figures/pnr_level.png

Figure 4: Spectral Efficiency vs. SNR for different PNR levels

figures/lest_dif.png

Figure 5: Spectral Efficiency vs. SNR for different Lest levels

6.2.2 Impact of Estimated Number of Paths Lest

Figure 5 investigates the effect of inaccuracies in estimating the channel sparsity (number of
paths, L) during the channel estimation step. The graph compares performance when the estimator
assumes Lest = 1 or Lest = 2 paths.

Page 20 of 22
Convergence Project A for Unmanned Vehicles (417306-001) Performance Analysis

As with PNR, SE increases with SNR for both Lest values. The key finding is that assuming
Lest = 1 consistently yields higher SE compared to assuming Lest = 1. The performance gap is
significant, reaching over 2 bits/s/Hz at high SNRs.
This demonstrates that the performance of the subsequent BFNN beamforming is sensitive to the
accuracy of the channel model parameters assumed by the channel estimator. When the estimator as-
sumes Lest = 1, it likely captures only the dominant path, resulting in a less accurate hest compared to
when it assumes Lest = 2, which can capture two paths. This more accurate hest (from Lest = 2) allows
the BFNN, trained on the true channel characteristics, to generate a more effective beamformer. While
the system shows robustness to this underestimation (performance with Lest = 1 is still reasonable),
striving for accurate sparsity estimation during the CSI acquisition phase is crucial for maximizing the
overall system performance enabled by the BFNN.

Page 21 of 22
Convergence Project A for Unmanned Vehicles (417306-001) Conclusion

7 Conclusion

Page 22 of 22
Convergence Project A for Unmanned Vehicles (417306-001) References

References
[1] T. Lin and Y. Zhu, “Beamforming design for large-scale antenna arrays using deep learning,”
IEEE Wireless Communications Letters, vol. 9, no. 1, pp. 60–64, 2020.
[2] A. Alkhateeb, O. E. Ayach, G. Leus, and R. W. Heath, “Channel estimation and hybrid precoding
for millimeter wave cellular systems,” IEEE Journal of Selected Topics in Signal Processing,
vol. 8, pp. 831–846, Oct. 2014.
[3] T. Lin, J. Cong, Y. Zhu, J. Zhang, and K. B. Letaief, “Hybrid beamforming for millimeter wave
systems using the mmse criterion,” IEEE Transactions on Communications, vol. 67, pp. 3693–
3708, May 2019.
[4] X. Yu, J. Shen, J. Zhang, and K. B. Letaief, “Alternating minimization algorithms for hybrid pre-
coding in millimeter wave mimo systems,” IEEE Journal of Selected Topics in Signal Processing,
vol. 10, pp. 485–500, Apr. 2016.
[5] F. Sohrabi and W. Yu, “Hybrid digital and analog beamforming design for large-scale antenna
arrays,” IEEE Journal of Selected Topics in Signal Processing, vol. 10, pp. 501–513, Apr. 2016.
[6] H. Ye, G. Y. Li, and B. Juang, “Power of deep learning for channel estimation and signal detection
in ofdm systems,” IEEE Wireless Communications Letters, vol. 7, pp. 114–117, Feb. 2018.
[7] S. Dörner, S. Cammerer, J. Hoydis, and S. Brink, “Deep learning based communication over
the air,” IEEE Journal of Selected Topics in Signal Processing, vol. 12, pp. 132–143, Feb. 2018.
[8] X. Gao, S. Jin, C. Wen, and G. Y. Li, “Comnet: Combination of deep learning and expert knowl-
edge in ofdm receivers,” IEEE Communications Letters, vol. 22, pp. 2627–2630, Dec. 2018.
[9] C. Wen, W. Shih, and S. Jin, “Deep learning for massive mimo csi feedback,” IEEE Wireless
Communications Letters, vol. 7, pp. 748–751, Oct. 2018.
[10] X. Gao, L. Dai, S. Han, C.-L. I, and X. Wang, “Reliable beamspace channel estimation for
millimeter-wave massive mimo systems with lens antenna array,” IEEE Transactions on Wireless
Communications, vol. 16, pp. 6010–6021, Sep. 2017.
[11] H. He, C. Wen, S. Jin, and G. Y. Li, “Deep learning-based channel estimation for beamspace
mmwave massive mimo systems,” IEEE Wireless Communications Letters, vol. 7, pp. 852–855,
Oct. 2018.
[12] I. Goodfellow, Y. Bengio, and A. Courville, Deep Learning. Cambridge, MA: MIT Press, 2016.
[13] H. Huang, Y. Song, J. Yang, G. Gui, and F. Adachi, “Deep-learning-based millimeter-wave mas-
sive mimo for hybrid precoding,” IEEE Transactions on Vehicular Technology, vol. 68, pp. 3027–
3032, Mar. 2019.

Page 23 of 22
Convergence Project A for Unmanned Vehicles (417306-001) References

[14] A. Alkhateeb, S. Alex, P. Varkey, Y. Li, Q. Qu, and D. Tujkovic, “Deep learning coordinated
beamforming for highly-mobile millimeter wave systems,” IEEE Access, vol. 6, pp. 37328–
37348, 2018.

[15] O. E. Ayach, S. Rajagopal, S. Abu-Surra, Z. Pi, and R. W. Heath, “Spatially sparse precoding
in millimeter wave mimo systems,” IEEE Transactions on Wireless Communications, vol. 13,
pp. 1499–1513, Mar. 2014.

Page 24 of 22