Ensemble Decision for Spam Detection Using

Term Space Partition Approach.

Ying Tan , Senior Member, IEEE, Quanbin Wang, and Guyue Mi

Abstract—This paper proposes an ensemble decision approach that combines global and
local features of e-mails together to detect spam effectively. In the proposed method, a
special feature construction method named term space partition (TSP) is utilized to divide
the whole term space into several subspaces and adopt different feature construction
strategies on each of them, respectively. This method can make each term play a distinct
and important role when conducting detection. This method is utilized and extended by
introducing the sliding window technique to extract local features from e-mails. The global
classifier and local classifiers are constructed on a global feature vector set and local feature
vector sets, respectively, and together make the ensemble decision by adopting the voting
technique. The principles of the TSP-based approach and the mechanism of the ensemble
decision method are presented in detail. Five different and standard benchmark corpora are
applied to experiments for the performance evaluation of this proposed method.
Comprehensive experimental results show that the proposed method brings significant
performance improvement and better robustness on the basis of the TSP-based approach.
In addition, the proposed method outperforms the current prevalent and state-of-the-art
approaches, especially when a comprehensive consideration of performance, efficiency, and
robustness is taken. This endows it with flexible capability and adaptivity in real-world
applications.

Advantages:
Improved accuracy: Ensemble methods often lead to better accuracy compared to
individual classifiers by combining multiple weak learners into a stronger one.
Robustness: By aggregating predictions from multiple models, ensemble methods are often
more robust to noise and outliers in the data.
Reduced overfitting: Ensemble methods can help reduce overfitting, especially when using
techniques like bagging or boosting.
Flexibility: The term space partition approach might offer flexibility in handling different
types of features or data representations.
Scalability: Depending on the implementation, ensemble methods can be scalable to large
datasets and high-dimensional feature spaces.
Disadvantages:
Complexity: Ensemble methods can be more complex to implement and understand
compared to individual classifiers.
Computational cost: Building and training multiple models can be computationally
expensive, especially for large datasets or complex models.
Increased training time: Ensemble methods typically require more training time compared
to single classifiers due to the need to train multiple models.
Potential for overfitting: While ensemble methods can reduce overfitting, there's still a risk
of overfitting, especially if not properly tuned or if using complex models.
Interpretability: Ensemble methods may sacrifice interpretability for improved
performance, making it harder to understand the reasoning behind predictions.

A Novel Stream Clustering Framework for Spam

Detection in Twitter
Hadi Tajalizadeh and Reza Boostani
Abstract—Stream clustering methods have been repeatedly used for spam filtering in
order to categorize input messages/tweets into spam and non-spam clusters. These
methods assume each cluster contains a number of neighbour small (micro) clusters, where
each micro-cluster has a symmetric distribution. Nonetheless, this assumption is not
necessarily correct, and big micro-clusters might have asymmetric distribution. To enhance
the assigning accuracy of former methods in their online phase, we suggest replacing the
Euclidean distance with a set of classifiers in order to assign incoming samples to the most
relative micro cluster with arbitrary distribution. Here, a set of incremental Naïve Bayes
(INB) classifiers is trained for micro-clusters whose population exceeds a threshold. These
INBs can capture the mean and boundary of micro-clusters, while the Euclidean distance
just considers the mean of clusters and acts inaccurately for asymmetric big micro-clusters.
In this paper, Den-Stream was promoted by the proposed framework, called here as INB-
Den-Stream. To show the effectiveness of INB-Den-Stream, state-of-the-art methods such as
Den-Stream, Stream KM++, and Clustream were applied to the Twitter datasets, and their
performance was determined in terms of purity, general precision, general recall, F1
measure, parameter sensitivity, and computational complexity. The compared results
implied the superiority of our method to the rivals in almost all the datasets.

Advantages:
Real-time detection: Stream clustering enables real-time detection of spam tweets as they
are posted, allowing for timely responses to emerging threats.
Scalability: Stream clustering frameworks are designed to handle large volumes of data,
making them suitable for processing the continuous stream of tweets on Twitter.
Adaptability: The framework may be designed to adapt to changes in spamming techniques
or the evolving nature of Twitter data, ensuring ongoing effectiveness.
Reduced manual effort: Automated clustering techniques can reduce the need for manual
labeling and intervention, saving time and resources in spam detection.
Efficient resource utilization: By processing tweets in real-time and identifying spam
clusters efficiently, the framework may help optimize resource utilization in spam detection
systems.

Disadvantages:
Complexity: Implementing and maintaining a stream clustering framework for spam
detection can be complex, requiring expertise in machine learning, data processing, and
Twitter-specific knowledge.
Parameter tuning: Stream clustering algorithms often have parameters that need to be
carefully tuned for optimal performance, which can be challenging, especially in dynamic
environments like Twitter.
Concept drift: The nature of Twitter data may lead to concept drift, where the
characteristics of spam tweets change over time, requiring continuous monitoring and
adaptation of the clustering framework.
Noise sensitivity: Stream clustering frameworks may be sensitive to noise in the data,
leading to false positives or missed spam tweets if not adequately addressed.
Evaluation challenges: Assessing the effectiveness of a stream clustering framework for
spam detection can be challenging, particularly in the absence of labeled data or ground
truth, making it difficult to validate performance objectively.

A Neural Network-Based Ensemble Approach for

Spam Detection in Twitter
Sreekanth Madisetty and Maunendra Sankar Desarkar
Abstract—As social networking sites get more popular, spammers target these sites to
spread spam posts. Twitter is one of the most popular online social networking sites where
users communicate and interact on various topics. Most of the current spam filtering
methods on Twitter focus on detecting spammers and blocking them. However, spammers
can create a new account and start posting new spam tweets again. So there is a need for
robust spam detection techniques to detect the spam at tweet level. These types of
techniques can prevent spam in real-time. To detect the spam at tweet level, features are
often defined, and appropriate machine-learning algorithms are applied in the literature.
Recently, deep learning methods have shown fruitful results on several natural language
processing tasks. We want to use the potential benefits of these two types of methods for
our problem. Toward this, we propose an ensemble approach for spam detection at the
tweet level. We develop various deep learning models based on convolutional neural
networks (CNNs). Five CNNs and one feature-based model are used in the ensemble. Each
CNN uses different word embeddings (Glove, Word2vec) to train the model. The feature-
based model uses content-based, user-based, and n-gram features. Our approach combines
both deep learning and traditional feature-based models using a multilayer neural network
which acts as a meta-classifier. We evaluate our method on two data sets, one data set is
balanced, and another one is imbalanced. The experimental results show that our proposed
method outperforms the existing methods.

Advantages:
Robustness: By combining deep learning models with traditional feature-based approaches,
the ensemble method can potentially capture a wider range of spam characteristics, making
it more robust to different types of spam.
Real-time prevention: Detecting spam at the tweet level allows for real-time prevention of
spam posts, enhancing the user experience and reducing the spread of harmful content on
Twitter.
Utilization of deep learning: Leveraging deep learning methods, such as convolutional
neural networks (CNNs), can exploit the intricate patterns and relationships in tweet data,
potentially leading to more accurate spam detection.
Flexibility: The ensemble approach allows for flexibility in combining multiple models, each
utilizing different word embeddings and features, which can improve the adaptability of the
spam detection system to varying types of spam.
Performance improvement: The experimental results indicate that the proposed ensemble
approach outperforms existing methods, suggesting its effectiveness in spam detection on
Twitter.

Disadvantages:
Complexity: Implementing and training multiple deep learning models, along with the
feature-based model, and integrating them into a multilayer neural network can introduce
complexity to the system, requiring expertise in machine learning and neural networks.
Computational resources: Training multiple deep learning models and conducting ensemble
learning may require significant computational resources, including powerful hardware and
potentially lengthy training times.
Feature engineering: Extracting and selecting relevant features for the feature-based model
can be challenging and time-consuming, requiring domain knowledge and experimentation
to identify the most effective features for spam detection.
Data imbalance: Addressing imbalanced datasets, as mentioned with the imbalanced
dataset used in the evaluation, requires careful consideration to avoid biased model
performance and to ensure fair evaluation of the spam detection system.
Interpretability: While the ensemble approach may achieve high performance, interpreting
the decisions made by the combined models, especially in the context of deep learning, may
be challenging, potentially limiting the transparency of the spam detection system.

Web Spam Detection: New Classification

Features Based on Qualified Link Analysis and
Language Models
Lourdes Araujo and Juan Martinez-Romo
Abstract—Web spam is a serious problem for search engines because the quality of their
results can be severely degraded by the presence of this kind of page. In this paper, we
present an efficient spam detection system based on a classifier that combines new link-
based features with language-model (LM)--based ones. These features are not only related
to quantitative data extracted from the Web pages but also to qualitative properties, mainly
of the page links. We consider, for instance, the ability of a search engine to find, using
information provided by the page for a given link, the page that the link actually points at.
This can be regarded as indicative of the link's reliability. We also check the coherence
between a page and another one pointed at by any of its links. Two pages linked by a
hyperlink should be semantically related, by at least a weak contextual relation. Thus, we
apply an LM approach to different sources of information from a web page that belongs to
the context of a link, in order to provide high-quality indicators of Web spam. We have
specifically applied the Kullback–Leibler divergence on different combinations of these
sources of information in order to characterize the relationship between two linked pages.
The result is a system that significantly improves the detection of Web spam using fewer
features, on two large and public datasets such as WEBSPAM-UK2006 and WEBSPAM-
UK2007.

Advantages:
Improved detection accuracy: Introducing new classification features based on qualified link
analysis and language models can potentially enhance the ability to differentiate between
legitimate web content and spam, leading to improved detection accuracy.
Robustness: By incorporating features from both link analysis and language models, the
detection system may become more robust against evolving spamming techniques, as it
considers multiple aspects of web content.
Adaptability: Language models can capture semantic and syntactic features of web content,
allowing the detection system to adapt to changes in spamming tactics and content trends
over time.
Efficiency: Effective feature selection and extraction techniques can help improve the
efficiency of the detection system, reducing computational overhead and resource
requirements.
Reduced false positives: By leveraging qualified link analysis and language models, the
detection system may be better equipped to distinguish between legitimate links and
spammy links, potentially reducing false positive detections.

Disadvantages:
Complexity: Implementing features based on qualified link analysis and language models
may introduce complexity to the detection system, requiring expertise in both web analysis
techniques and natural language processing.
Data availability and quality: Access to reliable data for training and evaluating the
detection system, especially for building language models and conducting qualified link
analysis, may pose challenges, potentially affecting the effectiveness of the system.
Computational resources: Processing and analyzing web content using language models and
link analysis techniques may require significant computational resources, particularly for
large-scale web datasets, which could impact the scalability of the detection system.
Interpretability: Incorporating complex features from language models may reduce the
interpretability of the detection system, making it challenging to understand the rationale
behind individual classification decisions.
Generalization: The effectiveness of the new classification features may depend on the
specific characteristics of the web content and the spamming techniques employed, limiting
the generalization of the detection system across different types of web spam.

Statistical Features Based Real-time Detection of

Drifted Twitter Spam
Chao Chen, Yu Wang, Jun Zhang,Yang Xiang,Wanlei Zhou, and Geyong Min
Abstract—Twitter Spam has become a critical problem nowadays. Recent works focus on
applying machine learning techniques for Twitter spam detection, which makes use of the
statistical features of tweets. In our labeled tweets dataset, however, we observe that the
statistical properties of spam tweets vary over time, and thus the performance of existing
machine learning-based classifiers decreases. This issue is referred to as “Twitter Spam
Drift”. In order to tackle this problem, we first carry out a deep analysis of the statistical
features of one million spam tweets and one million non-spam tweets and then propose a
novel Lfun scheme. The proposed scheme can discover “changed” spam tweets from
unlabelled tweets and incorporate them into the classifier’s training process. A number of
experiments are performed to evaluate the proposed scheme. The results show that our
proposed Lfun scheme can significantly improve spam detection accuracy in real-world
scenarios.

Advantages:
Real-time detection: The use of statistical features enables real-time detection of drifted
Twitter spam, allowing for timely responses to emerging spamming patterns and minimizing
the impact on users.
Adaptability to drift: Statistical features can capture changes in spamming characteristics
over time, making the detection system adaptable to drift in spamming behaviors without
the need for manual intervention.
Scalability: Statistical features typically require minimal computational resources, making
them suitable for processing large volumes of Twitter data in real-time, thereby enhancing
the scalability of the detection system.
Reduced false positives: By focusing on statistical features indicative of spam, the detection
system may be able to reduce false positive detections, thereby improving the overall
accuracy of spam detection on Twitter.
Efficiency: Statistical feature-based detection methods are often computationally efficient,
enabling rapid processing of Twitter data streams and facilitating quick decision-making in
detecting spam.

Disadvantages:
Limited feature representation: Statistical features may not capture all nuances of
spamming behaviors, potentially leading to missed detections or false negatives if important
characteristics are not adequately represented.
Vulnerability to evasion techniques: Sophisticated spammers may intentionally manipulate
their activities to evade detection based on statistical features, reducing the effectiveness of
the detection system over time.
Dependency on data quality: The effectiveness of statistical feature-based detection relies
heavily on the quality and reliability of the underlying Twitter data, which may vary due to
factors such as data noise, bias, or sampling issues.
Generalization: Statistical features may not generalize well across different types of Twitter
spam, leading to limitations in the detection system's ability to effectively identify diverse
spamming behaviors.
Interpretability: Statistical feature-based detection methods may lack interpretability,
making it challenging to understand the reasoning behind individual detection decisions and
hindering efforts to refine and improve the detection system over time.