AIoT-based Smart Cart System
AIoT-based Smart Cart System
2022 International Conference on Smart Technologies and Systems for Next Generation Computing (ICSTSN) | 978-1-6654-2111-9/22/$31.00 ©2022 IEEE | DOI: 10.1109/ICSTSN53084.2022.9761346
Abstract—- A supermarket is a place where customers go to and RFID technology to come up with a solution for this
purchase and pay for their daily necessities. As a result, particular problem and also to enhance the customer’s
calculating the number of things sold and generating a bill for shopping experience. The main motive of the discussed model
the consumer is required. Generally shopping in a mall is very is to provide a superior shopping environment that could allow
tedious where the right items needed are difficult to find. After the customer to access high-end technology to reduce
that, standing in line to bill all of the things are exhausting. For manpower and time [2]. One of the most significant challenges
this, the developed model here would be a solution which is a in implementing AIoT (Artificial Intelligence of Things) for
smart shopping cart system that would keep track of purchased these types of practical applications is the level of security it
products, as well as bill payment, is done using RFID (Radio
provides especially here we need to prevent shoplifting, which
Frequency Identification). In this paper, the automation of
invoicing and payment is done using image processing, RFID
is addressed by combining image processing to track and keep
scanners, and tags for consumers' convenience. In this case, note of the products to prevent this type of activity. The motive
RFID cards are being used, which are similar to the smart wallet of this paper is to build and create an Intelligent Smart
systems that are popular in malls these days. Since assigning Shopping Cart with the aim of reducing shopping time and
RFID tags to every item is a tiresome process, instead, they will enhancing the supermarket shopping experience.
only be assigned to a group of the same products, and the In this framework, we have decided to implement RFID for
number of items chosen from each group will be determined by
the sake of replacing the traditional payment method after
object detection and identification using the YOLO (You only
billing to create a customer-centered environment, where the
look once) neural network technique. The proposed technique
can be easily developed and tested on a commercial scale in a
customer can prepay the money to the digital wallet like the
real-world situation in the future. As a result, the proposed existing model present in many malls, with the help of RFID
model outperforms the alternatives. cards and RFID scanners [5]. This is more safe and encrypted
mode of payment which also could help the customers with the
Keywords— Super market; RFID; Smart Shopping Cart; budget. Since the customer will only add money to their wallet
YOLO neural network; Object detection based on their planned expenditure thereby helping to facilitate
an indirect mode of keeping up with their budget. The RFID
I. INTRODUCTION scanners are fixed in different places all over the supermarkets
A supermarket's customers are largely local households where the LCDs and scanners are connected to a
and small businesses that need to replenish their basic products microcontroller to display the indication of payment being
stock on a regular basis. A supermarket's suppliers are usually done. Also, this displays the bill and total price so the customer
domestic products makers who are positioned far distant from could be aware of the price they are going to pay. This is being
their final clients. The drastic growth and development in the done instead of having this system in every cart which could
field of IoT have paved the way for many cost-effective and turn into a very expensive model. Since this is only used to do
highly reliable models that can be easily implemented and payments after billing is done and a sufficient number of units
managed to address our day-to-day problems. Nowadays, will be installed so there won’t be any kind of delay happening
customers face a big problem in every supermarket or because of this choice of implementation. A different method
convenience store when it comes to queues in the billing is initiated for generating invoices and ensuring anti-thievery
counters which is both time-consuming and exhausting. Apart without a need for billing counters compared to other newly
from that, it reduces the enthusiasm of people to go shopping proposed methods which are done through RFID tags and
and also forces the store’s management to get equipped with scanners. It also contradicts the sole aim of the proposed
laborers more than they need. Thereby highly increasing the framework which is to save time and money. So instead, here
margins of the working capital. This paper, portrayals the image processing is used to do multiple object detection at the
incorporation of some applications of IoT, Image processing, same time in that way it is easy to identify and classify items
in the shopping cart. Using this method helps in adding and
Authorized licensed use limited to: UNIVERSITAETSBIBLIOTHEK CHEMNITZ. Downloaded on December 22,2023 at 17:08:19 UTC from IEEE Xplore. Restrictions apply.
removing items in between with ease. There is a small delay to the server which is interfaced with the cashier management
between taking the image input and starting to process them, system to send items data once the purchase is done and
which helps to ensure all products are immobile for proper receives an appropriate response from the cashier
detection as all of this is done in real-time [4]. This is done to management system. This system also helps in providing a
make a list of the products, count each product picked by the personalized experience to the customers by giving product
customers, then mention the price as per the data fed into the preferences based on their activity [2].
system, and also for summing the prices to get a total price of
all the items purchased. The bill will be stored in an CSV sheet The authors use ATMEGA 328 microcontroller as a
as a result of image processing. For object detection, to count
Center of data processing also acts as a connection to connect
the products under the same set and to find the total number of
products, here YOLO version 3 neural network architecture is all peripherals into one system [3]. Every cart has a dedicated
being used. This algorithm is preferred over the others for its barcode reader on its own to reduce the time by
faster and precise results [11]. To take the input for image simultaneously scanning while picking and dropping the
processing it is necessary to place a camera rite above the cart products. There is a DC motor that controls the flap cover
to provide the full view of things present in the cart. After the opening and closing in the cart which is automated with some
billing these details are needed to be sent to the delay or can be manually opened and closed just to make sure
microcontroller, this transmission is done through Bluetooth or only scanned items go inside the cart. As soon as one item is
Wi-Fi to RFID scanners [12] so that the payment process can scanned the LCD shows the product name and total price
be seamless. The contribution of this paper is the development which is just the sum of prices of all items in the cart at that
of a model that improves the shopping experience of the time. The total varies if one or more items are added or
customers and also helps the shop or supermarket to manage removed from the cart. In this way, customers can be kept
and monitor the customers easily and effectively. This posted about the status of the cart, their purchases, and their
technology could be the starting point for fully automated budget. Even the products can be removed by scanning it
supermarkets in the future also helps to reduce human-human again while taking it out, by doing this the amount will be
contact thereby avoiding contagiousness. subtracted from the bill. Then Wi-Fi module is used to send
II. LITERATURE SURVEY the billing data to the webserver which was done with the help
of HTML. This can be accessed through the exit counters for
In order to make shopping easier for the customers [1] quick payment when the customers are leaving the stores.
incorporates smart shopping carts which are equipped with
certain features. Firstly, a barcode scanner is used to avoid The authors in [4] proposed a model which is almost alike
separate billing counters using this, the customer could to [3] in terms of performance when it comes to using
directly scan the items as soon as they take it, the camera in ATMEGA controller while using servo motor instead of DC
the cart to monitor the customers if something looks motor but almost used very similarly for closing the flap
suspicious, a weight sensor which could calculate the total covers to ensure anti-thievery, avoid mistakes and to have
weight of all items so it could be later compared with the ideal LCD to display real-time billing. Here the scanning is done
weight of all that items and finally, there is a small computer with RFID scanners so multiple items can be scanned and
is used for computing purpose. To save the cost of displaying billed at the same time also customers can directly pay from
all the items data in a monitor, it is done through smartphones anywhere in the supermarket just by pressing the payment
of the customers who are connected to the cart via Wi-Fi. The button present in the cart. Here the RFID is used so the data
Wi-Fi is also used for connecting different carts to share the can be interfaced with both the server and customized mobile
bills and do parallel shopping which could save a lot of time. applications for the management and the customers. To
There is an app built for this purpose that asks customers to facilitate this, RFID tags are added to each product and the
log-in, so every item suggestion will be customized for tags are removed while going out. Also, here the customers
everyone based on their purchase history. This could help to can access the whole bill in real-time to ensure, helps to
clear the congestion in the billing counters in no time since modify, and can be used as a cashless online payment
only the payment is done there which can be comparatively platform after checking out the cart. This is a safer method
far efficient compared to the traditional methods of shopping. since all the customers are having a login id and password
applied with discounts while checking out in the app and it is
The automation in supermarkets is aimed to provide an in- seamlessly updating the data in-app and server at the same
person experience to buy items with a comfort level that is time.
equal to shopping from home to the customers and the model
suggested in this paper made it more attainable by Discussed framework in [5] consists of a Product
implementing the RFID tags and scanners for scanning and Identification Device (PID) is used that contains a
billing which is controlled by Arduino UNO microcontroller. microcontroller placed on every cart. Has an LCD to show
Also, the controller’s peripheral buses help to add Bluetooth live updates in the billing process, RFID reader for scanning
module to this system, enabling the connection of the product and feeding the data as a bill. This is possible with having all
identification system to the mobile application that was items in the supermarket hooked with RFID tags. EEPROM
created. This delivers features which help customers by and ZigBee module is used for data transmission wirelessly to
guiding them to different sections of the products in the the centralized database server which is used for bill payment
supermarket, if they are facing any problem with it. It also and management. The bills are updated in real-time that
remembers and manages the shopping list, by showing the bill affecting both displayed values in LCD and the data in the
thereby letting the customers know about any special server, this can also be modified if any of the items are needed
discounts available. Then the data from the app is transferred to be removed. This model also shows product info, expiry
Authorized licensed use limited to: UNIVERSITAETSBIBLIOTHEK CHEMNITZ. Downloaded on December 22,2023 at 17:08:19 UTC from IEEE Xplore. Restrictions apply.
date, better alternative, and customized discount based on
customer’s purchase history other than a bill with the ZigBee.
At last, while leaving the shop there are RFID readers placed
in the exits to identify unpaid items from the information from
those RFID tags to ensure anti-thievery.
Authorized licensed use limited to: UNIVERSITAETSBIBLIOTHEK CHEMNITZ. Downloaded on December 22,2023 at 17:08:19 UTC from IEEE Xplore. Restrictions apply.
consequence, a succession of bounding boxes with the network. A 2D image is taken as an input for this network, the
recognized categories appears. Every bounding box is convolution layer identifies and maps the image's
depicted by classes and coordinates (X Center, Y Center, characteristics, while the pooling layer down samples the
Width, Height). This architecture uses darknet which has a 53- input data of the image, decreasing machine complexity and
layer network trained on image net. For the task of detection, extracting the most important attributes [8].
double the layers area unit stacked onto it, giving us a layer
B. MICROCONTROLLER AND RFID
convolutional underlying design for YOLO v3. Parameters
used were the threshold for predicted object and the non-max IMPLEMENTATION
suppression threshold which helps to undertake the problem The system is designed to be cost-effective with in-expensive
of detecting multiple times of single object in an image and Hardware such as:
input size and shape. Darknet is used as a feature extractor Arduino UNO.
which consists of filters with connections like the residual EM-18 RF-ID module.
network present in ResNet [9]. YOLOv3 architecture mainly
16*2 LCD display.
includes two steps, they are bounding boxes and non-max
I2C interface module.
suppression. Object detection is done by YOLOv3 that
predicts and forms bounding boxes around the detected object
To carry out the data processing and to make the circuit
in an image. This bounding box is also known as the anchor
fully functional a microcontroller namely Arduino UNO
that predicts 3 boxes per cell. When there are several
board is used. Arduino UNO is chosen because of the compact
bounding boxes, the objects might be recognized multiple
nature of the board so that installation becomes simple on
times, resulting in a positive type detection. Non-maximum
every smart cart in the shopping mall. The board is also open
suppression helps to ignore this circumstance and only detects
source and modular so the hardware parts can be upgraded
the object if it hasn't already been detected.
easily and also replaced/repaired making it sustainable. The
programmability offered by the board is almost limitless and
With the help of threshold value and confidence threshold
can be done with ease giving it maximum debugging
value, non-max suppression is instrumented to prevent
capability and troubleshooting.
multiple detections. It is an indispensable part of using
YOLOv3 effectively [9]. Each anchor (bounding box)
In modern times cashless payment is becoming the
contains X axis center, Y axis center, width, height, and box
standard. So, the foundation behind the billing process for the
confidence scores. The confidence score is just the value of
smart cart system is RFID technology (Radio frequency
how probably a class or type is to be contained within the
identification). RFID is Automatic Identification technology
bounding box, as well as the precision of the anchor. The
that uses electromagnetic waves to automatically identify the
anchor with its width and height is 1st set to the width and
object installed with tags that are connected to the database.
height of the input image. Four anchor values are between
Two components of RFID are the integrator and transponder,
zero and one. Then, every cell has twenty conditional category
when the transponder comes to the proximity of the reader the
changes enforced by the YOLOv3 algorithmic program. The
tag will get scanned and tracked [7]. This scanning process
object detection algorithm is evaluated using IOU. IOU or
will work even when there is a barrier between tag and reader.
Intersection Over Union analyses how close the predicted box
In this proposed system the reader module chosen to be
is to the actual truth by calculating the crossover between the
implemented is the EM-18 RFID reader module.
actual truth and the predicted bounding box.
This is a passive RFID system as the cards obtain
P(Class j|Object)*P(Object)*IOU = P(Class j)*IOU (1)
operating power from the reader by receiving electromagnetic
waves and get induced with the current. And the 12 digits
Confidence is defined as in RHS of (1). Ultimately, the
hexadecimal code is transmitted by reflecting the RF signal
IOU between the projected box and any of the actual truth box
and modulating the signal for a certain frequency. RFID
is represented by the confidence prediction. Each grid cell
works in several frequencies varying with the distance of
additionally forecasts P (Class j|Object), which are C
operation as shown in table 1.
conditional class probabilities. The grid cell in which an Table. 1. Range of Operation for RFID’s
object is present determines the probability. We only expect
one group of related probabilities for each grid cell, Frequency range Distance of
irrespective of the number of boxes B. The box confidence operation
score can be improved by the conditional class probability if 120-140 KHZ (low frequency range) 10-20 cm
the category confidence score for each final boundary box 13.56 MHZ (High frequency range) 10-20 cm
used as a positive prediction is increased. 868-928 MHZ (Ultra High frequency range) 3 meters
2.45 &5.8 GHZ (Microwave) 3 meters
In this class, the conditional class probability is the
likelihood that the identified object belongs to a given type.
The h, w, and depth numbers in YOLOv3's prediction include Since this is a passive RFID system the principle of
mathematics, as well as the spatial dimensions of the images operation is inductive coupling which takes place HF and LF
and the tensors that are used to generate anchor predictions. frequency range [13], the operating frequency of the EM-18
Finally, the anchors with high confidence scores are made as module is given to be 125 kHz from this it can be said the
final detection. An input layer, convolutional layer, pooling circuit is operated at low-frequency range making it efficient
layer, and output layer make up a convolutional neural short-range with no loss in power.
Authorized licensed use limited to: UNIVERSITAETSBIBLIOTHEK CHEMNITZ. Downloaded on December 22,2023 at 17:08:19 UTC from IEEE Xplore. Restrictions apply.
C. WORKING OF BILL PAYMENT SYSTEM
V. RESULT
D. PRINCIPLE OF OPERATION The bill information is sent to the microcontroller and total
price is displayed as shown in Fig.6.Once the card gets
The micro-controller is connected with host computer
scanned the 12-digit code is sent from the serial port of the
with memory to store the data base for multiple 12- digit codes
EM-18 module to the digital serial port of Arduino and the
registered in the supermarket and it’s programmed to make
software will process the code to draw a comparison with the
the EM-18 RFID reader to detect all the codes by reading card
codes present in the database and will try to match the code.
and comparing it with the data base [10]. If any of the code
gets matched the authentication of payment for the specific
Once the match is found authentication of payment will be
customer is done. The reading process of the code from the
done and the amount will be deduced as show in Fig.7. A
card done by EM-18 module is done with the help of inductive
confirmation notification will be displayed on 16*2 LCD by
coupling.
sending the message ‘payment successful’ via serial data pin
As the reader generates strong electromagnetic wave this
of Arduino to the display as shown in Fig.7.
penetrates the antenna coil of the transponder. This induces
Authorized licensed use limited to: UNIVERSITAETSBIBLIOTHEK CHEMNITZ. Downloaded on December 22,2023 at 17:08:19 UTC from IEEE Xplore. Restrictions apply.
VI. CONCLUSION
A superior shopping experience is developed and delivered
through this contribution as intended. The developed model
can be implemented in any convenience store or supermarket
to improve the standard of shopping for the sake of customers.
Currently, the implemented project can only be used to
classify and assign items' prices without considering their
brands as it may lead to varied prices for the items under the
same class of products. This could be achieved by
incorporating the Artificial Intelligence of things which will
connect various components and control them based on our
requirements. The introduced framework in this paper
automates scanning items, generating bills, displaying bills
and total price, and also making quick RFID payments that
use image processing and IoT. The interesting feature of this
Fig. 4. Object detection using our YOLO model
project is the security and management are done without using
any actuators other than cameras for the input of image
processing thereby serving as one of the minimalistic
approaches to provide a smart shopping system.
ACKNOWLEDGMENT
We would like to thank our chairperson Dr. A. Manikandan
for his guidance throughout this project. Also, we are obliged
to acknowledge Amrita Vishwa Vidhyapeetham, Chennai
campus for their constant support.
REFERENCES
[1] Srinidhi Karjol, Anusha K. Holla, C. B. Abhilash, “An IOT Based Smart
Shopping Cart for Smart Shopping” Springer, 2018.
[2] Mobeen Shahroz, Muhammad Faheem Mushtaq, Maqsood Ahmad, Saleem
Fig. 5. Products sold presented in Excel sheet Ullah, Arif Mehmood, Gyu Sang Choi, “IoT-Based Smart Shopping Cart
Using Radio Frequency Identification” IEEE Access, 2020.
[3] Sujay Ekal, Sonal Suryawanshi, Siddhant Warke, Mrs. P.P. Gundewar,
“Smart Billing Cart” IRJET Vol.05 Issue 03, 2018.
[4] Manan Rao, “RFID Based Smart Trolley Using IOT” IJSR Vol.8 Issue 11,
2019.
[5] Ankush Yewatkar, Faiz Inamdar, Raj Singh, Ayushya, Amol Bandal,
“Smart Cart with Automatic Billing, Product Information, Product
Recommendation Using RFID & Zigbee with Anti-Theft” Procedia
Computer Science, 2016.
[6] Akindele E. Ayoola, Member, IAENG Awodeyi I. Afolabi, Victoria W.
Oguntosin, Olaitan A. Alashiri, Victor O. Matthews, Member, IAENG and
Odunayo O. Akande, “Development of an Intelligent Smart Shopping Cart
System” Proceedings of the World Congress on Engineering and Computer
Science, 2019.
[7] Davinder Parkash, Twinkle Kundu, Preet Kaur, “The Rfid Technology and
Fig. 6. Total bill is displayed to the customer Its Applications” IJECIERD Vol.2 Issue 3, 2012.
[8] Pan Weijun, Duan Yingjie, Zhang Qiang, Tang Jiahao, Zhou Jun, “Deep
Learning for Aircraft Wake Vortex Identification” IOP ATCES, 2019.
[9] Dr. S.V. Viraktamath, Madhuri Yavagal, Rachita Byahatti, “Object
Detection and Classification using YOLOv3” IJERT, 2021.
[10] Leo Louis, “Working Principle of Arduino and Using it as a Tool for Study
and Research” IJCACS, 2018.
[11] Shubham Shinde, Ashwin Kothari, Vikram Gupta, “YOLO based Human
Action Recognition and Localization” Procedia Computer Science, 2018.
[12] Jayaram Kizhekke Pakkathillam and M. Kanagasabai, "A Novel UHF
Near-Field RFID Reader Antenna Deploying CSRR Elements," in IEEE
Transactions on Antennas and Propagation, vol. 65, no. 4, pp. 2047-2050,
April 2017. doi: 10.1109/TAP.2017.2669722.
[13] Jayaram Kizhekke Pakkathillam, M. Kanagasabai and M. G. N. Alsath,
"Compact Multiservice UHF RFID Reader Antenna for Near-Field and
Far-Field Operations," in IEEE Antennas and Wireless Propagation
Fig. 7. Payment Status Letters, vol. 16, pp. 149-152, 2017. doi: 10.1109/LAWP.2016.2561960
Authorized licensed use limited to: UNIVERSITAETSBIBLIOTHEK CHEMNITZ. Downloaded on December 22,2023 at 17:08:19 UTC from IEEE Xplore. Restrictions apply.