Received March 4, 2022, accepted March 30, 2022, date of publication April 8, 2022, date of current version April

18, 2022.
Digital Object Identifier 10.1109/ACCESS.2022.3165797

Enabling the Gab Between RPA and Process

Mining: User Interface Interactions Recorder
DAEHYOUN CHOI 1, HIND R’BIGUI 2, AND CHIWOON CHO 1
1 School of Industrial Engineering, University of Ulsan, Ulsan 44610, Republic of Korea
2 Digital Enterprise Department, NSOFT Company Ltd., Ulsan 44776, Republic of Korea

Corresponding author: Chiwoon Cho (chiwoon6@ulsan.ac.kr)

This work was supported by the 2022 Research Fund of University of Ulsan.

ABSTRACT Robotic process automation (RPA) is a rapid-emerging technology for process automation that
is using software robots to replicate high-volume, manual, repeatable, routine, rule-based, and unmotivating
human tasks. The goal is not to replace human workers but to allow them focus more on difficult tasks by
delegating their tedious routine tasks to a digital force. RPA tools demonstrated powerful cost-saving and
other performances. Nevertheless, one of the main challenges of implementing RPA is the identification
of the suitable tasks for automation. Process mining is as an emerging technology for process discovery
and enhancement based on event log data. Since RPA operates on the user interface level; process mining
techniques can play a huge role in deciding the tasks that can be automated. However, process mining requires
an event log as input to be used. This paper presents a tool responsible for recording the interactions with
user interfaces and generating a UI log that can be used by process mining techniques for deciding the tasks
that can be automated with RPA.

INDEX TERMS RPA, robotic process automation, task mining, software robots, UI recorder, digital
transformation.

I. INTRODUCTION decades but failed because the automation turned out to be

Many changes in the global market today along with the too much expensive and due to the complexity of real pro-
rapid development of technologies, has led to the appear- cesses [5], [6]. However, RPA turned out to be cheaper than
ance of a new trend called digital transformation today [1]. the traditional automation solutions. RPA has lessened the
Digital transformation is defined as the process of changing threshold for process automation. Thus, the recent consid-
existing business models as well as creating new ones by eration for Robotic Process Automation has opened a new
implementing today’s digital technologies in the process to wave of automation initiatives. RPA is defined by Van der
meet changing business and market requirements [2]. Many Aalst as ‘‘an umbrella term for tools that operate on the
industries and organizations have started taking initiatives to user interface of other computer systems in the same way as
explore new digital technologies to crucially transform their humans’’ [7]. In RPA, repetitive tasks performed by people
business operations, processes, and management strategies, are entrusted to software robots. The most important thing
etc. [3]. Robotic process automation (RPA) is one of these is that RPA bots do not modify or replace any pre-existing
new digital transformation technologies that are rapidly and information system in the organization. They replace users by
increasingly drawing the attention of businesses. Robotic pro- interacting with the user interfaces of the same pre-existing
cess automation tools allow mimicking human digital tasks information system that human users were using before [5].
by providing a virtual workforce in the form of a software bot By automating repetitive tasks with RPA, people can focus
that automatizes manual, high-volume, repetitive, and routine more on difficult tasks and problem-solving. Many benefits
tasks [4]. related to RPA implementation within industries and orga-
Plenty of the traditional Workflow Management (WFM) nizations have been communicated [8]–[11]. However, the
systems initiatives for automation has been around for many implementation of RPA is still facing many challenges as the
research is still new. One of the most important challenges
The associate editor coordinating the review of this manuscript and is the determination of business tasks that can be automated
approving it for publication was Giuseppe Desolda . with RPA [12]. To automate user tasks with RPA, we need to

This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/
39604 VOLUME 10, 2022
D. Choi et al.: Enabling Gab Between RPA and Process Mining: User Interface Interactions Recorder

know beforehand the tasks that need to be automated. Process result or service with human exception management.’’ Tradi-
mining has been outlined that it can be used to identify tional definitions define RPA as a tool to be dedicated to being
the tasks performed by people to be automated [5]. Process used to replace repetitive, and rule-based tasks ([33], [34]).
mining provides a lot of techniques for process improvement In consequence, office employees can focus more on difficult
that is using event data stored in today’s information systems. tasks and problem solving rather than spending too much
An event log, where each event represents a task executed time on executing repetitive and tiring tasks. Implementing
either by people, a machine, or a system at a particular time; RPA allows organizations to efficaciously exploit their human
is the starting point of process mining [13]. Process mining resources. Thus, increasing the productivity [35]. However,
techniques exploit these event data to illustrate how people, RPA research is still new, and the implementation of RPA is
machines, and organizations are behaving. There are four facing some challenges ([12], [22], [36]). One of the most
main categories of process mining. 1) Process discovery tech- important challenge is to identify the tasks that need and can
niques automatically discover, from real process event data, be automated with RPA ([12], [4]) before starting to imple-
the process model which represent the real behavior of the ment RPA. A traditional way to identify the tasks that need
process. 2) Conformance verification techniques consists of to be automated is by knowledge by gathering information
determining and diagnosing the deviations between a process based on interviews and checklists [37]. The problem is that
model and reality. 3) Performance analysis techniques con- this task is time consuming. One way to automate this task is
sists of identifying bottlenecks, reworks, wastes, etc. in the by using process mining. Process mining ([13], [17]) has been
process. 4) Process reengineering techniques allows chang- proposed to tackle this challenge ([4], [21], [22]). Process
ing the existing process model. For more information on mining techniques are capable of extracting knowledge from
process mining refer to [13]–[20]. The previous works have event logs commonly existing in today’s information sys-
proposed [4], [21] methodologies to identify candidate digital tems to discover, monitor, and enhance processes in multiple
tasks for automation with RPA tools that is based on process application fields [39]. Process mining is used to discover the
mining techniques. R’bigui et al. [4] defined a digital task process model of the executed tasks for different purposes.
as a task that is performed by a user using a computer by Since RPA can automated repetitive task performed using
interacting with different graphical user interfaces of various a computer system, process mining techniques can be used
systems and applications. A user interface event log which to discover the actions performed on this computer while
corresponds to the events accruing while interacting with user interacting with different UIs. However, process mining tech-
interfaces of different applications or systems, is required as niques requires an event log as input to discover the executed
input to be able to identify the tasks that can be automated actions. therefore, the actions performed on computer by
with RPA using process mining techniques. The proposed interacting with different UIs of different systems need to be
approaches for detecting routine tasks for automation with recoded and the corresponding log need to be generated.
RPA using process mining suppose that the UI log already
exist or can be recorded. However, existing recording tools do B. LIMITATIONS OF EXISTING RECORDING TOOLS
not provide data from which process mining can discover the Existing UI actions recording tools such as WinParrot, JitBit
tasks performed on user interfaces, and existing approaches Macro Recorder, TodayDo can record only low-level actions
such as video recording is time-consuming [29]. Thus, the which refer only to pixels coordinates of mouse clicks which
adoption of process mining techniques for RPA is blocked is based on the window size and Ui resolution [29]. Some
by the absence of tools capable of recording the interactions tools like TodayDo and WinParrot can record information
with the user interface and generating UI logs providing related to where the actions have been performed. Never-
enough information as input for process mining techniques theless, actions such as copying, pasting and information
to discover digital tasks that can be automated with RPA. The related to button clicks types cannot be captured. Some tools
contribution of this work consists of presenting a tool, namely also do not record the timestamps, and none of these tools
User interface Interactions Recorder (UIIR), which fills the provides a log into a format supported by process mining
gap between robotic process automation and process mining. techniques (e.g., CSV, XES). Concerning RPA tools, they
provide recording function to produce the script of the bot, but
the problem is that the produced log cannot not be exported
II. RELATED WORK nor be readable outside of RPA environment [29]. Accord-
A. BACKGROUND ingly, there is a need to develop tools that can generate UI
RPA tools provides software bots that can mimic human logs that can be used by process mining techniques to support
actions performed on a computer while interacting with var- RPA implementation. Leno et al [29] introduced the first tool
ious user interfaces of different systems ([31], [32]). RPA is that produces a log containing information needed by process
defined by IEEE Standards Association [38] as ‘‘A preconfig- mining. The tool supports Excel application and chrome web
ured software instance that uses business rules and predefined browser. Our work is like the work presented in [29]. How-
activity choreography to complete the autonomous execution ever, we developed more functions such as differentiating
of a combination of processes, activities, transactions, and between different types of mouse clicks, and other advanced
tasks in one or more unrelated software system to deliver a functions. Moreover, our tool supports not only excel and

VOLUME 10, 2022 39605

 D. Choi et al.: Enabling Gab Between RPA and Process Mining: User Interface Interactions Recorder

FIGURE 1. User interface interaction recorder within RPA – research scope highlighted.

chrome web browser, but also the other Microsoft applica- automation with RPA. We proposed in the previous study [4]
tions (e.g., ppt, word, teams, etc.) and windows applications an approach for RPM consisting of four major steps.
(e.g., note, etc.).
1) UI INTERACTION RECORDING
III. USER INTERFACE INTERACTIONS RECORDING This step consists of recording the interactions (i.e., actions),
METHODOLOGY which is based on mouse clicks and the keyboard, of a human
In this section, we discuss the position of UI recorder within user with different applications such as web, desktop, system,
the framework of RPA, its architecture, the rules used for application, etc. while performing his administrative tasks.
recording, and the rule used for simplifying and reducing the This step is the major scope of this study. Fig. 1 shows the
generated UI log. position of UI interactions recording within the pipeline of
RPA framework.
A. USER INTERFACE INTERACTIONS RECORDER’S
POSITION WITHIN RPA FRAMEWORK 2) UI LOG TRANSFORMATION AND FILTERING
Robotic process automation tools are capable of automating This step consists of transforming the generated UI log into a
tasks belonging to a business process. However, employ- log supported by process mining tools. UI log transformation
ees are performing plenty of and various tasks within an rules is defined in [4]. The rules differ based on the type of
organization. Not all of them need to or can be automated the action performed. For more details on the transformation
with RPA tools. Thus, the main question is which of the rules refer to [4]. This step also addresses the filtering of
tasks performed by a user worker need to be automated to the generated UI log. While performing their tasks using a
enable business growth and can be automated with RPA. computer, employees can do other actions also which are not
A term called Robotic Process Mining (RPM) has been related to work on a computer such as opening personal email
introduced in [22] to refer to a category of techniques that or sending SNS messages. This type of actions needs to be
enables discovering and analyzing candidate tasks that can filtered to keep only actions relevant to work.
be automated with RPA robots from data collected during
the execution of user-based tasks. RPM techniques is a sub- 3) TASK DISCOVERY
class of process mining techniques. Process mining allow Each task performed using a computer is composed of a
discovering processes from an event log containing a chrono- set of performed actions. Thus, during this stage, the con-
logical order of executed tasks recorded with today’s infor- ducted tasks need to be identified or discovered based on
mation systems such as Enterprise Resource Planning (ERP), the sequence or the chronological order of the performed
Business Process Management (BPM) systems, etc. while actions. The process discovery category techniques of process
robotic process mining should allow discovering tasks from a mining [23]–[27] allow us to do this job based on UI log data.
graphical user interface log containing a chronological order
of executed actions performed on user interfaces of different 4) CANDIDATE TASKS SELECTION
systems and applications. RPM techniques need to be applied After discovering all tasks performed while interacting with
before implementing RPA to identify the tasks suitable for different applications and systems, the tasks that need to

39606 VOLUME 10, 2022

D. Choi et al.: Enabling Gab Between RPA and Process Mining: User Interface Interactions Recorder

UIIR tool allows recording all the data involved in the

context of each captured action. For instance, for an action
performed in web browser, UIIR records the information
about the URL link, the button clicked, active UI, the data
entered, etc., and for an action performed in an Excel file,
it captures information about the path of the spreadsheet,
the active sheet, the cell and its value, the button clicked,
etc. The tool generates a log in the format of Excel file
which can be converted easily into and CSV file that is one
of the formats required by process mining tools for further
analysis.

C. USER INTERFACE INTERACTIONS RECORDING RULES

FIGURE 2. User interface interactions recording process. To generate UI logs that can be processed with process mining
techniques to identify the digital tasks that can be/need to
be automated with RPA, the recorder tool should ensure that
and can be automated need to be determined. The iden- the recorded information is suitable for this analysis. First,
tifications of candidate tasks can be done using different two main questions need to be answered when developing
methods for instance criteria-based selection such periodicity, the user interface interactions recorder and logger: (1) What
frequency, etc. should be recorded? and (2) what should not be recorded? The
After candidate tasks are appropriately identified in the last tool should record only meaningful and value adding actions
step of RPM, RPA can be implemented by creating software and data. This work presents a set of rules applied to record
robots in charge of executing the selected tasks. As can be actions performed on different UIs.
seen, UI interactions recording can be considered the most
important step is it is the starting point of all the pipeline
of RPA Framework. Without a UI log, process mining tech-
niques cannot be used to enable RPA.

B. USER INTERFACE INTERACTIONS RECORDER

ARCHITECTURE
User Interface interaction recorder (UIIR) records the actions
performed on (i) chrome web browser, (ii) windows appli-
cations such as interaction with windows folders, notepad,
(iii) and Microsoft applications such as Excel, PowerPoint,
and Word. We developed a plugin for recording actions con-
ducted on the web browser as well as a windows program
FIGURE 3. User interface interactions recording architecture.
which records keyboard usage and mouse clicks performed
on windows and Microsoft applications. Both plugin and
windows program are surveying the events of the performed 1) R1. RECORDING SIGNIFICANT ACTIONS
actions and then sending the information to the logging com- To perform specific digital tasks, users perform many actions
ponent for generating and updating the UI interactions log in on different applications and UIs. However, while interacting
real-time. with these systems and UIs, not all performed actions are
Before starting the recording, the target user needs to sign significant.
in into the recoding tool UIIR with his id and password to There are two types of actions: actions performed with
differentiate between users using for example the same com- the mouse and actions performed with the keyboard. Each
puter. The recording of the actions performed on web browser of these actions is performed on a specific user interface.
and those performed on windows and Microsoft applications The actions that can be performed with a mouse are moving
starts automatically after sign in. The logs are stored directly the mouse, right click, left click, and scrolling. For instance,
to a server database as shown in Fig. 2. After stopping the actions of scrolling and moving the mouse are not meaningful
recording, one integrated UI log, which integrates both the as they do not impact the outcome of a task. Right and left
log generated from the web and the log generated from clicks can be irrelevant based on what has been clicked and
windows program, is generated. The log can be any time based on which UI the clicking action is performed. For
downloaded by user id and by date. Then the generated raw instance, clicking on the background of a desktop or of a
UI log is reduced and simplified with a filtering program that website, etc. is not meaningful. Hence, this type of actions
we developed as well. Fig. 3 shows the architecture of the should not be captured. However, for instance, button clicks
recorder UIIR. are relevant actions that need to be recorded. We define below

VOLUME 10, 2022 39607

 D. Choi et al.: Enabling Gab Between RPA and Process Mining: User Interface Interactions Recorder

FIGURE 4. Log exporting screen based on period selection.

an example of a set of actions that are essential and need to the performed actions, the recorder should also capture the
be captured in the log. data that supports them.

a: MOUSE CLICKS ACTIONS 3) R3. DIFFERENT APPLICATIONS ‘UIS RECORDING

We defined six types of mouse clicks that can be a part A task consists of a set of actions. One task can be performed
of a performing task and should be captured: button clicks, using different user interfaces such as web-based applications
checkbox clicks, text filed clicks, URL link clicks, selection and systems, Microsoft applications such as Excel, word, ppt,
related clicks, and general clicks (e.g., menu) actions. The etc., windows applications such as folders, etc. The inter-
recorder should differentiate between all these type of mouse actions between different UIs to perform a task need to be
clicks. recorded. For instance, filtering and copying data from an
Excel sheet and pasting it in a web-based system such as ERP.
b: COPY AND PASTE ACTIONS As can be seen, this task is performed through filter, copy, and
These two actions can be performed using only mouse clicks paste actions and through two user interfaces Excel, and web-
(i.e., left click + copy/paste button click) as well as they based system.
can be performed using the keyboard (i.e., copy = ctrl + c,
paste = ctrl + v). Performing the copy action is preceded by 4) R4. PRIVACY AWARE RECORDING
a selection action with the mouse which allows selecting the The goal of recording the interactions of users with different
content to be copied., and the paste action is preceded with a user interfaces of various systems and applications is to gen-
mouse click such as text field click which allows specifying erate a log that will be analyzed to discover the tasks that has
the place where the copied content should be pasted. The been performed and identify the ones that can be and need to
actions performed by typing ctrl + c/ctrl + v in the keyboard be automated with RPA. The generated log will be analyzed
are converted into copy and paste actions respectively, and the using different techniques such as process mining by man-
actions performed by left click action + copy/paste button agers who will decide the tasks to be automated. Since, all
click action are converted into one action copy and paste interactions will be recorded, private and personal data also
respectively. can be recorded. Therefore, there is a need to protect users
‘privacy. In this recorder, we took into consideration some of
2) R2. RECORDING RELEVANT DATA the privacy issues. For instance, all entered passwords are not
recorded as they are, but the entered passwords are recorded
Open, click button, copy etc. are the name of the recorded
with the word ‘‘password’’.
actions which is performed by a user. Considering only the
name of the actions conducted is not enough to identify the
performed task. Information related to what has been opened, D. UI LOG FILTERING – SIMPLIFICATION
what URL has been opened, which folder has been opened, Since, UIIR recorder records the performed actions in detail,
what is the path of the opened folder, which excel sheet has the generated log needs to be simplified. Therefore, we devel-
been opened, which cell has been modified, which button has oped another tool for filtering and simplifying the generated
been clicked, what content has been copied and pasted, what UI log. We define below some examples of simplification and
content is entered with the keyboard, etc. is necessary to be filtering that can be performed by the filtering tool.
able to extract the performed task and need to be captured.
Moreover, timestamps are the time at which the actions are 1) KEYBOARD ENTERING SIMPLIFICATION
performed. Hence, timestamp information is also essential in Recording some content and values entering with the key-
order to identify the order of actions. In conclusion, besides board can also be meaningful. however, each letter or

39608 VOLUME 10, 2022

D. Choi et al.: Enabling Gab Between RPA and Process Mining: User Interface Interactions Recorder

FIGURE 5. A screen of the UI log filtering with the simplification tool.

FIGURE 6. A fragememt of the recorded UI log.

number entered with the keyboard is recorded as one actions. Moreover, the set of {pressed, released} can be the result of
For instance, the actions of opening the following URL: a single click as can also be the result of a selection. The only
www.google.com by typing it with the keyboard will be difference is the position of the press and the release. Thus,
indeed recorded not in one action but into 14 actions, which the log can be simplified with the tool such that if the position
means 14 rows will be generated in the log (e.g., {w} in one of the pressing is equivalent to the position of the release, then
row,..{.} in one row, etc.). After filtering the raw log with the the action is converted to a click action and if the position of
developed filtering tool, the 14 actions are simplified into one the press is different than the position of the release, then the
action recorded in one row where the content of the action is action is converted to a selection action.
recorded as www.google.com.

2) MOUSE CLICKS SIMPLIFICATION AND FILTERING 3) SIMPLIFICATION RELATED TO COPY AND PASTE ACTIONS
Every mouse click is composed of a set of {pressed, released} When copying and pasting a content with CTRL+C, and
which means two actions/two rows are recorded. When we CTRL+V, the actions are recorded in two rows respectively.
press the mouse, it is recorded in one row and when we release The tool converts the CTRL+C into ‘‘copy’’ recorded in one
the mouse, it is recorded in one row. All released clicks are row and converts the CTRL+V into ‘‘paste’’ recorded in one
deleted to keep only meaningful rows. row.

VOLUME 10, 2022 39609

 D. Choi et al.: Enabling Gab Between RPA and Process Mining: User Interface Interactions Recorder

FIGURE 7. The set of actions (17 actions) of the performed task discovered with process mining from the recorded and filtered UI log.

4) REDUNDANT ACTIONS FILTERING A fragment of the generated log after filtering is shown in
Log reducer tool filters also redundant rows (i.e., deletes Fig. 6. To test the produced UI log, it needs to be used as input
rows having identical information in all columns), double by process discovery techniques. For this purpose, we used
copying, copying actions without the corresponding pasting the Disco tool [30], a process mining tool that is based on the
actions. fuzzy algorithm [24] that allows discovering process models
from an event log. In our case, the aim is to discover the
IV. RESULT AND DISCUSSION performed task that consists of a set of actions performed
This section presents a case study to demonstrate the using the keyboard and the mouse on different user interfaces
User Interface Interactions Recorder (UIIR) and to demon- of different applications.
strate that the generated log is useful for process mining Fig. 7 illustrates the model of the performed task that
investigations. has been discovered automatically with a process discovery
The case study consists of (1) loging to a web-based online technique from the recorded UI log depicted in Fig. 6. The
shopping system, (2) downloading all orders, (3) opening model shows the sequence of the actions performed while
the excel sheet of the orders, (4) filtering the orders based interacting with a web-based sales system and excel appli-
on delivery completed status, (5) copying filtered orders, cation interfaces.
and (6) pasting them in a new excel sheet called completed Based on the result, we can see that the produced model
orders. UIIR tool records the performed actions. The gener- is understandable. It shows that the performed actions are
ated log can be downloaded by selecting the date or the period discovered in the correct chronological order. Also, the infor-
as shown in Fig. 4. After downloading the produced log, mation provided in every discovered action, provides a full
it is filtered with the simplification tool as shown in Fig. 5. understanding of what action has been performed, on which

39610 VOLUME 10, 2022

D. Choi et al.: Enabling Gab Between RPA and Process Mining: User Interface Interactions Recorder

system or application the action is performed, which button [4] D. Choi, H. R’Bigui, and C. Cho, ‘‘Candidate digital tasks selection
has been clicked, which content has been entered with the methodology for automation with robotic process automation,’’ Sustain-
ability, vol. 13, no. 16, p. 8980, Aug. 2021, doi: 10.3390/su13168980.
keyboard, which folder and link address are involved, etc. [5] W. van der Aalst, ‘‘On the Pareto principle in process mining, task mining,
This work shows that the UI log generated by the user and robotic process automation,’’ in Proc. DATA, 2020, pp. 5–12.
interface interactions recorder (UIIR) and simplified with the [6] W. van der Aalst and K. van Hee, Workflow Management: Models, Meth-
ods, and Systems. Cambridge, MA, USA: MIT Press, 2004.
filter tool can successfully be used by process mining tech- [7] W. M. P. van der Aalst, M. Bichler, and A. Heinzl, ‘‘Robotic process
niques to discover the actions performed while interacting automation,’’ Bus. Inf. Syst. Eng., vol. 60, pp. 269–272, May 2018.
with different UIs of different systems and applications. [8] N. Nawaz, ‘‘Robotic process automation for recruitment process,’’ Int. J.
Adv. Res. Eng. Technol., vol. 10, no. 2, pp. 608–611, Apr. 2019.
[9] A. Choubey and M. Sharma, ‘‘Implementation of robotics and its impact
V. CONCLUSION on sustainable banking: A futuristic study,’’ J. Phys. Conf. Ser., vol. 1911,
RPA is a new and trending topic for automating repetitive no. 1, 2021, Art. no. 012013.
[10] A. Sobczak and L. Ziora, ‘‘The use of robotic process automation (RPA) as
and routine tasks. One of the most important challenges of an element of smart city implementation: A case study of electricity billing
RPA implementation is the determination of business tasks document management at Bydgoszcz city Hall,’’ Energies, vol. 14, no. 16,
that can be automated with RPA. We need to know before- p. 5191, Aug. 2021, doi: 10.3390/en14165191.
hand the tasks that need to be automated. Process mining [11] S. Aguirre and A. Rodriguez, ‘‘Automation of a business process using
robotic process automation (RPA): A case study,’’ in Proc. Workshop Eng.
can play a huge role in identifying the tasks performed by Appl., in Communications in Computer and Information Science, vol. 742,
people to be automated. However, the adoption of process 2017, pp. 65–71.
mining techniques for RPA is blocked by the absence of tools [12] D. Choi, H. R’bigui, and C. Cho, ‘‘Robotic process automation implemen-
tation challenges,’’ in Proceedings of International Conference on Smart
capable of recording the interactions with the user interface Computing and Cyber Security (Lecture Notes in Networks and Systems).
and generating UI logs providing enough information that can Sokcho, South Korea: Kyungdong Univ., Jul. 2020, pp. 297–304.
be used as input for process mining techniques to discover [13] W. M. P. Van der Aalst, Process Mining: Data Science in Action. Berlin,
Germany: Springer, 2016.
digital tasks that can be automated with RPA. In this work, [14] H. R’bigui, M. A. Al-Absi, and C. Cho, ‘‘Detecting complex control-
we presented a tool that can fill the gap between robotic flow constructs for choosing process discovery techniques,’’ Int. J. Innov.
process automation and process mining. We defined the rules Technol. Explor. Eng., vol. 9, no. 1, pp. 1389–1393, 2019.
[15] H. R’Bigui and C. Cho, ‘‘Purchasing process analysis with process mining
that allow capturing relevant information and data needed of a heavy manufacturing industry,’’ in Proc. Int. Conf. Inf. Commun. Tech-
by process mining techniques to correctly discover the per- nol. Converg. (ICTC), Jeju Island, South Korea, Oct. 2018, pp. 495–498.
formed tasks with a user interface. The tool has been tested [16] H. R’Bigui and C. Cho, ‘‘The state-of-the-art of business process min-
ing challenges,’’ Int. J. Bus. Process Integr. Manage., vol. 8, no. 4,
and evaluated with a case study and the results show that the pp. 285–303, 2017.
UI log generated by the user interface interactions recorder [17] W. M. P. Van der Aalst, Process Mining: Discovery, Conformance and
can successfully be used by process mining techniques to dis- Enhancement of Business Processes, 1st ed. Berlin, Germany: Springer,
cover the actions performed while interacting with different 2011.
[18] H. R’bigui and C. Cho, ‘‘Heuristic rule-based process discovery approach
UIs of different systems and applications. However, this topic from events data,’’ Int. J. Technol., Policy Manage., vol. 19, no. 4, p. 352,
is still new. In the future work, we will perform more further 2019.
investigations and evaluations to enhance the capability of the [19] H. R’Bigui and C. Cho, ‘‘Customer Oder fulfillment process analysis
with process mining: An industrial application in a heavy manufacturing
tool in capturing meaningful interactions. company,’’ in Proc. ACM Int. Conf. Proc. Ser., 2017, pp. 247–252.
[20] H. R’bigui and C. Cho, ‘‘Complex control-flow constructs detection from
AUTHOR CONTRIBUTION process related data,’’ in Proc. 21st Int. Conf. Adv. Commun. Technol.
(ICACT), Feb. 2019, pp. 579–582.
Hind R’bigui and Daehyoun Choi contributed to the main [21] J. Geyer-Klingeberg, J. Nakladal, F. Baldauf, and F. Veit, ‘‘Process mining
idea and the methodology of the research. Hind R’bigui and Robotic process automation: A perfect match,’’ in Proc. CEUR Work-
designed the experiment, performed the simulations, and shop, vol. 2196, 2018, pp. 124–131.
[22] V. Leno, A. Polyvyanyy, M. Dumas, M. La Rosa, and F. M. Maggi,
wrote the original manuscript. Hind R’bigui contributed sig- ‘‘Robotic process mining: Vision and challenges,’’ Bus. Inf. Syst. Eng.,
nificantly to improving the technical and grammatical con- vol. 63, no. 3, pp. 301–314, Jun. 2021.
tents of the manuscript. Hind R’bigui and Chiwoon Cho [23] W. van der Aalst, T. Weijters, and L. Maruster, ‘‘Workflow mining: Dis-
covering process models from event logs,’’ IEEE Trans. Knowl. Data Eng.,
reviewed the manuscript and provided valuable suggestions vol. 16, no. 9, pp. 1128–1142, Sep. 2004.
to further refine the manuscript. [24] C. W. Günther and W. M. P. van der Aalst, ‘‘Fuzzy mining—Adaptive
process simplification based on multi-perspective metrics,’’ in Business
Process Management (Lecture Notes in Computer Science), G. Alonso,
REFERENCES P. Dadam, and M. Rosemann, Eds. Berlin, Germany: Springer, 2007,
[1] A.-M. Stjepić, L. Ivančić, and D. S. Vugec, ‘‘Mastering digital transfor- pp. 328–343, doi: 10.1007/978-3-540-75183-0_24.
mation through business process management: Investigating alignments, [25] A. J. M. M. Weijters and J. T. S. Ribeiro, ‘‘Flexible heuristics miner
goals, orchestration, and roles,’’ J. Entrepreneurship, Manage., Innov., (FHM),’’ in Proc. IEEE Symp. Comput. Intell. Data Mining (CIDM), Paris,
vol. 16, no. 1, pp. 41–73, 2019, doi: 10.7341/20201612. France, Apr. 2011, pp. 310–317.
[2] M. P. Bach, M. Spremić, and D. S. Vugec, ‘‘Integrating digital trans- [26] W. M. P. Van der Aalst, A. K. Medeiros, and A. J. M. M. Weijters, ‘‘Genetic
formation strategies into firms: Values, routes and best practice exam- process mining,’’ Applications and Theory of Petri Nets 2005 (Lecture
ples,’’ in Management and Technological Challenges in the Digital Age, Notes in Computer Science), vol. 3536, G. Ciardo and P. Darondeau, Eds.
P. N. Melo and C. Machado, Eds. Boca Raton FL, USA: CRC Press, 2018, Berlin, Germany: Springer, 2005.
pp. 119–140. [27] H. Sun, W. Liu, L. Qi, Y. Du, X. Ren, and X. Liu, ‘‘A process mining
[3] C. Matt, T. Hess, and A. Benlian, ‘‘Digital transformation strategies,’’ Bus. algorithm to mixed multiple-concurrency short-loop structures,’’ Inf. Sci.,
Inf. Syst. Eng., vol. 57, no. 5, pp. 339–343, 2015. vol. 542, pp. 453–475, Jan. 2021.

VOLUME 10, 2022 39611

 D. Choi et al.: Enabling Gab Between RPA and Process Mining: User Interface Interactions Recorder

[28] Accessed: Feb. 24, 2022. [Online]. Available: https://www.w3schools. HIND R’BIGUI received the Ph.D. degree in
com/whatis/whatis_ajax.asp industrial engineering from the University of
[29] V. Leno, A. Polyvyanyy, M. La Rosa, M. Dumas, and F. M. Maggi, ‘‘Action Ulsan, South Korea, and the State Engineering
logger: Enabling process mining for robotic process automation,’’ in Proc. degree in industrial engineering from the National
CEUR Workshop, 2019, pp. 124–128. School of Applied Sciences of Fes, Morocco. She
[30] C. Gunther and A. Rozinat, ‘‘Disco: Discover your processes,’’ in Proc. is currently working as a Consultant and Solution
BPM (PhD/Demos), vol. 940, 2012, pp. 40–44. Architect of SIEMENS Smart Factory Solutions,
[31] W. M. P. Aalst, M. Bichler, and A. Heinzl, ‘‘Robotic Process Automation,’’
NSOFT Company Ltd., South Korea. Her research
Bus. Inf. Syst. Eng., vol. 60, no. 4, pp. 269–272, Aug. 2018.
interests include process mining, process model-
[32] C. Tornbohm and R. Dunie, ‘‘Gartner market guide for robotic
process automation software,’’ Gartner, Stamford, CT, USA, ing, BPM, robotic process automation, smart fac-
Tech. Rep. G00319864, 2017. tory, and digital transformation.
[33] L. P. Willcocks, M. Lacity, and A. Craig, ‘‘Robotic process automation
at Xchanging,’’ in The Outsourcing Unit Working Research. (The London
School of Econ. Political Sci.) London, U.K.:, Paper Series (15/03), 2015.
[34] L. Ivančić, D. S. Vugec, and V. B. Vukšić, ‘‘Robotic process automation:
Systematic literature review,’’ in Proc. BPM, 2019, pp. 280–295.
[35] S. Madakam, R. M. Holmukhe, and D. K. Jaiswal, ‘‘The future digital work
force: Robotic process automation (RPA),’’ J. Inf. Syst. Technol. Manage.,
vol. 16, pp. 1–17, Jan. 2019.
[36] A. Egger, A. H. M. T. Hofstede, W. Kratsch, S. J. J. Leemans, M. Röglinger,
and M. T. Wynn, ‘‘Bot Log mining: Using logs from robotic process
automation for process mining,’’ in Proc. 39th Int. Conf. Conceptual
Modeling (ER), Vienna, Austria, Nov. 2020, pp. 51–61.
[37] J. Wewerka and M. Reichert, ‘‘Checklist-based support of knowledge
workers in robotic process automation projects,’’ in Proc. IEEE 23rd Conf.
Bus. Informat. (CBI), Bolzano, Italy, Sep. 2021, pp. 52–61.
[38] IEEE Guide to Terms and Concepts in Intelligent Process Automation,
IEEE Standard 2755-2017, 2017.
[39] W. Van der Aalst et al., ‘‘Process mining manifesto,’’ in Proc. Bus. Pro-
cess Manage. Workshops (BPM) (Lecture Notes in Business Information
Processing), vol. 99, F. Daniel, K. Barkaoui, and S. Dustdar, Eds. Berlin,
Germany: Springer, 2012, doi: 10.1007/978-3-642-28108-2_19.

DAEHYOUN CHOI received the Ph.D. and mas- CHIWOON CHO is currently a Full Professor
ter’s degrees in industrial engineering from the in industrial engineering with the University of
University of Ulsan, South Korea. He has a lot of Ulsan, South Korea. He has a lot of industry
industry experiences and is currently the CEO of experiences, including Hyundai Heavy Industries,
NSOFT Company Ltd. LG CNS, and Samsung SDS.

39612 VOLUME 10, 2022