Measurement: Azri Mat Saad, Khairul Nizam Tahar
Measurement: Azri Mat Saad, Khairul Nizam Tahar
Measurement
journal homepage: www.elsevier.com/locate/measurement
a r t i c l e i n f o a b s t r a c t
Article history: Road development is very important in many countries. Many technologies have been used in road main-
Received 18 June 2018 tenance. The unmanned aerial vehicle (UAV) promises a rapid and cost saving technology and is widely
Received in revised form 22 December 2018 used in the mapping industry. The aim of this study is to assess rut and pothole on road surface by using
Accepted 29 January 2019
UAV. The methodology consists of four phases, namely site reconnaissance and planning, data acquisi-
Available online 2 February 2019
tion, data processing and result and data analysis. Data acquisition is based on difference altitudes to
determine the effect of resolution in rut and pothole extraction. In this study, the data processing is by
Keywords:
using a photogrammetric software which is based on structure from motion. The accuracy assessment
Road
UAV
in this study is based on actual and measured data comparison, which only concentrates on rut and pot-
Altitude hole samples. It was found that low altitude gives a better result as compared to high altitude. In conclu-
Pothole sion, this study demonstrated the capability of multirotor UAV image for rut and pothole extraction. This
Rut system provides a detailed and accurate measurement of road rut and pothole, and thus improves the
Identification efficiency of road condition monitoring.
Accuracy Ó 2019 Elsevier Ltd. All rights reserved.
1. Introduction details of the rut and pothole geometry, including shape, perime-
ter, area and depth to estimate the cold tar needed for road main-
Maintenance of roads throughout the country is run by the Pub- tenance at that specific location.
lic Works Department (PWD) [9]. PWD is responsible for planning, Nowadays, one of the fastest developed technologies and
designing and constructing of infrastructure projects, such as widely used in the mapping industry is an unmanned aerial vehicle
roads, government buildings, airports, harbours, piers and products (UAV). UAV is very popular for mapping applications among civil-
related engineering, road maintenance, and certain government ian and the Department of Survey and Mapping [1-3,5]. UAV is a
buildings, as well as technical advice to the federal, state and local new system technology which promises fast data acquisition for
governments. Road maintenance is one of the challenging tasks to decision making, especially for small aerial mapping with high res-
be carried out because it is very tedious and takes time for data olution imagery. There are many types of UAV available around the
acquisition. Previously, the local authority or PWD needs to assign globe and are used for the specific application [16, 17]. It does not
a dedicated person to survey and monitor the road condition. This require an onboard pilot, reduces cost, weighted, eminent of dan-
will involve a lot of time and cost to finish the data acquisition. ger and highly flexible. This method also gives users more advan-
Today, some developers use their creativity to develop a mobile tages, such as lower costs for the photogrammetric system and
application. A road user is able to report or complaint about their feasibility to fly autonomously based on designed waypoints and
road condition through apps. Then, the local authority or PWD will command in the photogrammetric applications. The high-
take action to repair the road surface. However, the road user can resolution images captured by UAV platform are able to extract
only report on the location and description of the road surface, but metric information of the Earth’s surface [6]. The recent technology
not about the rut and pothole geometric. Lack of technology to for road construction was introduced by the Road Surface Monitor-
assess rut and pothole geometric is one factor that causes a delay ing (RSM). RSM is used to process and detect distress on paved or
in road maintenance. A new method is needed to assess road con- unpaved road surface [4]. The researcher proposed to detect any
dition at the reported area. The system should be able to provide distress, such as road surface cracks, at early stages to apply main-
tenance on time. Early detection of road surface cracks can assist
maintenance before the repair costs become too high. Tiong et al.
⇑ Corresponding author. [14] studied on road surface assessment of pothole severity by
E-mail address: khairul0127@salam.uitm.edu.my (K.N. Tahar).
https://doi.org/10.1016/j.measurement.2019.01.093
0263-2241/Ó 2019 Elsevier Ltd. All rights reserved.
648 A.M. Saad, K.N. Tahar / Measurement 137 (2019) 647–654
close range digital photogrammetry method. This paper focuses on acquired data will be processed. Steps for data processing are such
the effectiveness of road surface assessment on the severity level of as adding photo, aligning photo, build dense cloud, build mesh,
pothole formation, based on the national guidelines published by build tiled model, build DEM and orthomosaic. Result and data
the PWD. Pothole samples were randomly chosen and their sever- analysis is the process of interpreting and analysing the data to
ity levels were assessed through both conventional visual approach obtain the result. This phase contains road condition parameters,
and photogrammetry method. The researcher revealed that digital such as length, size of ruts and potholes. Then, the actual measure-
close-range photogrammetry method can be used as an alternative ment is compared with the road parameters from UAV images. The
way for visual assessment of pothole severity by using a typical selected study area is on the roads located at Universiti Teknologi
digital camera. MARA (UiTM), in Shah Alam and some areas at Seksyen 7, also in
Many countries around the globe almost have the same issues Shah Alam. UiTM Shah Alam is located in Selangor Darul Ehsan
where most of the roads are paved but after a certain period, due and is the main campus in Selangor. The UiTM Shah Alam and Sek-
to weather and types of vehicles that accessed the road, can cause syen 7 roads were choosen because there is lack of vehicles passing
the possibility of damaged and cracked road. Sometimes, impass- through the roads as compared to the external roads. Other than
able roads were damaged by heavy vehicles, such as overloaded that, it is easier to do a study research on road pavement detection
with timber or cement trucks and so on. To overcome this problem, in UiTM Shah Alam (Fig. 1).
the use of UAVs is an alternative to analyse and control the situa- A paved road mostly illustrates various conditions. A newly
tion before it becomes increasingly damaged [15]. The use of UAVs constructed or well-maintained road should have excellent surface
will accelerate the effort to solve road condition problems, such as condition and provide smooth and safe traffic. However, the paved
cracks and puddles. In addition, UAVs usage can also reduce main- road surface was damaged by the traffic, weather and bad driving
tenance and labour costs, as well as time [7,8,10,11-13]. Also UAVs habit. More road usage and heavy traffic will cause more quick and
are new techniques that can be controlled away from site location. frequent road damages. Water trapped on the road can cause rut
UAVs can get information about the condition of crack and damage and pothole. The paved road had surface distress and created diffi-
of road; thus, this study concentrates on a novel method to identify culty to the local community in the day-to-day commune. There
and extract rut and pothole conditions on pavement road by using were a few samples of pothole at the selected study areas. The pot-
a multirotor UAV system. The validation is based on actual mea- holes were located on road surface of the paved road. Fig. 2 illus-
surement by using a conventional method. trates some samples of pothole in this study.
Equipment needed to do the survey includes thread, straight
2. Material and methods edge, metre tape to measure the length of ruts and ruler to mea-
sure the area of pothole. Apart from that, measuring time can be
There are four main phases. The first phase is about site recon- very tedious and labour extensive. The UAV system overview
naissance and planning which consist of several works, such as the includes UAV quadcopter, digital camera and on-board flight sys-
selection of study area, selection of equipment, camera calibration tem. A UAV system which uses DJI Phantom 3 Professional is com-
and software used. Next phase is data acquisition, involving a con- monly used in aerial photography. Nowadays, DJI Phantom series
ventional method measured by using metre tape and ruler and have become a benchmark for a consumer drone because this
capture image from UAV. During the data processing phase, all quadcopter is very simple to operate, easy to fly and relatively a
Fig. 2. Some samples; a) potholes b) Photo measure conventional method using meter tape and ruler.
cheap setup. Other than that, this quadcopter includes all compo- altitudes can produced a good pothole model results or not. The
nents, such as camera, GPS and remote controller, in a complete set accuracy assessment also will be conducted to determine the effect
without any modification Fig. 3. of these two altitudes. The altitude hold accuracy of the UAV dur-
Camera calibration is necessary for extracting accurate 3D ing flight affects the accuracy of the terrain model. Apart from that,
information from images. The camera used in this study is a phan- this mission also manually navigates by a human pilot. This is
tom camera with 12 mega pixel resolution. The aim for camera cal- because, for an altitude of 10 m from the ground, there are obsta-
ibration is to determine the inner orientation parameters. There cles such as high trees and buildings; thus, it is manually con-
are several commercial and open source software’s available in trolled by a human pilot. Fig. 4 is an example of the collected
the market and literature. The purpose of camera calibration is road imagery over the road pavement with rut and pothole.
important to define the geometric camera model that is described Agisoft Photoscan Version 1.2.4 was used to process the UAV
by interior parameters that include a computed principle distance images. The Agisoft Photoscan is the latest software to produce
of lens (c), principle point (xp, yp) and radial lens distortion (k1, k2, the image-based 3D modelling solution which is aimed at creating
k3) and tangential lens distortion (p1, p2). a professional quality 3D content from still images. With the latest
The Map Pilot for DJI is used to create and fly the optimal flight multi-view of 3D reconstruction technology, all photos can be
path. The Map Pilot is also used to calculate the area and estimate taken from any position and provides the objective to reconstruct
how many batteries are needed to operate. The software features at least two visible photos. Both image alignment and 3D model
an interface for mission plan, allowing for setting parameters. reconstruction are fully automated. Photoscan allows the genera-
The view is obtained from GPS will show the location for site area. tion and visualisation of a digital elevation model (DEM). A DEM
The interface also illustrates the user’s graphics, such as area, dis- represents a surface model as a regular grid of height values.
tance, maximum speed of aircraft, waypoint, altitude, resolution, DEM can be rasterised from a dense point cloud, a sparse point
duration time for flight planning and a connected display for air- cloud or a mesh. Most accurate results are calculated based on
craft. The Map Pilot can determine the exposure station pre- dense point cloud data. PhotoScan is able to perform DEM-based
launch based on the camera. The DJI Phantom 3 Professional can point, distance, area, volume measurements as well as generate
be navigated by both Map Pilot and human pilot. Usually the cross-sections for a part of a scene selected by the user. Addition-
take-off and landing are controlled by the human pilot, while the ally, contour lines can be calculated for the model and depicted
actual flight mission is navigated by the Map Pilot for DJI Phantom either over DEM or Orthomosaic in Ortho view within a PhotoScan
3 Professional in fully automated mode, following the planned environment (Fig. 5).
flight mission. In this study, the UAV travels at 4 m/s, while acquir- Rut and pothole models are measured by using a third party
ing images that are manually captured at 70% overlap along the software known as the Global Mapper software. The measurement
path. The focal length of phantom 3 pro camera is about of pavement road distress is based on a 3D model and orthoimages
3.61 mm. The estimation of ground sampling distance for altitude of the rut and pothole. A measuring tool is used to draw a polygon
40 m is about 17 mm and altitude 10 m is about 4 mm. In this on the 3D model and the tools will calculate the regional area and
study, the UAV flew at altitudes about 10 m and 40 m above
ground. The selection of this altitude is based on the different
ground sampling distance value to test whether these two
Fig. 3. UAV DJI Phantom 3 Professional with controller. Fig. 4. Sample UAV images at 10 m altitude above ground.
650 A.M. Saad, K.N. Tahar / Measurement 137 (2019) 647–654
volume of the pothole model. Fig. 6 illustrates the procedure to 3. Results and analysis
measure the maximum depth of a pothole. It also shows the mea-
suring tool used to measure the pothole depth. The DEM is used as Point cloud is a set of data points in some coordinates. In a 3D
input because it contains the elevation data for the study. A com- coordinate system, these points are usually defined by X, Y and Z
parison with field survey by using a tape was also conducted. coordinates and are often intended to represent the external sur-
The difference between image-based measurement and field sur- face of an object. 3D point cloud can be used to represent volumet-
vey is within the range of 1 cm. In addition to depth, other impor- ric data that are for example used in medical imaging [18]. Other
tant parameters in road condition assessment are the size and than that, it can be used to create 3D CAD models for manufactured
extension of pothole. parts, quality inspection and multitude of visualisation, animation,
rendering and mass customisation application [19]. Point cloud is vertices from 34 UAV images at altitude 10 m above the ground.
an important part in geographic information system whereby it Duration to process the 3D generation model took about 35 s.
is one of the sources used to make a digital elevation model The 3D model consists 83,673 faces of shape and about 42,097 ver-
(DEM) of a terrain. It is also used to generate 3D models of an urban tices from 28 UAV images at altitude 40 m. The duration to process
environment (Fig. 7a). A 3D model generation process is a repre- the 3D generation model took about 5 s. Fig. 8 shows the 3D model
sentative of any surface or object by manipulating polygons, edges generation created from a dense point cloud, where the site was at
and vertices in simulated 3D spaces. 3D modelling can be manually Jalan Bijak 2/2, near Palapes.
achieved with specialised 3D production software that allows an This study assess the accuracy of the generated pothole model
artist to create and deform polygonal surfaces or by scanning real based on a few samples. The comparison are between actual mea-
world objects into a set of data points that can be used to digitally surement and model measurement using third party software.
represent the object. It is widely used in engineering entertain- Third party software Global Mapper is used to visualize pothole
ment design, film visual effect, game development and commercial model developed from UAV images. This software also provide
advertising. specific tools for volume, perimeter and diameter measurement
The computer specification used in this study are intel core i7 compared processing software. The accuracy assessments are
(6th gen), 8 GB RAM and 1 TB storage and 2 GB graphics AMD divided into two road surface distress categories i.e. length of ruts
Radeon R5 M330 for image processing. Fig. 7b shows a 3D model and measurement of pothole samples. Table 1 illustrates the length
generation created from a dense point cloud. The 3D model con- of ruts measurement. There were five samples, named R1, R2, R3,
sists of 1,018,212 faces of shapes and about 511,286 vertices from R4 and R5. They were divided into two parts, which were in-situ
29 UAV images at altitude 10 m. To reconstruct, the parameters measurement and UAV measurement. UAV measurement was
include surface type and used field height. The duration to process divided into two altitudes, which were 10 m and 40 m altitude.
the 3D generation model takes about 1 min and 54 s. The 3D model Table 1 shows the comparison between actual and model measure-
consists of 607,721 faces of shape and about 304,895 vertices from ments for rut lengths at10m and 40 m altitudes. The results for
22 images at altitude 40 m. It is used the same parameters process- 10 m altitude, it was found that a higher difference was shown in
ing setting as 10 m altitude. The duration to process the 3D gener- Sample R4 and R5, which were about 0.003 m, while for the lowest
ation model takes about 35 s. The 3D model generation was difference, or error was sample R1, which about 0.001 m. Error is
created from a dense point cloud, where the site was at Jalan Bijak the difference between observed value of dependent variable (y)
2/2. The 3D model consists 77,632 faces of shape and about 39,369 and actual value (ŷ). Each data point has one error. Meanwhile,
Fig. 8. Image 3D model JalanBijak 2/2 near Palapes, UiTM Shah Alam with 10 m and 40 m altitudes.
652 A.M. Saad, K.N. Tahar / Measurement 137 (2019) 647–654
Table 1
Comparison between actual and measured for length of ruts for 10 m and 40 m altitudes.
Sample Number In-situ Measurement (m) UAV measurement (10 m) (m) Error (m) at 10 m UAV measurement (40 m) (m) Error (m) at 40 m
R1 9.920 9.921 0.001 9.902 0.018
R2 2.371 2.373 0.002 2.515 0.144
R3 1.922 1.924 0.002 1.898 0.024
R4 2.344 2.341 0.003 2.340 0.004
R5 2.153 2.150 0.003 2.120 0.033
Table 2
Measurement for pothole samples.
sample R4 and R5 illustrated the lower value from the actual of 10 m pothole diameter was sample P5, which was 0.010 m,
measurement. whereas the lowest value was Sample P2 and Sample P3. The high-
The results for 40 m altitude, it was found that a highest differ- est error for 40 m was sample P1 which recorded 0.020 m while
ence was shown in sample R2, which was about 0.144 m, while the the lowest error is 0.002 for sample P2.
lowest difference was sample R3, which was about 0.004 m. The The 10 m altitude had a slight difference as compared to the
standard deviation for 10 m and 40 m altitudes were ±0.002 m 40 m altitudes. The higher error for 10 m pothole perimeter was
and ±0.067 m, respectively. Table 2 illustrates the pothole sample sample P2 which was about 0.010 m, while the lowest value of
measurements. There were five samples of pothole, namely P1, error for 10 m altitude was sample P3, which about 0.001 m.
P2, P3, P4 and P5. They were divided into two parts which were The 40 m pothole perimeter has the highest value for error was
in-situ measurement and UAV measurement. UAV measurement sample P3, which was 0.044 m, while the lowest value was 0.001
was divided into two altitudes which were 10 m and 40 m altitude. for Sample P5. The standard deviation for diameter 10 m and
The pothole parameter measurement was taken by diameter (in 40 m are ±0.004 m and ±0.006 m respectively while standard devi-
metre), perimeter and area for each pothole. ation for perimeter 10 m and 40 m are ±0.004 m and ±0.017 m
Fig. 9 shows the difference in pothole diameters graph. There respectively. The mean absolute error for diameter for 10 m and
were five pothole samples. Blue colour shows the 10 m altitudes, 40 m are 0.003 m and 0.010 m respectively while the mean abso-
while red colour illustrates the 40 m altitude. The highest value lute error for 10 m and 40 m are 0.004 m and 0.023 m respectively.
Based on these results, it was found that the results from 10 m alti- Since the road data are documented in digital imagery format, re-
tudes provide an accurate results compared to 40 m altitude. measurement is possible, whenever necessary. The generated 3D
Fig. 10 shows the difference in pothole sample area. The error for road images enable better visualisation and inspection of road fea-
area at 10 m altitude has slight difference as compared to the tures. They can be easily integrated as video product for virtual
40 m altitude. The higher error for area 10 m altitude was for sam- road. In addition, the acquired digital imagery of roads, together
ple P1, which is about 0.006 m, while the lowest value of error for with extracted road condition data and the derived products can
10 m altitude was from sample P2 and P3, in which there was no be directly integrated with the road management system. In con-
difference in value. The error for area at 40 m altitude recorded clusion, this research has demonstrated the capability of UAV ima-
the highest value was from sample P1, which was 0.027 m while gery for data collection and detection of road pavement. This
the lowest value was 0.000 from sample P4. The standard deviation developed system can provide a detailed and accurate measure-
for area 10 m and 40 m are 0.003 m and 0.011 m respectively while ment of road surface distress, and thus improves the efficiency of
the mean absolute error for area 10 m and 40 m are 0.002 m and road condition data collection. The validation with actual measure-
0.009 m respectively. So, the 10 m also provide an accurate results ment by using a field survey is approved as compared to the UAV
in term area compared to 40 m. images. Future research can focus on the LiDAR UAV as a platform
to obtain the data. The advancement of UAV and other sensor tech-
4. Conclusions and future work nologies, which are more powerful, safer and inexpensive UAVs
will emerge in the market. Other technologies can also be used,
In conclusion, using UAV technique images is suitable to detect such as Lidar that can be integrated into the current system. This
will allow for more efficient road condition assessment in fully
rut and pothole in road pavement. The UAV images can determine
automatic mode.
various ruts and potholes for road pavement. It is easy to identify
the type of rut and pothole for road pavement by using the Agisoft
Photoscan because it has an automatic process to produce a 3D Acknowledgements
model generation and orthophoto product. This study has devel-
oped a UAV imagery for data acquisition in road pavement detec- Faculty of Architecture, Planning, and Surveying Universiti Tekno-
tion. This study has a cheap and flexible system that is easy to logi MARA (UiTM), Research Management Institute (RMi) and Min-
operate and can efficiently capture a fully automatic road imagery. istry of Higher Education (MOHE) are greatly acknowledged for
The images are automatically processed by using Agisoft Photoscan providing the fund BESTARI 600-IRMI/MyRA 5/3/BESTARI
and then analyses the acquired images for image orientation, gen- (001/2017) to enable this research to be carried out. The authors
eration of 3D models of road surface and ortho-rectified road would also like to thank the people who were directly or indirectly
images. This has allow for road surface distress extraction which involved in this research.
include potholes and ruts of road. Quantitative measurements
were conducted in office with a computer-aided technique. There-
fore, there is a difference in the field survey measurements that Authors’ contributions
used the conventional method. It is not necessary to go to the site
for data collection on road condition. This enables the local agen- AM Saad: Literature Search And Review, Manuscript Writing.
cies like the Public Works Department (PWD) to collect data for KN Tahar: Meta-Analysis, Content planning, Manuscript Writ-
the road pavement assessment more quickly, efficiently and safely. ing, Editing.
654 A.M. Saad, K.N. Tahar / Measurement 137 (2019) 647–654
References [10] S. Mansor, I.I. Ajibola, UAV-Based Imaging for Environment Sustainability-
Flash Floods Control Perspective. FIG Working Week 2013-Environment For
Sustainability (2013) 1–15.
[1] H. Bendea, P. Boccardo, S. Dequal, F.T. Giulio, D. Marenchino, M. Piras Part B8,
[11] S. Mansor, I.I. Ajibola, UAV-Based Imaging for the Environment Sustainability-
in: Low Cost Uav For Post-Disaster Assessment, The International Archives of
Flash Flood Control Perspective. FIG Working Week (2013) 1-15.
the Photogrammetry, Remote Sensing and Spatial Information Sciences,
[12] S. Nebiker, A. Annen, M. Scherrer, D. Oesch, A Light-Weight Multispectral
Beijing, 2008, pp. 1373–1380.
Sensor For Micro Uav –Oppoturnities For Very High Resolution Airborne
[2] H. Eisenbeiss, The Potential of Unmanned Aerial Vehicle for Mapping (2009)
Remote Sensing Part B1, The International Archives of the Photogrammetry,
135–144.
Remote Sensing and Spatial Information Sciences, Beijing, 2008, pp. 1193–
[3] J. Everaerts, The Use of Unmanned Aerial Vehicles (UAVs) for Remote Sensing
1200.
and Mapping Part B1, The International Archives of the Photogrammetry,
[13] F. Neitzel, J. Klonowski, Mobile 3d Mapping With A Low-Cost Uav System.
Remote Sensing and Spatial Information Sciences, 2008.
International Archives of the Photogrammetry, Remote Sensing and Spatial
[4] K.G. Fendi, M.S. Adam, N. Kokkas, M. Smith, An Approach to Produce a GIS
Information Sciences, 2011, pp. 39–44.
Database for Road Surface Monitoring, ICCEN, 2013.
[14] P.Y. Tiong, M. Mustaffar, M. Hainin, Road surface assessment of pothole
[5] Q. Feng, J. Liu, J. Gong, Urban Flood Mapping Base on Unmanned Aerial Vehicle
severity by close range photogrammetry method, World Appl. Sci. J. 867–873
Remote Sensing and Random Forest Classifier-A Case of Yuyao, China, Natl. Key
(2012).
Technol. R&D Program China (2015).
[15] P. Yee Tiong, M. Mustaffar, M. Hainin, Road surface assessment of pothole
[6] N. Graca, E. Mitishita, J. Goncalves, Photogrammetry mapping using unmanned
severity by, World Appl. Sci. J. 19 (2012) 867–873.
aerial vehicle. International archieves of the photogrammerty, Remote Sens.
[16] C. Zhang, Development of UAV-Based Remote Sensing System for Unpaved
Spatial Inf. Sci. XL-1 (2014) 129–133.
Road Condition Assessment, in: Annual Conference, ASPRS, Beijing, 2008, pp.
[7] G. Grenzdörffer, A. Engel, B. Teichert Part B1, in: The Photogrammetric
627–632.
Potential Of Low-Cost Uavs In Forestry, The International Archives of the
[17] C. Zhang, in: An UAV-Based Photogrammetric Mapping System For Road
Photogrammetry, Remote Sensing and Spatial Information Sciences, Beijing,
Condition Assessment, The International Archives of the Photogrammetry,
2008, pp. 1207–1214.
Remote Sensing and Spatial Information Sciences, 2008, pp. 627–632.
[8] H. Gyoo Sohn, J. Heo, H.-H. Yoo, S.S. Kim, H.-S. Cho, Hierarchical Multi-Sensor
[18] Darae Jeong, Yibao Li, Heon Ju Lee, et al., Efficient 3D volume reconstruction
Approach For. The International Archives of the Photogrammetry, Remote
from a point cloud using a phase-field method, Math. Problems Eng. 2018
Sensing and Spatial Information Sciences. Vol. XXXVII. Part B8 2008.
(2018) 9, https://doi.org/10.1155/2018/7090186.
[9] Hamizan, Road Network in Malaysia. Jabatan Kerja Raya Malaysia. <https://
[19] Yu et al., in: Three-dimensional model analysis and processing. Advanced
www.jkr.gov.my/app-jkr/index.php?setlang=en>, 2016 (accessed 26 February
topics in science and technology in China, Springer, Berlin, 2010, pp. 402–406.
2016).