GISTAM 2015 Proceedings
GISTAM 2015 Proceedings
Abstract: Drones have become ubiquitous in performing risky and labor intensive areal tasks cheaply and safely. To
allow them to be autonomous, their flight plan needs to be pre-built for them. Existing works do not precalcu-
late flight paths but instead focus on navigation through camera based image processing techniques, genetic or
geometric algorithms to guide the drone during flight. That makes flight navigation complex and risky. In this
paper we present automated flight plan builder DIFPL which pre-builds flight plans for drones to survey a large
area. The flight plans are built for subregions and fed into drones which allow them to navigate autonomously.
DIFPL employs distributed paradigm on Hadoop MapReduce framework. Distribution is achieved by process-
ing sections or subregions in parallel. Experiments performed with network and elevation datasets validate the
efficiency of DIFPL in building optimal flight plans.
1 INTRODUCTION
17
GISTAM2015-1stInternationalConferenceonGeographicalInformationSystemsTheory,ApplicationsandManagement
ing, linear inequalities, and spatial index to query el- drones and distributed platforms for general spatial
evation around waypoints. It is powerful as it can au- data processing and specific to drones.
tomate flight path building with only terrain data and Automated Flights. Several ways to automate the
pre-known drone hardware limitations. Focus of this flight of drones exist including using sensors(Visse
work is on drone flights to take pictures of vegetation et al., 2011), camera images(Bills and Saxena, 2011),
over electricity poles network. Image analysis during feeding their waypoints as a file(Babaei and Mor-
post-processing determines if the vegetation has over- tazavi, 2010) or automating from the control(Lugo
grown over poles and needs trimming. The applica- and Zell, 2014). Genetic algorithms have been used
tion can automate flights for many such tasks such as to trace flight paths(De Paula Santos et al., 2013)
determine flood damage, deforestation, pollution and along with ant colony algorithms in 3D route plan-
agricultural activities. The volume of terrain and net- ning(Deng et al., 2013). Optimization algorithms for
work lines data for large areas increases rapidly. In or- multi-objective drone route planning have been ex-
der to scale to the large terrain and networks datasets, plored(Li et al., 2013).
DIFPL uses distributed paradigm on Hadoop MapRe- Distributed Spatial Operations. With increase
duce framework. The contributions of the paper are: in spatial data distributed approaches are increas-
Create Subregions within Overall Area Dy- ingly being explored. Spatial data processing using
namically. DIFPL uses a novel way to divide MapReduce is explored(Cary et al., 2009). Tech-
a large area into subregions that can be covered niques for accelerated processing with MapReduce
with a single flight of a drone. Processing terrain have been proposed(Wang et al., 2010). Distributed
data by subregion provides flexibility in deciding spatial operations on Hadoop and SpatialHadoop as
which type of drone to assign the subregion. Hadoop extension for spatial operations have been ex-
plored(Eldawy and Mokbel, 2013). Computational
Model Terrain Scenarios with Multiple Drone geometry algorithms have been distributed using Spa-
Types and Their Hardware Limits. This work tialHadoop(Eldawy et al., 2013). Distribution of
combines terrain and network data with drone drone data analysis and multiple drone flight coordi-
flight constraints for multiple drone types applied nation tasks has begun to gather momentum(Chmaj
as linear inequalities. It maximally exploits capa- and Selvaraj, 2015). Hadoop based platforms that
bility of each drone type to determine subregion support spatial queries with MapReduce are pro-
size and type of drone to cover it. posed(Aji et al., 2013).
Distribute Flight Plan Creation. Flight plan As far as we know, Previous works did not con-
construction is distributed with MapReduce sider multiple drone types and variations in terrain to-
framework. Distribution eliminates any limita- gether in automating flight paths. DIFPL does not rely
tions on size of spatial index on a single node with on images or video to navigate. Flight paths are built
novel key-value pair based joins and scales hori- offline with terrain and network lines data and do not
zontally to larger terrains datasets. need to be adjusted dynamically as all the constraints
Optimize Subregions After Distributed Pro- are applied at the time of running the program. Our
cessing. DIFPL minimizes subregions by merg- distributed technique makes 2 passes on the data us-
ing as many boundary regions as possible after ing standard hadoop constructs and avoids building a
creating flight plans for subregions. This requires large index on a single node by splitting the data into
aligning incomplete subregions so adjacent ones sections or subregions that can be processed indepen-
can be merged optimizing subregions and its ef- dently.
fectiveness is validated by experiments.
Section 2 explores research related to this work. Sec-
tion 3 presents the preliminary design considerations 3 PRELIMINARIES
of flight plan builder. In Section 4 the MapReduce
framework based DIFPL implementation is described In this section we describe the background informa-
followed by experiments in Section 5. Conclusions tion to DIFPL including input and output data, con-
are presented in Section 6. straints on the hardware and subregion and waypoint
construction.
Research similar to this work can be broken into two DIFPL uses inputs (x,y) geocoordinate position of
categories, previous approaches to automate flight of network lines endpoints provided by aviation orga-
18
DIFPL-DistributedDroneFlightPathBuilderSystem
19
GISTAM2015-1stInternationalConferenceonGeographicalInformationSystemsTheory,ApplicationsandManagement
4.1 Architecture
The architecture of DIFPL is based on distributed
paradigm. The distribution approach in DIFPL
was implemented using Apache Hadoop MapReduce
framework.
Hadoop. Hadoop(Apache and Hadoop, 2014) is an
open source framework which facilitates distributed
computations on large clusters. A master node or-
chestrates data storage and computation distribution
over multiple slave nodes. Files are uploaded into
distributed file storage called HDFS, split into 64MB
blocks and then processed. Master node keeps track
of all the blocks of a file and where they are stored.
MapReduce(Dean and Ghemawat, 2008) allows mas- Figure 3: Distributed System Architecture.
ter node to break down computation tasks into map-
pers and reducers distributed over slave nodes. They present else reads the network line and elevation data
work on file blocks on slave nodes exploiting coloca- {networki } and {elevationi } and inserts them into a
tion of computation with data. Mappers read in in- spatial index Si. The builder then starts querying in-
put data as key value pairs <k1 , v1 >and emit interme- dex with conventional drone default Dc sized subre-
diate key value pairs <k2 , v2 >. Reducer receive the gions as range queries qi . Objects returned in result
intermediate key value pairs grouped by k2 and pro- set of the query ri include network lines and eleva-
cesses them to generate the final set of key value pairs tion points inside the subregion. It calculates way-
<k3 , v3 >. points along network lines in query and then eleva-
The distribution of flight path builder is necessi- tion of waypoints using kNN query and averaging the
tated due to memory limitations of indexing large ele- elevation of nearest neighbors. If the network lines li
vation and network datasets on a single node. Several satisfy the conventional drone autonomy constraint Ac
opportunities for distribution of the flight plan builder but fail to satisfy the conventional drone climbing an-
process are available. The identification of quad- gle constraint Cc , then it queries spatial index for the
copter subregions and the shrinking of quadcopter and subregion using default size of quadcopter Dq with
conventional drone subregions can be performed in queries qqi from left to right. Each consecutive quad-
parallel. An overview of the distributed system ar- copter size subregion that satisfies the climbing angle
chitecture is shown in Figure 3. The distributed ap- constraint is merged with previous one. Ones that do
plication runs on a cluster on Amazon Web Services not satisfy Cc are deemed to require quadcopter. If
(AWS). MapReduce jobs are run on AWS Elastic the length of network lines in ri are too large or less
MapReduce (EMR) and data is read from and written than % threshold of the autonomy of quadcopter or
to S3 buckets similar to HDFS. It uses Hadoop 2.5.1 conventional drone, then the subregion is shrunk or
and MapReduce2. The experiments were performed expanded iteratively till it satisfies the autonomy con-
on a 5 node Hadoop cluster with 1 master and 4 slave straint for the respective drone type. The output for
nodes. each subregion Oi as waypoints and landing point is
written out.
4.2 Algorithms
4.2.2 Sectionwise Distribution
The algorithms used in DIFPL include a base Build
Flight Plans algorithm that performs queries and ap- The first task of distribution is to split the entire area
plies constraints on results within a section or subre- into sections and query for subregions in spatial in-
gion. Two levels of distribution are built, one based dex built separately for each section. This allows for
on the flight path algorithm and another parallelized building smaller sections in spatial index instead of
on subregions. the entire area.
The distribution paradigm is broken down into 2
4.2.1 Flight Path Building within Section phases, each translating into a pass over the data.
Each pass incrementally identifies subregions for
Algorithm 1 shows the process of building the flight quadcopter or conventional drone and final pass re-
paths. The flight plan building is implemented in solves boundary issues and outputs final flight paths
3 steps. The builder reads index file from disk if for each subregion.
20
DIFPL-DistributedDroneFlightPathBuilderSystem
Algorithm 1: Build Fligh Plans. Assign and Process Section. The first pass labels
network and elevation data with the section they be-
Input: {networki },{elevationi } {network lines and elevation
data}
long to. The details are described in Algorithm 2.
Output: {subregioni ,waypointi } {each subregion and way- The data is processed by two mappers. First map-
points} per reads network lines data {networki } as text and
1: {step 0: setup index and subregion iterations} calculates section id based on the coordinates of the
2: if index spatial index file does not exist then network line. The key value pair emitted from the
3: Si {networki ,elevation j } {create spatial index with mapper are <sectionID j ,networklinel >. The second
network lines as 2D objects and elevation points}
4: else
mapper similarly reads elevation data {elevation j }
5: read Si from disk {read in spatial index from disk} and calculates section id and emits <K,V>pairs
6: end if <sectionID j ,elevationi >. The reducer reads the data
7: query qi for subregions subregioni starting from bottom and aggregates all elevation and network line obser-
left of region with size Dc vations for a section id.
8: while !at top right of region do
9: regionDone=false Algorithm 2: Assign and Process Section.
10: ri Si(qi ) {query Spatial index Si with query qi to
generate result set} Input: {elevationi ,networklinei }
11: generate wapypointi in ri Output: subregion j {xsr j ,ysr j ,xer j ,yer j ,
12: for all waypointi in ri do quadcopter|conventional,{network j },{elevation j },
13: kNN
elevationwi kwaypointi {waypoint j }
14: end for 1: mapper1:
15: if max C(ri ) Cc then 2: calculate sectionID j based on its coordinates
16: {step 1: Subregion can be covered with conven- 3: emit:sectionID j elevation j
tional drone} 4: mapper2:
17: {retrieved objects elevations satisfy conventional 5: calculate sectionID j based on its coordinates
drone climbing angle constraint} 6: emit:sectionID j networkline j
18: while {| lr |} does not satisfy Ac do 7: reducer:
19: {retrieved objects network lines do not satisfy 8: for all observations do
conventional drone autonomy constraints} 9: build section j
20: Si qi 10: end for
21: {reduce or expand window size and re-query} 11: APPLY Build Flight Plans Algorithm
22: end while 12: emit:subregion j {xsr j ,ysr j ,xer j ,yer j ,
23: if {| lr |} satisfies Ac then quadcopter|conventional,{network j },{elevation j },
24: {conventional drone constraints satisfied} {waypoint j }
25: {waypointi } Cq {create waypoints for conven-
tional drone subregion} Reducer builds spatial index for elevation points
26: write output Oi for the subregion with network and elevation data in
27: qi q j {move to next window}
memory and marks quadcopter and conventional sub-
28: end if
29: else region by elevation and length constraints. It shrinks,
30: {step 2: Subregion needs quadcopter} expands and merges subregions as needed. The final
31: for all qqi qi do emitted <K,V>pairs in the reducer are the details of
32: ri Si(qqi ) each subregion within a section such as subregion id
33: {requery with default quadcopter subregion subregion j , the subregion extent xsr j ,ysr j ,xer j ,yer j ,
sizes}
a flag indicating if its a quadcopter or conventional
34: if !maxC(ri ) Cc then
35: {region fails conventional drone climbing an- drone subregion, the network lines and elevation
gle constraints} points and waypoints for a quadcopter or conventional
36: {waypointi } Qq drone to follow along the network lines in the subre-
37: {build waypoints for quadcopter} gion waypointk ,...,waypointl .
38: qqi qq j Resolve Edge Effects. Second pass resolves edge ef-
39: {move to next window}
40: else
fects between sections as described in Algorithm 3.
41: Oi qi1 {merge into previous conventional The input to the mapper are the subregions subregion j
subregion} with their extent, the drone type needed, network lines
42: Write Output Oi and elevation points in the subregion and the way-
43: end if points. For the subregions that are along the verti-
44: end for cal edge of the sections associated with their section
45: end if
46: end while id, reducer pairs the corresponding left and right sub-
regions. The subregions could be merged together if
they are covered by same drone type. The output from
21
GISTAM2015-1stInternationalConferenceonGeographicalInformationSystemsTheory,ApplicationsandManagement
22
DIFPL-DistributedDroneFlightPathBuilderSystem
23
GISTAM2015-1stInternationalConferenceonGeographicalInformationSystemsTheory,ApplicationsandManagement
24
DIFPL-DistributedDroneFlightPathBuilderSystem
Figure 10: All the subregions for conventional drone and Figure 11: All the subregions in overall area shown in blue
for quadcopter broken down by the analysis type. for conventional drone and red for quadcopter after merging
subregions across the vertical section boundaries.
ventional default sized subregions into minimal num-
ber of quadcopter and conventional drone subregions
is very effective in minimizing subregions, in partic-
ular quadcopter subregions. The bulk of efficiency
comes from expanding subregions reducing the num-
ber of conventional drone subregions from 1553 to
809. The splitting of a conventional drone subregion
into pinpoint quadcopter and efficiently merged con-
ventional drone subregions minimizes the number of
quadcopter subregions.
5.2 Distributed Scenarios Figure 12: All the subregions in overall area shown in blue
for conventional drone and red for quadcopter after subre-
gionwise distributed processing and merge.
In distributed paradigm, the scenarios change as sub-
regions on the edges of sections can not expand which
as then resolved in an additional step. shrunk if its network lines length exceeds the auton-
Shrink, Expand, Merge or Split Subregions. The omy of a conventional drone or quadcopter. We mark
subregions are shrunk and expanded based on the the subregions that do not satisfy the % network line
same threshold % of autonomy as used in sequen- length constraint after splitting or shrinking as incom-
tial algorithm for each section. This invariably means plete. Subegions that satisfy the length constraint are
several subregions on the right edge of each section do emitted as complete.
not get to meet the threshold as they run out of room Merging Subregions. All incomplete subregions are
to expand. This issue is mitigated to some extent by then ordered and adjacent ones that can be merged
the edge effect job. together are emitted as new subregions. Figure 12
Merge Section Edge Subregions. Edge subregions shows the subregions created by the subregion par-
for each section are merged together if feasible. The allelization. The entire set of subregions that can be
result after merging in a 4-section distribution are processed together are merged if possible in a single
shown in Figure 11. If the subregions are being cov- pass.
ered by the same drone type on either side of an edge Running the scenarios in the distributed paradigm
and they have not been expanded they are likely to be produces results that are similar to the sequential pro-
merged together. Experiments find 4 such subregions. cessing results in terms of number of subregions of
quadcopter or conventional drone type. Merging re-
duces quadcopter subregions by 627 and conventional
5.3 Optimized Distributed Scenarios subregions by 385.
Comparisons of single node run with distributed
Optimized distribution focuses on processing each and optimized distributed results are shown in Figure
subregion in parallel. The subregions are then rec- 13. It clearly shows the impact of merge on reducing
onciled by merging with adjacent subregions. the count of quadcopter and conventional drone sub-
Splitting or Shrinking Subregion. Since a subre- regions in optimized distribution. After merge there
gion by itself can not be expanded, it can only be are 1239 conventional drone subregions and 1839
25
GISTAM2015-1stInternationalConferenceonGeographicalInformationSystemsTheory,ApplicationsandManagement
26