KLEF

Academic year: 2020-21

Sem-In Examinations-I,
B. Tech (Branch), 2018 Batch
III/IV, 2nd Semester
Subject Code: 18CS3064 Subject: Big Data Optimization
Time: 2 hours Max. Marks: 50
Key and Scheme of Evaluation

1. Define the terms: Local Maxima and Global Maxima.

Local Maxima definition 1.5M
Global Maxima definition 1.5M
Diagram 1.5M

Functions can have "hills and valleys": places where they reach a minimum or maximum
value.
It may not be the minimum or maximum for the whole function, but locally it is.

Local Maximum
First we need to choose an interval:
Then we can say that a local maximum is the point where:

The height of the function at "a" is greater than (or equal to) the height anywhere else in that
interval.
Or, more briefly: f(a) ≥ f(x) for all x in the interval

In other words, there is no height greater than f(a).

Note: a should be inside the interval, not at one end or the other.

Global (or Absolute) Maximum and Minimum

The maximum or minimum over the entire function is called an "Absolute" or "Global"
maximum or minimum.

There is only one global maximum (and one global minimum) but there can be more than one
local maximum or minimum.

Assuming this function continues downwards to left or right:

• The Global Maximum is about 3.7

• The Global Minimum is −Infinity

2. Fill in the blanks with appropriate answers:

(1). __________ is the class of the object defined by x <- c(4, TRUE).

(2). _______________ returns TRUE then X can be termed as a matrix data object.

(3). Suppose I have a list defined as x <- list(2, "a", "b", TRUE). How can I fetch character
vector "b" from the list?

(4). __________are the data types present in R.

(5). The __________ function returns a list of all the formal arguments of a function.

(6). The graph of frequency distribution is called __________

(7). Data frames can be converted to a matrix by calling __________ function.

(8). ___________ function is used to create a box plots for visualization in R programming
language

(9). Name the function used to extract the first name in the string “Mrs. Jake Luther”?

Each answer carries 0.5 Marks (0.5 * 9 = 4.5 M)

Answers

(1) numeric (2) is.matrix(X) (3) x[[3]] (4) Vector,List etc

(5) formals() (6) histogram or pie chart (7) matrix() (8) boxplot()
(9) substr()

3. Identify the different ways of representing solution and evaluation functions for bag price
problem and Sum of Bits problem.

Representation 4M

Evaluation function 4M

Representation of a problem

• A major decision when using modern optimization methods is related with how to
represent a possible solution.

• Such decision sets the search space and its size, thus producing an impact on how new
solutions are searched.

• To represent a solution, there are several possibilities. Binary, integer, character, real
value and ordered vectors, matrices, trees and virtually any computer based
representation form (e.g., computer program) can be used to encode solutions.

Evaluation function

• Another important decision for handling optimization tasks is the definition of the
evaluation function, which should translate the desired goal (or goals) to be maximized or
minimized.

• Such function allows to compare different solutions, by providing either a rank or a

quality measure score.
 • When considering numeric functions, the shape can be convex or non-convex, with
several local minima/maxima.

Bag prices –

Sum of bits -

4. What will be the output of the following R code snippet?

(a) adj <- list("red", "big", "tasty");

fruits <- list("apple", "banana", "cherry");

for (x in adj)
{
for (y in fruits)
{
print(paste(x, y));
}
}
(b) Outer_func <- function(x)
{
Inner_func <- function(y)
{
a <- x + y; return(a);
}
return (Inner_func);
}
output <- Outer_func(3) ;
output(5);
(c) st1 <- c(‘abcd’, ‘bdcd’, ‘abcdabcd’);
pattern <- ‘^abc’; print( grep(pattern, st1));
(d) mat <- matrix(data = seq(10, 21, by=1), nrow = 6, ncol =2) ;
for (r in 1:nrow(mat)) for (c in 1:ncol(mat))
print(paste("mat[", r, ",",c, "]=", mat[r,c]));
print(mat);

Each bit output carries 2 Marks ( 2*4 = 8 M)

Output
(a)
[1] "red apple"
[1] "red banana"
[1] "red cherry"
[1] "big apple"
[1] "big banana"
[1] "big cherry"
[1] "tasty apple"
[1] "tasty banana"
[1] "tasty cherry"
(b)
[1] 8
(c)
[1] 1 3
(d)
[1] "mat[ 1 , 1 ]= 10"
[1] "mat[ 1 , 2 ]= 16"
[1] "mat[ 2 , 1 ]= 11"
[1] "mat[ 2 , 2 ]= 17"
[1] "mat[ 3 , 1 ]= 12"
[1] "mat[ 3 , 2 ]= 18"
[1] "mat[ 4 , 1 ]= 13"
[1] "mat[ 4 , 2 ]= 19"
[1] "mat[ 5 , 1 ]= 14"
[1] "mat[ 5 , 2 ]= 20"
[1] "mat[ 6 , 1 ]= 15"
[1] "mat[ 6 , 2 ]= 21"
> print(mat)
[,1] [,2]
[1,] 10 16
[2,] 11 17
[3,] 12 18
[4,] 13 19
[5,] 14 20
[6,] 15 21

5. Answer 5a and 5b

5a. Outline about the different methods used in dealing with infeasible solutions.

Each method carries 1 Mark ( 1*4 = 4 M)

To deal with infeasible solutions, several methods can be adopted

- death-penalty

- penalty-weights
 - repair

- generate feasible solutions

• Death-penalty is a simple method, which involves assigning a very large penalty value,
such that infeasible solutions are quickly discarded by the search.

• This method is not very efficient and often puts the search engine effort in discarding
solutions and not finding the optimum value.

• Penalty-weights is a better solution, also easy to implement.

• For example, quite often, the shape of an evaluation function can be set within the form
f(s) = Objective(s) – Penalty(s).

• For instance, for a given business, a possible evaluation function could be f = w1 *

Profit(s) - w2 *Cost(s).

• The main problem with penalty-weights is that often it is difficult to find the ideal
weights, in particular when several constraints are involved.

• The repair approach transforms an infeasible solution into a feasible one.

• This is achieved by using domain dependent information or by applying a local search

(e.g., looking for a feasible solution in the solution space neighborhood).

• Finally, the approaches that only generate feasible solutions are based in decoders and
special operators.

• Decoders work only in a feasible search space, by adopting an indirectly representation,

while special operators use domain knowledge to create new solutions from previous
ones.

5b. Using R-Programming, write a user defined function that finds the sum of the digits in a
given number.

Partial Logic 5M
Correct Logic 8.5M
sumofdig=function(n)
{
s=0
while (n > 0) {
r = n %% 10
s=s+r
 n = n %/% 10
}
print(paste("Sum of the digits is :", s))
}
n = as.integer(readline(prompt = "Enter a number :"))
sumofdig(n)
Input
Enter a number :121
Output
"Sum of the digits is : 4"

6. Answer 6a and 6b

6a. Classify the various types of optimization methods.

Types of optimization methods 4M

6b. Using R programming create a dataframe for the following data and then find the details of
the student having lowest grade and also find out details of students who joined in CSE Branch.
Retrieving student having lowest grade 4.5M
Retrieving details of students who joined in CSE Branch 4M
retval <- subset(data, Grade == min(Grade))
print(retval)
Output
retval <- subset(data, Branch==”CSE”))
print(retval)
Output

7. Distinguish between Full Blind Search algorithm and Depth Full Search algorithm.

Full Blind Search algorithm 1.5M

Depth Full Search algorithm. 1.5M
Example 1.5M
Full Blind Search

• Here we will discuss two blind search functions: fsearch and dfsearch.

• The former is a simpler function that requires the search space to be explicitly defined in
a matrix in the format solutions D (argument Search), while the latter performs a
recursive implementation of the depth-first search and requires the definition of the
domain values for each variable to be optimized (argument domain).

• Both functions receive as arguments the evaluation function (FUN), the optimization type
(type, a character with "min" or "max") and extra arguments, (denoted by ... and that
might be used by the evaluation function FUN).

Example for sum of bits problem

Depth First Search

 • Depth-first search goes through the tree branch by branch, going all the way down to the
leaf nodes at the bottom of the tree before trying the next branch over.

• This strategy requires much less memory than breadth-first search, since it only needs to
store a single path from the root of the tree down to the leaf node.

• However, it is potentially incomplete, since it will keep going on down one branch until it
finds a dead-end, and it is nonoptimal -- if there's a solution at the fourth level in the first
branch tried, and a solution at the second level in the next one over, the solution at the
fourth level will be returned.

• The time complexity of depth-first search is O(b^m) where b is the branching factor (2 for
the binary trees below) and m is the maximum depth of the tree. Its space complexity is
only b*m.

Example for sum of bits problem

8. List out the limitations of Hill Climbing Algorithm.

Each limitation carries 1.5 Marks (1.5 * 3= 4.5 M)

Limitations of Hill Climbing Algorithm

• Local maximum : At a local maximum all neighboring states have a values which is
worse than than the current state. Since hill climbing uses greedy approach, it will not
move to the worse state and terminate itself. The process will end even though a better
solution may exist.

• Plateau : On plateau all neighbors have same value . Hence, it is not possible to select
the best direction.

• Ridge : Any point on a ridge can look like peak because movement in all possible
directions is downward. Hence the algorithm stops when it reaches this state.
9. Find maximum profit obtained for bag prices problem when D=2 with x= (413,395) and m=
(1.25,1).

X=413 4M

X=395 4M

Sales(413)= round(1000/ln(413+200)-141)*1.25)

=round(1000/6.418349 – 141)*1.25)

=round(155.80292 – 141)*1.25)

=round(18.50365)=19

Sales(395)= round(1000/ln(395+200)-141)*1)

=round(1000/6.38856 – 141)*1)

=round(156.529797 – 141)*1)

=round(15.529797)=16

Cost(413)= 100+15*19=385

Cost(395)=100+10*16=260

Fbagprices(x)=( 413*19 – 385 )+ (395 *16 – 260) = 7462 + 6060= 13522.

10. Illustrate the concept of Simulated Annealing Search technique with neat diagram.

Simulated Annealing Explanation 5M

Diagram 2M
Algorithm 1M
Simulated Annealing

• Developed to over come the limitation of local minima in hill climbing.

• We need some mechanism that can help us to escape from the trap of local minima.
Simulated Annealing is one of such methods.

• Simulated annealing is a variation of the hill climbing technique that was proposed in the
1980s and that is inspired in the annealing phenomenon of metallurgy, which involves
first heating a particular metal and then perform a controlled cooling.
 • This single-state method differs from the hill climbing search by adopting a control
temperature parameter (T ) that is used to compute the probability of accepting inferior
solutions.

• Annealing is the process in which materials are raised to high energy levels for melting
and are then cooled to solid state.

• Simulated Annealing allows downward steps or transition to weaker sections as well.

• Metropolis function has high probability to accept a point which is downhill.

• In contrast with the stochastic hill climbing, which adopts a fixed value for T , the
simulated annealing uses a variable temperature value during the search.

• The method starts with a high temperature and then gradually decreases (cooling process)
the control parameter until a small value is achieved (similar to the hill climbing).

• It should be noted that for high temperatures the method is almost equivalent to Monte
Carlo search, thus behaving more like a global search method, while for low temperatures
the method is similar to the hill climbing local search

Initial point: x1, f(x1)

New Point: X2, f(x2)

If f(x2)<f(x1) then no metropolis function is used & the point is accepted.

If f(x2) > f(x1) then apply metropolis function is used to go downhill.

11. Answer 11a and 11b

11a. Illustrate the Grid Search optimization algorithm with neat diagram.

Grid Search Explanation 1M

Diagram 1M
Algorithm 3M

• Grid search reduces the space of solutions by implementing a regular hyper dimensional
search with a given step size.

• Grid search is particularly used for hyper-parameter optimization of machine learning

algorithms, such as neural networks or support vector machines.

• Uniform design search (Huang et al. 2007) is similar to the standard grid search method,
except that it uses a different type of grid, with lesser search points.

• Nested grid search is another variant that uses several grid search levels.

• The first level is used with a large step size.

• Then, a second grid level is applied over the best point, searching over a smaller area and
with a lower grid size. And so on.
 • Nested search is not a pure blind method, since it incorporates a greedy heuristic, where
the next level search is guided by the result of the current level search.

Algorithm

gsearch=function(step,lower,upper,FUN,type="min",...)

{ D=length(step) # dimension

domain=vector("list",D) # domain values

L=vector(length=D) # auxiliary vector

for(i in 1:D)

{ domain[[i]]=seq(lower[i],upper[i],by=step[i])
L[i]=length(domain[[i]])

LS=prod(L) s=matrix(ncol=D,nrow=LS) # set the search space

for(i in 1:D)

{ if(i==1) E=1

else E=E*L[i-1]

s[,i]=rep(domain[[i]],length.out=LS,each=E)

fsearch(s,FUN,type,...) # best solution

11b. Using R-Programming, Apply Grid search for Bag Prices problem.

Grid search for Bag Prices problem 7.5M

gsearch=function(step,lower,upper,FUN,type="min",...)

{ D=length(step) # dimension

domain=vector("list",D) # domain values

L=vector(length=D) # auxiliary vector

for(i in 1:D)
 { domain[[i]]=seq(lower[i],upper[i],by=step[i])
L[i]=length(domain[[i]])

LS=prod(L) s=matrix(ncol=D,nrow=LS) # set the search space

for(i in 1:D)

{ if(i==1) E=1

else E=E*L[i-1]

s[,i]=rep(domain[[i]],length.out=LS,each=E)

fsearch(s,FUN,type,...) # best solution

PTM=proc.time() # start clock

S1=gsearch(rep(100,5),rep(1,5),rep(1000,5),profit,"max")

sec=(proc.time()-PTM)[3] # get seconds elapsed

cat("gsearch best s:",S1$sol,"f:",S1$eval,"time:",sec,"s\n")

Output

gsearch best s: 401 401 401 401 501 f: 43142 time: 4.149 s

12. Answer 12a and 12b

12a. Identify the differences between Tabu Search and Simulated Annealing Search technique.

Tabu Search 2.5M

Simulated Annealing Search 2.5M

Simulated Annealing

• Developed to over come the limitation of local minima in hill climbing.

• We need some mechanism that can help us to escape from the trap of local minima.
Simulated Annealing is one of such methods.
• Simulated annealing is a variation of the hill climbing technique that was proposed in the
1980s and that is inspired in the annealing phenomenon of metallurgy, which involves
first heating a particular metal and then perform a controlled cooling.

• This single-state method differs from the hill climbing search by adopting a control
temperature parameter (T ) that is used to compute the probability of accepting inferior
solutions.

• Annealing is the process in which materials are raised to high energy levels for melting
and are then cooled to solid state.

• Simulated Annealing allows downward steps or transition to weaker sections as well.

• Metropolis function has high probability to accept a point which is downhill.

• In contrast with the stochastic hill climbing, which adopts a fixed value for T , the
simulated annealing uses a variable temperature value during the search.

• The method starts with a high temperature and then gradually decreases (cooling process)
the control parameter until a small value is achieved (similar to the hill climbing).

• It should be noted that for high temperatures the method is almost equivalent to Monte
Carlo search, thus behaving more like a global search method, while for low temperatures
the method is similar to the hill climbing local search

Tabu Search

• Tabu search was created by Glover (1986) and uses the concept of “memory” to force the
search into new areas.

• The algorithm is a variation of the hill climbing method that includes in a tabu list of
length L, which stores the most recent solutions that become “tabu” and thus cannot be
used when selecting a new solution.

• The intention is to keep a short-term memory of recent changes, preventing future moves
from deleting these changes.

• It explores the solution space beyond local optimum by use of Tabu List.

• Tabu List: Contains set of solutions which are not to be used anymore or forbidden.

• Tabu Tenure: Don’t make same move for certain period of time.

• It uses flexible memory .

 • Similarly to the hill climbing method, the search of solutions within the neighborhood of
the current solution (function change) can be deterministic, including the entire
neighborhood, or stochastic (e.g., small random perturbation).

• Tabu Search algorithm is deterministic, except if a stochastic change is adopted.

• worsening moves can be accepted if no improving move is available (like when the
search is stuck at a strict local minimum).

• There are extensions of the original tabu search method.

• Tabu search was devised for discrete spaces and combinatorial problems (e.g., traveling
salesman problem).

• However, the method can be extended to work with real valued points if a similarity
function is used to check if a solution is very close to a member of the tabu list.

12b. Using R-Programming, Solve Bag Prices Problem using Simulated Annealing Search
Optimization technique.

Bag Prices Problem using Simulated Annealing Search 7.5M

eval=function(x) -profit(x) # optim minimizes!

# hill climbing for all bag prices, one run:

D=5;

C=list(maxit=10000,temp=1000,trace=TRUE,REPORT=10000)

s=sample(1:1000,D,replace=TRUE) # initial search

ichange=function(par) # integer value change

{ D=length(par) hchange(par,lower=rep(1,D),upper=rep(1000,D),rnorm,mean=0, sd=1)

s=optim(s,eval,gr=ichange,method="SANN",control=C)

cat("best:",s$par,"profit:",abs(s$value),"\n")

Output

sann objective function values

initial value -35982.000000

final value -39449.000000

sann stopped after 9999 iterations

best: 293 570 634 606 474 profit: 39449 s