Compiler Design Lab Practical 1: LEX

Name: Mushan Khan

Roll No: 47

Practical No. 1

Theory

LEX:
Lex is a program generator designed for lexical processing of character input streams. It accepts a
high level, problem-oriented specification for character string matching, and produces a program
in a general-purpose language which recognizes regular expressions. The regular expressions are
specified by the user in the source specifications given to Lex. The Lex written code recognizes
these expressions in an input stream and partitions the input stream into strings matching the
expressions. At the boundaries between strings program sections provided by the user are
executed. The Lex source file associates the regular expressions and the program fragments. As
each expression appears in the input to the program written by Lex, the corresponding fragment is
executed.
Lex is not a complete language, but rather a generator representing a new language feature
which can be added to different programming languages, called ``host languages.'' Just as general
purpose languages can produce code to run on different com puter hardware, Lex can write code
in different host languages.
Lex turns the user's expressions and actions (called source in this pic) into the host general-
purpose language; the generated program is named yylex. The yylex program will recognize
expressions in a stream (called input in this pic) and perform the specified actions for each
expression as it is detected.

Diagram of LEX
Compiler Design Lab Practical 1: LEX

Format for Lex file

The general format of Lex source is:
{definitions}
%%
{rules}
%%
{user subroutines}
where the definitions and the user subroutines are often omitted. The second %% is optional, but
the first is required to mark the beginning of the rules. The absolute minimum Lex program is thus
%% (no definitions, no rules) which translates into a program which copies the input to the output
unchanged.

Regular Expression
A regular expression (or RE) specifies a set of strings that matches it; the functions in this module
let you check if a particular string matches a given regular expression (or if a given regular
expression matches a particular string, which comes down to the same thing).

Regular expressions can be concatenated to form new regular expressions; if A and B are both
regular expressions, then AB is also a regular expression. In general, if a string p matches A and
another string q matches B, the string pqwill match AB. This holds unless A or B contain low
precedence operations; boundary conditions between A and B; or have numbered group
references. Thus, complex expressions can easily be constructed from simpler primitive
expressions. Regular expressions can contain both special and ordinary characters. Most ordinary
characters, like "A", "a", or "0", are the simplest regular expressions; they simply match
themselves. You can concatenate ordinary characters, so last matches the string 'last'. (In the rest
of this section, we'll write RE's in this special style, usually without quotes, and strings to be
matched 'in single quotes'.)

Some characters, like "|" or "(", are special. Special characters either stand for classes of ordinary
characters or affect how the regular expressions around them are interpreted.

Lex Library Routines

Lex library routines are those functions which have a detailed knowledge of the lex functionalities
and which can be called to implement various tasks in a lex program.
The following table gives a list of some of the lex routines.

Lex Routine Description

Main() Invokes the lexical analyzer by calling the yylex subroutine.

yywrap() Returns the value 1 when the end of input occurs.

yymore() Appends the next matched string to the current value of the yytext array
rather than replacing the contents of the yytext array.

yyless(int n) Retains n initial characters in the yytext array and returns the remaining
Compiler Design Lab Practical 1: LEX

characters to the input stream.

yyreject Allows the lexical analyzer to match multiple rules for the same input
string. (The yyreject subroutine is called when the special action REJECT
is used.)

yylex() The default main() contains the call of yylex()

Answer the Questions:

1. Use of yywrap: Returns the value 1 when the end of input occurs.

2. Use of yylex function: The default main() contains the call of yylex()

3. What does lex.yy.c. do? lex.yy.c is the gcc compiled c file to run the lexicl analyser

Practical No. E1

Aim :
Write a lex code to identify the tokens such as keywords, identifiers, operators, constants
(Int,
float & character), special symbols and strings for C language using LEX. Use File for
the input.

Program:
%{
#include <stdio.h>
#include <string.h>
extern FILE *yyin;
%}

%%

\"([A-Za-z0-9 ])*\" {
printf("%s is a String\n", yytext);
}

"main"|"printf" {
printf("%s is a Function\n", yytext);
}

"include"|"if"|"else" {
printf("%s is a KeyWord\n", yytext);
}
Compiler Design Lab Practical 1: LEX

"int"|"float"|"double"|"char" {
printf("%s is a Constant\n", yytext);
}

[\;\,\.?\(\)\{\}\[\]\#] {
printf("%s is a Special Symbol\n", yytext);
}

[A-Za-z_][A-Za-z0-9_]* {
printf("%s is a Variable\n", yytext);
}

[\n\t\ ] {

[0-9]* {
printf("%s is a Number\n", yytext);
}

[\>=\==\>\<\<=\!=] {
printf("%s is a Comparision Operator\n");
}

[\+\-\*\/\%\=] {
printf("%s is a Binary Operator\n", yytext);
}

%%

int main(void) {
yyin = fopen("code.txt", "r");
if (yyin == NULL) {
perror("Failed to open file");
return 1;
}

yylex();
fclose(yyin);
return 0;
}

int yywrap() {
Compiler Design Lab Practical 1: LEX

return 1;
}
Input:
#include <stdio.h>
void main(){
int i = 0;
if(i >= 1){
printf("I is Greater than 1");
}else{
printf("I is Smaller than 1");
}
}

Output:

# is a Special Symbol
include is a KeyWord
include < is a Comparision Operator
stdio is a Variable
. is a Special Symbol
h is a Variable
h> is a Comparision Operator
void is a Variable
main is a Function
( is a Special Symbol
) is a Special Symbol
{ is a Special Symbol
int is a Constant
i is a Variable
i = is a Comparision Operator
0 is a Number
; is a Special Symbol
if is a KeyWord
( is a Special Symbol
i is a Variable
i > is a Comparision Operator
i >= is a Comparision Operator
1 is a Number
) is a Special Symbol
{ is a Special Symbol
printf is a Function
( is a Special Symbol
"I is Greater than 1" is a String
) is a Special Symbol
; is a Special Symbol
} is a Special Symbol
Compiler Design Lab Practical 1: LEX

else is a KeyWord
{ is a Special Symbol
printf is a Function
( is a Special Symbol
"I is Smaller than 1" is a String
) is a Special Symbol
; is a Special Symbol
} is a Special Symbol
} is a Special Symbol

Practical No. E2

Aim:
E2: Question Paper Analyzer
Write a Lex program to find the parameters given below. Consider as input a question
paper of
an examination.
1. Count the number of questions.
2. Number of questions that have sub-part and how many donot.
3. Count the total marks.
4. Date of examination
5. Semester
6. Count different types of questions- Eg: What, Discuss, etc.
7. Numbers of words, lines, small letters, capital letters, digits, and special characters.

Program:
%{
#include <stdio.h>
#include <string.h>
#include <stdlib.h>
extern FILE *yyin;
int questionCount = 0;
int subQuestion = 0;
int totalMarks = 0;
char date[20];
char sem[2];
int questionTypes = 0;
int explainType = 0,whatType = 0, discusstype = 0;
int words = 0,lines = 0,small=0,capital=0,digits=0,special=0;
int hashmap[5] = {0,0,0,0,0};
%}

%%

"Explain"|"Discuss"|"What" {
char type = yytext[0];
Compiler Design Lab Practical 1: LEX

if(type == 'E'){
explainType++;
}
if(type == 'D'){
discusstype++;
}
if(type == 'W'){
whatType++;
}
}

[0-9]Marks {
int num = yytext[0] - '0';
totalMarks+=num;
}

Q[0-9]+ {
int num = yytext[1] - '0';
hashmap[num] += 1;
if(hashmap[num] == 1){
questionCount++;
}
}

"I"|"II"|"III"|"IV"|"V"|"VI" {
strcpy(sem, yytext);
}

Q[0-9]\(.*\) {
int num = yytext[1] - '0';
hashmap[num]++;
if(hashmap[num] == 1){
questionCount++;
}
subQuestion++;
}

\n {
lines++;
}

[ \t] {
words++;
}

[A-Z] {
Compiler Design Lab Practical 1: LEX

capital++;
}

[a-z] {
small++;
}

[0-9] {
digits++;
}

[\?\(\)\[\]\.\:\/] {
special++;
}

%%

int main(void) {
yyin = fopen("question.txt", "r");
if (yyin == NULL) {
perror("Failed to open file");
return 1;
}

yylex();

printf("Semester %s\n",sem);
printf("Word Count %d\n",words);
printf("Capital case letters %d\n",capital);
printf("Small Case letters %d\n",small);
printf("lines %d\n",lines);
printf("Special Charectors %d\n",special);
printf("Word Count %d\n",words);
printf("Question Count %d\n",questionCount);
printf("Question With SubQuestions %d\n",questionCount);
printf("Sub Question Count %d\n",subQuestion);
printf("Total Marks %d\n",totalMarks);
printf("Semester : %d\n",sem);
printf("Questions with What : %d\n",whatType);
printf("Questions with Explain : %d\n",explainType);
printf("Questions with Discuss : %d\n",discusstype);
fclose(yyin);
return 0;
}
Compiler Design Lab Practical 1: LEX

int yywrap() {
return 1;
}
Input:
Shri Ramdeobaba College of Engineering and Management
Sem: VI

12/12/2024
Solve as per questions.
Q1(a): What is a compiler? [5Marks]
Q1(b): What is a software? [5Marks]
Q2(a): Explain the need for an assembler. [3Marks]
Q2(b): Discuss phases of compiler. [2Marks]
Q3: What is a parser? [5Marks]
Q4(a): What is error correction? [2Marks]
Q4(b): Explain SDTS. [3Marks]

Output:
Semester VI
Word Count 45
Capital case letters 11
Small Case letters 147
lines 11
Special Charectors 32
Word Count 45
Question Count 4
Question With SubQuestions 4
Sub Question Count 6
Total Marks 25
Semester : 4227316
Questions with What : 4
Questions with Explain : 2
Questions with Discuss : 1

Practical No. E3

Aim:
E3: Program Cleaner
Write a Lex Program which takes C program from file and write the same C program in
another file after removing the comments.
Program:
%{
#include <stdio.h>
#include <string.h>
#include <stdlib.h>

extern FILE *yyin;

Compiler Design Lab Practical 1: LEX

FILE *file;
char *content;
size_t content_size = 0;
%}

%%

"//".* ;

"/*"([^*]|\*+[^*/])*\*+ "/" ;

[^/]+ {
size_t len = strlen(yytext);
char *temp = realloc(content, content_size + len + 1);
content = temp;
strcpy(content + content_size, yytext);
content_size += len;
}

"/" {
size_t len = strlen(yytext);
char *temp = realloc(content, content_size + len + 1);
content = temp;
strcpy(content + content_size, yytext);
content_size += len;
}
%%

int main(void) {
content = malloc(1);
content[0] = '\0';

yyin = fopen("testcode.c", "r");

yylex();
fclose(yyin);

file = fopen("testcode.c", "w");

fputs(content, file);
fclose(file);
free(content);
return 0;
}
Compiler Design Lab Practical 1: LEX

int yywrap() {
return 1;
}
Input:
#include <stdio.h>
#include <stdlib.h>

void main(){
int i = 0;
// This is a Sample Code that contains comments

// I want this comment to be removed

char * content = malloc(100000 * sizeof(char));

FILE * file = fopen("test.txt", "w");

// This is a comment that should be removed

if(i >= 1){
printf("I is Greater than 1");
}else{
printf("I is Smaller than 1");
}
}

Output:
#include <stdio.h>
#include <stdlib.h>

void main(){
int i = 0;

char * content = malloc(100000 * sizeof(char));

FILE * file = fopen("test.txt", "w");

if(i >= 1){

printf("I is Greater than 1");
}else{
printf("I is Smaller than 1");
}
}
Compiler Design Lab Practical 1: LEX

Practical No. E4

Aim:
E4: Do as directed
Write a LEX specification to take the contents from a file
1. Add 3 to number divisible by 7
2. Add 4 to number divisible by 2
3. Convert the alphabetical list to numbered list
Program:
%{
#include <stdio.h>
#include <string.h>
#include <stdlib.h>
extern FILE *yyin;
%}

%%
[a-z]")" {
int n = yytext[0] - 'a' + 1;
printf("%d)",n);
}
[0-9]+ {
int num = atoi(yytext);
int sum = 0;
if(num%7 == 0){
sum+=3;
}
if(num%2 == 0){
sum+=4;
}
printf("%d", num + sum);
}

. { printf("%s", yytext); } /* Print any other character as is */

%%

int main(void) {
yyin = fopen("sample4.txt", "r");
if (yyin == NULL) {
perror("Failed to open file");
return 1;
}

yylex();
fclose(yyin);
return 0;
Compiler Design Lab Practical 1: LEX

int yywrap() {
return 1;
}
Input:
14
16
a) Bread
b) Butter
c) Magnets
d) Paint Brush
Output:
21
20
1) Bread
2) Butter
3) Magnets
4) Paint Brush