Scribd
Upload
66 views2 pages

Llamacpp

The document provides instructions for compiling and installing CLBlast and llama.cpp libraries to run large language models using OpenCL. It clones repositories, cleans and builds projects, sets environment variables and runs models on Intel GPUs, measuring load and generation times.

Uploaded by

Sheik Mohamed Imran
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as TXT, PDF, TXT or read online on Scribd
66 views2 pages

Llamacpp

The document provides instructions for compiling and installing CLBlast and llama.cpp libraries to run large language models using OpenCL. It clones repositories, cleans and builds projects, sets environment variables and runs models on Intel GPUs, measuring load and generation times.

Uploaded by

Sheik Mohamed Imran
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as TXT, PDF, TXT or read online on Scribd
You are on page 1/ 2

git clone https://github.com/CNugteren/CLBlast.

git
cd CLBlast
rm -rf build
mkdir build
cd build
make clean
cmake .. -DOPENCL_INCLUDE_DIRS=/opt/intel/oneapi/compiler/latest/linux/include/sycl
-DOPENCL_LIBRARIES=/opt/intel/oneapi/compiler/latest/linux/lib/libOpenCL.so -
DCMAKE_EXE_LINKER_FLAGS="-Wl,-rpath,/opt/intel/oneapi/compiler/latest/linux/
compiler/lib/intel64_lin" -DCMAKE_INSTALL_PREFIX=./Release -DBUILD_SHARED_LIBS=OFF
-DTUNERS=OFF
make
make install

cd ~/wrkdir

git clone https://github.com/ggerganov/llama.cpp.git

mkdir llama.cpp/build
cd llama.cpp/build
export LD_LIBRARY_PATH=/opt/intel/oneapi/compiler/latest/linux/include/:
$LD_LIBRARY_PATH

make clean
cmake .. -DLLAMA_CLBLAST=ON
-DCLBlast_DIR=/home/imran/wrkdir/CLBlast/build/Release/lib/cmake/CLBlast -
DCMAKE_EXE_LINKER_FLAGS="-Wl,-rpath,/opt/intel/oneapi/compiler/latest/linux/
compiler/lib/intel64_lin"
cmake --build . --config Release

export LD_LIBRARY_PATH=/opt/intel/oneapi/compiler/latest/linux/compiler/lib/
intel64_lin:$LD_LIBRARY_PATH

base) imran@flex-r3-u44:~/wrkdir/llama.cpp/build/bin$ clinfo -l


Platform #0: Intel(R) OpenCL Graphics
+-- Device #0: Intel(R) Data Center GPU Flex 170
`-- Device #1: Intel(R) Data Center GPU Flex 170

download llama-2-13b.ggmlv3.q4_0.bin
from https://huggingface.co/TheBloke/Llama-2-13B-GGML/tree/main

GGML_OPENCL_PLATFORM="Intel(R) OpenCL Graphics" GGML_OPENCL_DEVICE=0 ./main -m


~/wrkdir/llama-2-13b.ggmlv3.q4_0.bin -n 128 -ngl 40 -p "Building a website can be
done in 10 simple steps:"
GGML_OPENCL_PLATFORM="Intel(R) OpenCL Graphics" GGML_OPENCL_DEVICE=1 ./main -m
~/wrkdir/llama-2-13b.ggmlv3.q4_0.bin -n 128 -ngl 40 -p "Building a website can be
done in 10 simple steps:"
GGML_OPENCL_PLATFORM="Intel(R) OpenCL Graphics" GGML_OPENCL_DEVICE="#1" ./main -m
~/wrkdir/llama-2-13b.ggmlv3.q4_0.bin -n 128 -ngl 40 -p "Building a website can be
done in 10 simple steps:"
GGML_OPENCL_PLATFORM="Intel(R) OpenCL Graphics" GGML_OPENCL_DEVICE=0 ./main -m
~/wrkdir/llama-2-7b.ggmlv3.q4_0.bin -n 128 -ngl 40 -p "Building a website can be
done in 10 simple steps:"
GGML_OPENCL_PLATFORM="Intel(R) OpenCL Graphics" GGML_OPENCL_DEVICE=0,1 ./main --
main-gpu 0 --tensor-split 30,30 -m ~/wrkdir/llama2-22b-daydreamer-
v2.ggmlv3.q8_0.bin -n 128 -ngl 40 -p "Building a website can be done in 10 simple
steps:"

(base) hcp@scrappie1:~/projects/wizard/llama.cpp$ ./main -ngl 80 --main-gpu 0 --


tensor-split 0,12,12,12,12,12,12,8 -m ~/Desktop/models/alpaca-lora-
65B.ggmlv3.q4_0.bin -p "Instruction: Make a list of 10 imaginary fruits with a
description including shape, color and taste. List: "

load time: time it takes for the model to load.


sample time: time it takes to "tokenize" (sample) the prompt message for it to be
processed by the program.
prompt eval time: time it takes to process the tokenized prompt message. If this
isn't done, there would be no context for the model to know what token to predict
next.
eval time: time needed to generate all tokens as the response to the prompt
(excludes all pre-processing time, and it only measures the time since it starts
outputting tokens).

set(CLBlast_INCLUDE_DIRS "${CMAKE_CURRENT_LIST_DIR}/../../../include")
set(CLBlast_LIBRARIES "${CMAKE_CURRENT_LIST_DIR}/../../../lib/libclblast.a")

You might also like

pFad - Phonifier reborn

Pfad - The Proxy pFad of © 2024 Garber Painting. All rights reserved.

Note: This service is not intended for secure transactions such as banking, social media, email, or purchasing. Use at your own risk. We assume no liability whatsoever for broken pages.


Alternative Proxies:

Alternative Proxy

pFad Proxy

pFad v3 Proxy

pFad v4 Proxy