termux build issues

[SubhranshuSharma]

pip install llama-cpp-python doesn't work on termux and gives this error, even if ninja is installed it gives same error, might be a hardcoded absolute path problem

i have tried to use portable venv setup on my linux machine but running it on termux gave a dependency not found error, so maybe some paths in the source code were still absolute(even after the correction attempts in the blog post)

tried using pyinstaller, but it doesn't support this library yet, same missing dependency issue

another option is to use docker on termux but that requires root privileges and custom kernel

i hv tried to look into the source code of this repo but donno where to start, any hint on where to start?

the original llama.cpp library works fine on termux but doesn't have a server inbuilt, and doesn't work well unless using bash

should i make a pull request editing the readme, linking to the docker workaround on rooted phones

[aseok]

I resolved ninja installation by termux-chroot. Also added following to
project CMakeLists.txt:
set(CMAKE_C_COMPILER "/data/data/com.termux/files/usr/bin/clang")
set(CMAKE_CXX_COMPILER "/data/data/com.termux/files/usr/bin/clang++")
And passing opencl args:
CMAKE_ARGS="-DLLAMA_CLBLAST=on -DLLAMA_OPENBLAS=on" FORCE_CMAKE=1 pip install llama-cpp-python
Here's output eror:
../CMake-src/Utilities/cmlibarchive/libarchive/archive.h:101:10: fatal error: 'android_lf.h' file not found
#include "android_lf.h"

[abetlen]

Looks to be related to scikit-build/cmake-python-distributions#223

The longer term solution seems to be migrating the project to scikit-build-core (something I'm in the process of doing). However someone in the thread mentioned they were able to get their example to work by building cmake and ninja from source on the android.

[aseok]

Also tried following:
cmake .. -DLLAMA_CLBLAST=on -DLLAMA_OPENBLAS=on
make
make install

Here's output:
CMake Error: Target DependInfo.cmake file not found
[100%] Built target run
Install the project...
-- Install configuration: "RelWithDebInfo"
-- Up-to-date: /usr/local/llama_cpp/libllama.so

Is installing successful?

Python:

import llama_cpp
Traceback (most recent call last):
File "", line 1, in
ModuleNotFoundError: No module named 'llama_cpp'

[SubhranshuSharma]

while trying to make it work on termux i ended up writing a small llama.cpp wrapper myself, instructions to use which are here, at the time llama.cpp cache files could be read but not generated/edited in termux, but that problem is sorted now, so put cache_is_supported_trust_me_bro=True in discord/termux/settings.py to use it.

these highlighted lines are all you need to make a wrapper of your own

[aseok]

Is it compatible with babyagi and other stuff that needs llama python wrapper?

…

On Wed, Jun 28, 2023, 10:37 PM Subhranshu Sharma ***@***.***> wrote: while trying to make it work on termux i ended up writing a small llama.cpp wrapper <https://github.com/SubhranshuSharma/The_Elite_Order_of_Pure_Thought/tree/cb063f5ac0d8c8a26bc75f4f0de18cc8cee062ef/discord/termux> myself, instructions to use which are here <https://github.com/SubhranshuSharma/The_Elite_Order_of_Pure_Thought/tree/cb063f5ac0d8c8a26bc75f4f0de18cc8cee062ef#linuxtermux>, at the time llama.cpp cache files could be read but not generated/edited in termux, but that problem is sorted now, so put cache_is_supported_trust_me_bro=True in discord/termux/settings.py to use it. these highlighted lines <https://github.com/SubhranshuSharma/The_Elite_Order_of_Pure_Thought/blob/cb063f5ac0d8c8a26bc75f4f0de18cc8cee062ef/discord/termux/the_elite_bot_termux.py#L40C1-L51C71> are all you need to make a wrapper of your own — Reply to this email directly, view it on GitHub <#389 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AD5KKLEJQUGTJKV45ZHQCJTXNR6I5ANCNFSM6AAAAAAZKDJVOQ> . You are receiving this because you commented.Message ID: ***@***.***>

[SubhranshuSharma]

Is it compatible with babyagi and other stuff that needs llama python wrapper?

nope, my wrapper is tightly integrated with my usecase and isn't a separate installable package, was thinking of doing so, but would be too much of a headache

[abetlen]

@SubhranshuSharma so for your wrapper you just built the libllama.so seperately correct, did you just build llama.cpp with the Makefile? Maybe one solution is to avoid building llama.cpp on install by setting an environment variable / path to a pre-built library.

[SubhranshuSharma]

@abetlen Yeah, I run git clone https://github.com/ggerganov/llama.cpp ~/llama.cpp;cd ~/llama.cpp;make

I suggest, try building llama.cpp but don't crash if u can't, just check if the main file exists at a pre defined/user inputted path, if not use if os.system('git clone https://github.com/ggerganov/llama.cpp ~/llama.cpp;cd ~/llama.cpp;make')!=0:print('windows is for pussies, install git and cmake');exit() to clone and make it, way more simple and easy to maintain.

[Freed-Wu]

The longer term solution seems to be migrating the project to scikit-build-core

Any progress about it?

[SubhranshuSharma]

Any progress about it?

seems related

[abetlen]

@SubhranshuSharma @Freed-Wu implementing in #499 but I just have some issues with Macos still

[SubhranshuSharma]

unrelated question: is there any way of storing cache files on disk for quick reboot in the api

implementing in #499 but I just have some issues with Macos still

i would still suggest treating this repo and llama.cpp as different things and not letting failure in one stop the other (for as long as its possible), so make the compilation a try except pass, if compile fails, force user to set a system variable pointing to llama.cpp, I would also suggest keeping the system variable at first priority, as in if the system variable is set, it will be given the first priority. That way the project will be more robust, letting people find workarounds to issues that originate in llama.cpp.

[abetlen]

@SubhranshuSharma sorry for this very late reply but I finally merged in #499.

You can now set CMAKE_ARGS="-DLLAMA_BUILD=OFF pip install llama-cpp-python to avoid building from source, then just set LLAMA_CPP_LIB to the path to the shared library to use a pre-built library.