Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .gitmodules
Original file line number Diff line number Diff line change
@@ -1,3 +1,3 @@
[submodule "src/llama.cpp"]
path = src/llama.cpp
url = git@github.com:Bip-Rep/llama.cpp.git
url = https://github.com/ggerganov/llama.cpp.git
11 changes: 5 additions & 6 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,17 +1,16 @@
# Demo App for llama.cpp Model
This app is a demo of the llama.cpp model that tries to recreate an offline chatbot, working similar to OpenAI's ChatGPT. The source code for this app is available on GitHub.

# Now it works with Vicuna !!!
You can use the latest models on the app.

# Works on multiple devices :
Windows, mac and android !
[Releases page](https://github.com/Bip-Rep/sherpa/releases)


The app was developed using Flutter and implements ggerganov/llama.cpp, recompiled to work on mobiles. Please note that Meta officially distributes the LLaMA models, and they will not be provided by the app developers.
The app was developed using Flutter and is built upon ggerganov/llama.cpp, recompiled to work on mobiles.

The app will prompt you for a model file, which you need to provide. This must be a GGML model is compatible with llama.cpp as of June 30th 2022. Models from previous versions of Sherpa (which used an older version of llama.cpp) may no longer be compatible until conversion. The llama.cpp repository provides tools to convert ggml models to the latest format, as well as to produce ggml models from the original datasets.

To run this app, you need to download the 7B llama model from Meta for research purposes. You can choose the target model (should be a xxx.bin) from the app.
You can experiment with [Orca Mini 3B](https://huggingface.co/TheBloke/orca_mini_3B-GGML). These models (e.g. orca-mini-3b.ggmlv3.q4_0.bin from June 24 2023) are directly compatible with this version of Sherpa.

Additionally, you can fine-tune the ouput with preprompts to improve its performance.

Expand All @@ -23,7 +22,7 @@ It shows a OnePlus 7 with 8Gb running Sherpa without speed up.
## Usage
To use this app, follow these steps:

1. Download the `ggml-model.bin` from Meta for research purposes.
1. Download the GGML model from your chosen source.
2. Rename the downloaded file to `ggml-model.bin`.
3. Place the file in your device's download folder.
4. Run the app on your mobile device.
Expand Down
47 changes: 21 additions & 26 deletions android/app/build.gradle
Original file line number Diff line number Diff line change
Expand Up @@ -48,6 +48,10 @@ android {
}

defaultConfig {
ndk {
abiFilters 'arm64-v8a', 'x86_64'
}

// TODO: Specify your own unique Application ID (https://developer.android.com/studio/build/application-id.html).
applicationId "com.biprep.sherpa"
// You can update the following values to match your application needs.
Expand All @@ -59,32 +63,23 @@ android {
}

signingConfigs {

release {
storeFile keystoreProperties['storeFile'] ? file(keystoreProperties['storeFile']) : null
storePassword keystoreProperties['storePassword']
keyAlias keystoreProperties['releaseAlias']
keyPassword keystoreProperties['releasePassword']
}

debug {
storeFile keystoreProperties['storeFile'] ? file(keystoreProperties['storeFile']) : null
storePassword keystoreProperties['storePassword']
keyAlias keystoreProperties['releaseAlias']
keyPassword keystoreProperties['releasePassword']
}

}

buildTypes {
release {
signingConfig signingConfigs.release
proguardFiles getDefaultProguardFile("proguard-android.txt"), "proguard-rules.pro"
}

debug {
signingConfig signingConfigs.release
proguardFiles getDefaultProguardFile("proguard-android.txt"), "proguard-rules.pro"
release {
keyAlias keystoreProperties['keyAlias']
keyPassword keystoreProperties['keyPassword']
storeFile keystoreProperties['storeFile'] ? file(keystoreProperties['storeFile']) : null
storePassword keystoreProperties['storePassword']
}
}

buildTypes {
release {
signingConfig signingConfigs.release
}
}

externalNativeBuild {
cmake {
path "../../src/CMakeLists.txt"
}
}
}
Expand Down
Binary file removed assets/libs/libllama.so
Binary file not shown.
Binary file removed assets/libs/llama.dll
Binary file not shown.
Loading