Total time is about 30min, depending on your hardware and internet connection.
This will install PrivateGPT on a Windows 2019/2022 machine and run everything (including the LLMs) locally. After installation, no further Internet connection required for most operations.
Install the following pre-requisite software:
Git from https://git-scm.com/download/win
Python 3.11.7 from https://www.python.org/downloads/release/python-3117/
Cmake from https://cmake.org/download/
If you’re using a Nvidia GPU, also install:
MS Visual Studio Community 2022 from https://visualstudio.microsoft.com/vs/community/
Nvidia cuda_12.2.0_536.25_windows.exe from https://developer.download.nvidia.com/compute/cuda/12.2.0/local_installers/cuda_12.2.0_536.25_windows.exe
Open an administrative Powershell window and type:
c:\git clone https://github.com/imartinez/privateGPT
pip install poetry
python.exe -m pip install –upgrade pip
Set-ExecutionPolicy Bypass -Scope Process -Force; [System.Net.ServicePointManager]::SecurityProtocol = [System.Net.ServicePointManager]::SecurityProtocol -bor 3072; iex ((New-Object System.Net.WebClient).DownloadString(‘https://community.chocolatey.org/install.ps1’))
choco install make
poetry install –with ui,local
The following may take a while:
poetry run python scripts/setup
To enable PrivateGPT to use the Nvidia GPU, type:
$env:CMAKE_ARGS=’-DLLAMA_CUBLAS=on’; $Env:LLAMA_CUBLAS=”1″; $Env:FORCE_CMAKE=”1″; poetry run pip install –force-reinstall llama-cpp-python
Add some file processors for .docx, .html, ebooks, sound:
poetry add docx2txt ebookLib html2text whisper
Start the Private-GPT with:
make run
If all went well, after a minute, you’ll have an output that shows something like:
Open a browser on the machine that Private-GPT was installed on and type:
http://localhost:8001
Note: To remove uploaded files, stop the server and type:
make wipe