Conda使用
下载
https://docs.conda.io/projects/conda/en/latest/user-guide/install/index.html设置环境
(使用Conda Prompt)
https://docs.conda.io/projects/conda/en/latest/user-guide/tasks/manage-environments.html
conda create -n leffa python==3.10
conda env list
conda activate leffa
conda deactivate leffa
conda env remove --name leffa
安装项目依赖
cd Leffa(需要在Anaconda PowerShell Prompt 里在启用环境时就cd，Anaconda Prompt的cmd无法cd)
pip install -r requirements.txt
在python 3.10版本下安装triton：
pip install https://huggingface.co/madbuda/triton-windows-builds/resolve/main/triton-3.0.0-cp310-cp310-win_amd64.whl
参考: https://huggingface.co/madbuda/triton-windows-builds/blob/main/README.md?code=true#L3运行AI模型
python app.py
报错“Torch not compiled with CUDA enabled”
原因是此时的pytorch版本是cpu的，
import torch; print(torch.version.cuda)
None
print(torch.version)
2.5.1+cpu
要下载有cuda support的版本, 且需要先卸载再下载:
https://blog.csdn.net/qq_66608435/article/details/141606279
- Google Cloud使用
登录到 Google Cloud Console。 在右上角，点击 "Activate Cloud Shell" 按钮。这将打开一个新的 Cloud Shell 会话:
https://shell.cloud.google.com/?show=ide%2Cterminal
Google Cloud Run - 构建和部署 Flask 应用
https://developers.google.cn/learn/pathways/cloud-run-serverless-computing?hl=zh-cn
https://cloud.google.com/run/docs/configuring/services/gpu-best-practices?hl=zh-cn#model-internet
如果是谷歌AI模型库里有的模型的话，很简单，Python的Web服务器Flask+ Vertex AI就可以了：
使用 GenAI 和 Cloud Run 构建知识问答生成器
如果模型库里还没有，就复杂一些:
https://cloud.google.com/storage/docs/reference/libraries#client-libraries-install-python
https://developer.nvidia.com/blog/machine-learning-in-practice-deploy-an-ml-model-on-google-cloud-platform/
https://medium.com/google-cloud/serving-ml-models-and-monitoring-predictions-in-google-cloud-vertex-ai-148b4e910e14
https://medium.com/publicis-sapient-france/how-to-deploy-your-own-ml-model-to-gcp-in-5-simple-steps-bf2b5898c1ab
或者可以先把model导入到Vertex AI :
https://huggingface.co/blog/alvarobartt/deploy-from-hub-to-vertex-ai
https://www.googlecloudcommunity.com/gc/AI-ML/How-do-I-deploy-my-custom-model-I-have-trained-on-workbench-GCP/m-p/551556
https://www.youtube.com/watch?v=Lw55J-koBT4
https://cloud.google.com/vertex-ai/docs/model-registry/import-model?hl=zh-cn
https://cloud.google.com/vertex-ai/docs/general/deployment?hl=zh-cn
https://medium.com/@yorrr78/guide-to-deploying-a-custom-containerized-ml-model-on-vertex-ai-8499d6eaa9ab
https://cloud.google.com/vertex-ai/docs/predictions/custom-prediction-routines?hl=zh-cn
