基于pytorch实现bert4ner、图像分类、yolv5目标检测小工具

代码地址

压缩后放在百度网盘
链接: https://pan.baidu.com/s/1r4JhMSGD5XyFIJVvZ2zjMQ 提取码: i418

总体界面

image.png

预测结果

image.png

server {
        listen       80;
    listen  [::]:80;
        server_name  localhost;

        #charset koi8-r;

        #access_log  logs/host.access.log  main;

        location / {
            root   /app/www;
            autoindex on;  
        }
        
        location /images/ {
            alias /app/images/;
            autoindex on;
        }
    
        location /ner/ {
            add_header Cache-Control no-cache;
            add_header Cache-Control private;
            expires -1s;
            proxy_pass http://localhost:28582/;
        }

        location /detection/ {
            add_header Cache-Control no-cache;
            add_header Cache-Control private;
            expires -1s;
            proxy_pass http://localhost:28382/;
        }

        location /img/ {
            add_header Cache-Control no-cache;
            add_header Cache-Control private;
            expires -1s;
            proxy_pass http://localhost:28482/;
        }

docker容器

Dockerfile

FROM nvidia/cuda:11.4.0-base-ubuntu20.04
RUN sed -i 's/archive.ubuntu.com/mirrors.ustc.edu.cn/g' /etc/apt/sources.list
RUN mkdir /app
RUN apt update
RUN apt upgrade -y
RUN apt install build-essential -y
RUN apt install libtool -y
RUN apt-get install libpcre3 libpcre3-dev -y
RUN apt-get install zlib1g-dev -y
RUN apt-get install openssl -y
ADD nginx-1.20.0.tar.gz /usr/local/src/
RUN apt-get install gcc -y
RUN apt-get install g++ -y
RUN apt install libgl1-mesa-glx -y
RUN apt-get install make -y
RUN apt install wget -y
RUN apt install vim -y
WORKDIR /usr/local/src/nginx-1.20.0/
RUN ./configure --prefix=/usr/local/nginx --without-http_gzip_module && make && make install
ENV PATH /usr/local/nginx/sbin:$PATH
WORKDIR /app
RUN apt-get install -y python3 python3-pip
RUN ln -s /usr/bin/python3 /usr/bin/python
COPY requirements.txt .
COPY torch-1.8.2+cu111-cp38-cp38-linux_x86_64.whl .
COPY torchaudio-0.8.2-cp38-cp38-linux_x86_64.whl .
COPY torchvision-0.9.2+cu111-cp38-cp38-linux_x86_64.whl .
RUN pip install -r requirements.txt -i https://pypi.tuna.tsinghua.edu.cn/simple
RUN pip install torch-1.8.2+cu111-cp38-cp38-linux_x86_64.whl
RUN pip install torchaudio-0.8.2-cp38-cp38-linux_x86_64.whl
RUN pip install torchvision-0.9.2+cu111-cp38-cp38-linux_x86_64.whl
RUN pip install virtualenv==20.0.23 -i https://pypi.tuna.tsinghua.edu.cn/simple
RUN /bin/bash -c "source /etc/profile"
RUN /bin/bash -c "source ~/.bashrc"
RUN virtualenv /app/photo-categories-venv
RUN virtualenv /app/dd-content-venv
RUN virtualenv /app/yolov5-prj-venv
COPY ./photo-categories ./photo-categories
COPY ./dd-content ./dd-content
COPY ./yolov5-prj ./yolov5-prj
COPY ./dd-deep-front/vue-element-admin/dist ./www
ADD ./nginx.conf /usr/local/nginx/conf/
COPY start.sh .
RUN chmod +x start.sh
ENTRYPOINT ["/app/start.sh"]
EXPOSE 80

构建镜像

docker build -t dddeep:v1 .

Removing intermediate container 866de083c375
 ---> f0803340cb54
Step 38/46 : COPY ./photo-categories ./photo-categories
 ---> 0d917f6795d5
Step 39/46 : COPY ./dd-content ./dd-content
 ---> 88f8565cb740
Step 40/46 : COPY ./yolov5-prj ./yolov5-prj
 ---> 9f441e497fe6
Step 41/46 : COPY ./dd-deep-front/vue-element-admin/dist ./www
 ---> 0ee9dbcc63f5
Step 42/46 : ADD ./nginx.conf /usr/local/nginx/conf/
 ---> 875a0d8fe504
Step 43/46 : COPY start.sh .
 ---> 1cad813647db
Step 44/46 : RUN chmod +x start.sh
 ---> Running in 355ad926fd74
Removing intermediate container 355ad926fd74
 ---> 488b58ac5c68
Step 45/46 : ENTRYPOINT ["/app/start.sh"]
 ---> Running in 935b571763b5
Removing intermediate container 935b571763b5
 ---> c27a972960a0
Step 46/46 : EXPOSE 80
 ---> Running in 609c2736b3d5
Removing intermediate container 609c2736b3d5
 ---> 588251e4a32f
Successfully built 588251e4a32f
Successfully tagged dddeep:v1

创建容器

docker run --name dddeep_1 -d --gpus all -p 80:80 dddeep:v1

进入容器

docker exec -it dddeep_1 /bin/bash
查看启动的python服务，缺少yolov5的服务

root@f74abc4faa27:/app# ps -ef|grep python3
root           8       1 21 12:10 ?        00:00:06 /usr/bin/python3 /usr/local/bin/uvicorn main:app --host 0.0.0.0 --port 28582
root          10       1 14 12:10 ?        00:00:04 /usr/bin/python3 /usr/local/bin/uvicorn main:app --host 0.0.0.0 --port 28482

尝试启动yolov5服务

cd /app/yolov5-prj
source /app/yolov5-prj-venv/bin/activate
export PYTHONIOENCODING=utf-8
export PYTHONPATH=/app/yolov5-prj/backend/src:/app/yolov5-prj/yolov5
uvicorn main:app --host 0.0.0.0 --port 28382

出现缺少库错误

root@f74abc4faa27:/app# cd /app/yolov5-prj
root@f74abc4faa27:/app/yolov5-prj# source /app/yolov5-prj-venv/bin/activate
(yolov5-prj-venv) root@f74abc4faa27:/app/yolov5-prj# export PYTHONIOENCODING=utf-8
(yolov5-prj-venv) root@f74abc4faa27:/app/yolov5-prj# export PYTHONPATH=/app/yolov5-prj/backend/src:/app/yolov5-prj/yolov5
(yolov5-prj-venv) root@f74abc4faa27:/app/yolov5-prj# uvicorn main:app --host 0.0.0.0 --port 28382
Traceback (most recent call last):
  File "/usr/local/bin/uvicorn", line 8, in <module>
    sys.exit(main())
  File "/usr/local/lib/python3.8/dist-packages/click/core.py", line 1128, in __call__
    return self.main(*args, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/click/core.py", line 1053, in main
    rv = self.invoke(ctx)
  File "/usr/local/lib/python3.8/dist-packages/click/core.py", line 1395, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/usr/local/lib/python3.8/dist-packages/click/core.py", line 754, in invoke
    return __callback(*args, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/uvicorn/main.py", line 425, in main
    run(app, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/uvicorn/main.py", line 447, in run
    server.run()
  File "/usr/local/lib/python3.8/dist-packages/uvicorn/server.py", line 68, in run
    return asyncio.run(self.serve(sockets=sockets))
  File "/usr/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/dist-packages/uvicorn/server.py", line 76, in serve
    config.load()
  File "/usr/local/lib/python3.8/dist-packages/uvicorn/config.py", line 448, in load
    self.loaded_app = import_from_string(self.app)
  File "/usr/local/lib/python3.8/dist-packages/uvicorn/importer.py", line 24, in import_from_string
    raise exc from None
  File "/usr/local/lib/python3.8/dist-packages/uvicorn/importer.py", line 21, in import_from_string
    module = importlib.import_module(module_str)
  File "/usr/lib/python3.8/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1014, in _gcd_import
  File "<frozen importlib._bootstrap>", line 991, in _find_and_load
  File "<frozen importlib._bootstrap>", line 975, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 671, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 848, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "/app/yolov5-prj/backend/src/main.py", line 2, in <module>
    from api import api_yolov5
  File "/app/yolov5-prj/backend/src/api/api_yolov5.py", line 16, in <module>
    import detect
  File "/app/yolov5-prj/yolov5/detect.py", line 14, in <module>
    import cv2
  File "/usr/local/lib/python3.8/dist-packages/cv2/__init__.py", line 180, in <module>
    bootstrap()
  File "/usr/local/lib/python3.8/dist-packages/cv2/__init__.py", line 152, in bootstrap
    native_module = importlib.import_module("cv2")
  File "/usr/lib/python3.8/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
ImportError: libgthread-2.0.so.0: cannot open shared object file: No such file or directory

安装库libglib2.0-dev

(yolov5-prj-venv) root@f74abc4faa27:/app/yolov5-prj# apt-get install libglib2.0-dev -y

安装过程需要选择时区，选择“亚洲”> “上海”

Please select the geographic area in which you live. Subsequent configuration questions will narrow this down by presenting a list of cities, representing the time zones in which
they are located.

  1. Africa  2. America  3. Antarctica  4. Australia  5. Arctic  6. Asia  7. Atlantic  8. Europe  9. Indian  10. Pacific  11. SystemV  12. US  13. Etc
Geographic area: 6

Please select the city or region corresponding to your time zone.

  1. Aden      10. Bahrain     19. Chongqing  28. Harbin       37. Jerusalem    46. Kuala_Lumpur  55. Novokuznetsk  64. Qyzylorda      73. Taipei         82. Ulaanbaatar
  2. Almaty    11. Baku        20. Colombo    29. Hebron       38. Kabul        47. Kuching       56. Novosibirsk   65. Rangoon        74. Tashkent       83. Urumqi
  3. Amman     12. Bangkok     21. Damascus   30. Ho_Chi_Minh  39. Kamchatka    48. Kuwait        57. Omsk          66. Riyadh         75. Tbilisi        84. Ust-Nera
  4. Anadyr    13. Barnaul     22. Dhaka      31. Hong_Kong    40. Karachi      49. Macau         58. Oral          67. Sakhalin       76. Tehran         85. Vientiane
  5. Aqtau     14. Beirut      23. Dili       32. Hovd         41. Kashgar      50. Magadan       59. Phnom_Penh    68. Samarkand      77. Tel_Aviv       86. Vladivostok
  6. Aqtobe    15. Bishkek     24. Dubai      33. Irkutsk      42. Kathmandu    51. Makassar      60. Pontianak     69. Seoul          78. Thimphu        87. Yakutsk
  7. Ashgabat  16. Brunei      25. Dushanbe   34. Istanbul     43. Khandyga     52. Manila        61. Pyongyang     70. Shanghai       79. Tokyo          88. Yangon
  8. Atyrau    17. Chita       26. Famagusta  35. Jakarta      44. Kolkata      53. Muscat        62. Qatar         71. Singapore      80. Tomsk          89. Yekaterinburg
  9. Baghdad   18. Choibalsan  27. Gaza       36. Jayapura     45. Krasnoyarsk  54. Nicosia       63. Qostanay      72. Srednekolymsk  81. Ujung_Pandang  90. Yerevan
Time zone: 70

安装成功后，再执行启动yolov5服务

uvicorn main:app --host 0.0.0.0 --port 28382

(yolov5-prj-venv) root@f74abc4faa27:/app/yolov5-prj# uvicorn main:app --host 0.0.0.0 --port 28382
INFO:     Started server process [332]
INFO:     Waiting for application startup.
INFO:     Application startup complete.
INFO:     Uvicorn running on http://0.0.0.0:28382 (Press CTRL+C to quit)
^CINFO:     Shutting down
INFO:     Waiting for application shutdown.
INFO:     Application shutdown complete.
INFO:     Finished server process [332]

退出容器，停止容器，重新启动容器

(yolov5-prj-venv) root@f74abc4faa27:/app/yolov5-prj# exit

docker ps
CONTAINER ID   IMAGE       COMMAND           CREATED          STATUS          PORTS                               NAMES
f74abc4faa27   dddeep:v1   "/app/start.sh"   16 minutes ago   Up 16 minutes   0.0.0.0:80->80/tcp, :::80->80/tcp   dddeep_1

docker stop dddeep_1 

docker start dddeep_1

人面猴
序言：七十年代末，一起剥皮案震惊了整个滨河市，随后出现的几起案子，更是在滨河造成了极大的恐慌，老刑警刘岩，带你破解...
沈念sama阅读 212,332评论 6赞 493
死咒
序言：滨河连续发生了三起死亡事件，死亡现场离奇诡异，居然都是意外死亡，警方通过查阅死者的电脑和手机，发现死者居然都...
沈念sama阅读 90,508评论 3赞 385
救了他两次的神仙让他今天三更去死
文/潘晓璐我一进店门，熙熙楼的掌柜王于贵愁眉苦脸地迎上来，“玉大人，你说我怎么就摊上这事。” “怎么了？”我有些...
开封第一讲书人阅读 157,812评论 0赞 348
道士缉凶录：失踪的卖姜人
文/不坏的土叔我叫张陵，是天一观的道长。经常有香客问我，道长，这世上最难降的妖魔是什么？我笑而不...
开封第一讲书人阅读 56,607评论 1赞 284
港岛之恋（遗憾婚礼）
正文为了忘掉前任，我火速办了婚礼，结果婚礼上，老公的妹妹穿的比我还像新娘。我一直安慰自己，他们只是感情好，可当我...
茶点故事阅读 65,728评论 6赞 386
恶毒庶女顶嫁案：这布局不是一般人想出来的
文/花漫我一把揭开白布。她就那样静静地躺着，像睡着了一般。火红的嫁衣衬着肌肤如雪。梳的纹丝不乱的头发上，一...
开封第一讲书人阅读 49,919评论 1赞 290
城市分裂传说
那天，我揣着相机与录音，去河边找鬼。笑死，一个胖子当着我的面吹牛，可吹牛的内容都是我干的。我是一名探鬼主播，决...
沈念sama阅读 39,071评论 3赞 410
双鸳鸯连环套：你想象不到人心有多黑
文/苍兰香墨我猛地睁开眼，长吁一口气：“原来是场噩梦啊……” “哼！你这毒妇竟也来了？” 一声冷哼从身侧响起，我...
开封第一讲书人阅读 37,802评论 0赞 268
万荣杀人案实录
序言：老挝万荣一对情侣失踪，失踪者是张志新（化名）和其女友刘颖，没想到半个月后，有当地人在树林里发现了一具尸体，经...
沈念sama阅读 44,256评论 1赞 303
护林员之死
正文独居荒郊野岭守林人离奇死亡，尸身上长有42处带血的脓包…… 初始之章·张勋以下内容为张勋视角年9月15日...
茶点故事阅读 36,576评论 2赞 327
白月光启示录
正文我和宋清朗相恋三年，在试婚纱的时候发现自己被绿了。大学时的朋友给我发了我未婚夫和他白月光在一起吃饭的照片。...
茶点故事阅读 38,712评论 1赞 341
活死人
序言：一个原本活蹦乱跳的男人离奇死亡，死状恐怖，灵堂内的尸体忽然破棺而出，到底是诈尸还是另有隐情，我是刑警宁泽，带...
沈念sama阅读 34,389评论 4赞 332
日本核电站爆炸内幕
正文年R本政府宣布，位于F岛的核电站，受9级特大地震影响，放射性物质发生泄漏。R本人自食恶果不足惜，却给世界环境...
茶点故事阅读 40,032评论 3赞 316
男人毒药：我在死后第九天来索命
文/蒙蒙一、第九天我趴在偏房一处隐蔽的房顶上张望。院中可真热闹，春花似锦、人声如沸。这庄子的主人今日做“春日...
开封第一讲书人阅读 30,798评论 0赞 21
一桩弑父案，背后竟有这般阴谋
文/苍兰香墨我抬头看了看天上的太阳。三九已至，却和暖如春，着一层夹袄步出监牢的瞬间，已是汗流浃背。一阵脚步声响...
开封第一讲书人阅读 32,026评论 1赞 266
情欲美人皮
我被黑心中介骗来泰国打工，没想到刚下飞机就差点儿被人妖公主榨干…… 1. 我叫王不留，地道东北人。一个月前我还...
沈念sama阅读 46,473评论 2赞 360
代替公主和亲
正文我出身青楼，却偏偏与公主长得像，于是被迫代替她去往敌国和亲。传闻我的和亲对象是个残疾皇子，可洞房花烛夜当晚...
茶点故事阅读 43,606评论 2赞 350

基于pytorch实现bert4ner、图像分类、yolv5目标检测小工具

基于pytorch实现bert4ner、图像分类、yolv5目标检测小工具

代码地址

总体界面

预测结果

目录

总体思路

项目目录结构

模型预测地址（训练和删除功能被屏蔽）

部署架构

基本流程

BERT实现的NER句子内容提取

数据格式

使用 BOI 格式

数据目录

预测

图像分类

数据目录

预测

yolov5目标检测

数据目录

预测

预测结果

VUE-ELEMENT-ADMIN前端

来自VUE-ELEMENT-ADMIN，模板修改

使用nginx做前端服务器和后台服务代理

docker容器

Dockerfile

构建镜像

创建容器

进入容器

推荐阅读更多精彩内容