在项目里快速运行史上最强开源大模型「Llama 3.1」

7月份，Meta正式发布迄今为止最强大的开源模型——Llama 3.1，包含了405B、70B和8B三个版本的模型。

作为开发者，如何在项目里快速运行LLaMa3.1大模型？

今天，我们结合英智未来自己的LLM推理API平台，给大家介绍一套免费实践流程。

免费使用地址：https://cognihub.baystoneai.com

方式一：不使用流式传输，直接让大模型返回最终结果

请求：

# sh

export API_BASE_URL="https://api.baystoneai.com/cognihub/service/66b97dd5fb19cc7397384f42"

export API_KEY="Sy0FY5eH1jOaKtcfVvIxN1zsgRWED4IgfoFA"

curl -X POST $API_BASE_URL/v1/chat/completions -H "Content-Type: application/json" -H "Authorization: Bearer $API_KEY" -d '{

    "model": "llama3.1",

    "stream": false,

    "messages": [

        {"role": "system", "content": "You are a helpful assistant."},

        {"role": "user", "content": "英智人工智能智汇API是什么？"}

    ]

}'

应答：

{

  "id": "chatcmpl-754",

  "object": "chat.completion",

  "created": 1723433933,

  "model": "llama3.1",

  "system_fingerprint": "fp_ollama",

  "choices": [

    {

      "index": 0,

      "message": {

        "role": "assistant",

        "content": "英智人工智能智汇API（Application Programming Interface）是指一种开放的开发接口，让第三方服务或应用能够访问和调用英智的人工智能技术。通过这些API，开发者可以整合英智的人工智能功能，如自然语言处理、图像识别和情绪分析，来提高自己的产品或服务。\n\n例如，一家移动应用开发商可能会使用英智的语音识别API，让用户能够通过语音说话给应用进行交互。或者，一家教育平台可能会利用英智的情绪分析API，对学生的情绪变化进行实时监测和反馈等应用。\n\n通过API接口，第三方开发者可以使用英智的人工智能能力，为更加强大和有趣的应用或服务提供支持。这也促进了应用和服务之间的快速创新与集成。"

      },

      "finish_reason": "stop"

    }

  ],

  "usage": {

    "prompt_tokens": 31,

    "completion_tokens": 186,

    "total_tokens": 217

  }

}

方式二：设置stream（流式传输）

请求：

# sh

export API_BASE_URL="https://api.baystoneai.com/cognihub/service/66b97dd5fb19cc7397384f42"

export API_KEY="Sy0FY5eH1jOaKtcfVvIxN1zsgRWED4IgfoFA"

curl -X POST $API_BASE_URL/v1/chat/completions -H "Content-Type: application/json" -H "Authorization: Bearer $API_KEY" -d '{

    "model": "llama3.1",

    "stream": true,

    "messages": [

        {"role": "system", "content": "You are a helpful assistant."},

        {"role": "user", "content": "英智人工智能智汇API是什么？"}

    ]

}'

应答：

data: {"id":"chatcmpl-514","object":"chat.completion.chunk","created":1723434271,"model":"llama3.1","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":"英"},"finish_reason":null}]}

data: {"id":"chatcmpl-514","object":"chat.completion.chunk","created":1723434271,"model":"llama3.1","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":"智"},"finish_reason":null}]}

data: {"id":"chatcmpl-514","object":"chat.completion.chunk","created":1723434271,"model":"llama3.1","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":"人"},"finish_reason":null}]}

data: {"id":"chatcmpl-514","object":"chat.completion.chunk","created":1723434271,"model":"llama3.1","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":"工"},"finish_reason":null}]}

data: {"id":"chatcmpl-514","object":"chat.completion.chunk","created":1723434271,"model":"llama3.1","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":"智能"},"finish_reason":null}]}

data: {"id":"chatcmpl-514","object":"chat.completion.chunk","created":1723434271,"model":"llama3.1","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":"智"},"finish_reason":null}]}

data: {"id":"chatcmpl-514","object":"chat.completion.chunk","created":1723434271,"model":"llama3.1","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":"汇"},"finish_reason":null}]}

data: {"id":"chatcmpl-514","object":"chat.completion.chunk","created":1723434271,"model":"llama3.1","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":"API"},"finish_reason":null}]}

data: {"id":"chatcmpl-514","object":"chat.completion.chunk","created":1723434271,"model":"llama3.1","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":"是一"},"finish_reason":null}]}

data: {"id":"chatcmpl-514","object":"chat.completion.chunk","created":1723434271,"model":"llama3.1","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":"种"},"finish_reason":null}]}

data: {"id":"chatcmpl-514","object":"chat.completion.chunk","created":1723434271,"model":"llama3.1","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":"可以"},"finish_reason":null}]}

data: {"id":"chatcmpl-514","object":"chat.completion.chunk","created":1723434271,"model":"llama3.1","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":"为"},"finish_reason":null}]}

data: {"id":"chatcmpl-514","object":"chat.completion.chunk","created":1723434272,"model":"llama3.1","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":"网站"},"finish_reason":null}]}

data: {"id":"chatcmpl-514","object":"chat.completion.chunk","created":1723434272,"model":"llama3.1","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":"、"},"finish_reason":null}]}

data: {"id":"chatcmpl-514","object":"chat.completion.chunk","created":1723434272,"model":"llama3.1","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":"应用"},"finish_reason":null}]}

data: {"id":"chatcmpl-514","object":"chat.completion.chunk","created":1723434272,"model":"llama3.1","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":"程序"},"finish_reason":null}]}

data: {"id":"chatcmpl-514","object":"chat.completion.chunk","created":1723434272,"model":"llama3.1","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":"和"},"finish_reason":null}]}

data: {"id":"chatcmpl-514","object":"chat.completion.chunk","created":1723434272,"model":"llama3.1","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":"其他"},"finish_reason":null}]}

data: {"id":"chatcmpl-514","object":"chat.completion.chunk","created":1723434272,"model":"llama3.1","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":"设备"},"finish_reason":null}]}

data: {"id":"chatcmpl-514","object":"chat.completion.chunk","created":1723434272,"model":"llama3.1","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":"提供"},"finish_reason":null}]}

data: {"id":"chatcmpl-514","object":"chat.completion.chunk","created":1723434272,"model":"llama3.1","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":"话"},"finish_reason":null}]}

data: {"id":"chatcmpl-514","object":"chat.completion.chunk","created":1723434272,"model":"llama3.1","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":"语"},"finish_reason":null}]}

data: {"id":"chatcmpl-514","object":"chat.completion.chunk","created":1723434272,"model":"llama3.1","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":"建议"},"finish_reason":null}]}

data: {"id":"chatcmpl-514","object":"chat.completion.chunk","created":1723434272,"model":"llama3.1","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":"、"},"finish_reason":null}]}

data: {"id":"chatcmpl-514","object":"chat.completion.chunk","created":1723434272,"model":"llama3.1","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":"文"},"finish_reason":null}]}

data: {"id":"chatcmpl-514","object":"chat.completion.chunk","created":1723434272,"model":"llama3.1","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":"本"},"finish_reason":null}]}

data: {"id":"chatcmpl-514","object":"chat.completion.chunk","created":1723434272,"model":"llama3.1","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":"分析"},"finish_reason":null}]}

data: {"id":"chatcmpl-514","object":"chat.completion.chunk","created":1723434272,"model":"llama3.1","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":"等"},"finish_reason":null}]}

data: {"id":"chatcmpl-514","object":"chat.completion.chunk","created":1723434272,"model":"llama3.1","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":"服务"},"finish_reason":null}]}

data: {"id":"chatcmpl-514","object":"chat.completion.chunk","created":1723434272,"model":"llama3.1","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":"的"},"finish_reason":null}]}

data: {"id":"chatcmpl-514","object":"chat.completion.chunk","created":1723434273,"model":"llama3.1","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":"软件"},"finish_reason":null}]}

data: {"id":"chatcmpl-514","object":"chat.completion.chunk","created":1723434273,"model":"llama3.1","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":"接"},"finish_reason":null}]}

data: {"id":"chatcmpl-514","object":"chat.completion.chunk","created":1723434273,"model":"llama3.1","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":"口"},"finish_reason":null}]}

data: {"id":"chatcmpl-514","object":"chat.completion.chunk","created":1723434273,"model":"llama3.1","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":"。"},"finish_reason":null}]}

data: {"id":"chatcmpl-514","object":"chat.completion.chunk","created":1723434273,"model":"llama3.1","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":""},"finish_reason":"stop"}]}

data: [DONE]

下面，给大家介绍一下「英智LLM推理API平台」，以及平台具有的优势。

英智LLM推理API平台

英智LLM推理API是一种基于人工智能大语言模型的推理API服务，它可以帮助开发者简单灵活地使用市面上的所有人工智能大语言模型，为企业定制专属的智能化解决方案。

目前已支持包括Llama 3.1、Mistral Large 2、Qwen 2等在内的主流开源大模型的最新版本，同时兼容OpenAI API。


平台优势

免费注册：申请服务的免费套餐后可获取专属您的“API_KEY”，即可接入本服务来免费使用API。

在线使用：支持在线使用开源大模型。

兼容能力强：提供API调用方式，与OpenAI API接口兼容(只要支持OpenAI API的应用，都可以使用)。

推理速度快：基于NVIDIA高性能GPU算力，推理速度更快。

自然语言处理：通过对大量文本数据进行分析和理解，实现对自然语言的智能处理，如情感分析、文本分类、关键词提取等。

语义理解：通过对自然语言的深入理解，实现对用户意图的准确识别，从而提供更精准的信息服务。

智能问答：通过对大量知识库的学习和理解，实现对用户提问的智能回答，提高用户满意度。

文本生成：通过对大量文本数据的学习，实现对特定主题或场景的文本生成，如新闻稿、产品描述等。

语音识别与合成：通过对语音数据的分析，实现对语音的识别和合成，提高人机交互的自然度和便捷性。

图像识别与处理：通过对图像数据的分析和理解，实现对图像内容的识别和处理，如人脸识别、物体检测等。

©著作权归作者所有,转载或内容合作请联系作者
  • 人面猴
    序言：七十年代末，一起剥皮案震惊了整个滨河市，随后出现的几起案子，更是在滨河造成了极大的恐慌，老刑警刘岩，带你破解...
    沈念sama阅读 215,384评论 6赞 497
  • 死咒
    序言：滨河连续发生了三起死亡事件，死亡现场离奇诡异，居然都是意外死亡，警方通过查阅死者的电脑和手机，发现死者居然都...
    沈念sama阅读 91,845评论 3赞 391
  • 救了他两次的神仙让他今天三更去死
    文/潘晓璐 我一进店门，熙熙楼的掌柜王于贵愁眉苦脸地迎上来，“玉大人，你说我怎么就摊上这事。” “怎么了？”我有些...
    开封第一讲书人阅读 161,148评论 0赞 351
  • 道士缉凶录：失踪的卖姜人
    文/不坏的土叔 我叫张陵，是天一观的道长。 经常有香客问我，道长，这世上最难降的妖魔是什么？ 我笑而不...
    开封第一讲书人阅读 57,640评论 1赞 290
  • 港岛之恋（遗憾婚礼）
    正文 为了忘掉前任，我火速办了婚礼，结果婚礼上，老公的妹妹穿的比我还像新娘。我一直安慰自己，他们只是感情好，可当我...
    茶点故事阅读 66,731评论 6赞 388
  • 恶毒庶女顶嫁案：这布局不是一般人想出来的
    文/花漫 我一把揭开白布。 她就那样静静地躺着，像睡着了一般。 火红的嫁衣衬着肌肤如雪。 梳的纹丝不乱的头发上，一...
    开封第一讲书人阅读 50,712评论 1赞 294
  • 城市分裂传说
    那天，我揣着相机与录音，去河边找鬼。 笑死，一个胖子当着我的面吹牛，可吹牛的内容都是我干的。 我是一名探鬼主播，决...
    沈念sama阅读 39,703评论 3赞 415
  • 双鸳鸯连环套：你想象不到人心有多黑
    文/苍兰香墨 我猛地睁开眼，长吁一口气：“原来是场噩梦啊……” “哼！你这毒妇竟也来了？” 一声冷哼从身侧响起，我...
    开封第一讲书人阅读 38,473评论 0赞 270
  • 万荣杀人案实录
    序言：老挝万荣一对情侣失踪，失踪者是张志新（化名）和其女友刘颖，没想到半个月后，有当地人在树林里发现了一具尸体，经...
    沈念sama阅读 44,915评论 1赞 307
  • 护林员之死
    正文 独居荒郊野岭守林人离奇死亡，尸身上长有42处带血的脓包…… 初始之章·张勋 以下内容为张勋视角 年9月15日...
    茶点故事阅读 37,227评论 2赞 331
  • 白月光启示录
    正文 我和宋清朗相恋三年，在试婚纱的时候发现自己被绿了。 大学时的朋友给我发了我未婚夫和他白月光在一起吃饭的照片。...
    茶点故事阅读 39,384评论 1赞 345
  • 活死人
    序言：一个原本活蹦乱跳的男人离奇死亡，死状恐怖，灵堂内的尸体忽然破棺而出，到底是诈尸还是另有隐情，我是刑警宁泽，带...
    沈念sama阅读 35,063评论 5赞 340
  • 日本核电站爆炸内幕
    正文 年R本政府宣布，位于F岛的核电站，受9级特大地震影响，放射性物质发生泄漏。R本人自食恶果不足惜，却给世界环境...
    茶点故事阅读 40,706评论 3赞 324
  • 男人毒药：我在死后第九天来索命
    文/蒙蒙 一、第九天 我趴在偏房一处隐蔽的房顶上张望。 院中可真热闹，春花似锦、人声如沸。这庄子的主人今日做“春日...
    开封第一讲书人阅读 31,302评论 0赞 21
  • 一桩弑父案，背后竟有这般阴谋
    文/苍兰香墨 我抬头看了看天上的太阳。三九已至，却和暖如春，着一层夹袄步出监牢的瞬间，已是汗流浃背。 一阵脚步声响...
    开封第一讲书人阅读 32,531评论 1赞 268
  • 情欲美人皮
    我被黑心中介骗来泰国打工， 没想到刚下飞机就差点儿被人妖公主榨干…… 1. 我叫王不留，地道东北人。 一个月前我还...
    沈念sama阅读 47,321评论 2赞 368
  • 代替公主和亲
    正文 我出身青楼，却偏偏与公主长得像，于是被迫代替她去往敌国和亲。 传闻我的和亲对象是个残疾皇子，可洞房花烛夜当晚...
    茶点故事阅读 44,248评论 2赞 352

推荐阅读更多精彩内容