书生大模型包括一系列开源的高质量的大模型,同时为了便利开发者和使用者,提供了全栈的开发工具。
GitHub地址为:https://github.com/internLM/
书生大模型全链路开源体系总共分三大部分
模型
- InternLM: 一系列的基础大模型和对话大模型.
- InternLM-Math: 强大的、专业的数学大模型.
- InternLM-XComposer: 基于InternLM开发的,支持视觉-语言复合数据的大模型.
InternLM
GitHub地址:https://github.com/InternLM/InternLM
书生基础大模型有以下版本
InternLM3
Model | Transformers | ModelScope | Modelers | Release Date |
---|---|---|---|---|
InternLM3-8B-Instruct | internlm3_8B_instruct | internlm3_8b_instruct | Open in Modelers | 2025-01-15 |
InternLM2.5
Model | Transformers(HF) | ModelScope(HF) | Release Date |
---|---|---|---|
InternLM2.5-1.8B | 🤗internlm2_5-1_8b | internlm2_5-1_8b | 2024-08-05 |
InternLM2.5-1.8B-Chat | 🤗internlm2_5-1_8b-chat | internlm2_5-1_8b-chat | 2024-08-05 |
InternLM2.5-7B | 🤗internlm2_5-7b | internlm2_5-7b | 2024-07-03 |
InternLM2.5-7B-Chat | 🤗internlm2_5-7b-chat | internlm2_5-7b-chat | 2024-07-03 |
InternLM2.5-7B-Chat-1M | 🤗internlm2_5-7b-chat-1m | internlm2_5-7b-chat-1m | 2024-07-03 |
InternLM2.5-20B | 🤗internlm2_5-20b | internlm2_5-20b | 2024-08-05 |
InternLM2.5-20B-Chat | 🤗internlm2_5-20b-chat | internlm2_5-20b-chat | 2024-08-05 |
InternLM-Math
InternLM2-Math-Plus
GitHub地::https://github.com/InternLM/InternLM-Math
Model | Model Type | Transformers(HF) | ModelScope | Release Date |
---|---|---|---|---|
InternLM2-Math-Plus-1.8B | Chat | 🤗internlm/internlm2-math-plus-1_8b | Shanghai_AI_Laboratory/internlm2-math-plus-1_8b | 2024-05-27 |
InternLM2-Math-Plus-7B | Chat | 🤗internlm/internlm2-math-plus-7b | Shanghai_AI_Laboratory/internlm2-math-plus-7b | 2024-05-27 |
InternLM2-Math-Plus-20B | Chat | 🤗internlm/internlm2-math-plus-20b | Shanghai_AI_Laboratory/internlm2-math-plus-20b | 2024-05-27 |
InternLM2-Math-Plus-Mixtral8x22B | Chat | 🤗internlm/internlm2-math-plus-mixtral8x22b | Shanghai_AI_Laboratory/internlm2-math-plus-mixtral8x22b | 2024-05-27 |
InternLM2-Math-Base
Model | Model Type | Transformers(HF) | ModelScope | Release Date |
---|---|---|---|---|
InternLM2-Math-Base-7B | Base | 🤗internlm/internlm2-math-base-7b | internlm2-math-base-7b | 2024-01-23 |
InternLM2-Math-Base-20B | Base | 🤗internlm/internlm2-math-base-20b | internlm2-math-base-20b | 2024-01-23 |
InternLM2-Math-7B | Chat | 🤗internlm/internlm2-math-7b | internlm2-math-7b | 2024-01-23 |
InternLM2-Math-20B | Chat | 🤗internlm/internlm2-math-20b | internlm2-math-20b | 2024-01-23 |
InternLM-XComposer
书生.浦语-灵笔
GitHub地址:https://github.com/InternLM/InternLM-XComposer
Model | Usage | Transformers(HF) | ModelScope(HF) | Release Date |
---|---|---|---|---|
InternLM-XComposer-2.5 | Video Understanding, Multi-image Multi-tune Dialog, 4K Resolution Understanding, Web Craft, Article creation, Benchmark | 🤗internlm-xcomposer2.5 | internlm-xcomposer2.5 | 2024-07-03 |
InternLM-XComposer2-4KHD | 4K Resolution Understanding, Benchmark, VL-Chat | 🤗internlm-xcomposer2-4khd-7b | internlm-xcomposer2-4khd-7b | 2024-04-09 |
InternLM-XComposer2-VL-1.8B | Benchmark, VL-Chat | 🤗internlm-xcomposer2-vl-1_8b | internlm-xcomposer2-vl-1_8b | 2024-04-09 |
InternLM-XComposer2 | Text-Image Composition | 🤗internlm-xcomposer2-7b | internlm-xcomposer2-7b | 2024-01-26 |
InternLM-XComposer2-VL | Benchmark, VL-Chat | 🤗internlm-xcomposer2-vl-7b | internlm-xcomposer2-vl-7b | 2024-01-26 |
InternLM-XComposer2-4bit | Text-Image Composition | 🤗internlm-xcomposer2-7b-4bit | internlm-xcomposer2-7b-4bit | 2024-02-06 |
InternLM-XComposer2-VL-4bit | Benchmark, VL-Chat | 🤗internlm-xcomposer2-vl-7b-4bit | internlm-xcomposer2-vl-7b-4bit | 2024-02-06 |
InternLM-XComposer | Text-Image Composition, VL-Chat | 🤗internlm-xcomposer-7b | internlm-xcomposer-7b | 2023-09-26 |
InternLM-XComposer-4bit | Text-Image Composition, VL-Chat | 🤗internlm-xcomposer-7b-4bit | internlm-xcomposer-7b-4bit | 2023-09-26 |
InternLM-XComposer-VL | Benchmark | 🤗internlm-xcomposer-vl-7b | internlm-xcomposer-vl-7b | 2023-09-26 |
工具链
- InternEvo: 支持大模型预训练和微调的轻量级框架。
- XTuner: 高效的大模型微调工具,支持多种大模型和多种调优方法。
- LMDeploy: 用来压缩、部署和使用大模型的工具。
- Lagent: 让用户高效率的开发agent的轻量级框架.
- AgentLego: 扩展和增强agent的一组类库和工具
- OpenCompass: 大模型评测平台.
- OpenAOE: 大模型比对工具.
InternEvo
XTuner
GitHub地址:https://github.com/InternLM/InternEvo/
支持多种大模型的预训练和微调
LMDeploy
GitHub地址:https://github.com/InternLM/lmdeploy
LLMs
Llama (7B - 65B)
Llama2 (7B - 70B)
Llama3 (8B, 70B)
Llama3.1 (8B, 70B)
Llama3.2 (1B, 3B)
InternLM (7B - 20B)
InternLM2 (7B - 20B)
InternLM3 (8B)
InternLM2.5 (7B)
Qwen (1.8B - 72B)
Qwen1.5 (0.5B - 110B)
Qwen1.5 - MoE (0.5B - 72B)
Qwen2 (0.5B - 72B)
Qwen2-MoE (57BA14B)
Qwen2.5 (0.5B - 32B)
Baichuan (7B)
Baichuan2 (7B-13B)
Code Llama (7B - 34B)
ChatGLM2 (6B)
GLM4 (9B)
CodeGeeX4 (9B)
Falcon (7B - 180B)
YI (6B-34B)
Mistral (7B)
DeepSeek-MoE (16B)
DeepSeek-V2 (16B, 236B)
DeepSeek-V2.5 (236B)
Mixtral (8x7B, 8x22B)
Gemma (2B - 7B)
Dbrx (132B)
StarCoder2 (3B - 15B)
Phi-3-mini (3.8B)
Phi-3.5-mini (3.8B)
Phi-3.5-MoE (16x3.8B)
MiniCPM3 (4B)
Lagent
GitHub地址:https://github.com/InternLM/lagent
高效、轻量级的开发工具,大大提高agent的开发效率。
AgentLego
提供了多种类库,支持开发强大的智能体
General ability
- Calculator: Calculate by Python interpreter.
- GoogleSearch: Search on Google.
Speech related
- TextToSpeech: Speak the input text into audio.
- SpeechToText: Transcribe an audio into text.
Image-processing related
- ImageDescription: Describe the input image.
- OCR: Recognize the text from a photo.
- VQA: Answer the question according to the image.
- HumanBodyPose: Estimate the pose or keypoints of human in an image.
- HumanFaceLandmark: Estimate the landmark or keypoints of human faces in an image.
- ImageToCanny: Extract the edge image from an image.
- ImageToDepth: Generate the depth image of an image.
- ImageToScribble: Generate a sketch scribble of an image.
- ObjectDetection: Detect all objects in the image.
- TextToBbox: Detect specific objects described by the given text in the image.
- Segment Anything series
- SegmentAnything: Segment all items in the image.
- SegmentObject: Segment the certain objects in the image according to the given object name.
AIGC related
- TextToImage: Generate an image from the input text.
- ImageExpansion: Expand the peripheral area of an image based on its content.
- ObjectRemove: Remove the certain objects in the image.
- ObjectReplace: Replace the certain objects in the image.
- ImageStylization: Modify an image according to the instructions.
- ControlNet series
- CannyTextToImage: Generate an image from a canny edge image and a description.
- DepthTextToImage: Generate an image from a depth image and a description.
- PoseToImage: Generate an image from a human pose image and a description.
- ScribbleTextToImage: Generate an image from a sketch scribble image and a description.
- ImageBind series
- AudioToImage: Generate an image according to audio.
- ThermalToImage: Generate an image according a thermal image.
- AudioImageToImage: Generate am image according to a audio and image.
- AudioTextToImage: Generate an image from a audio and text prompt.
OpenCompass
GitHub地址: https://github.com/open-compass/opencompass
OpenAOE
What can you get from OpenAOE?
OpenAOE can:
- return one or more LLMs' answers at the same time by a single prompt.
- provide access to commercial LLM APIs, with default support for gpt3.5, gpt4, Google Palm, Minimax, Claude, Spark, etc., and also support user-defined access to other large model APIs. (API keys need to be prepared in advanced)
- provide access to open-source LLM APIs. ( We recommend to use LMDeploy to deploy with one click)
- provide backend APIs and a WEB-UI to meet the needs of different requirements.
应用
- HuixiangDou茴香豆: 企业级知识库搭建工具.
- MindSearch: 基于大模型的互联网搜索引擎.
HuixiangDou茴香豆
GitHub地址:https://github.com/InternLM/HuixiangDou
HuixiangDou1 is a professional knowledge assistant based on LLM.
Advantages:
- Design three-stage pipelines of preprocess, rejection and response
-
chat_in_group
copes with group chat scenario, answer user questions without message flooding, see 2401.08772, 2405.02817, Hybrid Retrieval and Precision Report -
chat_with_repo
for real-time streaming chat
-
- No training required, with CPU-only, 2G, 10G, 20G and 80G configuration
- Offers a complete suite of Web, Android, and pipeline source code, industrial-grade and commercially viable
Check out the scenes in which HuixiangDou are running and join WeChat Group to try AI assistant inside.
If this helps you, please give it a star
MindSearch
GitHub地址: https://github.com/InternLM/MindSearch
MindSearch 是一个开源的 AI 搜索引擎框架,具有与 Perplexity.ai Pro 相同的性能。您可以轻松部署它来构建您自己的搜索引擎,可以使用闭源 LLM(如 GPT、Claude)或开源 LLM(InternLM2.5 系列模型经过专门优化,能够在 MindSearch 框架中提供卓越的性能;其他开源模型没做过具体测试)。其拥有以下特性:
- 🤔 任何想知道的问题:MindSearch 通过搜索解决你在生活中遇到的各种问题
- 📚 深度知识探索:MindSearch 通过数百网页的浏览,提供更广泛、深层次的答案
- 🔍 透明的解决方案路径:MindSearch 提供了思考路径、搜索关键词等完整的内容,提高回复的可信度和可用性。
- 💻 多种用户界面:为用户提供各种接口,包括 React、Gradio、Streamlit 和本地调试。根据需要选择任意类型。
- 🧠 动态图构建过程:MindSearch 将用户查询分解为图中的子问题节点,并根据 WebSearcher 的搜索结果逐步扩展图