# 安裝 SGLang
pip install "sglang[all]>=0.4.3" --find-links https://flashinfer.ai/whl/cu124/torch2.5/flashinfer-python
# 啟動(dòng)推理服務(wù)
python3 -m sglang.launch_server --model /Your_Model_Path/DeepSeek-V3-0324 --tp 8 --trust-remote-code

import openai
client = openai.Client(
  base_url="http://127.0.0.1:30000/v1", 
  api_key="EMPTY")


  # Chat completion
response = client.chat.completions.create(
  model="default",
  messages=[
    {"role": "system", "content": "You are a helpful AI assistant"},
    {"role": "user", "content": "List 3 countries and their capitals."},
  ],
  temperature=0,
  max_tokens=64,
)
print(response)

file_template = """[file name]: {file_name}[file content begin]{file_content}[file content end]{question}"""

search_answer_zh_template = \
'''# 以下內(nèi)容是基于用戶發(fā)送的消息的搜索結(jié)果:
{search_results}
在我給你的搜索結(jié)果中，每個(gè)結(jié)果都是
[webpage X begin]...[webpage X end]格式的，X代表每篇文章的數(shù)字索引。請(qǐng)?jiān)谶m當(dāng)?shù)那闆r下在句子末尾引用上下文。請(qǐng)按照引用編號(hào)[citation:X]的格式在答案中對(duì)應(yīng)部分引用上下文。如果一句話源自多個(gè)上下文，請(qǐng)列出所有相關(guān)的引用編號(hào)，例如[citation:3][citation:5]，切記不要將引用集中在最后返回引用編號(hào)，而是在答案對(duì)應(yīng)部分列出。
在回答時(shí)，請(qǐng)注意以下幾點(diǎn)：
- 今天是{cur_date}。
- 并非搜索結(jié)果的所有內(nèi)容都與用戶的問題密切相關(guān)，你需要結(jié)合問題，對(duì)搜索結(jié)果進(jìn)行甄別、篩選。
- 對(duì)于列舉類的問題（如列舉所有航班信息），盡量將答案控制在10個(gè)要點(diǎn)以內(nèi)，并告訴用戶可以查看搜索來源、獲得完整信息。優(yōu)先提供信息完整、最相關(guān)的列舉項(xiàng)；如非必要，不要主動(dòng)告訴用戶搜索結(jié)果未提供的內(nèi)容。
- 對(duì)于創(chuàng)作類的問題（如寫論文），請(qǐng)務(wù)必在正文的段落中引用對(duì)應(yīng)的參考編號(hào)，例如[citation:3][citation:5]，不能只在文章末尾引用。你需要解讀并概括用戶的題目要求，選擇合適的格式，充分利用搜索結(jié)果并抽取重要信息，生成符合用戶要求、極具思想深度、富有創(chuàng)造力與專業(yè)性的答案。你的創(chuàng)作篇幅需要盡可能延長(zhǎng)，對(duì)于每一個(gè)要點(diǎn)的論述要推測(cè)用戶的意圖，給出盡可能多角度的回答要點(diǎn)，且務(wù)必信息量大、論述詳盡。
- 如果回答很長(zhǎng)，請(qǐng)盡量結(jié)構(gòu)化、分段落總結(jié)。如果需要分點(diǎn)作答，盡量控制在5個(gè)點(diǎn)以內(nèi)，并合并相關(guān)的內(nèi)容。
- 對(duì)于客觀類的問答，如果問題的答案非常簡(jiǎn)短，可以適當(dāng)補(bǔ)充一到兩句相關(guān)信息，以豐富內(nèi)容。
- 你需要根據(jù)用戶要求和回答內(nèi)容選擇合適、美觀的回答格式，確保可讀性強(qiáng)。
- 你的回答應(yīng)該綜合多個(gè)相關(guān)網(wǎng)頁來回答，不能重復(fù)引用一個(gè)網(wǎng)頁。
- 除非用戶要求，否則你回答的語言需要和用戶提問的語言保持一致。
# 用戶消息為：
{question}'''

search_answer_en_template = \
'''# The following contents are the search results related to the user's message:
{search_results}
In the search results I provide to you, each result is formatted as [webpage X begin]...[webpage X end], where X represents the numerical index of each article. Please cite the context at the end of the relevant sentence when appropriate. Use the citation format [citation:X] in the corresponding part of your answer. If a sentence is derived from multiple contexts, list all relevant citation numbers, such as [citation:3][citation:5]. Be sure not to cluster all citations at the end; instead, include them in the corresponding parts of the answer.
When responding, please keep the following points in mind:
- Today is {cur_date}.
- Not all content in the search results is closely related to the user's question. You need to evaluate and filter the search results based on the question.
- For listing-type questions (e.g., listing all flight information), try to limit the answer to 10 key points and inform the user that they can refer to the search sources for complete information. Prioritize providing the most complete and relevant items in the list. Avoid mentioning content not provided in the search results unless necessary.
- For creative tasks (e.g., writing an essay), ensure that references are cited within the body of the text, such as [citation:3][citation:5], rather than only at the end of the text. You need to interpret and summarize the user's requirements, choose an appropriate format, fully utilize the search results, extract key information, and generate an answer that is insightful, creative, and professional. Extend the length of your response as much as possible, addressing each point in detail and from multiple perspectives, ensuring the content is rich and thorough.
- If the response is lengthy, structure it well and summarize it in paragraphs. If a point-by-point format is needed, try to limit it to 5 points and merge related content.
- For objective Q&A, if the answer is very brief, you may add one or two related sentences to enrich the content.
- Choose an appropriate and visually appealing format for your response based on the user's requirements and the content of the answer, ensuring strong readability.
- Your answer should synthesize information from multiple relevant webpages and avoid repeatedly citing the same webpage.
- Unless the user requests otherwise, your response should be in the same language as the user's question.
# The user's message is:
{question}'''