openai

Module Contents

Classes

GPTAPI

Model wrapper around OpenAI's models.

AsyncGPTAPI

Model wrapper around OpenAI's models.

AsyncOpenAIWrapper

Attributes

OPENAI_API_BASE

lagent.llms.openai.OPENAI_API_BASE = 'https://api.openai.com/v1/chat/completions'
class lagent.llms.openai.GPTAPI(model_type='gpt-3.5-turbo', retry=2, json_mode=False, key='ENV', org=None, meta_template=[dict(role='system', api_role='system'), dict(role='user', api_role='user'), dict(role='assistant', api_role='assistant'), dict(role='environment', api_role='system')], api_base=OPENAI_API_BASE, proxies=None, **gen_params)

Bases: lagent.llms.base_api.BaseAPILLM

Model wrapper around OpenAI’s models.

参数:

  • model_type (str) – The name of OpenAI’s model.

  • retry (int) – Number of retires if the API call fails. Defaults to 2.

  • key (str or List[str]) – OpenAI key(s). In particular, when it is set to “ENV”, the key will be fetched from the environment variable $OPENAI_API_KEY, as how openai defaults to be. If it’s a list, the keys will be used in round-robin manner. Defaults to ‘ENV’.

  • org (str or List[str], optional) – OpenAI organization(s). If not specified, OpenAI uses the default organization bound to each API key. If specified, the orgs will be posted with each request in round-robin manner. Defaults to None.

  • meta_template (Dict, optional) – The model’s meta prompt template if needed, in case the requirement of injecting or wrapping of any meta instructions.

  • api_base (str) – The base url of OpenAI’s API. Defaults to ‘https://api.openai.com/v1/chat/completions’.

  • gen_params – Default generation configuration which could be overridden on the fly of generation.

  • json_mode (bool) –

  • proxies (Optional[Dict]) –

is_api: bool = True
logger
invalid_keys
key_ctr = 0
org_ctr = 0
url = 'https://api.openai.com/v1/chat/completions'
model_type = 'gpt-3.5-turbo'
proxies = None
json_mode = False
chat(inputs, **gen_params)

Generate responses given the contexts.

参数:

  • inputs (Union[List[dict], List[List[dict]]]) – a list of messages or list of lists of messages

  • gen_params – additional generation configuration

返回:

generated string(s)

返回类型:

Union[str, List[str]]

stream_chat(inputs, **gen_params)

Generate responses given the contexts.

参数:

  • inputs (List[dict]) – a list of messages

  • gen_params – additional generation configuration

返回:

generated string

返回类型:

str

generate_request_data(model_type, messages, gen_params, json_mode=False)

Generates the request data for different model types.

参数:

  • model_type (str) – The type of the model (e.g., ‘gpt’, ‘internlm’, ‘qwen’).

  • messages (list) – The list of messages to be sent to the model.

  • gen_params (dict) – The generation parameters.

  • json_mode (bool) – Flag to determine if the response format should be JSON.

返回:

A tuple containing the header and the request data.

返回类型:

tuple

tokenize(prompt)

Tokenize the input prompt.

参数:

prompt (str) – Input string.

返回:

token ids

返回类型:

list

class lagent.llms.openai.AsyncGPTAPI(model_type='gpt-3.5-turbo', retry=2, json_mode=False, key='ENV', org=None, meta_template=[dict(role='system', api_role='system'), dict(role='user', api_role='user'), dict(role='assistant', api_role='assistant'), dict(role='environment', api_role='system')], api_base=OPENAI_API_BASE, proxies=None, **gen_params)

Bases: lagent.llms.base_api.AsyncBaseAPILLM

Model wrapper around OpenAI’s models.

参数:

  • model_type (str) – The name of OpenAI’s model.

  • retry (int) – Number of retires if the API call fails. Defaults to 2.

  • key (str or List[str]) – OpenAI key(s). In particular, when it is set to “ENV”, the key will be fetched from the environment variable $OPENAI_API_KEY, as how openai defaults to be. If it’s a list, the keys will be used in round-robin manner. Defaults to ‘ENV’.

  • org (str or List[str], optional) – OpenAI organization(s). If not specified, OpenAI uses the default organization bound to each API key. If specified, the orgs will be posted with each request in round-robin manner. Defaults to None.

  • meta_template (Dict, optional) – The model’s meta prompt template if needed, in case the requirement of injecting or wrapping of any meta instructions.

  • api_base (str) – The base url of OpenAI’s API. Defaults to ‘https://api.openai.com/v1/chat/completions’.

  • gen_params – Default generation configuration which could be overridden on the fly of generation.

  • json_mode (bool) –

  • proxies (Optional[Dict]) –

is_api: bool = True
logger
invalid_keys
key_ctr = 0
org_ctr = 0
url = 'https://api.openai.com/v1/chat/completions'
model_type = 'gpt-3.5-turbo'
proxies
json_mode = False
async chat(inputs, session_ids=None, **gen_params)

Generate responses given the contexts.

参数:

  • inputs (Union[List[dict], List[List[dict]]]) – a list of messages or list of lists of messages

  • gen_params – additional generation configuration

  • session_ids (Union[int, List[int]]) –

返回:

generated string(s)

返回类型:

Union[str, List[str]]

async stream_chat(inputs, **gen_params)

Generate responses given the contexts.

参数:

  • inputs (List[dict]) – a list of messages

  • gen_params – additional generation configuration

返回:

generated string

返回类型:

str

generate_request_data(model_type, messages, gen_params, json_mode=False)

Generates the request data for different model types.

参数:

  • model_type (str) – The type of the model (e.g., ‘gpt’, ‘internlm’, ‘qwen’).

  • messages (list) – The list of messages to be sent to the model.

  • gen_params (dict) – The generation parameters.

  • json_mode (bool) – Flag to determine if the response format should be JSON.

返回:

A tuple containing the header and the request data.

返回类型:

tuple

tokenize(prompt)

Tokenize the input prompt.

参数:

prompt (str) – Input string.

返回:

token ids

返回类型:

list

class lagent.llms.openai.AsyncOpenAIWrapper(model, base_url, api_key=None, sample_params=None, proxy=None, timeout=600, max_retry=50, sleep_interval=5, extra_body=None)
参数:

  • model (str) –

  • base_url (str | List[str]) –

  • api_key (str) –

  • sample_params (dict) –

  • proxy (str) –

  • timeout (int) –

  • max_retry (int) –

  • sleep_interval (int) –

  • extra_body (dict) –

model
clients
sample_params
timeout = 600
max_retry = 50
sleep_interval = 5
extra_body = None
async chat(messages, session_id=None, **kwargs)
参数:

  • messages (list[dict]) –

  • session_id (str | int) –

返回类型:

openai.types.chat.ChatCompletion