openai

Module Contents

Classes

GPTAPI

Model wrapper around OpenAI's models.

Attributes

OPENAI_API_BASE

lagent.llms.openai.OPENAI_API_BASE = 'https://api.openai.com/v1/chat/completions'
class lagent.llms.openai.GPTAPI(model_type='gpt-3.5-turbo', query_per_second=1, retry=2, key='ENV', org=None, meta_template=[dict(role='system', api_role='system'), dict(role='user', api_role='user'), dict(role='assistant', api_role='assistant')], openai_api_base=OPENAI_API_BASE, **gen_params)

Bases: lagent.llms.base_api.BaseAPIModel

Model wrapper around OpenAI’s models.

Parameters:

  • model_type (str) – The name of OpenAI’s model.

  • query_per_second (int) – The maximum queries allowed per second between two consecutive calls of the API. Defaults to 1.

  • retry (int) – Number of retires if the API call fails. Defaults to 2.

  • key (str or List[str]) – OpenAI key(s). In particular, when it is set to “ENV”, the key will be fetched from the environment variable $OPENAI_API_KEY, as how openai defaults to be. If it’s a list, the keys will be used in round-robin manner. Defaults to ‘ENV’.

  • org (str or List[str], optional) – OpenAI organization(s). If not specified, OpenAI uses the default organization bound to each API key. If specified, the orgs will be posted with each request in round-robin manner. Defaults to None.

  • meta_template (Dict, optional) – The model’s meta prompt template if needed, in case the requirement of injecting or wrapping of any meta instructions.

  • openai_api_base (str) – The base url of OpenAI’s API. Defaults to ‘https://api.openai.com/v1/chat/completions’.

  • gen_params – Default generation configuration which could be overridden on the fly of generation.

is_api: bool = True
chat(inputs, **gen_params)

Generate responses given the contexts.

Parameters:

  • inputs (Union[List[dict], List[List[dict]]]) – a list of messages or list of lists of messages

  • gen_params – additional generation configuration

Returns:

generated string(s)

Return type:

Union[str, List[str]]

tokenize(prompt)

Tokenize the input prompt.

Parameters:

prompt (str) – Input string.

Returns:

token ids

Return type:

list