base_api

Module Contents

Classes

APITemplateParser

Intermidate prompt template parser, specifically for API models.

BaseAPILLM

Base class for API model wrapper.

AsyncBaseAPILLM

Base class for API model wrapper.
class lagent.llms.base_api.APITemplateParser(meta_template=None)

Intermidate prompt template parser, specifically for API models.

参数:

meta_template (Dict) – The meta template for the model.

meta_template = None
class lagent.llms.base_api.BaseAPILLM(model_type, retry=2, template_parser=APITemplateParser, meta_template=None, *, max_new_tokens=512, top_p=0.8, top_k=40, temperature=0.8, repetition_penalty=0.0, stop_words=None)

Bases: lagent.llms.base_llm.BaseLLM

Base class for API model wrapper.

参数:

  • model_type (str) – The type of model.

  • retry (int) – Number of retires if the API call fails. Defaults to 2.

  • meta_template (Dict, optional) – The model’s meta prompt template if needed, in case the requirement of injecting or wrapping of any meta instructions.

  • template_parser (APITemplateParser) –

  • max_new_tokens (int) –

  • top_p (float) –

  • top_k (int) –

  • temperature (float) –

  • repetition_penalty (float) –

  • stop_words (Union[List[str], str]) –

is_api: bool = True
model_type
meta_template = None
retry = 2
gen_params
class lagent.llms.base_api.AsyncBaseAPILLM(model_type, retry=2, template_parser=APITemplateParser, meta_template=None, *, max_new_tokens=512, top_p=0.8, top_k=40, temperature=0.8, repetition_penalty=0.0, stop_words=None)

Bases: lagent.llms.base_llm.AsyncLLMMixin, BaseAPILLM

Base class for API model wrapper.

参数:

  • model_type (str) – The type of model.

  • retry (int) – Number of retires if the API call fails. Defaults to 2.

  • meta_template (Dict, optional) – The model’s meta prompt template if needed, in case the requirement of injecting or wrapping of any meta instructions.

  • template_parser (APITemplateParser) –

  • max_new_tokens (int) –

  • top_p (float) –

  • top_k (int) –

  • temperature (float) –

  • repetition_penalty (float) –

  • stop_words (Union[List[str], str]) –