base_llm

Module Contents

Classes

LMTemplateParser

Intermidate prompt template parser, specifically for language models.

BaseModel

Base class for model wrapper.
class lagent.llms.base_llm.LMTemplateParser(meta_template=None)

Intermidate prompt template parser, specifically for language models.

Parameters:

meta_template (list of dict, optional) – The meta template for the model.

class lagent.llms.base_llm.BaseModel(path, tokenizer_only=False, template_parser=LMTemplateParser, meta_template=None, *, max_tokens=512, top_p=0.8, top_k=None, temperature=0.8, repetition_penalty=1.0, stop_words=None)

Base class for model wrapper.

Parameters:

  • path (str) – The path to the model.

  • max_seq_len (int) – The maximum sequence length of the model. Defaults to 2048.

  • tokenizer_only (bool) – If True, only the tokenizer will be initialized. Defaults to False.

  • meta_template (list of dict, optional) – The model’s meta prompt template if needed, in case the requirement of injecting or wrapping of any meta instructions.

  • template_parser (LMTemplateParser) –

  • max_tokens (int) –

  • top_p (float) –

  • top_k (float) –

  • temperature (float) –

  • repetition_penalty (float) –

  • stop_words (Union[List[str], str]) –

is_api: bool = False
abstract generate(inputs, **gen_params)

Generate results given a str (or list of) inputs.

Parameters:

  • inputs (Union[str, List[str]]) –

  • gen_params (dict) – The input params for generation.

Returns:

A (list of) generated strings.

Return type:

Union[str, List[str]]

eg.

batched = True if isinstance(inputs, str):

inputs = [inputs] batched = False

response = [‘’] if batched:

return response

return response[0]

abstract stream_generate(inputs, **gen_params)

Generate results as streaming given a str inputs.

Parameters:

  • inputs (str) –

  • gen_params (dict) – The input params for generation.

Returns:

A generated string.

Return type:

str

chat(inputs, **gen_params)

Generate completion from a list of templates.

Parameters:

  • inputs (Union[List[dict], List[List[dict]]]) –

  • gen_params (dict) – The input params for generation.

Returns:

generate_from_template(inputs, **gen_params)
Parameters:

inputs (Union[List[dict], List[List[dict]]]) –

abstract stream_chat(inputs, **gen_params)

Generate results as streaming given a list of templates.

Parameters:

  • inputs (Union[List[dict]) –

  • gen_params (dict) – The input params for generation.

Returns:

abstract tokenize(prompts)

Tokenize the input prompts.

Parameters:

prompts (str | List[str]) – user’s prompt, or a batch prompts

Returns:

prompt’s token ids, ids’ length and requested output length

Return type:

Tuple(numpy.ndarray, numpy.ndarray, numpy.ndarray)

update_gen_params(**kwargs)