codenib.llm

LLM module for CodeNib.

Modules:

Name	Description
diagnostics	Static diagnostics for LiteLLM-backed product configuration.
litellm_chat	Lightweight LLM chat wrapper around LiteLLM.
options	Validated LiteLLM request options shared by CLI and web runtimes.
usage	Token usage and cost tracking for LLM calls.

Classes:

Name	Description
ChatMessage	A single chat message with role and content.
LiteLLMChat	Thin wrapper around litellm.completion.
RetryConfig	Exponential-backoff retry policy for transient LLM errors.

ChatMessage dataclass

ChatMessage(role: str, content: str)

A single chat message with role and content.

LiteLLMChat dataclass

LiteLLMChat(
    model: str,
    temperature: float | None = 0.0,
    max_tokens: int | None = 8192,
    api_key: str | None = None,
    api_base: str | None = None,
    extra_kwargs: dict[str, Any] = dict(),
    retry: RetryConfig = RetryConfig(),
)

Thin wrapper around litellm.completion.

Provides .invoke() and .with_structured_output() matching the interface that callers already use via LangChain chat models.

Methods:

Name	Description
invoke	Send messages and return the assistant content string.
complete	Complete raw LiteLLM message dictionaries and return text content.
with_structured_output	Return a callable that parses LLM responses into schema.

Attributes:

Name	Type	Description
cache_identity	str	Stable non-secret identity for model-dependent artifact caches.

cache_identity property

cache_identity: str

Stable non-secret identity for model-dependent artifact caches.

invoke

invoke(messages: list[ChatMessage]) -> str

Send messages and return the assistant content string.

Source code in codenib/llm/litellm_chat.py

def invoke(self, messages: List[ChatMessage]) -> str:
    """Send messages and return the assistant content string."""
    response = self._call(messages)
    return response.choices[0].message.content

complete

complete(messages: list[dict[str, Any]], **overrides: Any) -> str

Complete raw LiteLLM message dictionaries and return text content.

Source code in codenib/llm/litellm_chat.py

def complete(
    self,
    messages: List[Dict[str, Any]],
    **overrides: Any,
) -> str:
    """Complete raw LiteLLM message dictionaries and return text content."""

    response = self._call_raw(messages, **overrides)
    return (response.choices[0].message.content or "").strip()

with_structured_output

with_structured_output(schema: Type[BaseModel]) -> _StructuredLLM

Return a callable that parses LLM responses into schema.

Source code in codenib/llm/litellm_chat.py

def with_structured_output(self, schema: Type[BaseModel]) -> _StructuredLLM:
    """Return a callable that parses LLM responses into *schema*."""
    return _StructuredLLM(chat=self, schema=schema)

RetryConfig dataclass

RetryConfig(
    max_retries: int = 2,
    base_delay: float = 0.5,
    max_delay: float = 8.0,
    jitter: float = 0.1,
)

Exponential-backoff retry policy for transient LLM errors.

max_retries is the number of additional attempts after the first, so the call is made at most max_retries + 1 times. base_delay is the first backoff in seconds; each subsequent wait is base_delay * 2**attempt capped at max_delay. A small random jitter spreads retries out so concurrent callers don't synchronize.

Methods:

Name	Description
backoff	Seconds to sleep before retry attempt (0-based).

backoff

backoff(attempt: int) -> float

Seconds to sleep before retry attempt (0-based).

Source code in codenib/llm/litellm_chat.py

def backoff(self, attempt: int) -> float:
    """Seconds to sleep before retry ``attempt`` (0-based)."""
    delay = min(self.base_delay * (2**attempt), self.max_delay)
    if self.jitter:
        delay += random.uniform(0, self.jitter * delay)
    return delay