TopGit tracks dzhng/llm-api on GitHub. The project has 171 stars. Fully typed & consistent chat APIs for OpenAI, Anthropic, Groq, and Azure's chat models for browser, edge, and node environments.
Snapshot summary built from the project's own GitHub metadata — there's no written TopGit review yet. The page will update automatically when a full review is published.
WHY NO REVIEW YET
TopGit writes full reviews for the most-starred, most-requested repositories. This page is a snapshot until then — see the READ ME tab for the original README in full.
options
You can override the default request options via this parameter. A request will automatically be retried if there is a ratelimit or server error.
type ModelRequestOptions = {
// set to automatically add system message (only relevant when using textCompletion)
systemMessage?: string | (() => string);
// send a prefix to the model response so the model can continue generating from there, useful for steering the model towards certain output structures.
// the response prefix WILL be appended to the model response.
// for Anthropic's models ONLY
responsePrefix?: string;
// function related parameters are for OpenAI's models ONLY
functions?: ModelFunction[];
// force the model to call the following function
callFunction?: string;
// default: 3
retries?: number;
// default: 30s
retryInterval?: number;
// default: 60s
timeout?: number;
// the minimum amount of tokens to allocate for the response. if the request is predicted to not have enough tokens, it will automatically throw a 'TokenError' without sending the request
// default: 200
minimumResponseTokens?: number;
// the maximum amount of tokens to use for response
// NOTE: in OpenAI models, setting this option also requires contextSize in ModelConfig to be set
maximumResponseTokens?: number;
};
Response
Completion responses are in the following format:
interface ModelResponse {
content?: string;
// used to parse function responses
name?: string;
arguments?: JsonValue;
usage?: {
promptTokens: number;
completionTokens: number;
totalTokens: number;
};
// function to send another message in the same 'chat', this will automatically append a new message to the messages array
respond: (
message: ChatCompletionRequestMessage,
opt?: ModelRequestOptions,
) => Promise<ModelResponse>;
}
📃 Token Errors
A common error with LLM APIs is token usage - you are only allowed to fit a certain amount of data in the context window.
If you set a contextSize key, llm-api will automatically determine if the request will breach the token limit BEFORE sending the actual request to the model provider (e.g. OpenAI). This will save one network round-trip call and let you handle these type of errors in a responsive manner.
const openai = new OpenAIChatApi(
{ apiKey: 'YOUR_OPENAI_KEY' },
{ model: 'gpt-4-0613', contextSize: 8129 },
);
try {
const res = await openai.textCompletion(...);
} catch (e) {
if (e instanceof TokenError) {
// handle token errors...
}
}
🔷 Azure
llm-api also comes with support for Azure's OpenAI models. The Azure version is usually much faster and more reliable than OpenAI's own API endpoints. In order to use the Azure endpoints, you must include 2 Azure specific options when initializing the OpenAI model, azureDeployment and azureEndpoint. The apiKey field will also now be used for the Azure API key.
You can find the Azure API key and endpoint in the Azure Portal. The Azure Deployment must be created under the Azure AI Portal.
Note that the model parameter in ModelConfig will be ignored when using Azure. This is because in the Azure system, the model is selected on deployment creation, not on run time.
const openai = new OpenAIChatApi({
apiKey: 'AZURE_OPENAI_KEY',
azureDeployment: 'AZURE_DEPLOYMENT_NAME',
azureEndpoint: 'AZURE_ENDPOINT',
// optional, defaults to 2023-06-01-preview
azureApiVersion: 'YYYY-MM-DD',
});
🔶 Anthropic
Anthropic's models have the unique advantage of a large 100k context window and extremely fast performance. If no explicit model is specified, llm-api will default to the Claude Sonnet model.
const anthropic = new AnthropicChatApi(params: AnthropicConfig, config: ModelConfig);
🔶 Groq
Groq is a new LLM inference provider that provides the fastest inference speed on the market. They currently support Meta's Llama 2 and Mistral's Mixtral models.
const groq = new GroqChatApi(params: GroqConfig, config: ModelConfig);
The most recent commit recorded on dzhng/llm-api was 2.5 years ago, based on the GitHub push timestamp. The repository has 11 forks — one of the better signals of community interest.
How many stars does dzhng/llm-api have?
dzhng/llm-api has 171 GitHub stars — refresh the page for the live number, or check github.com/dzhng/llm-api. TopGit mirrors GitHub's count but does not claim minute-by-minute accuracy.
Is dzhng/llm-api open source?
Yes — dzhng/llm-api ships under the MIT license, which makes its source code freely readable (and, depending on license terms, forkable and reusable). Source: github.com/dzhng/llm-api.
What is dzhng/llm-api?
dzhng/llm-api (dzhng/llm-api) is a TypeScript project on GitHub. From the project's own README: Fully typed & consistent chat APIs for OpenAI, Anthropic, Groq, and Azure's chat models for browser, edge, and node environments.
Where do I read more about dzhng/llm-api?
This TopGit page is a snapshot — the READ ME tab shows the project's own README content (links stripped, images preserved). The GitHub repository at github.com/dzhng/llm-api is the definitive source.
Read full README in the tab above.
Want a second opinion on llm-api?
Ask an AI that can read this page — one click and you get its take on llm-api.