AI SDK - GitHub Models

The GitHub Models provider for the AI SDK gives you access to the GitHub Models catalog of large language models, designed for building modern web applications. GitHub Models has support for multiple model families, like xAI, OpenAI and Meta.

Setup

The GitHub Models provider is available in the @github/models NPM package. You can install it with:

npm i @github/models

Quick Start

To get started, you'll need a GitHub PAT (personal access token).

import {githubModels} from '@github/models'
import {generateText} from 'ai'

const result = await generateText({
  model: githubModels('meta/meta-llama-3.1-8b-instruct'),
  prompt: 'Write a haiku about programming.',
})

console.log(result.text)

Tip

GitHub Models includes free usage for all accounts. For higher usage limits, you can opt into paid usage.

If you need a customized setup, you can import createGithubModels from @github/models and create a provider instance with your settings:

import {createGithubModels} from '@github/models'

const githubModels = createGithubModels({
  org: 'my-organization',
})

Configuration Options

• apiKey string

  Your GitHub Personal Access Token or Fine-Grained Token with read:models scope. Defaults to the GITHUB_TOKEN environment variable.

• org string

  Organization to attribute API usage to (optional).

• baseURL string

  Custom API endpoint URL. Defaults to https://models.github.ai/inference.

• headers Record<string, string>

  Additional headers to include with requests.

• fetch (input: RequestInfo, init?: RequestInit) => Promise

  Custom fetch implementation. Defaults to the global fetch function. You can use it as a middleware to intercept requests, or to provide a custom fetch implementation for e.g. testing.

API Reference

The GitHub Models provider uses the GitHub Models Inference API.

License

Distributed under the MIT License. See LICENSE for more information.