FastAPI-BitNet

Overview

FastAPI-BitNet provides a robust REST API built with FastAPI to manage and interact with BitNet model instances. It allows developers to programmatically control llama-cli and llama-server processes for automated testing, benchmarking, and interactive chat sessions. The server integrates with VS Code Copilot via the Model Context Protocol, enabling seamless model interaction within development workflows.

Try asking AI

After installing, here are 5 things you can ask your AI assistant:

• create_session

  Start a new llama-cli or llama-server session

  Try:Create a new BitNet sessionTry:Start a chat session with BitNet
• stop_session

  Stop a running llama-cli or llama-server session

  Try:Stop my current BitNet sessionTry:End the active chat session
• check_session_status

  Check the status of a running session

  Try:What's the status of my BitNet session?Try:Is the chat session still running?
• chat_with_session

  Send a prompt to a running BitNet session and receive a response

  Try:Ask BitNet what the capital of France isTry:Send a message to my active chat session
• initialize_multiple_instances

  Initialize multiple BitNet instances simultaneously

  Try:Start 3 BitNet instances for testingTry:Initialize multiple chat sessions at once
• shutdown_multiple_instances

  Shut down multiple BitNet instances in a single API call

  Try:Stop all running BitNet instancesTry:Shutdown all chat sessions
• run_benchmark

  Run a benchmark test on a GGUF model

  Try:Benchmark the BitNet model performanceTry:Test the speed and quality of the model
• calculate_perplexity

  Calculate perplexity scores for a model on test data

  Try:Calculate perplexity for the BitNet modelTry:Measure model uncertainty with perplexity
• estimate_server_capacity

  Estimate maximum number of BitNet instances the server can handle

  Try:How many BitNet instances can this server run?Try:Estimate server capacity for BitNet models

Note: Tool names inferred from feature descriptions in the README, as no explicit 'Tools' section was found. The README describes functionality for session management, chat operations, and benchmarking, which were mapped to tool names.

Comparable tools

Installation

{
  "mcpServers": {
    "fastapi-bitnet": {
      "command": "http",
      "args": ["http://localhost:8080/mcp"]
    }
  }
}

FAQ

What models are supported?

Currently supports Microsoft's BitNet-b1.58-2B-4T model in GGUF format.

How do I integrate with VS Code?

Run the server and configure VS Code Copilot to use 'http://localhost:8080/mcp' as an HTTP MCP server.

Compare FastAPI-BitNet with