Add Large Language Models to Your .NET Apps with Aspose.LLM

Most .NET teams that add AI features today end up writing the same plumbing: an HTTP client, a retry policy, secret management for an API key, and a per-token billing line on someone’s credit card. Aspose.LLM takes a different route. It is a NuGet package that runs large language model inference inside your own process - no service to call, no key to rotate, and no request leaving the machine.

This article introduces the library, explains the three concepts its API is built on, and walks you through a working chat application. If you can add a package reference, you can be sending prompts in a few dozen lines.

Why a local model matters

Running the model in-process changes what you can build.

• Data stays put. Prompts never leave the host, which matters when you are working with contracts, medical notes, source code, or anything else covered by a data-residency policy.
• No per-token cost. Once the model is on disk, inference is a CPU or GPU cost you already own, not a metered API call.
• No network dependency at runtime. Desktop tools, on-premises services, and air-gapped environments all work the same way.
• One deployment artifact. The model configuration ships with your application instead of living in a separate account’s dashboard.

The trade-off is honest and worth stating up front: you provide the hardware. A model that runs locally needs local memory, and the first run needs to download several gigabytes.

Aspose.LLM for .NET in three concepts

The surface area is deliberately small. Almost everything you do with Aspose.LLM involves these three types.

A preset is the interesting one. Rather than making you wire up model paths, tokenizer settings, and sampler values by hand, a preset packages a known-good combination. Qwen25Preset is the default starting point, and you can adjust its parameters or swap in a different preset later without touching the rest of your code.

Prerequisites

Before writing any code:

1. Install the package - Install-Package Aspose.LLM
2. Place your license - put Aspose.LLM.lic somewhere the executable can read at runtime.
3. Check your memory budget - the Qwen25 preset wants at least 8 GB of free RAM.

For product details, see the Aspose.LLM for .NET product page.

Your first chat

Here is the shortest program that holds a real conversation. Note the second and third calls: the API instance tracks context on its own, so the model still knows who Alice is.

What the code does

1. Creates a License and points it at your license file.
2. Instantiates a Qwen25Preset with its default configuration.
3. Creates one AsposeLLMApi instance from that preset.
4. Sends several prompts, all sharing the same conversation.

using Aspose.LLM;
using Aspose.LLM.Abstractions.Parameters.Presets;

var license = new Aspose.LLM.License();
license.SetLicense("Aspose.LLM.lic");

var preset = new Qwen25Preset();
using var api = AsposeLLMApi.Create(preset);

string reply = await api.SendMessageAsync("What is 2 + 2?");
Console.WriteLine(reply);
// Output: 2 + 2 equals 4.

await api.SendMessageAsync("My name is Alice.");
string reply2 = await api.SendMessageAsync("What is my name?");
Console.WriteLine(reply2);
// Output: Your name is Alice.

AsposeLLMApi.Create(preset) returns a session-aware client. Each SendMessageAsync call appends the prompt to the running conversation, so you get multi-turn behavior without managing any state yourself. The using declaration matters - the instance owns native resources and needs to be disposed.

Note: These samples follow the official Aspose documentation and have not been executed in a sandbox. Verify them in your own environment before shipping.

Tuning behavior for a single request

Conversational answers benefit from some randomness. Arithmetic does not. Instead of reconfiguring the instance, build a second preset and pass it to the one call that needs it.

What the code does

1. Creates the default preset and the API instance as usual.
2. Builds strictPreset with a low sampler temperature.
3. Passes it to SendMessageAsync through the preset argument.
4. Leaves later calls on the original default.

using Aspose.LLM.Abstractions.Parameters.Presets;

// Default preset used by the API instance
var defaultPreset = new Qwen25Preset();
using var api = AsposeLLMApi.Create(defaultPreset);

// A more deterministic preset, for this question only
var strictPreset = new Qwen25Preset();
strictPreset.SamplerParameters.Temperature = 0.1f;

string reply = await api.SendMessageAsync(
    "What is the square root of 144?",
    preset: strictPreset);

Console.WriteLine(reply);
// Output: The square root of 144 is 12.

SamplerParameters.Temperature controls how much randomness the sampler introduces. Values near 0.1 produce near-deterministic output; higher values produce more varied phrasing. Because the preset is scoped to the call, the override does not leak into the rest of the conversation.

Cancellation and timeouts

Generation length is not fully predictable, and a request that runs long will block whatever is awaiting it. SendMessageAsync accepts a CancellationToken, so give it one.

What the code does

1. Creates a CancellationTokenSource with a 30-second budget.
2. Passes cts.Token into SendMessageAsync.
3. Catches OperationCanceledException and degrades gracefully.

using var cts = new CancellationTokenSource(TimeSpan.FromSeconds(30));

try
{
    string reply = await api.SendMessageAsync(
        "Write a 500-word essay about migration patterns of the Arctic tern.",
        cancellationToken: cts.Token);
    Console.WriteLine(reply);
}
catch (OperationCanceledException)
{
    Console.WriteLine("Generation timed out.");
}

The cancellation leaves session state intact, so the conversation continues normally on the next call. In an ASP.NET Core handler, forward the request’s own RequestAborted token here as well - there is no reason to keep generating for a client that has already disconnected.

A complete console application

Putting it together: a REPL that keeps one session alive for the life of the process and recovers from per-message failures.

What the code does

1. Sets up the license and default preset once, at startup.
2. Creates a single AsposeLLMApi instance for the whole program.
3. Loops on user input until the user types quit.
4. Sends each line and prints the reply.
5. Catches exceptions per message so one failure does not end the session.

using Aspose.LLM;
using Aspose.LLM.Abstractions.Parameters.Presets;

internal class SimpleChatDemo
{
    public static async Task Main()
    {
        var license = new Aspose.LLM.License();
        license.SetLicense("Aspose.LLM.lic");

        var preset = new Qwen25Preset();
        using var api = AsposeLLMApi.Create(preset);

        Console.WriteLine("Ask me anything. Type 'quit' to exit.");
        while (true)
        {
            Console.Write("> ");
            string? line = Console.ReadLine();
            if (line == null || line.Equals("quit", StringComparison.OrdinalIgnoreCase))
                break;

            try
            {
                string reply = await api.SendMessageAsync(line);
                Console.WriteLine(reply);
            }
            catch (Exception ex)
            {
                Console.WriteLine($"Error: {ex.Message}");
            }
        }
    }
}

Two design points are worth carrying into real applications. First, the instance is created once and reused - creation is expensive and, as the next section explains, concurrent instances are not allowed. Second, the try-catch sits inside the loop rather than around it, so a single bad message costs you one turn instead of the whole session.

Evaluate the library with a temporary license from the Aspose temporary license page.

Errors you are likely to hit first

That second row deserves emphasis in service scenarios. If you are wiring Aspose.LLM into a DI container, register AsposeLLMApi as a singleton and serialize access to it - a scoped or transient registration will fail as soon as two requests overlap.

Additional resources

• Aspose.LLM Documentation
• API Reference
• Free Online Apps

Conclusion

Aspose.LLM reduces “add an LLM to this .NET application” to three types and one method. A license activates the library, a preset defines the model and how it generates, and an AsposeLLMApi instance carries the conversation. From there, SendMessageAsync handles a single question, a multi-turn session, a per-call parameter override, and a cancellable long-running generation.

Start with the console sample, confirm the model behaves the way your feature needs, then move on to multi-session and persistence work once the fundamentals are solid.

FAQs

1. What is Aspose.LLM for .NET?

A NuGet library that runs large language model inference inside your own .NET process. You create a preset, create an API instance, and send prompts - no external service, no API key.

2. Do I need an internet connection to use it?

Only for the first run. The initial Create call downloads the native engine and the model files defined by the preset, and the assets are cached locally afterwards.

3. What are the prerequisites for the code samples?

The Aspose.LLM NuGet package, a valid Aspose.LLM.lic file, and enough free RAM for your chosen preset - plan on at least 8 GB for Qwen25.

4. How do I change the model’s behavior for a single request?

Build a second preset, adjust its parameters (such as SamplerParameters.Temperature), and pass it to SendMessageAsync via the preset argument. Later calls revert to the instance’s default preset.

5. How do I stop a generation that is taking too long?

Pass a CancellationToken to SendMessageAsync and catch OperationCanceledException. Session state survives the cancellation, so the conversation continues normally.

6. Why is the first response so slow?

The first Create call downloads native binaries and model weights, which can take 5-15 minutes on a clean machine. Warm the instance up at application startup rather than on a user’s first request.

Read More

• Introducing AI-Powered Search in Aspose.Words for .NET Documentation
• Use Self-hosted LLM Implementations with Aspose.Words for .NET AI Functionality
• Build a Simple AI Chat in C# with Aspose.LLM