Skip to main content

Explore and test local modals using Ollama and OpenWebUI

If you are following my blog you probably noticed that I'm experimenting a lot with Large Language models locally. I typically expose these LLM's locally through Ollama and use either Semantic Kernel or the API directly to test and interact with these models.

Recently I discovered OpenWebUI, an open source web interface designed primarily for interacting with AI language models. It offers a clean, intuitive interface that makes it easy to have conversations with AI models while providing advanced features for developers and power users.

Some of the key features of OpenWebUI are:

  • OpenAI API Integration: Effortlessly integrate OpenAI-compatible APIs for versatile conversations alongside Ollama models.

  • Granular Permissions and User Groups: Create detailed user roles and permissions for a secure and customized user environment.

  • Full Markdown and LaTeX Support: Comprehensive Markdown and LaTeX capabilities for enriched interaction.

  • Model Builder: Easily create Ollama models directly from Open WebUI.

  • Local and Remote RAG Integration: Cutting-edge Retrieval Augmented Generation (RAG) technology within chats.

  • Web Search for RAG: Perform web searches and inject results directly into your chat experience.

  • Web Browsing Capabilities: Seamlessly integrate websites into your chat experience.

  • Image Generation Integration: Incorporate image generation capabilities using various APIs.

  • Concurrent Model Utilization: Engage with multiple models simultaneously for optimal responses.

  • Role-Based Access Control (RBAC): Ensure secure access with restricted permissions..

  • Pipelines Plugin Framework: Integrate custom logic and Python libraries into Open WebUI.

We’ll explore some of these features in later post, but today we focus on how to get started.

Getting started

The easiest way to get started with OpenWebUI is through Docker. So let’s run the following command to run OpenWebUI:

docker run -d -p 3000:8080 --add-host=host.docker.internal:host-gateway -v open-webui:/app/backend/data --name open-webui --restart always ghcr.io/open-webui/open-webui:cuda

Remark: There are multiple parameters that can be provided. Check out the readme file for more details.

After the container has started, you can browse to http://localhost:3000. The first time you run the container, you get a welcome page:

After clicking on Get started, you need to create an admin account:

Now we arrive on the main OpenWebUI screen. If you used ChatGPT or Microsoft Copilot before, the experience feels familiar.

You can select a different model by clicking on the dropdown in the left corner:

Now we can send a message to the model and wait for some results:

The output is shown in the chat interface:

Configure OpenWebUI

There are a lot of things we can configure when using OpenWebUI. To access the admin settings, click on the user icon in the top right corner and choose Admin Panel:

On the Admin panel, you can further click on the Settings tab:

Now we get an overview of all available settings. For example in the connections section, we can see and manage the list of available connections:

By clicking on the configure icon next to a specific connection, we can configure it further:

 

Conclusion

OpenWebUI makes it really easy to test AI models while giving users and organizations the control they need. Whether you're an individual developer, researcher, or enterprise user, I would recommend giving it a try.

More information

open-webui/open-webui: User-friendly AI Interface (Supports Ollama, OpenAI API, ...)

Home | Open WebUI

Labels:

Popular posts from this blog

Podman– Command execution failed with exit code 125

After updating WSL on one of the developer machines, Podman failed to work. When we took a look through Podman Desktop, we noticed that Podman had stopped running and returned the following error message: Error: Command execution failed with exit code 125 Here are the steps we tried to fix the issue: We started by running podman info to get some extra details on what could be wrong: >podman info OS: windows/amd64 provider: wsl version: 5.3.1 Cannot connect to Podman. Please verify your connection to the Linux system using `podman system connection list`, or try `podman machine init` and `podman machine start` to manage a new Linux VM Error: unable to connect to Podman socket: failed to connect: dial tcp 127.0.0.1:2655: connectex: No connection could be made because the target machine actively refused it. That makes sense as the podman VM was not running. Let’s check the VM: >podman machine list NAME         ...
Read more

Azure DevOps/ GitHub emoji

I’m really bad at remembering emoji’s. So here is cheat sheet with all emoji’s that can be used in tools that support the github emoji markdown markup: All credits go to rcaviers who created this list.
Read more

VS Code Planning mode

After the introduction of Plan mode in Visual Studio , it now also found its way into VS Code. Planning mode, or as I like to call it 'Hannibal mode', extends GitHub Copilot's Agent Mode capabilities to handle larger, multi-step coding tasks with a structured approach. Instead of jumping straight into code generation, Planning mode creates a detailed execution plan. If you want more details, have a look at my previous post . Putting plan mode into action VS Code takes a different approach compared to Visual Studio when using plan mode. Instead of a configuration setting that you can activate but have limited control over, planning is available as a separate chat mode/agent: I like this approach better than how Visual Studio does it as you have explicit control when plan mode is activated. Instead of immediately diving into execution, the plan agent creates a plan and asks some follow up questions: You can further edit the plan by clicking on ‘Open in Editor’: ...
Read more