Read this

Leitfaden zum Prompt-Engineering

Prompting Techniques and Best Practices for Journalists

Prompting Fundamentals and How to Apply them Effectively

Beginner’s prompt handbook: ChatGPT for local news publishers

More technical

Llama 2 is here – get it on Hugging Face: How to prompt llama

A Developer’s Guide to Simplifying Environment Management

As developers, managing virtual environments is a crucial part of our workflow. With Python projects constantly shifting between dependencies and Python versions, using tools that streamline this process is key. Enter uv: a tool designed to simplify the creation, activation, and management of virtual environments and to manage python packages and projects.

In this post, I’ll introduce you to uv, walk you through its installation, and provide some tips to help you get started.

What is uv?

uv is an extremely fast Python package and project manager, written in Rust. It is a powerful tool that allows developers to manage Python virtual environments effortlessly. It provides functionality to create, activate, and switch between virtual environments in a standardized way.

By using uv, you can ensure that your virtual environments are consistently created and activated across different projects without the need to manually deal with multiple commands.

Why Use uv?

Managing Python projects often involves juggling various dependencies, versions, and configurations. Without proper tooling, this can become a headache. uv helps by:

• Standardizing virtual environments across projects, ensuring consistency.
• Simplifying project setup, requiring fewer manual steps to get your environment ready.
• Minimizing errors by automating activation and management of virtual environments.

Hint

In our examples, before each command you will see our shell prompt:

❯ 

Don’t type the ❯ when you enter the command. So, when seeing

❯  uv init

just type

uv init

In addition, when we activate the virtual environment, you will see a changed prompt:

✦ ❯ 

Installation and Setup

Getting started with uv is easy. Below are the steps for installing and setting up uv for your Python projects.

1. Install uv

With MacOS or Linux, you can install uv from the website:

❯ curl -LsSf https://astral.sh/uv/install.sh | sh

Alternatively, you can install uv using pip. You’ll need to have Python 3.8+ installed on your system.

❯ pip install uv

2. Create a New Virtual Environment

Once installed, you can use uv to create a virtual environment for your project. Simply navigate to your project directory and run:

❯ uv new

This command will create a new virtual environment inside the .venv folder within your project.

3. Activate the Virtual Environment

After creating the virtual environment, you can easily activate it using the following command:

uv activate

No need to worry about different activation scripts for Windows, Linux, or macOS. uv handles that for you.

4. Install Your Dependencies

Once the environment is active, you can install your project’s dependencies as you normally would:

❯ pip install -r requirements.txt

uv ensures that your dependencies are installed in the correct environment without any extra hassle.

You can also switch to a pyproject.toml file to manage your dependencies.

First you have to initialize the project:

❯ uv init

Then, add the dependency:

❯ uv add requests

Tips with virtual environments

When you create a virtual environment, the corresponding folder should be in your PATH.

Normally this is .venv/bin, when you create it with uv init. This path is added to your $PATH variable when you run uv activate.

But, if you want to choose a different folder, you must set the variable UV_PROJECT_ENVIRONMENT to this path:

❯ mkdir playground
❯ cd playground
❯ /usr/local/bin/python3.12 -m venv .venv/python/3.12
❯ . .venv/python/3.12/bin/activate

✦ ❯ which python
.../Playground/.venv/python/3.12/bin/python

✦ ❯ export UV_PROJECT_ENVIRONMENT=$PWD/.venv/python/3.12

✦ ❯ pip install uv
Collecting uv
  Downloading uv-0.4.25-py3-none-macosx_10_12_x86_64.whl.metadata (11 kB)
Downloading uv-0.4.25-py3-none-macosx_10_12_x86_64.whl (13.2 MB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 13.2/13.2 MB 16.5 MB/s eta 0:00:00
Installing collected packages: uv
Successfully installed uv-0.4.25

✦ ❯ which uv
.../Playground/.venv/python/3.12/bin/uv

✦ ❯ uv init
Initialized project `playground`

So, with the default settings, you will get an error because uv is searching the virtual environment in .venv.

✦ ❯ uv add requests
warning: `VIRTUAL_ENV=.venv/python/3.12` does not match the project environment path `.../.venv/python/3.12` and will be ignored

Use the environment variable to tell uv where the virtual environment is installed.

✦ ❯ export UV_PROJECT_ENVIRONMENT=$PWD/.venv/python/3.12

✦ ❯ uv add requests
Resolved 6 packages in 0.42ms
Installed 5 packages in 8ms
 + certifi==2024.8.30
 + charset-normalizer==3.4.0
 + idna==3.10
 + requests==2.32.3
 + urllib3==2.2.3

Tip

Use direnv to automatically set your environment:

• Install direnv: https://direnv.net/docs/installation.html

• Set .envrc file:

✦ ❯ . .venv/python/3.12/bin/activate
✦ ❯ export UV_PROJECT_ENVIRONMENT=$PWD/.venv/python/3.12

• Allow the .envrc file:

✦ ❯ direnv allow

Common uv Commands

Here are a few more useful uv commands to keep in mind:

• Deactivate the environment: uv deactivate
• Remove the environment: uv remove
• List available virtual environments in your project: uv list

Tips for Using uv Effectively

1. Consistent Environment Names: By default, uv uses .venv as the folder name for virtual environments. Stick to this default to keep things consistent across your projects.
2. Integrate uv into your CI/CD pipeline: Ensure that your automated build tools use the same virtual environment setup by adding uv commands to your pipeline scripts.
3. Use uv in combination with pyproject.toml: If your project uses pyproject.toml for dependency management, uv can seamlessly integrate, ensuring your environment is always up to date.
4. Quick Switching: If you manage multiple Python projects, uv‘s environment activation and deactivation commands make it easy to switch between projects without worrying about which virtual environment is currently active.
5. Automate Activation: Combine uv with direnv or add an activation hook in your shell to automatically activate the correct environment when you enter a project folder.

Cheatsheet

uv Command Cheatsheet

General Commands

Working with Dependencies

Environment Management

Cleanup and Miscellaneous

Using Python and Pip Inside Virtual Environment

Helper Commands

More to read

Here is a shot list of websites with documentation or other information about uv:

• https://docs.astral.sh/uv

• https://github.com/fpgmaas/cookiecutter-uv
• https://github.com/a5chin/python-uv

Responsive Design

React on Size Change

<script>
export default {
    data() {
        return {
            isMobile: false,
            isDesktop: false,

            windowWidth: window.innerWidth,
            windowHeight: window.innerHeight,
        };
    },

    created() {
        this.updateWindowSize();
        window.addEventListener('resize', this.updateWindowSize);
    },

    methods: {
        updateWindowSize() {
            // console.log("updateWindowSize())");

            this.windowWidth = window.innerWidth;
            this.windowHeight = window.innerHeight;
            this.checkIsMobile();
        },

        checkIsMobile() {
            this.isMobile = this.windowWidth <= 768;
            // console.log(`checkIsMobile(): windowWidth = ${this.windowWidth} isMobile=${this.isMobile}`)
        },

        beforeUnmount() {
            console.log("beforeUnmount()");

            window.removeEventListener('resize', this.updateWindowSize);
        },

    },
};
</script>

Debugging

<script setup>
import {
    onActivated,
    onBeforeMount,
    onBeforeUnmount,
    onBeforeUpdate,
    /*  onCreated, */
    onDeactivated,
    onErrorCaptured,
    onMounted,
    /*  onRenderTracked,*/
    onRenderTriggered,
    onScopeDispose,
    onServerPrefetch,
    onUnmounted,
    onUpdated,
    /*  onWatcherCleanup, */

} from 'vue';

onActivated(() => { console.log('onActivated() called'); });
onBeforeMount(() => { console.log(`onBeforeMount():`) })
onBeforeUnmount(() => { console.log('onBeforeUnmount() called'); });
onBeforeUpdate(() => { console.log(`onBeforeUpdate():`) })
onDeactivated(() => { console.log('onDeactivated() called'); });
onErrorCaptured((err, instance, info) => { console.log('onErrorCaptured() called'); console.error(err); return false; });
onMounted(() => { console.log(`onMounted():`) })
onRenderTriggered((e) => { console.log('onRenderTriggered() called', e); });
onUnmounted(() => { console.log(`onUnmounted():`) })
onUpdated(() => { console.log('onUpdated() called'); });
onScopeDispose(() => { console.log('onScopeDispose() called'); });
onServerPrefetch(() => { console.log('onServerPrefetch() called'); });

</script>

Routing

Alle Routen anzeigen

php artisan route:list

Routen dynamisch erzeugen

composer require illuminate/support

use Illuminate\Support\Facades\File;

function generateRoutes($basePath, $baseNamespace = 'Pages', $routePrefix = '/')
{
    $files = File::allFiles($basePath);

    foreach ($files as $file) {
        $relativePath = str_replace([$basePath, '.vue'], '', $file->getRelativePathname());
        $routeName = str_replace(DIRECTORY_SEPARATOR, '.', $relativePath);
        $routeUri = str_replace(DIRECTORY_SEPARATOR, '/', $relativePath);

        // Example: if file is `resources/js/Pages/Examples/layout-discord.vue`
        // $routeName = 'Examples.layout-discord';
        // $routeUri = 'examples/layout-discord'

        Route::get($routePrefix . $routeUri, function () use ($relativePath, $baseNamespace) {
            return Inertia::render($baseNamespace . str_replace('/', '\\', $relativePath));
        })->name($routeName);
    }
}

generateRoutes(resource_path('js/Pages'));

Mail / SMTP

Lokaler Mailserver für SMTP Testing

MailHog: Web and API based SMTP testing

Allgemeines

Vue and CSS

Styling with CSS Variables

<script setup>
const theme = {
    "menu": {
        "background": 'black',
        "item": {
            "background": "green"
        },
        "subitem": {
            "background": "green"
        }
    }
}
</script>

<style scoped>
.menu {
    background-color: v-bind('theme.menu.background');
}

Using PrimeVue

Installation

❯ pnpm add primevue @primevue/themes
❯ pnpm add primevue @primevue/icons

Allgemeines

Installation

Installation mit der vue-cli

yarn global add @vue/cli
# OR
npm install -g @vue/cli

Neues Projekt erstellen

vue create my-project
# OR
vue ui

Installation mit Vite

Vite

Vite ist ein Build-Tool für die Webentwicklung, das aufgrund seines nativen ES-Modul-Importansatzes eine blitzschnelle Bereitstellung von Code ermöglicht.

Installation mit npm:

npm init @vitejs/app <project-name>
cd <project-name>
npm install
npm run dev

Oder mit Yarn

$ yarn create @vitejs/app <project-name>
$ cd <project-name>
$ yarn
$ yarn dev

Wenn der Projektname Leerzeichen ehthält, kann es zu Fehlern kommen. Dann hilft das nachfolgende Kommando

$ create-vite-app <project-name>

Vue Frameworks

Description:

Setting up VS Code Environment

Debugging

Create a file launch.json and add this to configuration

"configurations": [
        {
            "name": "Streamlit",
            "type": "debugpy",
            "request": "launch",
            "module": "streamlit",
            "args": [
                "run",
                "${file}",
                "--server.port",
                "2000"
            ]
        }
    ]

Ollama und Python

Beispiele

• Ollama Python Library

• Ollama JavaScript Library

Getting Started

_Python

pip install ollama

import ollama

response = ollama.chat(model='llama2', messages=[
  {
    'role': 'user',
    'content': 'Why is the sky blue?',
  },
])

print(response['message']['content'])

_JavaScript

npm install ollama

import ollama from 'ollama'

const response = await ollama.chat({
  model: 'llama2',
  messages: [{ role: 'user', content: 'Why is the sky blue?' }],
})
console.log(response.message.content)

Use cases

Both libraries support Ollama’s full set of features. Here are some examples in Python:

Streaming

for chunk in chat('mistral', messages=messages, stream=True):
    print(chunk['message']['content'], end='', flush=True)

Multi-modal

with open('image.png', 'rb') as file:
  response = ollama.chat(
    model='llava',
    messages=[
      {
        'role': 'user',
        'content': 'What is strange about this image?',
        'images': [file.read()],
      },
    ],
  )

print(response['message']['content'])

Text Completion

result = ollama.generate(
  model='stable-code',
  prompt='// A c function to reverse a string\n',
)

print(result['response'])

Creating custom models

modelfile='''
FROM llama2
SYSTEM You are mario from super mario bros.
'''

ollama.create(model='example', modelfile=modelfile)

Custom client

ollama = Client(host='my.ollama.host')

More examples are available in the GitHub repositories for the Python and JavaScript libraries.

Tipps und Tricks

ollama serve

OLLAMA_ORIGINS=https://webml-demo.vercel.app
OLLAMA_HOST=127.0.0.1:11435 ollama serve

Docker

Run Ollama in Docker container

CPU only

docker run -d -v ollama:/root/.ollama -p 11434:11434 --name ollama ollama/ollama

Nvidia GPU

Install the Nvidia container toolkit.
Run Ollama inside a Docker container

docker run -d --gpus=all -v ollama:/root/.ollama -p 11434:11434 --name ollama ollama/ollama

Run a model

Now you can run a model like Llama 2 inside the container.

docker exec -it ollama ollama run llama2

OpenAI

OpenAI Compatibility

from openai import OpenAI

client = OpenAI(
    base_url = 'http://localhost:11434/v1',
    api_key='ollama', # required, but unused
)

response = client.chat.completions.create(
  model="llama2",
  messages=[
    {"role": "system", "content": "You are a helpful assistant."},
    {"role": "user", "content": "Who won the world series in 2020?"},
    {"role": "assistant", "content": "The LA Dodgers won in 2020."},
    {"role": "user", "content": "Where was it played?"}
  ]
)

print(response.choices[0].message.content)

Using Streamlit

LangChain

from langchain_community.llms import Ollama
llm = Ollama(model="gemma2")
llm.invoke("Why is the sky blue?")

LlamaIndex

from llama_index.llms.ollama import Ollama

llm = Ollama(model="gemma2")
llm.complete("Why is the sky blue?")

LLMs and Models by Example

wizard-math

ollama run wizard-math 'Expand the following expression: $7(3y+2)$'

Introduction

Large Language Models (LLMs) have revolutionized the field of Natural Language Processing (NLP) by providing powerful capabilities for understanding and generating human language. Open-source LLMs have democratized access to these technologies, allowing developers and researchers to innovate and apply these models in various domains. In this blog post, we will explore Ollama, a framework for working with LLMs, and demonstrate how to load webpages, parse them, build embeddings, and query the content using Ollama.

Understanding Large Language Models (LLMs)

LLMs are neural networks trained on vast amounts of text data to understand and generate human language. They can perform tasks such as translation, summarization, question answering, and more. Popular LLMs include GPT-3, BERT, and their open-source counterparts like GPT-Neo and BERT variants. These models have diverse applications, from chatbots to automated content generation.

Introducing Ollama

Ollama is an open-source framework designed to simplify the use of LLMs in various applications. It provides tools for training, fine-tuning, and deploying LLMs, making it easier to integrate these powerful models into your projects. With Ollama, you can leverage the capabilities of LLMs to build intelligent applications that understand and generate human language.

Example

The following example from the ollama documentation demonstrates how to use the LangChain framework in conjunction with the Ollama library to load a web page, process its content, create embeddings, and perform a query on the processed data. Below is a detailed explanation of the script’s functionality and the technologies used.

Technologies Used

1. LangChain: A framework for building applications powered by large language models (LLMs). It provides tools for loading documents, splitting text, creating embeddings, and querying data.
2. Ollama: A library for working with LLMs and embeddings. In this script, it’s used to generate embeddings for text data.
3. BeautifulSoup (bs4): A library used for parsing HTML and XML documents. It’s essential for loading and processing web content.
4. ChromaDB: A vector database used for storing and querying embeddings. It allows efficient similarity searches.

Code Breakdown

Imports and Setup

The script starts by importing the necessary modules and libraries, including sys, Ollama, WebBaseLoader, RecursiveCharacterTextSplitter, OllamaEmbeddings, Chroma, and RetrievalQA.

from langchain_community.llms import Ollama

from langchain_community.document_loaders import WebBaseLoader
from langchain.text_splitter import RecursiveCharacterTextSplitter
from langchain_community.embeddings import OllamaEmbeddings
from langchain_community.vectorstores import Chroma
from langchain.chains import RetrievalQA

Loading the Web Page

The script uses WebBaseLoader to load the content of a webpage. In this case, it loads the text of “The Odyssey” by Homer from Project Gutenberg.

print("- get web page")

loader = WebBaseLoader("https://www.gutenberg.org/files/1727/1727-h/1727-h.htm")
data = loader.load()

Splitting the Document

Due to the large size of the document, it is split into smaller chunks using RecursiveCharacterTextSplitter. This ensures that the text can be processed more efficiently.

print("- split documents")

text_splitter=RecursiveCharacterTextSplitter(chunk_size=500, chunk_overlap=0)
all_splits = text_splitter.split_documents(data)

Creating Embeddings and Storing Them

The script creates embeddings for the text chunks using the Ollama library and stores them in ChromaDB, a vector database. This step involves instantiating an embedding model (nomic-embed-text) and using it to generate embeddings for each text chunk.

print("- create vectorstore")

oembed = OllamaEmbeddings(base_url="http://localhost:11434", model="nomic-embed-text")
vectorstore = Chroma.from_documents(documents=all_splits, embedding=oembed)

Performing a Similarity Search

A question is formulated, and the script uses the vector database to perform a similarity search. It retrieves chunks of text that are semantically similar to the question.

print("- ask for similarities")

question="Who is Neleus and who is in Neleus' family?"
docs = vectorstore.similarity_search(question)
nrofdocs=len(docs)
print(f"{question}: {nrofdocs}")

Creating an Ollama Instance and Defining a Retrieval Chain

The script initializes an instance of the Ollama model and sets up a retrieval-based question-answering (QA) chain. This chain is used to process the question and retrieve the relevant parts of the document.

print("- create ollama instance")
ollama = Ollama(
    base_url='http://localhost:11434',
    model="llama3"
)

print("- get qachain")
qachain=RetrievalQA.from_chain_type(ollama, retriever=vectorstore.as_retriever())

Running the Query

Finally, the script invokes the QA chain with the question and prints the result.

print("- run query")
res = qachain.invoke({"query": question})

print(res['result'])

Result

Now lets look at the impresiv result:

Try another example: ask wikipedia page

In this example, we are going to use LangChain and Ollama to learn about something just a touch more recent. In August 2023, there was a series of wildfires on Maui. There is no way an LLM trained before that time can know about this, since their training data would not include anything as recent as that.

So we can find the Wikipedia article about the fires and ask questions about the contents.

url = "https://en.wikipedia.org/wiki/2023_Hawaii_wildfires"

question="When was Hawaii's request for a major disaster declaration approved?"

General

With Open Source Toools, it is easy to analyse images.

Just install Ollama, download the llava image and run this command:

❯ ollama run llava:latest "Beschreibe das Bild <path to image>"

Try this image: Statue of LIberty

❯ ollama run llava:latest "Beschreibe das Bild /tmp/statue-liberty-liberty-island-new-york.jpg"

Added image '/tmp/statue-liberty-liberty-island-new-york.jpg'
The image shows the Statue of Liberty, an iconic landmark in New York Harbor. This neoclassical statue is a symbol of freedom and democracy, and it has become a universal symbol of the United States. The statue is situated on Liberty Island, which is accessible via ferries from Manhattan.

In the background, you can see a clear sky with some clouds, indicating good weather. The surrounding area appears to be lush with greenery, suggesting that the photo was taken in spring or summer when vegetation is abundant. There are also people visible at the base of the statue, which gives a sense of scale and demonstrates the size of the monument.

Introduction

The GitHub Copilot extension is an AI pair programmer tool that helps you write code faster and smarter. 

We want to use this feature with Open Source Tools:

• Ollama
• VS Code Extension Continue
• Phi3 Model

Setup

Install Ollama

Download Ollama and install it.

To start Ollama, you have two possibilities:

From the command line

Using the Icon from the Installation

With MacOS, you could start Ollama

You should the the runnning Ollama instance in the header

Pull Phi3 Model

Run

ollama pull phi3

Install VS Code

Install VS Code Extension Continue

Start Model

Configure VS Code Extension