Skip to content

Python

Python Repo Templating

We have a template for creating Python repositories using the cookiecutter package.

dt-template-python-repo

This template provides a standardized structure for Python projects, including:

  • Project structure and organization
  • Configuration files
  • CI/CD pipelines

Usage

Detailed usage instructions are included in the README file of the template repository.

Best Practices

Code Style

  • Minimal, DRY code - Avoid duplication and keep code concise
  • Self-documenting names - Prefer clear variable/function names over explanatory comments
  • No unnecessary docstrings - Only add docstrings for public APIs that need documentation

Python Preferences

  • Type hints - Always use type annotations
  • Comprehensions - Prefer list/dict comprehensions over loops when readable
  • f-strings - Use f-strings for string formatting
  • Early returns - Favor early returns to reduce nesting
  • pathlib - Use pathlib over os.path
# Good
def get_user_path(user_id: str) -> Path:
    if not user_id:
        return None
    return Path(f"/users/{user_id}")

# Avoid
def get_user_path(user_id):
    """Get the path for a user."""  # unnecessary docstring
    if user_id:
        path = "/users/" + user_id  # use f-strings instead
        return path
    else:
        return None

uv

Project Structure

my-project/
├── pyproject.toml       # Project configuration and dependencies
├── uv.lock              # Locked dependencies (commit this)
├── src/
│   └── my_project/      # Main package
│       ├── __init__.py
│       └── main.py
└── src/tests/           # Tests directory
    └── test_main.py

Running Code

Always run Python scripts through uv run to ensure reproducible execution with locked dependencies:

# Run scripts
uv run script.py

# Run any command in the uv environment
uv run pytest
Danger

Never use python directly - always prefix with uv run.

Update Dependencies

  1. Add/modify dependencies → Edit pyproject.toml
  2. Update lock file → uv lock --upgrade
  3. Sync environment → uv sync

Linting & Type Checking

We use ruff for formatting and linting, and pyrefly for type checking.

Development Workflow

After modifying any Python code, always run:

uv run ruff format .      # Format code
uv run ruff check --fix . # Lint and auto-fix
uv run pyrefly check      # Type check
uv run pytest src/tests/  # Run tests

Automation tip

Add these commands to a Makefile or pre-commit hooks to automate the workflow.

ruff Configuration

Add a ruff.toml file to your project root:

[lint]
select = ["ALL"]
preview = false
ignore = [
    "D100", # Missing docstring in public module
    "D101", # Missing docstring in public class
    "D102", # Missing docstring in public method
    "D103", # Missing docstring in public function
    "D104", # Missing docstring in public package
    "D107", # Missing docstring in __init__
    "COM812", # Trailing commas (handled by formatter)
    "G004", # Logging statement uses f-string
    "PLE1205", # Too many arguments for logging (false positive with loguru)
]
dummy-variable-rgx = "^(_+|(_+[a-zA-Z0-9_]*[a-zA-Z0-9]+?))$"

[lint.per-file-ignores]
"src/tests/**" = ["S101"]  # Allow assert in tests

[lint.pydocstyle]
convention = "google"

Testing

Tests should be placed in src/tests/ and run with pytest:

uv run pytest src/tests/
uv run pytest src/tests/ -v          # Verbose output
uv run pytest src/tests/ --cov       # With coverage

CI/CD Pipelines

Your GitHub Actions workflow should include:

name: CI

on:
  push:
    branches: [master]
  pull_request:
    branches: [master]

jobs:
  lint-and-test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: astral-sh/setup-uv@v5
      - run: uv sync
      - run: uv run ruff format --check .
      - run: uv run ruff check .
      - run: uv run pyrefly check
      - run: uv run pytest src/tests/

Dockerfile

Example Dockerfile for Python projects:

FROM minoritycr.azurecr.io/datateam-dag-base-python313:latest

ARG GITHUB_TOKEN

WORKDIR /build

# Enable bytecode compilation
ENV UV_COMPILE_BYTECODE=1

# Disable runtime sync - dependencies are already installed at build time
ENV UV_NO_SYNC=1

COPY pyproject.toml .
COPY uv.lock .
COPY README.md .
COPY src/ src/

# Configure git to use the token for private GitHub dependencies
RUN git config --global url."https://x-access-token:${GITHUB_TOKEN}@github.com/".insteadOf "https://github.com/"

RUN uv sync --frozen --no-dev

Build with GitHub token

Pass the token when building: docker build --build-arg GITHUB_TOKEN=$GITHUB_TOKEN .

Renovate

Add a renovate.json file to keep dependencies updated:

{
  "$schema": "https://docs.renovatebot.com/renovate-schema.json",
  "extends": ["config:recommended"],
  "lockFileMaintenance": {
    "enabled": true
  }
}

README Template

Your README should answer these key questions:

# Project Name

Brief description of what the project does.

## Setup

\`\`\`bash
uv sync
\`\`\`

## Running

\`\`\`bash
uv run your-command
\`\`\`

## How to Fix Issues

Document common issues and their solutions.

## Is it Safe to Rerun?

Describe idempotency - can this be safely rerun without side effects?

## Local Development

Instructions for running locally with any required environment variables.