python-packaging

Python Packaging

Comprehensive guide to creating, structuring, and distributing Python packages using modern packaging tools, pyproject.toml, and publishing to PyPI.

When to Use This Skill

• Creating Python libraries for distribution
• Building command-line tools with entry points
• Publishing packages to PyPI or private repositories
• Setting up Python project structure
• Creating installable packages with dependencies
• Building wheels and source distributions
• Versioning and releasing Python packages
• Creating namespace packages
• Implementing package metadata and classifiers

Core Concepts

1. Package Structure

• Source layout: src/package_name/ (recommended)
• Flat layout: package_name/ (simpler but less flexible)
• Package metadata: pyproject.toml, setup.py, or setup.cfg
• Distribution formats: wheel (.whl) and source distribution (.tar.gz)

2. Modern Packaging Standards

• PEP 517/518: Build system requirements
• PEP 621: Metadata in pyproject.toml
• PEP 660: Editable installs
• pyproject.toml: Single source of configuration

3. Build Backends

• setuptools: Traditional, widely used
• hatchling: Modern, opinionated
• flit: Lightweight, for pure Python
• poetry: Dependency management + packaging

4. Distribution

• PyPI: Python Package Index (public)
• TestPyPI: Testing before production
• Private repositories: JFrog, AWS CodeArtifact, etc.

Quick Start

Minimal Package Structure

my-package/
├── pyproject.toml
├── README.md
├── LICENSE
├── src/
│   └── my_package/
│       ├── __init__.py
│       └── module.py
└── tests/
    └── test_module.py

Minimal pyproject.toml

[build-system]
requires = ["setuptools>=61.0"]
build-backend = "setuptools.build_meta"
 
[project]
name = "my-package"
version = "0.1.0"
description = "A short description"
authors = [{name = "Your Name", email = "you@example.com"}]
readme = "README.md"
requires-python = ">=3.8"
dependencies = [
    "requests>=2.28.0"

Package Structure Patterns

Pattern 1: Source Layout (Recommended)

my-package/
├── pyproject.toml
├── README.md
├── LICENSE
├── .gitignore
├── src/
│   └── my_package/
│       ├── __init__.py
│       ├── core.py
│       ├── utils.py
│       └── py.typed          # For type hints
├── tests/
│   ├── __init__.py
│   ├── test_core.py
│   └── test_utils.py
└── docs/
    └── index.md

Advantages:

• Prevents accidentally importing from source
• Cleaner test imports
• Better isolation

pyproject.toml for source layout:

[tool.setuptools.packages.find]
where = ["src"]

Pattern 2: Flat Layout

my-package/
├── pyproject.toml
├── README.md
├── my_package/
│   ├── __init__.py
│   └── module.py
└── tests/
    └── test_module.py

Simpler but:

• Can import package without installing
• Less professional for libraries

Pattern 3: Multi-Package Project

project/
├── pyproject.toml
├── packages/
│   ├── package-a/
│   │   └── src/
│   │       └── package_a/
│   └── package-b/
│       └── src/
│           └── package_b/
└── tests/

Complete pyproject.toml Examples

Pattern 4: Full-Featured pyproject.toml

Pattern 5: Dynamic Versioning

[build-system]
requires = ["setuptools>=61.0", "setuptools-scm>=8.0"]
build-backend = "setuptools.build_meta"
 
[project]
name = "my-package"
dynamic = ["version"]
description = "Package with dynamic version"
 
[tool.setuptools.dynamic]
version = {attr = "my_package.__version__"}

In init.py:

# src/my_package/__init__.py
__version__ = "1.0.0"
 
# Or with setuptools-scm
from importlib.metadata import version
__version__ = version("my-package")

Command-Line Interface (CLI) Patterns

Pattern 6: CLI with Click

# src/my_package/cli.py
import click
 
@click.group()
@click.version_option()
def cli():
    """My awesome CLI tool."""
    pass
 
@cli.command()
@click.argument("name")
@click.option("--greeting", default="Hello", help

Register in pyproject.toml:

[project.scripts]
my-tool = "my_package.cli:main"

Usage:

pip install -e .
my-tool greet World
my-tool greet Alice --greeting="Hi"
my-tool repeat --count=3

Pattern 7: CLI with argparse

# src/my_package/cli.py
import argparse
import sys
 
def main():
    """Main CLI entry point."""
    parser = argparse.ArgumentParser(
        description="My awesome tool",
        prog="my-tool"
    )
 

Building and Publishing

Pattern 8: Build Package Locally

# Install build tools
pip install build twine
 
# Build distribution
python -m build
 
# This creates:
# dist/
#   my-package-1.0.0.tar.gz (source distribution)
#   my_package-1.0.0-py3-none-any.whl (wheel)
 
# Check the distribution
twine check dist/*

Pattern 9: Publishing to PyPI

# Install publishing tools
pip install twine
 
# Test on TestPyPI first
twine upload --repository testpypi dist/*
 
# Install from TestPyPI to test
pip install --index-url https://test.pypi.org/simple/ my-package
 
# If all good, publish to PyPI
twine upload dist/*

Using API tokens (recommended):

# Create ~/.pypirc
[distutils]
index-servers =
    pypi
    testpypi
 
[pypi]
username = __token__
password = pypi-...your-token...
 
[testpypi]
username = __token__
password = pypi-...your-test-token...

Pattern 10: Automated Publishing with GitHub Actions

# .github/workflows/publish.yml
name: Publish to PyPI
 
on:
  release:
    types: [created]
 
jobs:
  publish:
    runs-on: ubuntu-latest
 
    steps:
      - uses: actions/checkout@v3
 

For advanced patterns including data files, namespace packages, C extensions, version management, testing installation, documentation templates, and distribution workflows, see references/advanced-patterns.md