Part 017 — Packaging, Dependency Management, dan Reproducible Builds

1. Tujuan Part Ini

Banyak engineer bisa menulis Python, tetapi bingung saat project harus:

• diinstall secara konsisten;
• dijalankan di CI;
• dibagikan ke teammate;
• dipaketkan sebagai library;
• dibuat CLI command;
• dipisahkan dev dependency dan runtime dependency;
• dikunci dependency version-nya;
• dibangun sebagai wheel;
• dideploy ke environment lain;
• direproduksi setelah beberapa bulan;
• diaudit dependency dan supply chain risk-nya.

Packaging bukan topik “administratif”. Packaging adalah bagian dari engineering reliability.

Dalam Python, packaging historically punya banyak tool dan transisi standar:

• setup.py;
• requirements.txt;
• setup.cfg;
• pyproject.toml;
• build backend;
• wheel;
• sdist;
• virtual environment;
• lockfile;
• dependency groups;
• extras;
• editable install;
• package index;
• modern tools seperti uv, Poetry, Hatch, PDM, pip-tools.

Part ini membangun mental model yang stabil supaya kamu tidak hanya menghafal command.

Target setelah part ini:

1. Memahami perbedaan project, package, distribution, module.
2. Memahami pyproject.toml.
3. Memahami runtime dependency vs dev dependency.
4. Memahami optional dependency/extras.
5. Memahami dependency groups.
6. Memahami editable install.
7. Memahami wheel dan source distribution.
8. Memahami build backend.
9. Memahami lockfile dan reproducibility.
10. Memahami basic supply-chain risk.
11. Menerapkan packaging yang sehat ke case-tracker.

2. Vocabulary: Project, Package, Module, Distribution

Istilah packaging sering membingungkan.

Contoh:

case-tracker/                  # project
  src/
    case_tracker/              # import package
      domain.py                # module

Nama project di packaging bisa case-tracker, sedangkan import package bisa case_tracker.

Ini umum karena:

• distribution name sering memakai hyphen;
• import package memakai underscore.

3. Why Packaging Matters

Tanpa packaging yang benar, kamu akan melihat masalah seperti:

ModuleNotFoundError: No module named 'case_tracker'

atau:

Works locally, fails in CI

atau:

I installed pytest but Python still cannot find it

Packaging yang baik menjawab:

• Python version apa yang didukung?
• Dependency apa yang dibutuhkan saat runtime?
• Dependency apa yang hanya untuk development/test?
• Bagaimana menjalankan CLI?
• Bagaimana package dibangun?
• Bagaimana package diinstall?
• Bagaimana environment direproduksi?
• Bagaimana dependency dikunci?
• Bagaimana source layout mencegah import accident?
• Bagaimana CI menginstall project?

4. The Modern Center: pyproject.toml

pyproject.toml adalah file konfigurasi project Python modern.

Ia bisa memuat:

• metadata project;
• dependencies;
• optional dependencies;
• scripts/entry points;
• build system;
• config tools seperti pytest, Ruff, mypy;
• dependency groups.

Example minimal:

[project]
name = "case-tracker"
version = "0.1.0"
description = "A small CLI case tracker."
readme = "README.md"
requires-python = ">=3.12"
dependencies = []

[project.scripts]
case-tracker = "case_tracker.cli:main"

[build-system]
requires = ["hatchling"]
build-backend = "hatchling.build"

4.1 TOML Sections

5. Project Metadata

Example:

[project]
name = "case-tracker"
version = "0.1.0"
description = "A small CLI case tracker for learning Python engineering fundamentals."
readme = "README.md"
requires-python = ">=3.12"
authors = [
    { name = "Your Name" }
]
license = "MIT"
keywords = ["python", "cli", "case-management"]
classifiers = [
    "Programming Language :: Python :: 3",
]
dependencies = []

Important fields:

For internal application, metadata can be minimal. For library distribution, metadata matters more.

6. Runtime Dependencies

Runtime dependencies are required when user runs/imports your package.

Example:

[project]
dependencies = [
    "pydantic>=2.0",
    "httpx>=0.27",
]

For early case-tracker, runtime dependencies can be empty because we use standard library.

This is good.

Rule:

Do not add external dependency until standard library is insufficient or dependency gives clear leverage.

Every dependency brings:

• version compatibility risk;
• transitive dependency risk;
• security risk;
• upgrade cost;
• packaging/deployment complexity;
• possible license consideration;
• supply-chain surface.

7. Development Dependencies

Dev dependencies are needed for development, not runtime.

Examples:

• pytest;
• Ruff;
• mypy;
• pyright;
• coverage;
• Hypothesis;
• pre-commit.

Modern dependency groups:

[dependency-groups]
dev = [
    "pytest",
    "ruff",
    "mypy",
    "hypothesis",
]

Dependency groups are meant for internal development use cases and are not exposed as installable extras for package users.

If your tool supports dependency groups, this is clean.

If not, alternatives:

• requirements-dev.txt;
• tool-specific groups;
• extras like [project.optional-dependencies] dev = [...];
• lockfile tool configuration.

8. Optional Dependencies / Extras

Extras are optional dependencies exposed to users.

Example library:

[project.optional-dependencies]
postgres = [
    "psycopg[binary]>=3",
]
redis = [
    "redis>=5",
]
dev = [
    "pytest",
    "ruff",
]

Install:

python -m pip install "my-package[postgres]"

Use extras when user of the package may choose optional features.

Use dependency groups when dependencies are for internal dev workflow.

8.1 Extras vs Dependency Groups

For application repositories, dependency groups often fit dev dependencies well.

9. Build System

[build-system] tells build tools how to build your project.

Example with Hatchling:

[build-system]
requires = ["hatchling"]
build-backend = "hatchling.build"

Other build backends exist:

• setuptools;
• hatchling;
• flit;
• poetry-core;
• pdm-backend.

For a simple package, Hatchling or setuptools are common choices.

Do not confuse:

• build backend: creates artifacts;
• installer: installs artifacts;
• environment manager: creates/uses environment;
• dependency resolver: resolves compatible versions.

One tool may provide multiple roles, but concepts remain separate.

10. Editable Install

During development:

python -m pip install -e .

Editable install means:

• package registered in environment;
• source changes reflected without reinstall;
• CLI entry point available;
• imports work like installed package.

After editable install:

case-tracker --help

works if [project.scripts] is configured.

This is better than relying on current directory import behavior.

11. CLI Entry Points

In pyproject.toml:

[project.scripts]
case-tracker = "case_tracker.cli:main"

This creates command:

case-tracker

It calls:

case_tracker.cli.main

Recommended main:

def main(argv: list[str] | None = None) -> int:
    ...

Entry point tools may call main() without arguments.

If main returns int, entry point wrappers usually handle process exit, but explicit python -m case_tracker can use:

from case_tracker.cli import main

raise SystemExit(main())

12. Source Layout

Recommended:

case-tracker/
  pyproject.toml
  src/
    case_tracker/
      __init__.py
      cli.py
      domain.py
  tests/

Why src layout?

• prevents accidental import from project root;
• tests mimic installed package behavior;
• catches packaging mistakes earlier;
• better for real distribution.

Without src, package at root:

case_tracker/
tests/

can also work, but src layout is safer for learning packaging discipline.

13. Building Artifacts

Install build tool:

python -m pip install build

Build:

python -m build

Output:

dist/
  case_tracker-0.1.0-py3-none-any.whl
  case_tracker-0.1.0.tar.gz

Wheel:

• built distribution;
• faster install;
• common deployment artifact.

sdist:

• source distribution;
• can be built on target;
• includes source files per backend config.

13.1 Testing Built Package

A good packaging check:

python -m pip install dist/case_tracker-0.1.0-py3-none-any.whl
python -c "import case_tracker; print(case_tracker)"
case-tracker --help

CI can build package to catch packaging errors.

14. Versioning

For learning project:

version = "0.1.0"

Semantic versioning idea:

MAJOR.MINOR.PATCH

• MAJOR: breaking changes;
• MINOR: backwards-compatible features;
• PATCH: bug fixes.

For internal applications, versioning may tie to deployment version/build SHA.

For libraries, versioning communicates compatibility.

Do not obsess over semver before public API exists.

15. Dependency Specifiers

Examples:

dependencies = [
    "httpx>=0.27,<1.0",
    "pydantic>=2.8",
]

Specifier styles:

For libraries:

• avoid over-pinning;
• allow compatible ranges;
• do not force exact pins unnecessarily.

For applications:

• lock exact resolved versions through lockfile;
• reproducibility matters more.

16. Requirements Files

requirements.txt is still common.

Example:

pytest==8.0.0
ruff==0.5.0

Use cases:

• deployment compatibility;
• legacy workflows;
• simple pinned install;
• generated lock output;
• platform requiring requirements file.

But for modern project metadata, pyproject.toml should define project dependencies.

A common pattern:

• declare dependencies in pyproject.toml;
• lock with tool;
• export requirements if deployment needs it.

Avoid manually maintaining multiple conflicting dependency lists.

17. Lockfiles

A lockfile records exact resolved dependency versions.

Purpose:

• reproducible installs;
• consistent CI;
• predictable deployment;
• easier auditing;
• controlled upgrades.

Tools have different lockfiles:

• uv.lock;
• poetry.lock;
• pdm.lock;
• requirements.txt generated with hashes;
• tool-specific formats.

Library vs application:

For case-tracker, lockfile becomes useful once dev dependencies and tools are managed by a tool like uv.

18. Reproducible Builds

Reproducibility has levels.

For most application teams, good baseline:

• define dependencies;
• lock dependencies;
• build in CI;
• run tests against installed package;
• deploy built artifact or image;
• avoid manual server installs.

19. Supply Chain Risk

Dependencies create supply-chain risk.

Risk sources:

• vulnerable package version;
• malicious package;
• typosquatting;
• compromised maintainer account;
• transitive dependency risk;
• license incompatibility;
• abandoned package;
• overly broad version range;
• dependency confusion;
• build-time code execution.

Mitigations:

1. Minimize dependencies.
2. Prefer mature packages.
3. Pin/lock for applications.
4. Audit dependencies.
5. Review new dependencies.
6. Use private index carefully.
7. Avoid installing from random URLs.
8. Separate dev and runtime dependencies.
9. Upgrade regularly.
10. Watch transitive dependency changes.

Dependency decision should be architectural, not casual.

20. Dependency Decision Framework

Before adding dependency, ask:

1. Is this in the standard library?
2. Is the problem core to our domain?
3. Is the package actively maintained?
4. Is the API stable?
5. How many transitive dependencies?
6. What is the license?
7. What is the security posture?
8. Can we replace it later?
9. Does it work with our Python versions?
10. Does it support our deployment platform?
11. Is the functionality worth the dependency cost?
12. Will it simplify code enough?

Example:

• For CLI parsing, argparse is enough.
• For rich CLI UI, Typer/Click/Rich may be justified.
• For JSON, standard json is enough.
• For data validation in API, Pydantic may be justified.
• For HTTP client, standard library possible, but httpx/requests often higher leverage.

21. Environment Management

Basic:

python -m venv .venv
source .venv/bin/activate
python -m pip install -e .

Modern tools can combine workflows.

Example with uv:

uv init
uv add pytest --dev
uv run pytest
uv lock
uv sync

Conceptually:

• environment;
• dependencies;
• lock;
• command runner;
• project metadata.

Even if using a modern tool, understand the underlying model.

22. Application vs Library Packaging

22.1 Application

Goal:

• deploy/run consistently;
• lock dependencies;
• build image/artifact;
• expose CLI/server;
• prioritize reproducibility.

Application can pin tightly.

22.2 Library

Goal:

• be installed by many environments;
• declare compatibility ranges;
• avoid forcing exact versions;
• maintain public API;
• test across Python versions;
• publish wheel/sdist.

Library should not over-pin runtime dependencies because it can create conflicts for users.

case-tracker is currently an application/learning project.

23. Internal Packages

In companies, internal packages are common.

Risks:

• too many tiny packages;
• versioning overhead;
• dependency diamond conflicts;
• private index complexity;
• unclear ownership;
• slow release cadence;
• breaking changes across teams.

Before extracting internal package, ask:

1. Is there real reuse across projects?
2. Is API stable?
3. Who owns it?
4. How are breaking changes handled?
5. Is copy-paste temporarily cheaper?
6. Is monorepo module enough?
7. Are security/licensing policies clear?
8. Is documentation ready?

Packaging creates distribution boundary. Distribution boundaries have cost.

24. Build Backend Selection

For simple projects:

• Hatchling: modern, lightweight.
• setuptools: widely supported, mature.
• Flit: simple for pure Python packages.
• Poetry-core: if using Poetry.

Selection criteria:

• team familiarity;
• ecosystem support;
• pure Python vs compiled extensions;
• config simplicity;
• compatibility with tooling;
• release workflow.

Do not switch build backend casually. It affects packaging behavior.

For case-tracker, Hatchling or setuptools is fine.

25. Package Data

If package needs non-Python files:

• templates;
• config defaults;
• static assets;
• schema files.

You need package data configuration. This depends on build backend.

Avoid reading files by relative filesystem paths from current working directory.

Use importlib.resources for package resources.

Example later:

from importlib.resources import files

template = files("case_tracker.templates").joinpath("report.txt").read_text()

This works better when package is installed as wheel.

26. Entry Point vs python -m

Support both:

pyproject.toml:

[project.scripts]
case-tracker = "case_tracker.cli:main"

__main__.py:

from case_tracker.cli import main

raise SystemExit(main())

Now users can run:

case-tracker

or:

python -m case_tracker

This improves development and installed usage.

27. Packaging Tests

Add tests/checks:

python -m build
python -m pip install dist/*.whl
python -c "import case_tracker"
case-tracker --help

In CI, use a clean environment to test the built wheel.

Why?

Editable installs can hide packaging mistakes.

Wheel install catches:

• missing package data;
• wrong package discovery;
• broken entry point;
• missing runtime dependency;
• import path errors.

28. Case Tracker pyproject.toml v2

[project]
name = "case-tracker"
version = "0.1.0"
description = "A small CLI case tracker for learning Python engineering fundamentals."
readme = "README.md"
requires-python = ">=3.12"
dependencies = []

[project.scripts]
case-tracker = "case_tracker.cli:main"

[dependency-groups]
dev = [
    "build",
    "pytest",
    "ruff",
    "mypy",
    "hypothesis",
]

[build-system]
requires = ["hatchling"]
build-backend = "hatchling.build"

[tool.pytest.ini_options]
testpaths = ["tests"]
pythonpath = ["src"]
addopts = ["-ra"]

[tool.ruff]
line-length = 100
target-version = "py312"

[tool.ruff.lint]
select = ["E", "F", "I", "B", "UP", "SIM", "PT"]

[tool.mypy]
python_version = "3.12"
mypy_path = "src"
strict = true

If your installer/tool does not support dependency groups yet, keep dev dependency installation documented separately or use your chosen tool's supported format.

29. CI Packaging Stage

Add to CI:

      - name: Build package
        run: python -m build

      - name: Inspect dist
        run: ls -la dist

Better with clean install:

  package:
    runs-on: ubuntu-latest
    needs: quality

    steps:
      - uses: actions/checkout@v4

      - uses: actions/setup-python@v5
        with:
          python-version: "3.12"

      - name: Build package
        run: |
          python -m pip install --upgrade pip
          python -m pip install build
          python -m build

      - name: Install wheel
        run: |
          python -m pip install dist/*.whl
          python -c "import case_tracker"
          case-tracker --help

This checks installability.

30. Common Packaging Mistakes

30.1 Running from Source but Never Installing

python src/case_tracker/cli.py

This bypasses package behavior.

Use:

python -m case_tracker

after editable install or proper path setup.

30.2 Missing build-system

Some tools can infer, but explicit is better.

30.3 Dev Dependencies in Runtime Dependencies

Bad:

dependencies = ["pytest", "ruff"]

Runtime users should not install test/lint tools.

30.4 Over-Pinning Library Dependencies

Bad for libraries:

dependencies = ["requests==2.31.0"]

Can cause conflicts.

30.5 No Lockfile for Application

Deployment becomes non-reproducible.

30.6 Import Package Name Mismatch Confusion

Distribution:

case-tracker

Import:

case_tracker

This is normal. Document it.

30.7 Assuming Current Working Directory

Bad:

Path("templates/report.txt").read_text()

Use package resources or configurable paths.

30.8 Publishing Secrets or Test Data

Check package contents before publishing.

31. Practice: Editable Install

From project root:

python -m pip install -e .

Then:

python -c "import case_tracker; print(case_tracker)"
case-tracker --help
python -m case_tracker --help

If one works and another fails, inspect:

• [project.scripts];
• __main__.py;
• package import path;
• environment;
• editable install output.

32. Practice: Build Package

python -m pip install build
python -m build

Inspect:

ls dist

Then create clean venv:

python -m venv /tmp/case-tracker-test
source /tmp/case-tracker-test/bin/activate
python -m pip install dist/*.whl
case-tracker --help

This simulates user install.

33. Practice: Separate Dependencies

Move dev tools out of runtime dependencies.

Bad:

[project]
dependencies = [
    "pytest",
    "ruff",
]

Better:

[project]
dependencies = []

[dependency-groups]
dev = [
    "pytest",
    "ruff",
]

If using a tool that does not support dependency groups, document alternative.

34. Practice: Dependency Decision Record

Before adding a dependency, write:

# Dependency Decision: <package>

## Problem

What problem does this solve?

## Alternatives

- Standard library:
- Internal implementation:
- Other packages:

## Decision

Why this package?

## Risks

- Security:
- License:
- Maintenance:
- Transitive dependencies:
- Replacement cost:

## Usage Boundary

Where may this dependency be imported?

This is especially useful for core dependencies.

35. Practice: Packaging Checklist

Check:

1. pyproject.toml exists.
2. [project] metadata is valid.
3. requires-python set.
4. Runtime dependencies minimal.
5. Dev dependencies separated.
6. [project.scripts] works.
7. [build-system] set.
8. src layout works.
9. python -m case_tracker works.
10. Editable install works.
11. python -m build works.
12. Wheel install works in clean environment.
13. Tests pass after install.
14. README setup is accurate.
15. CI builds package.

36. Self-Check

Jawab tanpa melihat materi:

1. Apa beda module, package, project, distribution?
2. Apa fungsi pyproject.toml?
3. Apa itu runtime dependency?
4. Apa itu dev dependency?
5. Apa beda optional dependencies dan dependency groups?
6. Apa fungsi [build-system]?
7. Apa itu editable install?
8. Apa itu wheel?
9. Apa itu sdist?
10. Kenapa src layout membantu?
11. Kenapa aplikasi butuh lockfile?
12. Kenapa library tidak sebaiknya over-pin dependency?
13. Apa risiko supply chain dependency?
14. Apa pertanyaan sebelum menambah dependency?
15. Apa beda app packaging dan library packaging?
16. Apa fungsi [project.scripts]?
17. Kenapa package data tidak boleh diasumsikan dari cwd?
18. Apa packaging mistake paling umum?
19. Bagaimana mengetes wheel install?
20. Apa level reproducibility yang cukup untuk aplikasi kecil?

37. Definition of Done Part 017

Kamu selesai part ini jika bisa:

1. Menjelaskan vocabulary packaging.
2. Membuat pyproject.toml valid.
3. Memisahkan runtime dan dev dependencies.
4. Menjelaskan extras vs dependency groups.
5. Menambahkan [build-system].
6. Menjalankan editable install.
7. Menambahkan CLI entry point.
8. Menjalankan python -m case_tracker.
9. Build wheel dan sdist.
10. Install wheel di clean environment.
11. Menjelaskan lockfile.
12. Menjelaskan supply-chain risk.
13. Membuat dependency decision record.
14. Menambahkan packaging stage di CI.
15. Menghindari packaging mistakes umum.

38. Ringkasan

Packaging adalah reliability boundary.

Inti part ini:

• module, package, project, dan distribution adalah konsep berbeda;
• pyproject.toml adalah pusat metadata modern;
• runtime dependency berbeda dari dev dependency;
• extras untuk optional user features;
• dependency groups untuk internal dev workflows;
• build backend membuat artifact;
• editable install membuat development workflow benar;
• wheel adalah artifact install yang penting;
• lockfile mendukung reproducibility;
• aplikasi dan library punya strategi dependency berbeda;
• dependency membawa supply-chain risk;
• package harus dites sebagai installed artifact, bukan hanya source tree;
• CI harus membangun dan menginstall package untuk menangkap packaging bug.

Part berikutnya akan membahas standard library sebagai engineering leverage: modul-modul bawaan Python yang harus kamu kuasai sebelum menambah dependency eksternal.

39. Referensi

• Python Packaging User Guide — Writing your pyproject.toml.
• Python Packaging User Guide — Dependency Groups.
• Python Packaging User Guide — Packaging and distributing projects.
• Python Documentation — venv.
• Python Documentation — importlib.resources.
• PyPA build documentation.
• pip documentation.