Week 8: Packaging, CI, and Code Quality#
Week 4 read a workflow and week 5 deployed one. This week turns to where most workflows in the wild live: Python packages. Every package you have installed this quarter was built, tested on several versions of Python, documented, and released by one. We look at how, then at the checks that run before a push, Ruff and pre-commit, and close by drawing the line between a CI service and an orchestrator such as Airflow, which comes after the exam in week 9.
Announcements#
HW 4, the class report, is due next Tuesday, November 24, at 9:30 am. Only pull requests merged by then count. Your pull request should be open today, so that your two reviewers have the week.
HW 5: TBD.
No class next week. November 24 falls in Thanksgiving break. Week 9 meets on December 1.
The exam is December 1, in the first 75 minutes of class, covering weeks 0 through 8, today included. Practice questions and their answer key are handed out today; see Exam Preparation.
Final project work should be underway. Your project is due 24 hours before your final project presentation in week 10. I run and grade the code before we meet, so it must run from a clean clone by following your README alone.
Final project presentation sign-ups. Book your group’s 30-minute slot for week 10 at Microsoft Bookings. The last day is Friday, December 11.
Objectives#
Explain the anatomy of a published Python package:
pyproject.toml, a build backend, versioning, a test matrix in CI, and a release: Writing and Publishing Your Own Python Packages.Say what Sphinx builds, and why documentation generated from the code stays current: Sphinx.
Lint and format code with Ruff, run it on every commit with pre-commit, and enforce the same checks in CI.
Use a matrix and a cache in a workflow.
Say clearly where GitHub Actions stops and an orchestrator starts, and defend the choice either way.
Explain what a monetary policy surprise is, how it is extracted from fed funds futures, and what Bernanke and Kuttner (2005) find about the stock market’s reaction to it.
Agenda Item 1: Where CI/CD Lives — Python Packages#
Every package you have installed this quarter was built, tested, and
released by a workflow.
Writing and Publishing Your Own Python Packages:
pyproject.toml as the single source of truth, a build backend, dynamic
versioning, semantic versioning as a contract.
The demo is chartbook, the package
you have been running all quarter: its pyproject.toml, hatch build
producing the wheel, hatch test running the suite across a matrix of
Python versions in GitHub Actions, and a release cut from the
changelog.
You will not build a package in this course. You will see how one is built,
because this is what “software packaging” means in the postings, and
because HW 3 itself grew out of a package two of last summer’s students
wrote for their final project, crashbounds, which is credited in the
assignment.
Agenda Item 2: Documentation with Sphinx#
Sphinx builds the documentation for pandas and for this book: pages written in reStructuredText or Markdown, reference pages generated from the code’s own docstrings, and a static site published the same way you published yours in week 4.
Agenda Item 3: Checks Before the Push — Ruff and pre-commit#
CI catches a problem after you push. The cheapest checks run before that.
Ruff is a Python linter and formatter.
ruff checkflags unused imports, undefined names, and other mistakes;ruff formatsettles style once, so a code review never argues about it. Both are configured inpyproject.toml.pre-commit runs those checks as a git hook on every commit, from a
.pre-commit-config.yamlchecked into the repository, so every contributor runs the same checks.Then run the same hooks in a workflow, so the check that guards
mainis the one you already ran on your laptop. Add both to your final project.
Agenda Item 4: Matrices, Caching, and the Boundary#
Matrices and caching. A
strategy: matrix:runs the same job on several versions of Python, which catches the bug that only one version has; a cache keeps a slow workflow from reinstalling everything on every run.The boundary. You have now used three kinds of automation:
Triggered by |
Knows about |
Reach for it when |
|
|---|---|---|---|
|
you, at a prompt |
one project’s file graph |
building or rebuilding a project |
Actions / cron |
a push, or a clock |
one repository |
tests on every change; one job on a schedule |
Airflow |
a clock, or data landing |
many projects, with history |
pipelines depend on each other, and you need to know what ran |
Agenda Item 5: The Paper — Bernanke and Kuttner (2005)#
Bernanke and Kuttner (2005), What Explains the Stock Market’s Reaction to Federal Reserve Policy?, splits a rate change into the part the market expected and the part it did not, using the futures price change on the announcement day, and shows that equities respond to the surprise and essentially not at all to the expected part. It is the natural paper for this week because its entire empirical content depends on a point-in-time measurement: what was priced before the announcement. The FedWatch case study computes these surprises from the same futures as HW 2, and a monitor like the one you deployed in week 5 is the machine that records them every day, without being asked.
Looking ahead to Week 9#
The exam is in the first 75 minutes. After it, Apache Airflow: an orchestrator that runs many pipelines on a schedule or when their inputs land, pointed at the class report. Then a short look at experiment tracking with MLflow. It meets on December 1, after the break.