Running Ruff and mypy in CI with Inline Annotations

A lint failure that only says "exit code 1" in a CI log is a small tax on every contributor: open the job, scroll to the step, find the file and line, switch back to the editor. Multiply by every pull request and the linters start to feel like obstacles. GitHub Actions can do much better — a step can emit workflow commands that attach a finding to a specific line of a specific file, so the problem appears directly in the pull request's diff view, next to the code that caused it. Ruff speaks that format natively. mypy does not, but its JSON output converts to it in twenty lines. This guide builds a lint job that runs Ruff's linter and formatter check and mypy, annotates every finding inline, stays fast, and optionally feeds GitHub's code-scanning dashboard through SARIF. It belongs to the linting and type checking topic.

Prerequisites

• Ruff and mypy configured for the project — see configuring Ruff for a CLI project and type-checking Click and Typer code with mypy.
• A GitHub Actions workflow; other CI systems have equivalents (GitLab code quality reports, for example).

How inline annotations work

Any step in a GitHub Actions job can print a line in a special format to its standard output:

::error file=src/mytool/cli.py,line=14,col=5,title=mypy (return-value)::Incompatible return value type

The runner intercepts it, shows it in the job summary, and — when the file is part of the pull request's diff — attaches it to that line in the "Files changed" view. ::warning and ::notice work the same way with different severities.

The recipe

Ruff: native GitHub output

uvx ruff check --output-format github .
uvx ruff format --check --diff .

--output-format github makes every lint finding a workflow command. On a module with an unused import it prints:

::error title=ruff (F401),file=src/mytool/bad.py,line=1,col=8,endLine=1,endColumn=10::src/mytool/bad.py:1:8: F401 `os` imported but unused

The format check has no annotation mode, but --diff prints exactly what would change, which is usually enough to fix it at a glance — and the fix is always the same: run ruff format locally.

mypy: JSON to annotations

Recent mypy versions emit one JSON object per finding with --output json:

{"file": "src/mytool/bad.py", "line": 4, "column": 11, "end_line": 4, "end_column": 12, "message": "Incompatible return value type (got \"int\", expected \"str\")", "hint": null, "code": "return-value", "severity": "error"}

A small script turns those lines into workflow commands, escaping the characters the format reserves:

# scripts/mypy_annotations.py
"""Read `mypy --output json` on stdin; print GitHub workflow annotations."""
from __future__ import annotations

import json
import sys


def escape_data(text: str) -> str:
    return text.replace("%", "%25").replace("\r", "%0D").replace("\n", "%0A")


def escape_property(text: str) -> str:
    return escape_data(text).replace(":", "%3A").replace(",", "%2C")


def annotation(finding: dict) -> str:
    level = {"error": "error", "note": "notice"}.get(finding.get("severity"), "warning")
    props = {
        "file": finding["file"],
        "line": finding.get("line", 1),
        "col": max(finding.get("column", 0), 0) + 1,
        "title": f"mypy ({finding.get('code') or 'misc'})",
    }
    if finding.get("end_line"):
        props["endLine"] = finding["end_line"]
    message = finding["message"]
    if finding.get("hint"):
        message += "\n" + finding["hint"]
    rendered = ",".join(f"{k}={escape_property(str(v))}" for k, v in props.items())
    return f"::{level} {rendered}::{escape_data(message)}"


def main() -> int:
    errors = 0
    for line in sys.stdin:
        line = line.strip()
        if not line.startswith("{"):
            print(line)                       # pass through summaries and crashes
            continue
        finding = json.loads(line)
        errors += finding.get("severity") == "error"
        print(annotation(finding))
    return 1 if errors else 0


if __name__ == "__main__":
    raise SystemExit(main())

mypy reports zero-based columns in JSON while annotations are one-based, hence the + 1. Lines that are not JSON — a crash, a configuration error — are passed through unchanged, so a broken mypy setup still fails visibly instead of producing an empty, green run. The exit code comes from the findings, because mypy's own exit status is lost in the pipe.

The workflow

# .github/workflows/lint.yml
name: lint
on: [push, pull_request]

permissions:
  contents: read

jobs:
  ruff:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: astral-sh/setup-uv@v6
      - run: uvx ruff@0.16.10 check --output-format github .
      - run: uvx ruff@0.16.10 format --check --diff .

  mypy:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: astral-sh/setup-uv@v6
        with:
          enable-cache: true
      - run: uv sync --locked --group lint
      - run: uv run mypy --output json src | python scripts/mypy_annotations.py

Ruff runs in its own job without installing the project — it needs no dependencies — so lint results arrive within seconds of a push. mypy needs the project and its dependencies installed to resolve types, so it gets the cached uv sync. Pin both tool versions (here via ruff@… and the lint dependency group) so a new release with new rules lands as a deliberate change rather than a surprise failure on someone's unrelated pull request.

Optional: SARIF for code scanning

For repositories with GitHub code scanning enabled, Ruff can write SARIF, which feeds the Security tab's alert list with history and dismissal tracking:

      - run: uvx ruff@0.16.10 check --output-format sarif -o ruff.sarif . || true
      - uses: github/codeql-action/upload-sarif@v3
        with:
          sarif_file: ruff.sarif
          category: ruff

That job needs security-events: write permission. SARIF suits security-relevant rules (Ruff's S rules from flake8-bandit, for example) better than style findings, which belong in the pull request annotations.

UX considerations

The users of a lint job are contributors trying to get a pull request green:

• Fail with the fix. Ruff's messages include a "help" line; the format check's diff shows the exact change. A contributor should never have to guess what the linter wants.
• Match local and CI. The same Ruff and mypy versions and configuration in pre-commit and CI, so "it passed locally" means something.
• Keep it fast. Separate jobs, no project install for Ruff, cached environments for mypy. A lint job that finishes before tests start is one people do not resent.
• Mind the annotation limits. GitHub shows a limited number of annotations per step (currently 10 errors and 10 warnings per step in the diff view); fix a large backlog in a dedicated pull request rather than letting annotations scroll off.
• Use notice for advice. Not every finding should block: emit non-blocking rules as notices so they inform without failing.

Testing the behaviour

The converter is plain Python and worth a few tests, because a bug in it silently hides type errors:

# tests/test_mypy_annotations.py
import io
import json
import sys

from scripts.mypy_annotations import annotation, main

FINDING = {"file": "src/a,b.py", "line": 4, "column": 10, "end_line": 4, "end_column": 12,
           "message": "Incompatible return value\nsecond line", "hint": None,
           "code": "return-value", "severity": "error"}


def test_annotation_escapes_and_offsets_columns():
    out = annotation(FINDING)
    assert out.startswith("::error file=src/a%2Cb.py,line=4,col=11,")
    assert "title=mypy (return-value)" in out
    assert out.endswith("::Incompatible return value%0Asecond line")


def test_exit_code_reflects_errors(monkeypatch, capsys):
    monkeypatch.setattr(sys, "stdin", io.StringIO(json.dumps(FINDING) + "\n"))
    assert main() == 1
    monkeypatch.setattr(sys, "stdin", io.StringIO("Success: no issues found in 3 source files\n"))
    assert main() == 0
    assert "Success" in capsys.readouterr().out

Then prove the whole path once on a throwaway branch: introduce an unused import and a wrong return type, open a pull request, and check that both appear on the right lines in the diff view.

Conclusion

Inline annotations make linters feel like reviewers rather than gatekeepers. Run Ruff with --output-format github and format --check --diff in a fast job of its own, pipe mypy --output json through a small converter that escapes correctly and keeps the exit code, pin both tools, and optionally send SARIF to code scanning for security rules. Contributors then see each problem next to the line that caused it, with the fix one glance away.

Frequently asked questions

Related

• Up: Linting and type checking CLI code
• Configuring Ruff for a CLI project
• Type-checking Click and Typer code with mypy
• Running pre-commit in CI
• Caching uv dependencies in CI