Skip to content

Commit

Permalink
Stabilize support for Jupyter Notebooks (#12878)
Browse files Browse the repository at this point in the history
Co-authored-by: Alex Waygood <[email protected]>
Closes: #12456
Closes: astral-sh/ruff-vscode#546
  • Loading branch information
2 people authored and MichaReiser committed Aug 14, 2024
1 parent 15aa5a6 commit 73160dc
Show file tree
Hide file tree
Showing 13 changed files with 68 additions and 160 deletions.
3 changes: 1 addition & 2 deletions crates/ruff/src/commands/check.rs
Original file line number Diff line number Diff line change
Expand Up @@ -268,8 +268,7 @@ mod test {

// Run
let diagnostics = check(
// Notebooks are not included by default
&[tempdir.path().to_path_buf(), notebook],
&[tempdir.path().to_path_buf()],
&pyproject_config,
&ConfigArguments::default(),
flags::Cache::Disabled,
Expand Down
64 changes: 1 addition & 63 deletions crates/ruff/tests/lint.rs
Original file line number Diff line number Diff line change
Expand Up @@ -1806,7 +1806,7 @@ select = ["UP006"]
}

#[test]
fn checks_notebooks_in_preview_mode() -> anyhow::Result<()> {
fn checks_notebooks_in_stable() -> anyhow::Result<()> {
let tempdir = TempDir::new()?;
std::fs::write(
tempdir.path().join("main.ipynb"),
Expand Down Expand Up @@ -1853,7 +1853,6 @@ fn checks_notebooks_in_preview_mode() -> anyhow::Result<()> {
.args(STDIN_BASE_OPTIONS)
.arg("--select")
.arg("F401")
.arg("--preview")
.current_dir(&tempdir)
, @r###"
success: false
Expand All @@ -1867,64 +1866,3 @@ fn checks_notebooks_in_preview_mode() -> anyhow::Result<()> {
"###);
Ok(())
}

#[test]
fn ignores_notebooks_in_stable() -> anyhow::Result<()> {
let tempdir = TempDir::new()?;
std::fs::write(
tempdir.path().join("main.ipynb"),
r#"
{
"cells": [
{
"cell_type": "code",
"execution_count": null,
"id": "ad6f36d9-4b7d-4562-8d00-f15a0f1fbb6d",
"metadata": {},
"outputs": [],
"source": [
"import random"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3 (ipykernel)",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.12.0"
}
},
"nbformat": 4,
"nbformat_minor": 5
}
"#,
)?;

assert_cmd_snapshot!(Command::new(get_cargo_bin(BIN_NAME))
.args(STDIN_BASE_OPTIONS)
.arg("--select")
.arg("F401")
.current_dir(&tempdir)
, @r###"
success: true
exit_code: 0
----- stdout -----
All checks passed!
----- stderr -----
warning: No Python files found under the given path(s)
"###);
Ok(())
}
Original file line number Diff line number Diff line change
Expand Up @@ -60,6 +60,7 @@ file_resolver.force_exclude = false
file_resolver.include = [
"*.py",
"*.pyi",
"*.ipynb",
"**/pyproject.toml",
]
file_resolver.extend_include = []
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,11 @@ use super::super::helpers::at_last_top_level_expression_in_cell;
/// assert foo == bar, "`foo` and `bar` should be equal."
/// ```
///
/// ## Notebook behavior
/// For Jupyter Notebooks, this rule is not applied to the last top-level expression in a cell.
/// This is because it's common to have a notebook cell that ends with an expression,
/// which will result in the `repr` of the evaluated expression being printed as the cell's output.
///
/// ## References
/// - [Python documentation: `assert` statement](https://docs.python.org/3/reference/simple_stmts.html#the-assert-statement)
#[violation]
Expand All @@ -43,9 +48,6 @@ impl Violation for UselessComparison {
/// B015
pub(crate) fn useless_comparison(checker: &mut Checker, expr: &Expr) {
if expr.is_compare_expr() {
// For Jupyter Notebooks, ignore the last top-level expression for each cell.
// This is because it's common to have a cell that ends with an expression
// to display it's value.
if checker.source_type.is_ipynb()
&& at_last_top_level_expression_in_cell(
checker.semantic(),
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,11 @@ use super::super::helpers::at_last_top_level_expression_in_cell;
/// foo = 1 + 1
/// ```
///
/// ## Notebook behavior
/// For Jupyter Notebooks, this rule is not applied to the last top-level expression in a cell.
/// This is because it's common to have a notebook cell that ends with an expression,
/// which will result in the `repr` of the evaluated expression being printed as the cell's output.
///
/// ## Known problems
/// This rule ignores expression types that are commonly used for their side
/// effects, such as function calls.
Expand Down Expand Up @@ -81,9 +86,6 @@ pub(crate) fn useless_expression(checker: &mut Checker, value: &Expr) {
return;
}

// For Jupyter Notebooks, ignore the last top-level expression for each cell.
// This is because it's common to have a cell that ends with an expression
// to display it's value.
if checker.source_type.is_ipynb()
&& at_last_top_level_expression_in_cell(
checker.semantic(),
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -6,8 +6,7 @@ use ruff_text_size::Ranged;
use crate::checkers::ast::Checker;

/// ## What it does
/// Checks for imports that are not at the top of the file. For Jupyter notebooks, this
/// checks for imports that are not at the top of the cell.
/// Checks for imports that are not at the top of the file.
///
/// ## Why is this bad?
/// According to [PEP 8], "imports are always put at the top of the file, just after any
Expand Down Expand Up @@ -36,6 +35,9 @@ use crate::checkers::ast::Checker;
/// a = 1
/// ```
///
/// ## Notebook behavior
/// For Jupyter notebooks, this rule checks for imports that are not at the top of a *cell*.
///
/// [PEP 8]: https://peps.python.org/pep-0008/#imports
#[violation]
pub struct ModuleImportNotAtTopOfFile {
Expand Down
3 changes: 3 additions & 0 deletions crates/ruff_linter/src/rules/pydocstyle/rules/not_missing.rs
Original file line number Diff line number Diff line change
Expand Up @@ -53,6 +53,9 @@ use crate::registry::Rule;
/// def calculate_speed(distance: float, time: float) -> float: ...
/// ```
///
/// ## Notebook behavior
/// This rule is ignored for Jupyter Notebooks.
///
/// ## References
/// - [PEP 257 – Docstring Conventions](https://peps.python.org/pep-0257/)
/// - [PEP 287 – reStructuredText Docstring Format](https://peps.python.org/pep-0287/)
Expand Down
12 changes: 3 additions & 9 deletions crates/ruff_workspace/src/configuration.rs
Original file line number Diff line number Diff line change
Expand Up @@ -230,15 +230,9 @@ impl Configuration {
extend_exclude: FilePatternSet::try_from_iter(self.extend_exclude)?,
extend_include: FilePatternSet::try_from_iter(self.extend_include)?,
force_exclude: self.force_exclude.unwrap_or(false),
include: FilePatternSet::try_from_iter(self.include.unwrap_or_else(|| {
let mut include = INCLUDE.to_vec();

if global_preview.is_enabled() {
include.push(FilePattern::Builtin("*.ipynb"));
}

include
}))?,
include: FilePatternSet::try_from_iter(
self.include.unwrap_or_else(|| INCLUDE.to_vec()),
)?,
respect_gitignore: self.respect_gitignore.unwrap_or(true),
project_root: project_root.to_path_buf(),
},
Expand Down
6 changes: 2 additions & 4 deletions crates/ruff_workspace/src/options.rs
Original file line number Diff line number Diff line change
Expand Up @@ -241,13 +241,11 @@ pub struct Options {
/// included here not for configuration but because we lint whether e.g. the
/// `[project]` matches the schema.
///
/// If [preview](https://docs.astral.sh/ruff/preview/) is enabled, the default
/// includes notebook files (`.ipynb` extension). You can exclude them by adding
/// `*.ipynb` to [`extend-exclude`](#extend-exclude).
/// Notebook files (`.ipynb` extension) are included by default on Ruff 0.6.0+.
///
/// For more information on the glob syntax, refer to the [`globset` documentation](https://docs.rs/globset/latest/globset/#syntax).
#[option(
default = r#"["*.py", "*.pyi", "**/pyproject.toml"]"#,
default = r#"["*.py", "*.pyi", "*.ipynb", "**/pyproject.toml"]"#,
value_type = "list[str]",
example = r#"
include = ["*.py"]
Expand Down
1 change: 1 addition & 0 deletions crates/ruff_workspace/src/settings.rs
Original file line number Diff line number Diff line change
Expand Up @@ -137,6 +137,7 @@ pub(crate) static EXCLUDE: &[FilePattern] = &[
pub(crate) static INCLUDE: &[FilePattern] = &[
FilePattern::Builtin("*.py"),
FilePattern::Builtin("*.pyi"),
FilePattern::Builtin("*.ipynb"),
FilePattern::Builtin("**/pyproject.toml"),
];

Expand Down
88 changes: 39 additions & 49 deletions docs/configuration.md
Original file line number Diff line number Diff line change
Expand Up @@ -339,23 +339,9 @@ For example, `ruff check /path/to/excluded/file.py` will always lint `file.py`.

### Default inclusions

By default, Ruff will discover files matching `*.py`, `*.ipy`, or `pyproject.toml`.
By default, Ruff will discover files matching `*.py`, `*.pyi`, `*.ipynb`, or `pyproject.toml`.

To lint or format files with additional file extensions, use the [`extend-include`](settings.md#extend-include) setting.

=== "pyproject.toml"

```toml
[tool.ruff]
extend-include = ["*.ipynb"]
```

=== "ruff.toml"

```toml
extend-include = ["*.ipynb"]
```

You can also change the default selection using the [`include`](settings.md#include) setting.


Expand All @@ -378,78 +364,82 @@ You can also change the default selection using the [`include`](settings.md#incl

## Jupyter Notebook discovery

Ruff has built-in support for [Jupyter Notebooks](https://jupyter.org/).

!!! info
Notebooks are linted and formatted by default when using [preview mode](preview.md).
You can opt-out of notebook linting and formatting by adding `*.ipynb` to [`extend-exclude`](settings.md#extend-exclude).
Ruff has built-in support for linting and formatting [Jupyter Notebooks](https://jupyter.org/),
which are linted and formatted by default on version `0.6.0` and higher.

To opt in to linting and formatting Jupyter Notebook (`.ipynb`) files, add the `*.ipynb` pattern to
your [`extend-include`](settings.md#extend-include) setting, like so:
If you'd prefer to either only lint or only format Jupyter Notebook files, you can use the
section-specific `exclude` option to do so. For example, the following would only lint Jupyter
Notebook files and not format them:

=== "pyproject.toml"

```toml
[tool.ruff]
extend-include = ["*.ipynb"]
[tool.ruff.format]
exclude = ["*.ipynb"]
```

=== "ruff.toml"

```toml
extend-include = ["*.ipynb"]
[format]
exclude = ["*.ipynb"]
```

This will prompt Ruff to discover Jupyter Notebook (`.ipynb`) files in any specified
directories, then lint and format them accordingly.

If you'd prefer to either only lint or only format Jupyter Notebook files, you can use the
section specific `exclude` option to do so. For example, the following would only lint Jupyter
Notebook files and not format them:
And, conversely, the following would only format Jupyter Notebook files and not lint them:

=== "pyproject.toml"

```toml
[tool.ruff]
extend-include = ["*.ipynb"]

[tool.ruff.format]
[tool.ruff.lint]
exclude = ["*.ipynb"]
```

=== "ruff.toml"

```toml
extend-include = ["*.ipynb"]

[format]
[lint]
exclude = ["*.ipynb"]
```

And, conversely, the following would only format Jupyter Notebook files and not lint them:
You can completely disable Jupyter Notebook support by updating the
[`extend-exclude`](settings.md#extend-exclude) setting:

=== "pyproject.toml"

```toml
[tool.ruff]
extend-include = ["*.ipynb"]

[tool.ruff.lint]
exclude = ["*.ipynb"]
extend-exclude = ["*.ipynb"]
```

=== "ruff.toml"

```toml
extend-include = ["*.ipynb"]
extend-exclude = ["*.ipynb"]
```

[lint]
exclude = ["*.ipynb"]
If you'd like to ignore certain rules specifically for Jupyter Notebook files, you can do so by
using the [`per-file-ignores`](settings.md#per-file-ignores) setting:

=== "pyproject.toml"

```toml
[tool.ruff.lint.per-file-ignores]
"*.ipynb" = ["T20"]
```

=== "ruff.toml"

```toml
[lint.per-file-ignores]
"*.ipynb" = ["T20"]
```

Alternatively, pass the notebook file(s) to `ruff` on the command-line directly. For example,
`ruff check /path/to/notebook.ipynb` will always lint `notebook.ipynb`. Similarly,
`ruff format /path/to/notebook.ipynb` will always format `notebook.ipynb`.
Some rules have different behavior when applied to Jupyter Notebook files. For
example, when applied to `.py` files the
[`module-import-not-at-top-of-file` (`E402`)](rules/module-import-not-at-top-of-file.md)
rule detect imports at the top of a file, but for notebooks it detects imports at the top of a
**cell**. For a given rule, the rule's documentation will always specify if it has different
behavior when applied to Jupyter Notebook files.

## Command-line interface

Expand Down
26 changes: 2 additions & 24 deletions docs/faq.md
Original file line number Diff line number Diff line change
Expand Up @@ -398,30 +398,8 @@ them. You can find the supported settings in the [API reference](settings.md#lin

## Does Ruff support Jupyter Notebooks?

Ruff has built-in support for linting [Jupyter Notebooks](https://jupyter.org/).

To opt in to linting Jupyter Notebook (`.ipynb`) files, add the `*.ipynb` pattern to your
[`extend-include`](settings.md#extend-include) setting, like so:

=== "pyproject.toml"

```toml
[tool.ruff]
extend-include = ["*.ipynb"]
```

=== "ruff.toml"

```toml
extend-include = ["*.ipynb"]
```

This will prompt Ruff to discover Jupyter Notebook (`.ipynb`) files in any specified
directories, then lint and format them accordingly.

Alternatively, pass the notebook file(s) to `ruff` on the command-line directly. For example,
`ruff check /path/to/notebook.ipynb` will always lint `notebook.ipynb`. Similarly,
`ruff format /path/to/notebook.ipynb` will always format `notebook.ipynb`.
Ruff has built-in support for linting and formatting [Jupyter Notebooks](https://jupyter.org/). Refer to the
[Jupyter Notebook section](configuration.md#jupyter-notebook-discovery) for more details.

Ruff also integrates with [nbQA](https://github.com/nbQA-dev/nbQA), a tool for running linters and
code formatters over Jupyter Notebooks.
Expand Down
Loading

0 comments on commit 73160dc

Please sign in to comment.