## Summary
Fixes part of #15874: UP032 emitted broken fixes for str.format calls
with no f-string equivalent. Non-identifier attributes, quote-clashing
or backslash keys, unknown conversions, and mixed numbering are now
skipped. Signed indices like {+0} were a parser bug (leading +
accepted), which also affected F523/F524
## Test Plan
New mdtest at crates/ruff_linter/resources/mdtest/pyupgrade/f-string.md
plus parser unit tests
Canonical equivalence can make distinct string values look the same. ty
escapes their displays to distinguish them, but the current approach
relies on ICU4X's deprecated `NfcInert` property, which [ICU4X plans to
remove](https://github.com/unicode-org/icu4x/issues/7892), and can
escape characters that do not need escaping.
Replace inertness checks with NFC checks on runs of unescaped text. When
adding a character would make a run non-NFC, escape that character and
continue checking after it. The resulting string literal is in NFC and
evaluates to the original value, so canonically equivalent values remain
distinguishable.
For example, `\u212b` (the Angstrom sign) is canonically equivalent to
`Å`. ty now keeps the unrelated `é` visible while still distinguishing
the two values:
```py
reveal_type("éÅ")
# main: Literal["éÅ"]
# this PR: Literal["éÅ"]
reveal_type("é\u212b")
# main: Literal["\xe9\u212b"]
# this PR: Literal["é\u212b"]
```
The change also uses Unicode grapheme-break properties to keep
characters visually separate from the quotes and escape sequences around
them. It escapes the braille blank because it can look like an ordinary
space.
This PR contains the following updates:
| Package | Type | Update | Change |
|---|---|---|---|
| [icu_normalizer](https://icu4x.unicode.org)
([source](https://redirect.github.com/unicode-org/icu4x)) |
workspace.dependencies | minor | `2.2.0` → `2.3.0` |
---
### Configuration
📅 **Schedule**: (UTC)
- Branch creation
- "before 4am on Wednesday"
- Automerge
- At any time (no schedule defined)
🚦 **Automerge**: Disabled by config. Please merge this manually once you
are satisfied.
♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the
rebase/retry checkbox.
🔕 **Ignore**: Close this PR and you won't be reminded about this update
again.
---
- [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check
this box
---
This PR was generated by [Mend Renovate](https://mend.io/renovate/).
View the [repository job
log](https://developer.mend.io/github/astral-sh/ruff).
<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiI0NC4xMzQuMSIsInVwZGF0ZWRJblZlciI6IjQ0LjEzNC4xIiwidGFyZ2V0QnJhbmNoIjoibWFpbiIsImxhYmVscyI6WyJpbnRlcm5hbCJdfQ==-->
---------
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Co-authored-by: Brent Westbrook <brentrwestbrook@gmail.com>
## Summary
Noticed case below with possible false positive/negative when handling
`%b` format character by rules
https://docs.astral.sh/ruff/rules/bad-string-format-character/ and
https://docs.astral.sh/ruff/rules/bad-string-format-type/
```python
# False negative: not flagged by bad-string-format-character
# Runtime: ValueError: unsupported format character 'b' (0x62) at index 7
a = "hello %b" % 25
# False positive: bad-string-format-type
# Runtime: ValueError: unsupported format character 'b' (0x62) at index 7
# It should report nothing, since the problem is not mismatching
# formatter and provided value, but use of invalid format character in general.
a = "hello %b" % "23"
```
This fix resolves both issues - first case is now reported and second
results in no diagnostic.
The root cause for was `bad-string-format-character` issue was
`CFormatString` parser always parsing `%b` as `Bytes` type, while for
bytes literals it's actually just an invalid character. Added
`CFormatContext` enum, so parser can parse `%b` only when invoked from
`CFormatBytes` and not from `CFormatString` and report issue when finds
`%b` in string literals formatters.
Second issue was caused by `bad-string-format-type` assuming `%b` is
allowed only for integers, so it was reporting any other type as a
mismatch. Now it just ignores `%b`, since it will be handled by
`bad-string-format-character`.
https://github.com/astral-sh/ruff/blob/17a00de2e298612201a8fe30790e9399204af1b9/crates/ruff_linter/src/rules/pylint/rules/bad_string_format_type.rs#L100
For completeness I also refactored `FormatType::from` in that rule to
rely on `CFormatType` variants instead of hardcoding formatter
characters It really just made it more clean without any functional
changes:
- `Ascii` is now handled explicitly as `Repr`, instead of assuming it's
unknown
- dropped non-existing formatters `n` and `%`, they are present from the
original implementation https://github.com/astral-sh/ruff/pull/2572 and
are not valid formatters (possibly `%` is artifact from old cformat
parser quirks).
## Test Plan
Added tests for both rules, updated snapshots. All previous tests pass
too.
## Summary
This PR uses Hawk 0.1.11 to narrow internal declarations to the
visibility they actually need.
Here, I've intentionally _only_ made visibility changes; unused API
removals are split into follow-up PRs. In total, 1,201 declaration
visibility modifiers are narrowed. Public generated formatter APIs and
other declarations whose narrower visibility would activate dead-code or
API-shape lints remain unchanged.
## Summary
Publish Ruff's production Rust crates to crates.io as part of the
existing release workflow. This follows uv's setup: workspace
dependencies carry registry-compatible version pins, internal crates
receive generated READMEs and an explicit versioning policy, CI dry-runs
the full workspace publish, and release jobs publish missing crate
versions idempotently through crates.io trusted publishing.
The publishing set contains 32 packages: Ruff's CLI, WASM package,
server, linter, formatter, parser, and the internal Ruff component
crates required by them. It also includes `ty_module_resolver`,
`ty_site_packages`, and `ty_static`, because those three crates are part
of Ruff's normal dependency closure.
The publishing set deliberately excludes:
- Test-only crates: `mdtest`, `ruff_mdtest`,
`ruff_python_ast_integration_tests`, and
`ruff_python_trivia_integration_tests`.
- Benchmark and development crates: `ruff_benchmark` and `ruff_dev`.
- All remaining `ty_*` crates.
These packages retain `publish = false`, so bootstrap, version bumping,
README generation, CI dry-runs, and release publishing skip them through
Cargo metadata instead of a separate hard-coded allowlist.
The crates.io job is intentionally independent of release announcement,
so a crates.io outage cannot block the existing binary, PyPI, npm, or
GitHub release outputs. A bootstrap script reserves new crate names,
configures the release workflow as each crate's trusted publisher,
enables trusted-publishing-only releases, and records configured crates
for idempotent reruns.
The full workspace publish dry-run packages and verifies all 32 selected
crates.
Closes https://github.com/astral-sh/ruff/issues/43.
**Summary:**
Fixes UP032 autofix incorrectly converting raw strings with `\N{...}` to
f-strings, which changes semantics and causes runtime errors.
Fixes#22060
## Test Plan
- Added test case for raw strings with \N{...}
- Regular strings with \N{...} still autofix correctly
- All 119 pyupgrade tests pass
---------
Co-authored-by: Brent Westbrook <brentrwestbrook@gmail.com>
## Summary
Fixes https://github.com/astral-sh/ruff/issues/19771
Fixes incorrect parsing of Unicode named escape sequences like `Hey
\N{snowman}` in `FormatString`, which were being incorrectly split into
separate literal and field parts instead of being treated as a single
literal unit.
## Problem
The `FormatString` parser incorrectly handles Unicode named escape
sequences:
- **Current**: `Hey \N{snowman}` is parsed into 2 parts `Literal("Hey
\N")` & `Field("snowman")`
- **Expected**: `Hey \N{snowman}` should be parsed into 1 part
`Literal("Hey \N{snowman}")`
This affects f-string conversion rules when fixing `UP032` that rely on
proper format string parsing.
## Solution
I modified `parse_literal` to detect and handle Unicode named escape
sequences before parsing single characters:
- Introduced a flag to track when a backslash is "available" to escape
something.
- When the flag is `true`, and the text starts with `N{`, try to parse
the complete Unicode escape sequence as one unit, and set the flag to
`false` after parsing successfully.
- Set the flag to `false` when the backslash is already consumed.
## Manual Verification
`"\N{angle}AOB = {angle}°".format(angle=180)`
**Result**
```bash
def foo():
- "\N{angle}AOB = {angle}°".format(angle=180)
+ f"\N{angle}AOB = {180}°"
Would fix 1 error.
```
`"\N{snowman} {snowman}".format(snowman=1)`
**Result**
```bash
def foo():
- "\N{snowman} {snowman}".format(snowman=1)
+ f"\N{snowman} {1}"
Would fix 1 error.
```
`"\\N{snowman} {snowman}".format(snowman=1)`
**Result**
```bash
def foo():
- "\\N{snowman} {snowman}".format(snowman=1)
+ f"\\N{1} {1}"
Would fix 1 error.
```
## Test Plan
- Added test cases (happy case, invalid case, edge case) for
`FormatString` when parsing Unicode escape sequence.
- Updated snapshots.
## Summary
This is a follow-up to #15726, #15778, and #15794 to preserve the triple
quote and prefix flags in plain strings, bytestrings, and f-strings.
I also added a `StringLiteralFlags::without_triple_quotes` method to
avoid passing along triple quotes in rules like SIM905 where it might
not make sense, as discussed
[here](https://github.com/astral-sh/ruff/pull/15726#discussion_r1930532426).
## Test Plan
Existing tests, plus many new cases in the `generator::tests::quote`
test that should cover all combinations of quotes and prefixes, at least
for simple string bodies.
Closes#7799 when combined with #15694, #15726, #15778, and #15794.
---------
Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
## Summary
I used `cargo-shear` (see
[tweet](https://twitter.com/boshen_c/status/1770106165923586395)) to
remove some unused dependencies that `cargo udeps` wasn't reporting.
<!-- What's the purpose of the change? What does it do, and why? -->
## Test Plan
`cargo test`
## Summary
This PR modifies our `Cargo.toml` files to use workspace dependencies
for _all_ dependencies, rather than the status quo of sporadically
trying to use workspace dependencies for those dependencies that are
used across multiple crates. I find the current situation more confusing
and harder to manage, since we have a mix of workspace and crate-local
dependencies, whereas this setup consistently uses the same approach for
all dependencies.
This PR removes several uses of `unsafe`. I generally limited myself to
low hanging fruit that I could see. There are still a few remaining uses
of `unsafe` that looked a bit more difficult to remove (if possible at
all). But this gets rid of a good chunk of them.
I put each `unsafe` removal into its own commit with a justification for
why I did it. So I would encourage reviewing this PR commit-by-commit.
That way, we can legislate them independently. It's no problem to drop a
commit if we feel the `unsafe` should stay in that case.
Update to [Rust
1.74](https://blog.rust-lang.org/2023/11/16/Rust-1.74.0.html) and use
the new clippy lints table.
The update itself introduced a new clippy lint about superfluous hashes
in raw strings, which got removed.
I moved our lint config from `rustflags` to the newly stabilized
[workspace.lints](https://doc.rust-lang.org/stable/cargo/reference/workspaces.html#the-lints-table).
One consequence is that we have to `unsafe_code = "warn"` instead of
"forbid" because the latter now actually bans unsafe code:
```
error[E0453]: allow(unsafe_code) incompatible with previous forbid
--> crates/ruff_source_file/src/newlines.rs:62:17
|
62 | #[allow(unsafe_code)]
| ^^^^^^^^^^^ overruled by previous forbid
|
= note: `forbid` lint level was set on command line
```
---------
Co-authored-by: Charlie Marsh <charlie.r.marsh@gmail.com>
## Summary
This is a follow-up to #7469 that attempts to achieve similar gains, but
without introducing malachite. Instead, this PR removes the `BigInt`
type altogether, instead opting for a simple enum that allows us to
store small integers directly and only allocate for values greater than
`i64`:
```rust
/// A Python integer literal. Represents both small (fits in an `i64`) and large integers.
#[derive(Clone, PartialEq, Eq, Hash)]
pub struct Int(Number);
#[derive(Debug, Clone, PartialEq, Eq, Hash)]
pub enum Number {
/// A "small" number that can be represented as an `i64`.
Small(i64),
/// A "large" number that cannot be represented as an `i64`.
Big(Box<str>),
}
impl std::fmt::Display for Number {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
match self {
Number::Small(value) => write!(f, "{value}"),
Number::Big(value) => write!(f, "{value}"),
}
}
}
```
We typically don't care about numbers greater than `isize` -- our only
uses are comparisons against small constants (like `1`, `2`, `3`, etc.),
so there's no real loss of information, except in one or two rules where
we're now a little more conservative (with the worst-case being that we
don't flag, e.g., an `itertools.pairwise` that uses an extremely large
value for the slice start constant). For simplicity, a few diagnostics
now show a dedicated message when they see integers that are out of the
supported range (e.g., `outdated-version-block`).
An additional benefit here is that we get to remove a few dependencies,
especially `num-bigint`.
## Test Plan
`cargo test`
Closes#6767
Replaces https://github.com/astral-sh/ruff/pull/6773 (this cherry-picks
some parts from there)
Alternative to the approach introduced in #6616 which added support for
placeholders in format specifications while retaining parsing of other
format specification parts.
The idea is that if there are placeholders in a format specification we
will not attempt to glean semantic meaning from the other parts of the
format specification we'll just extract all of the placeholders ignoring
other characters. The dynamic content of placeholders can drastically
change the meaning of the format specification in ways unknowable by
static analysis. This change prevents false analysis and will ensure
safety if we build other rules on top of this at the cost of missing
detection of some bad specifications.
Minor note: I've use "replacements" and "placeholders" interchangeably
but am trying to go with "placeholder" as I think it's a better term for
the static analysis concept here