Files
Darren Schroeder 973c590918 Fix the bugs found by the nu-conformance open questions (#19271)
## Description

The [nu-conformance](https://github.com/Fikoko/nu-conformance) suite
(#19206) recorded 35 behaviors as open questions. This PR fixes the ones
that are bugs, plus five that needed a decision (T2, T4, T8, T12 and
T13). The IDs are the question IDs in that repo's
[`open-questions.md`](https://github.com/Fikoko/nu-conformance/blob/master/open-questions.md).

### Before and after

| | Code | main | branch |
|---|---|---|---|
| S2 | `"\xFF"` | error `nu::parser::parse_mismatch`: expected any shape
| error `nu::parser::error`: invalid UTF-8 from `\x` escapes in string |
| S3 | `r#'abc` | error `nu::parser::unexpected_eof`: expected closing
'# | error `nu::parser::unclosed_delimiter`: expected `'#` here |
| T1 | `inf == 1` | `true` | `false` |
| T1 | `inf > 1` | `false` | `true` |
| T1 | `[1.0 inf] \| math max` | `1.0` | `inf` |
| T2 | `let x = 5; -$x` | `-$x` | `-5` |
| T2 | `let x = 5; 1 + -$x` | error
`nu::parser::operator_incompatible_types`: string | `-4` |
| T2 | `let x = 5sec; -$x` | `-$x` | `-5sec` |
| T2 | `-(2 + 3) \| describe` | `string` | `int` |
| T2 | `-(1)px` | `-1px` | `-1px` |
| T3 | `-0x10` | `-0x10` | `-16` |
| T3 | `+0b11` | error `nu::shell::external_command`: Command `+0b11`
not found | `3` |
| T4 | `9223372036854775808` | `9223372036854775808.0` | error
`nu::parser::error`: number too large (add `.0` for a float) in int |
| T4 | `9223372036854775808.0` | `9223372036854775808.0` |
`9223372036854775808.0` |
| T6 | `[INF Nan Infinity] \| to nuon` | `[inf, NaN, inf]` | `["INF",
"Nan", "Infinity"]` |
| T6 | `[inf nan NaN] \| to nuon` | `[inf, NaN, NaN]` | `[inf, NaN,
NaN]` |
| T7 | `9223372036854775808ns` | `15250wk 1day 23hr 47min 16sec 854ms
775µs 807ns` | error `nu::parser::literal_too_large`: duration too large
|
| T7 | `8EiB` | `9.2 EB` | error `nu::parser::literal_too_large`:
filesize too large |
| T8 | `[2023-02-29] \| to nuon` | `["2023-02-29"]` | error
`nu::parser::error`: not a valid date in datetime |
| T8 | `[2023-02-29.log] \| to nuon` | `["2023-02-29.log"]` |
`["2023-02-29.log"]` |
| T10 | `+5sec` | error `nu::shell::external_command`: Command `+5sec`
not found | `5sec` |
| T11 | `[1 2]] \| to nuon` | `[1, "2]"]` | error
`nu::parser::unbalanced_delimiter`: unexpected `]` (unbalanced with `[`)
|
| T12 | `{true: 1, null: 2} \| to nuon` | error
`nu::parser::parse_mismatch_with_full_string_msg`: expected string |
`{"true": 1, "null": 2}` |
| T12 | `[[1 2]; [3 4]] \| to nuon` | error `nu::parser::error`: must be
a string | `[["1", "2"]; [3, 4]]` |
| T13 | `print before; {a: 1, a: 2}` | error
`nu::shell::column_defined_twice`: field first defined here | error
`nu::parser::column_defined_twice`: field redefined here |
| O1 | `not "a" == "b"` | error `nu::shell::cant_convert`: can't convert
string to boolean | `true` |
| O1 | `let x = 1; not $x == 2` | error `nu::shell::cant_convert`: can't
convert int to boolean | `true` |
| O3 | `(-9223372036854775808) mod -1` | error
`nu::shell::operator_overflow`: modulo operation overflowed | `0` |
| O4 | `1 in [1.0]` | error `nu::parser::operator_incompatible_types`:
list<float> | `true` |
| D1 | `const x = 1; unlet $x; $x` | `1` | error
`nu::compile::invalid_literal`: constants cannot be deleted |
| D2 | `unlet $in` | `nothing` | error `nu::compile::invalid_literal`:
'$in' is a built-in variable and cannot be deleted |
| D3 | `mut x = 0; [1 2 3] \| where {\|v\| $x += 1; true } \| ignore;
$x` | `0` | error `nu::parser::expected_keyword`: capture of mutable
variable |
| F2 | `while false { } \| describe` | `nothing` | error
`nu::parser::unexpected_keyword`: not allowed in pipeline |
| F3 | `if 1 { }` | error `nu::shell::cant_convert`: can't convert int
to boolean | error `nu::shell::type_mismatch`: expected bool |
| F4 | `2.0 in 1.0..2.0` | `false` | `true` |
| F4 | `2.0 in 1.0..<2.0` | `true` | `false` |
| C1 | `def f [...r, a] { {r: $r, a: $a} }; f 1 2 3 \| to nuon` | `{r:
[2, 3], a: 1}` | error `nu::parser::positional_after_rest`: parameter a
after the rest parameter |
| C2 | `def f [a, a] { $a }; f 1 2` | `2` | error
`nu::parser::duplicate_parameter`: `a` uses a name that is already taken
|
| C2 | `def f [--x, --x] { $x }; f --x` | `false` | error
`nu::parser::duplicate_parameter`: `--x` uses a name that is already
taken |
| C2 | `def f [--help] { 'mine' }; f --help \| lines \| first` |
`Usage:` | error `nu::parser::error`: every command already has `--help`
|

C3, a script that calls two deprecated commands:

```nushell
$ nu two_deprecated.nu    # main
Warning: nu::parser::deprecated
   : `-- g is deprecated and will be removed in a future release.
```
```nushell
$ nu two_deprecated.nu    # branch
Warning: nu::parser::deprecated
   : `-- g is deprecated and will be removed in a future release.
Warning: nu::parser::deprecated
   : `-- k is deprecated and will be removed in a future release.
```
## User-facing changes (Release notes)

### Breaking changes

- `not` now has the precedence `help operators` lists: `not $a == $b`
means `not ($a == $b)`. Before, `not` applied to the next value only.
- `-$x` and `-(expr)` negate their value, also as a flag's value or an
argument after `--`. Before, `-$x` was the string `"-$x"` and `-(2 + 3)`
the string `"-5"`. Arguments to external commands are unchanged.
- An integer literal too large for 64 bits, such as
`9223372036854775808`, is an error. Add `.0` to write a float.
- Filesize and duration literals that don't fit are a parse error
(`nu::parser::literal_too_large`) instead of saturating to the largest
value.
- A word written like a date that isn't a valid date, such as
`2023-02-29`, is an error.
- `INF`, `Nan`, `Infinity` and other spellings of special floats are
strings. Use `inf`, `nan` or `NaN`. `from nuon` reads them the same way.
- A record or table literal that names a field twice is a parse error.
- `[1 2]]` is an error instead of `[1, "2]"]`.
- In a custom command, repeated parameter or flag names, repeated short
flags, a parameter after `...rest`, and a `--help` flag are errors.
- `while` and `loop` can't be piped, like `for`.
- `unlet` errors on constants and on `$in`.
- Assigning to a `mut` variable inside a `where` condition is an error;
the assignment never took effect.

### Fixes

- `inf` compares as larger than every finite number, and `[1.0 inf] |
math max` is `inf`.
- `2.0 in 1.0..2.0` is `true`, and `2.0 in 1.0..<2.0` is `false`.
- `-0x10`, `+0b11`, `+5sec` and `+1kB` are numbers.
- `1 in [1.0]` is `true`.
- `{true: 1}` and `[[1 2]; [3 4]]` parse, with the keys `"true"`, `"1"`
and `"2"`.
- `(-9223372036854775808) mod -1` is `0`.
- `if 1 { }` reports the same type mismatch as `while 1 { }`, and an
`if` or `while` condition that fails while it runs reports its own
error.
- `9223372036854775807ns` and `9223372036854775807b` are exact, and
integer filesize and duration literals no longer lose precision.
- A script or `nu -c` command shows every parse warning once, not only
the first.
- Clearer errors for invalid UTF-8 from `\x` escapes and for unclosed
raw strings.

## Additional notes
2026-10-17 14:54:47 -05:00

234 lines
6.8 KiB
Rust

use crate::repl::tests::{TestResult, fail_test, run_test};
use nu_test_support::prelude::*;
use pretty_assertions::assert_matches;
use rstest::rstest;
#[test]
fn cjk_in_substrings() -> TestResult {
run_test(
"let s = '[Rust 程序设计语言](title-page.md)'; let start = ($s | str index-of '('); let end = ($s | str index-of ')'); $s | str substring ($start + 1)..<($end)",
"title-page.md",
)
}
#[test]
fn string_not_in_string() -> TestResult {
run_test("'d' not-in 'abc'", "true")
}
#[test]
fn string_in_string() -> TestResult {
run_test("'z' in 'abc'", "false")
}
#[test]
fn non_string_in_string() -> TestResult {
fail_test("42 in 'abc'", "nu::parser::operator_incompatible_types")
}
#[test]
fn string_in_record() -> TestResult {
run_test(r#""a" in ('{ "a": 13, "b": 14 }' | from json)"#, "true")
}
#[test]
fn non_string_in_record() -> TestResult {
fail_test(
r#"4 in ('{ "a": 13, "b": 14 }' | from json)"#,
"nu::shell::operator_incompatible_types",
)
}
#[test]
fn unbalance_string() -> TestResult {
fail_test(r#""aaaab"cc"#, "invalid characters")?;
fail_test("'aaaab'cc", "invalid characters")?;
fail_test(r#"$"aaaab"cc"#, "invalid characters")?;
fail_test("$'aaaab'cc", "invalid characters")
}
#[test]
fn string_in_valuestream() -> TestResult {
run_test(
r#"
'Hello' in ("Hello
World" | lines)"#,
"true",
)
}
#[test]
fn single_tick_interpolation() -> TestResult {
run_test("$'(3 + 4)'", "7")
}
#[test]
fn unclosed_interpolation_subexpression() -> TestResult {
fail_test("$\"foo (2 + 3\"", "expected `)`")
}
#[test]
fn detect_newlines() -> TestResult {
run_test("'hello\r\nworld' | lines | get 0 | str length", "5")
}
#[test]
fn case_insensitive_sort() -> TestResult {
run_test(
"[a, B, d, C, f] | sort -i | to json --raw",
"[\"a\",\"B\",\"C\",\"d\",\"f\"]",
)
}
#[test]
fn case_insensitive_sort_columns() -> TestResult {
run_test(
r#"[[version, package]; ["two", "Abc"], ["three", "abc"], ["four", "abc"]] | sort-by -i package version | to json --raw"#,
r#"[{"version":"four","package":"abc"},{"version":"three","package":"abc"},{"version":"two","package":"Abc"}]"#,
)
}
#[test]
fn raw_string() -> TestResult {
run_test(r#"r#'abcde""fghi"''''jkl'#"#, r#"abcde""fghi"''''jkl"#)?;
run_test(r#"r##'abcde""fghi"''''#jkl'##"#, r#"abcde""fghi"''''#jkl"#)?;
run_test(
r#"r###'abcde""fghi"'''##'#jkl'###"#,
r#"abcde""fghi"'''##'#jkl"#,
)?;
run_test("r#''#", "")?;
run_test(
"r#'a string with sharp inside # and ends with #'#",
"a string with sharp inside # and ends with #",
)
}
#[test]
fn raw_string_inside_parentheses() -> TestResult {
let (left, right) = ('(', ')');
run_test(
&format!(r#"{left}r#'abcde""fghi"''''jkl'#{right}"#),
r#"abcde""fghi"''''jkl"#,
)?;
run_test(
&format!(r#"{left}r##'abcde""fghi"''''#jkl'##{right}"#),
r#"abcde""fghi"''''#jkl"#,
)?;
run_test(
&format!(r#"{left}r###'abcde""fghi"'''##'#jkl'###{right}"#),
r#"abcde""fghi"'''##'#jkl"#,
)?;
run_test(&format!("{left}r#''#{right}"), "")?;
run_test(
&format!("{left}r#'a string with sharp inside # and ends with #'#{right}"),
"a string with sharp inside # and ends with #",
)
}
#[test]
fn raw_string_inside_list() -> TestResult {
let (left, right) = ('[', ']');
run_test(
&format!(r#"{left}r#'abcde""fghi"''''jkl'#{right} | get 0"#),
r#"abcde""fghi"''''jkl"#,
)?;
run_test(
&format!(r#"{left}r##'abcde""fghi"''''#jkl'##{right} | get 0"#),
r#"abcde""fghi"''''#jkl"#,
)?;
run_test(
&format!(r#"{left}r###'abcde""fghi"'''##'#jkl'###{right} | get 0"#),
r#"abcde""fghi"'''##'#jkl"#,
)?;
run_test(&format!("{left}r#''#{right} | get 0"), "")?;
run_test(
&format!("{left}r#'a string with sharp inside # and ends with #'#{right} | get 0"),
"a string with sharp inside # and ends with #",
)
}
#[test]
fn raw_string_inside_closure() -> TestResult {
let (left, right) = ('{', '}');
run_test(
&format!(r#"do {left}r#'abcde""fghi"''''jkl'#{right}"#),
r#"abcde""fghi"''''jkl"#,
)?;
run_test(
&format!(r#"do {left}r##'abcde""fghi"''''#jkl'##{right}"#),
r#"abcde""fghi"''''#jkl"#,
)?;
run_test(
&format!(r#"do {left}r###'abcde""fghi"'''##'#jkl'###{right}"#),
r#"abcde""fghi"'''##'#jkl"#,
)?;
run_test(&format!("do {left}r#''#{right}"), "")?;
run_test(
&format!("do {left}r#'a string with sharp inside # and ends with #'#{right}"),
"a string with sharp inside # and ends with #",
)
}
#[test]
fn incomplete_string() -> TestResult {
fail_test("r#abc", "expected '")?;
fail_test("r#'bc", "expected `'#`")?;
fail_test("'ab\"", "expected `'`")?;
fail_test("\"ab'", "expected `\"`")?;
fail_test(
r#"def func [] {
{
"A": ""B" # <- the quote is bad
}
}
"#,
"expected `\"`",
)
}
/// Text after a string's last quote is an error.
#[rstest]
#[case::single_then_bare("'a'b")]
#[case::single_then_double(r#"'a'"b""#)]
fn text_after_closing_quote(#[case] code: &str) -> Result {
test()
.run(code)
.expect_error_code_eq("nu::parser::extra_token_after_closing_delimiter")
}
/// A quoted word runs to its last quote of the same kind, and the quotes of that kind inside it
/// are part of the string. Scripts rely on this for JSON and HTML text.
#[rstest]
#[case::single("'a''b'", "a''b")]
#[case::double(r#""[{a: "b"}]""#, r#"[{a: "b"}]"#)]
#[case::interpolation(r#"let c = 'x'; $"<p class="($c)">""#, r#"<p class="x">"#)]
fn same_quotes_inside_a_string(#[case] code: &str, #[case] expected: &str) -> Result {
test().run(code).expect_value_eq(expected)
}
/// An escaped quote and a quote inside an interpolated subexpression don't close the string.
#[rstest]
#[case::escaped_quote(r#""a\"b""#, "a\"b")]
#[case::subexpression_quote(r#"$"a(")")b""#, "a)b")]
fn quotes_that_do_not_close(#[case] code: &str, #[case] expected: &str) -> Result {
test().run(code).expect_value_eq(expected)
}
/// `\x` escapes add raw bytes, which must form valid UTF-8.
#[test]
fn hex_escapes_invalid_utf8() -> Result {
let err = test().run(r#""\xFF""#).expect_parse_error()?;
assert_matches!(err, ParseError::InvalidLiteral(msg, ..) if msg.contains("UTF-8"));
test().run(r#""\xC3\xA9""#).expect_value_eq("é")
}
/// An unclosed raw string reports the closer it needs, like any other unclosed string.
#[rstest]
#[case::one_hash("r#'abc", "'#")]
#[case::two_hashes("r##'abc", "'##")]
fn unclosed_raw_string(#[case] code: &str, #[case] closer: &str) -> Result {
let err = test().run(code).expect_parse_error()?;
assert_matches!(err, ParseError::Unclosed(delimiter, ..) if delimiter == closer);
Ok(())
}