The Sharpy Formatter¶

sharpyc format and the LSP's Format Document rewrite a file into one canonical layout. Both use the same formatter, so they produce the same text; Format Selection applies part of that text (see In the editor). This page states what formatting guarantees, and what it does when it cannot keep a guarantee.

Usage¶

sharpyc format file.spy             # rewrite file.spy in place
sharpyc format src/                 # every .spy file under src/
sharpyc format file.spy -o out.spy  # write the result to out.spy; file.spy is untouched
sharpyc format src/ --check         # CI mode: exit 1 if any file would change; nothing is written
sharpyc format file.spy --diff      # print a unified diff; nothing is written

Without -o, --check or --diff, the file is rewritten in place. The output always uses \n line endings: a file with \r\n line endings comes back with \n. The program is unaffected, because a line break inside a string literal is \n in either spelling.

Indentation is always 4 spaces¶

Sharpy indentation is exactly four spaces per level, and tabs are not allowed (Indentation). So the formatter always writes four spaces.

--indent 4 is accepted and changes nothing. --indent N with any other N, and --tabs, are usage errors: sharpyc exits with code 2 and touches no file.

$ sharpyc format indent.spy --indent 2
Error: --indent 2 is not supported: Sharpy indentation is exactly 4 spaces per level (docs/language_specification/indentation.md).

The LSP ignores the editor's tabSize and insertSpaces settings for Sharpy documents.

What formatting guarantees¶

Formatting changes layout only. For every file it formats:

String values are untouched. Every string literal, f-string and t-string denotes exactly the same text after formatting: its trailing spaces, its indentation and the blank lines inside a triple-quoted string are all kept. The formatter may re-spell a string's quotes and escapes in a canonical form without changing what it denotes: '''x''' is written "x", and '\x41' is written "A". Replacement fields ({x!r:>4}) are written back exactly as written.
Every comment survives, in its place. The output's comments are the input's comments, by text and in order. Each one stays where it was written: at the end of its line, before the statement it preceded, on a decorator or a clause header (elif, else, except, finally, case), or inside the block it closes. An end-of-block comment stays inside that block, at the block's indentation, even when several blocks close at once. A comment-only line carries no indentation meaning, so it is written at the indentation of the code it belongs to. Trailing spaces after a comment are removed; they are not part of the comment.
Every backtick escape survives. A name written with backticks (`class`, `count`) is written back with them at every position: parameters, keyword arguments, imports and their aliases, except … as, patterns, member access, decorators and type names. This matters for meaning, not just spelling: obj.`count` and obj.count can bind different .NET members.
The program is the same. The output parses to the same syntax tree as the input, and compiles to the same C# apart from the #line directives, which record line numbers.
Formatting is idempotent. Formatting a formatted file changes nothing.

For example, this file

def scale(`class`: int) -> int:
    return `class` * 2
def main():
    x   =   scale(`class`=3)  # escaped keyword argument
    if x > 5:
        print("big")
        # still inside the if body
    else:  # the else header
        print("small")

formats (sharpyc format guarantees.spy -o formatted.spy) to

def scale(`class`: int) -> int:
    return `class` * 2


def main():
    x = scale(`class`=3)  # escaped keyword argument
    if x > 5:
        print("big")
        # still inside the if body
    else:  # the else header
        print("small")

and both files print big.

These guarantees are checked over the whole test corpus on every build: each fixture as written, each fixture with a comment added at every position, and each fixture with every name backtick-escaped (FormatterMeaningPreservationSweepTests, FormatterEmitInvarianceSweepTests).

Constructs with comments inside are written as you wrote them¶

A statement, a clause header (elif, except, case) or a decorator that has a comment inside it — inside its brackets, or inside an f-string replacement field — is written verbatim: exactly as it appears in the source, re-indented as a unit. Its continuation lines move with its first line. A continuation line that starts inside a multi-line string or f-string is never moved, because its leading spaces belong to the string.

def main():
        xs = [1,  # one
              2]
        print(xs)

formats (sharpyc format verbatim.spy -o verbatim_out.spy) to

def main():
    xs = [1,  # one
          2]
    print(xs)

Both print [1, 2].

Verbatim also means the layout inside the construct is kept: [1,2] stays [1,2] and is not rewritten to [1, 2]. The formatter has no comment-aware layout for bracketed code, so it does not re-lay out anything that carries a comment inside. A construct without inner comments is formatted normally, and its trailing comment stays on its line.

Canonical spellings¶

Some constructs have one written form, so formatting may change their spelling without changing the program. For example, an except clause that catches several exception types is always written with parentheses: except KeyError, IndexError: becomes except (KeyError, IndexError):.

Vertical spacing¶

The formatter writes blank lines by two rules.

Canonical spacing. Two blank lines come before a top-level definition (a function, class, struct, interface, enum, union or delegate), whatever precedes it: code, the module docstring or a comment. Two blank lines follow a top-level definition, whatever follows it, an import included. One blank line follows the module docstring before anything that is not a definition — a comment, an import, a statement. One blank line separates two items of a body when either is a method, property or nested definition, in every body. One blank line follows the import block.
Your blank lines. A blank line you wrote between two statements is kept as one blank line, however many you wrote. The same holds between two comments, two enum members, two union cases, and two case arms of a match. Canonical spacing is a minimum, not an addition: blank lines you wrote before a top-level definition still give two, never more.

Comments directly above a top-level definition, with no blank line between them and it, belong to the definition: its two blank lines go above them. A comment that a blank line separates from the definition belongs to the module instead: the definition gets its two blank lines directly above it, and the comment keeps the spacing above it that any statement would have. This is the layout black writes.

A blank line you wrote is never kept at the start of a body (before its first statement, docstring or comment), between a decorator and what it decorates, or at the start or end of the file. A blank line between a comment and the statement below it is kept as one blank line — a comment line between a decorator and its definition counts too: a blank line below that comment is kept. A blank line inside brackets is not kept, unless the construct is written as you wrote it because it has a comment inside (see the previous sections). A blank line inside a triple-quoted string is part of the string and is never changed (guarantee 1).

When the formatter declines: SPY0912¶

Before it writes anything, the formatter re-reads its own output and checks guarantees 2–4. If the output would drop or move a comment, drop a backtick escape, fail to parse, or change the program's structure, the formatter declines:

the file is left exactly as it was, and
error SPY0912 names the first problem.

$ sharpyc format declined.spy
Error: declined.spy: SPY0912: formatting declined: the output would not re-parse (first error at formatted line 9: SPY0104 Expected RightBracket, got Colon); the file was left unchanged

sharpyc format exits with code 2, and the file is byte-for-byte unchanged. A declined file has not been damaged.

SPY0912 always means a formatter bug: your file is valid and its meaning is safe, but the formatter could not lay it out without changing it. Please report it (see below). Known cases: a string containing a \r escape, a generic constraint intersection [T: A & B], and a with … as (a, *rest): target are declined, because the formatter cannot yet write them back unchanged (#2169).

In the editor (LSP)¶

Format Document uses the same formatter as sharpyc format and produces the same text.
If the formatter declines (SPY0912), the editor receives no edits and the document is unchanged, as with the CLI.
Format Selection formats the whole file and applies only the changes the selection touches. A change the selection touches is applied whole, so it can reach past the selection in either direction, to both ends of its run of changed lines, because part of a change can be wrong on its own. The text it would produce is checked the same way as the formatter's own output: if it would not parse, or would change the program, Format Selection applies nothing. The file keeps its line endings: a line outside a change keeps its own, the last line of a change keeps the ending it had, and every other line break a change writes is the file's first one — so a file with one kind of line ending keeps it, and a file that mixes them may see a changed line's ending become the file's first kind.
Format on type aligns the first line of a statement to its block's indentation, and nothing else: never a line that continues a statement (inside brackets, or after \), a comment line, or a line inside a string. It changes how far the line is indented, never which block it is in, so it leaves an unexpected indent alone and does not dedent an else: typed at its body's indentation. It applies the new indentation only if the file stays the same program; while the file does not parse, the indentation-only check below decides instead.
If the document does not parse, for example in the middle of typing, Format Document and Format Selection fall back to an indentation-only pass. It re-indents lines to four-space levels and changes nothing else. It applies its result only if every line keeps its text, every line inside a string is unchanged, every statement stays in the block it was in, and no new indentation error appears; otherwise it applies nothing. A line that belongs to no block, because it is indented with a tab or to a width no enclosing block uses, is the mistake the pass repairs: it moves to the level the editor guesses for it, which may not be the block you meant. This fallback is used only for documents that fail to parse, never for a document the formatter declined.
The indentation-only pass and format on type leave a comment-only line exactly as you wrote it, even when they re-indent the code around it. A comment has no indentation the lexer reads, so the editor cannot tell which block it belongs to while the file does not parse. After a repair, a comment can therefore sit at the old indentation: in a body repaired from 2 spaces to 4, its comments keep their 2 spaces. Format Document on a file that parses moves each comment-only line to the indentation of its code (guarantee 2).
While a bracket is left open — never closed before the end of the file, or still open at the end of a line the lexer could not read — the editor reads every line below the line that opened it as part of that bracket, so the indentation-only pass and format on type change none of them, and while any non-blank line follows the bracket they leave the line that opened it alone too. A bracket closed inside an unfinished string or backtick-delimited name counts as open, and so does a bracket opened on a line the lexer could not read, even when a later line closes it. Lines above the bracket can still be repaired or re-indented, but never in a way that puts the bracket's line in a different block, so in a file indented 8 spaces per level the block holding the bracket keeps its 8 spaces.
The line after a line the lexer could not read (one with a character such as $ that is not Sharpy) starts a new statement, so it can be repaired like any other line. The unreadable line may have opened a block the editor cannot see: when it has a : outside brackets, strings, backtick-delimited names and comments (if ready($):), or it ends inside an unfinished string or backtick-delimited name (if name == "abc:), the lines directly below it that are indented deeper are never moved out of that block, however deep they are. When the unreadable line continues a statement begun above it (for i in range(1, and then $): on the next line), it is that statement's line: the lines below are measured against the statement's first line, and its own indentation is only ever aligned with the block it continues.
If any line of the file is indented with whitespace other than spaces and tabs (a form feed, a vertical tab or a no-break space), the lexer cannot read that line's indentation, so format on type and the indentation-only pass change nothing until it is replaced with spaces.
While a triple-quoted string is unfinished, or cannot be read as a string, the editor cannot tell which lines are code. A string cannot be read as one when a stray """ above it has paired with its opening quotes, when its opening """ sits on a line the lexer could not read at all (a line indented with a tab, or to a width no enclosing block uses), or when an error earlier on its opening line (an unexpected character, an invalid escape, the f-string's own quote inside a format spec) made the lexer skip the rest of that line. Format on type and the indentation-only pass change nothing until the string can be read again.
The editor reads the whole file, however many errors it has. The compiler stops at 25 lexer errors, but the indentation-only pass and format on type still read every later line and every string in it.
A few re-paired strings look exactly like a string that is fine, so the editor cannot tell them apart: a stray """ whose partner sits inside a comment, two stray delimiters, an opening """ inside an unfinished short string or backtick-delimited name, or an unfinished short string holding a stray """ followed on its line by a complete string (t = '""" + "\n" while the closing ' is still to be typed). There the indentation-only pass and format on type may re-indent lines of the string. Fix the stray quote first, or check the edit.

Selecting the body of main (lines 4–5) in

def helper() -> int:
    return 2
def main():
    x   =   helper()
    print(x)

applies only the change inside the selection:

def helper() -> int:
    return 2
def main():
    x = helper()
    print(x)

Format Document would also add two blank lines before def main():. They are outside the selection, so Format Selection does not add them.

In this file, formatting re-indents both statements of main. Selecting line 2 alone applies nothing: re-indenting s = """ without print(s) would leave a file that does not parse, and the change print(s) needs lies past the string.

def main():
        s = """
abc
"""
        print(s)

Format on type on line 4 of

def main():
    x = 1
    if x > 0:
            print(x)

re-indents that line:

def main():
    x = 1
    if x > 0:
        print(x)

In this file, indented 8 spaces per level, format on type on line 4 applies nothing: re-indenting print(2) alone would move it out of the if block, and the program would print 2.

def main():
        if False:
                print(1)
                print(2)

Here a docstring has been opened in f but not closed yet. Its """ pairs with the one that opens s in g, so key: value is read as code. Format Document, Format Selection of line 6 and format on type on line 6 all apply nothing: the file stays exactly as written until the docstring is closed.

def f() -> int:
    """
    return 1
def g() -> str:
    s = """
        key: value
    """
    return s

Here a stray """ at the top of the file pairs with the one that opens s, so key: value is read as code, and the string's closing """ pairs with the quotes inside '"""'. Format Document, Format Selection of line 4 and format on type on line 4 all apply nothing:

"""
def main():
    s = """
      key: value
    """
    t = '"""'
    print(s, t)

Here the line that opens the string is indented two spaces, a width no block in the file uses, so the lexer skips the whole line and never sees its """. Format Document, Format Selection of line 3 and format on type on line 3 all apply nothing, so key: value keeps its eight spaces:

def main():
  s = """
        key: value
  """
  print(s)

Here xs = [1, is never closed, so the editor reads every line below it as part of the list. Format Document, Format Selection of line 5 and format on type on line 5 all apply nothing, and print(y) stays in the if block:

def main():
    xs = [1,
    y = 2
    if y:
        print(y)

Here the for header spans two lines and the lexer cannot read the second one ($):), so the editor cannot tell what block the header opens. Format Document, Format Selection of line 4 and format on type on line 4 all apply nothing, and print(i) stays in the for block:

def main():
    for i in range(1,
$):
        print(i)
    print(0)

Format on type on line 3 of

def main():
    x = $
      y = 1
    print(x)

repairs the line after the one the lexer could not read:

def main():
    x = $
    y = 1
    print(x)

In this file the body of f is indented six spaces, a width no block uses, so the file does not parse. Format Document repairs y = 1 and leaves the comment in main as it is:

def f():
      y = 1

def main():
    # say hello
    print("hello")

def f():
    y = 1

def main():
    # say hello
    print("hello")

A file with 25 or more lexer errors is read to its end: past the compiler's limit, the editor re-indents lines and keeps every string it can read exactly as it does in a file with fewer errors. The compiler itself still stops at 25 errors; sharpyc emit diagnostics --max-errors N lists the errors past the first 25.

Format Document keeps one blank line wherever you wrote blank lines between two statements, none at the start of a body or after a decorator, and gives a definition its two blank lines below a comment a blank line separates from it (see Vertical spacing):

class Greeter:

    @staticmethod

    def hello() -> str:
        return "hello"
# the entry point

def main():

    a = 1

    b = 2
    # section two


    print(a + b)
x = 1

y = 2

class Greeter:
    @staticmethod
    def hello() -> str:
        return "hello"


# the entry point


def main():
    a = 1

    b = 2
    # section two

    print(a + b)


x = 1

y = 2

Reporting a problem¶

If formatting declines a file (SPY0912), changes what a program prints, or moves or drops a comment or an escape, please open an issue at github.com/antonsynd/sharpy with:

the smallest input that shows it (the file, or a few lines from it),
the sharpyc format output (the SPY0912 message, or the -o result),
sharpyc --version.

Open formatter issues are tracked on #2169 (constructs the formatter declines). The guarantees on this page were established by #2062.