SYNOPSIS

pytogo [-v] [-d] <python-source >go-source

DESCRIPTION

pytogo is a fast, crude, incomplete Python-to-Go translator meant to assist a human programmer in translating large volumes of Python to idiomatic Go, preserving comments and structure. It does the mechanical things that are easy for software but hard for humans, leaving a human free to concentrate on the hard parts like type annotations.

It exists because at the time it was written the only automated translators available either produced rather horrifyingly unidiomatic generated Go or wouldn’t translate standalone Python code to standalone Go, requiring a heavyweight support package.

One of the main thing this tool does is deduce open and close braces from Python indentation with ":" and leading whitespace, a tedious and error-prone task for humans.

It is conservative, never actually throwing away information from the input Python except where explicity noted below. But it assumes its Python input is well-formed, and can have point failures on inputs that are sufficiently weird.

Here are some things to do before running this tool. The early steps preserve the correctness of the Python.

  1. Have a strong test suite. Run it before starting, and after each preparation step to check that you still have a working Python program.

  2. If your code does not use a PEP8-standard 4-space indent, you should reformat it to do so before applying this tool; we recommend black(1). Tabs are treated as equivalent to 8 spaces. The output will retain 4-space indents and should be fixed up with "go fmt" or equivalent.

  3. Move any print statements you still have in Python 2 format (no parens around the argument) to Python 3 format. If your base code is Python 2, apply the "from future import print_function" special import.

  4. It is advisable to decorate your Python code with PEP484 function type annotations (which pytogo can use directly) before applying pytogo. Without these pytogo must fall back on its own rules, which are rather weak. Note that this implies giving up on Python 2 compatibility - you can remove the future import you may have added in an earlier step.

  5. Add explicit parentheses to compound expressions that don’t already have them. Python and Go have different operator-precedence rules, diverging around logical operators (&& ||) bitwise operators (&, |, ^) and relationals (< > <= >= == !=). If you make operation grouping explicit with parentheses you can avoid introducing subtle logic errors.

  6. Unroll list and set comprehensions to explicit for loops. By doing this early and applying your test suite you will verify that the logic is correct before translating it in a correctness-preserving way.

  7. Move any nested class definitions out to top level, the parser is not smart enough to handle these.

  8. In Python, when you iterate over a list of objects, the iteration variable is a reference and can be be used to modify fields in the current object on the list. In Go, when you iterate over a list of structs, the iteration variable is a value copy of the current struct

    • modifying it will not change the list contents. You’ll need to rewite such loops to index to the list elemant to be modified. Do this in Python before applying pytogo so you check correctness with your test suite.

  9. The following other Python constructions can also confuse the indent tracker: "with", "try/finally", and "try/except". Consider commenting out with, try and except lines with an indication of the indent level. (This step does not preserve correctness.)

Here are the supported transformations:

  • Opens Python block scopes with { and closes them with }.

  • Changes docstrings to comments. The first line of a function’s docstring is moved to the godoc-preferred position just above the function signature line.

  • Changes comments from #-led to //-led

  • Translates Python boolean constants True, False to Go true, false; alse Python None to Go nil.

  • Translates Python "in range" to Go ":= range". Some common for-loops using range that iterate over lists and maps are translated; others you will need to fix up by hand.

  • Maps Python single-quote literals to Go double-quote literals. Python r-prefix string literals and multiline strings become Go backtick literals. You’ll need to fix up backslash escapes in multiline literals yourself.

  • Translates Python logical connectives and or not to Go && || !.

  • Changes "def " to "func ".

  • "= 1" and "-= 1" become + and --.

  • A recognizable leading Python shebang line is converted to "package main".

  • Common cases of various Python string library methods - capitalize(), count(), endswith(), find(), join(), lower(), lstrip(), rfind(), replace(), rstrip(), split(), startswith(), split(), upper() - are translated to Go string library calls.

  • Some common Python standard library calls and variables, notably from os and os.filepath and sys, are translated into Go equivalents. The result is not guaranteed to be perfectly correct; a Python call that throws a signal on failure may map into a Go function with an error return. But it will always be a step in the right direction, and the Go compiler will respond with indicative error messages.

  • math and cmath library calls are also translated; Python float is assumed to map to Go float64 and complex to complex128. The Go compiler will refuse to do certain implicit conversions that Python takes for granted, and will throw errors; you must fix those up by hand.

  • The append() method of Python is translated to a call to Go’s append intrinsic.

  • Python’s "del foo[bar]" is translated to Go "delete(foo, bar)". Note that this can false-match on a list del with a numeric key; this will throw a compilation error in Go and you can hand-fix it.

  • Trailing line-continuation backslashes are removed.

  • Python try/finally/catch become pseudo-statements with delimited block scopes. You will need to fix these up by hand.

  • Single-line % expressions with a string-literal left hand and either tuple or single-variable right hands are translated into fmt.Sprintf calls. You will need to translate %-cookies by hand; %r → %v or %q is the usual trouble spot.

  • Many, but not necessarily all, cases where Go ":=" assignment can replace a plain "=" assignment in Python are translated.

  • If your function signatures have type hints in PEP484 format, pytogo will use them to create Go function signatures. pytogo knows that str should map to Go string, float to float64, and complex to complex128; other types it leaves alone.

  • In the absence of PEP484 hints, pytogo will deduce some return types. The rules for this are weak and only work on functions for which the type of an expression in a return statement can be deduced by pattern-matching on the syntax. They will fail when a function can return any of several types.

  • The Python "*args" form for variadics is translated as Go "args …​".

  • Import lines are massaged into a Go-like form. Some recognizable Python library names are either (a) mapped to the corresponding Go library (if there is one), or discarded (if there is not). The remainder are passed through unaltered and must be fixed up by hand.

  • Changes if statements testing for a literal string in an object reference to call the string.Contains() function. It does not, however, handle list or tuple literals on the right-hand side.

  • Python class method declarations get properly decorated with the classname attached to the "self" instance-variable placeholder. It’s generated in reference form, e.g. with a leading asterisk so the instance is modifiable; if the method requires only read access to the instance you can remove the asterisk.

  • Strips out slots declarations. This is the only case in which it might be information is thrown away, but these are just an optimization hack that doesn’t change code semantics.

  • Translates Python function, variable, and field names with embedded underscores into golint-friendly camelCased names by removing the embedded underscores and uppercasing the following letter. The logic for this avoids tricky cases where it could potentially do damage, so you might still get some golint warnings afterwards.

Things it doesn’t do:

Lexically unwrap methods of classes, or handle static class definitions. Know anything about decorators. Handle iterators, lambda, filter, comprehensions, raise statements, operator overloading, or context managers. Translate dictionary arguments, translate PEP484 type aliases or interpret PEP484 type-composition syntax. Translate Python backslash references in substitution strings to Go-style {} cookies.

This tool is probably best used by filtering sections of files for incremental conversion rather than trying to do entire files at once.

OPTIONS

-v

Report pytogo’s version and exit.

-d

Enable verbose debugging of translation stages.

BUGS

See the preparation steps above for Python cases that can trip you up. In addition to those, the following bugs are known:

If your Python code has a double-quoted string literal ending in "r", and there is another double-quoted string literal following it on the same line, the code for translating such literals will false-match and garble the code between the literals.

A "#" inside a Python docstring will be turned into a Go //.

If a docstring is immediately followed by Python comments it will turn into an ordinary Go comment.

The distinction between triple-doublquote and triple-singlequote literals is lost; both turn into Go backquote literals.

Report bugs to Eric S. Raymond <esr@thyrsus.com>. The project page is at http://catb.org/~esr/pytogo