Skip to content

Commit 6ea8916

Browse files
ezyangfacebook-github-bot
authored andcommittedAug 31, 2020
Rewrite of ATen code generator (pytorch#42629)
Summary: Pull Request resolved: pytorch#42629 How to approach reviewing this diff: - The new codegen itself lives in `tools/codegen`. Start with `gen.py`, then read `model.py` and them the `api/` folder. The comments at the top of the files describe what is going on. The CLI interface of the new codegen is similar to the old one, but (1) it is no longer necessary to explicitly specify cwrap inputs (and now we will error if you do so) and (2) the default settings for source and install dir are much better; to the extent that if you run the codegen from the root source directory as just `python -m tools.codegen.gen`, something reasonable will happen. - The old codegen is (nearly) entirely deleted; every Python file in `aten/src/ATen` was deleted except for `common_with_cwrap.py`, which now permanently finds its home in `tools/shared/cwrap_common.py` (previously cmake copied the file there), and `code_template.py`, which now lives in `tools/codegen/code_template.py`. We remove the copying logic for `common_with_cwrap.py`. - All of the inputs to the old codegen are deleted. - Build rules now have to be adjusted to not refer to files that no longer exist, and to abide by the (slightly modified) CLI. - LegacyTHFunctions files have been generated and checked in. We expect these to be deleted as these final functions get ported to ATen. The deletion process is straightforward; just delete the functions of the ones you are porting. There are 39 more functions left to port. Signed-off-by: Edward Z. Yang <[email protected]> Test Plan: Imported from OSS Reviewed By: bhosmer Differential Revision: D23183978 Pulled By: ezyang fbshipit-source-id: 6073ba432ad182c7284a97147b05f0574a02f763
1 parent 576880f commit 6ea8916

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

49 files changed

+8579
-4024
lines changed
 

‎.circleci/scripts/cpp_doc_push_script.sh

+2-7
Original file line numberDiff line numberDiff line change
@@ -47,16 +47,11 @@ sudo apt-get -y install doxygen
4747
# Generate ATen files
4848
pushd "${pt_checkout}"
4949
pip install -r requirements.txt
50-
time python aten/src/ATen/gen.py \
50+
time python -m tools.codegen.gen \
5151
-s aten/src/ATen \
52-
-d build/aten/src/ATen \
53-
aten/src/ATen/Declarations.cwrap \
54-
aten/src/THCUNN/generic/THCUNN.h \
55-
aten/src/ATen/nn.yaml \
56-
aten/src/ATen/native/native_functions.yaml
52+
-d build/aten/src/ATen
5753

5854
# Copy some required files
59-
cp aten/src/ATen/common_with_cwrap.py tools/shared/cwrap_common.py
6055
cp torch/_utils_internal.py tools/shared
6156

6257
# Generate PyTorch files

‎.github/workflows/lint.yml

+2-6
Original file line numberDiff line numberDiff line change
@@ -131,13 +131,9 @@ jobs:
131131
time python setup.py --cmake-only build
132132
133133
# Generate ATen files.
134-
time python aten/src/ATen/gen.py \
134+
time python -m tools.codegen.gen \
135135
-s aten/src/ATen \
136-
-d build/aten/src/ATen \
137-
aten/src/ATen/Declarations.cwrap \
138-
aten/src/THCUNN/generic/THCUNN.h \
139-
aten/src/ATen/nn.yaml \
140-
aten/src/ATen/native/native_functions.yaml
136+
-d build/aten/src/ATen
141137
142138
# Generate PyTorch files.
143139
time python tools/setup_helpers/generate_code.py \

0 commit comments

Comments
 (0)
Please sign in to comment.