1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
|
# C++ / Python API parity tracker
## torch::nn
API | Implementation Parity | Doc Parity
------------- | ------------- | -------------
torch::nn::Sequential|Yes|No
torch::nn::ModuleList|Yes|No
torch::nn::ModuleDict|No|No
torch::nn::ParameterList|No|No
torch::nn::ParameterDict|No|No
torch::nn::Conv1d|Yes|No
torch::nn::Conv2d|Yes|No
torch::nn::Conv3d|Yes|No
torch::nn::ConvTranspose1d|Yes|No
torch::nn::ConvTranspose2d|Yes|No
torch::nn::ConvTranspose3d|Yes|No
torch::nn::Unfold|Yes|No
torch::nn::Fold|Yes|No
torch::nn::MaxPool1d|Yes|No
torch::nn::MaxPool2d|Yes|No
torch::nn::MaxPool3d|Yes|No
torch::nn::MaxUnpool1d|Yes|No
torch::nn::MaxUnpool2d|Yes|No
torch::nn::MaxUnpool3d|Yes|No
torch::nn::AvgPool1d|Yes|No
torch::nn::AvgPool2d|Yes|No
torch::nn::AvgPool3d|Yes|No
torch::nn::FractionalMaxPool2d|Yes|No
torch::nn::FractionalMaxPool3d|Yes|No
torch::nn::LPPool1d|Yes|No
torch::nn::LPPool2d|Yes|No
torch::nn::AdaptiveMaxPool1d|Yes|No
torch::nn::AdaptiveMaxPool2d|Yes|No
torch::nn::AdaptiveMaxPool3d|Yes|No
torch::nn::AdaptiveAvgPool1d|Yes|No
torch::nn::AdaptiveAvgPool2d|Yes|No
torch::nn::AdaptiveAvgPool3d|Yes|No
torch::nn::ReflectionPad1d|Yes|No
torch::nn::ReflectionPad2d|Yes|No
torch::nn::ReflectionPad3d|Yes|No
torch::nn::ReplicationPad1d|Yes|No
torch::nn::ReplicationPad2d|Yes|No
torch::nn::ReplicationPad3d|Yes|No
torch::nn::ZeroPad2d|Yes|No
torch::nn::ConstantPad1d|Yes|No
torch::nn::ConstantPad2d|Yes|No
torch::nn::ConstantPad3d|Yes|No
torch::nn::ELU|Yes|No
torch::nn::Hardshrink|Yes|No
torch::nn::Hardtanh|Yes|No
torch::nn::LeakyReLU|Yes|No
torch::nn::LogSigmoid|Yes|No
torch::nn::Mish|Yes|No
torch::nn::MultiheadAttention|No|No
torch::nn::PReLU|Yes|No
torch::nn::ReLU|Yes|No
torch::nn::ReLU6|Yes|No
torch::nn::RReLU|Yes|No
torch::nn::SELU|Yes|No
torch::nn::CELU|Yes|No
torch::nn::GELU|Yes|No
torch::nn::SiLU|Yes|No
torch::nn::Sigmoid|Yes|No
torch::nn::Softplus|Yes|No
torch::nn::Softshrink|Yes|No
torch::nn::Softsign|Yes|No
torch::nn::Tanh|Yes|No
torch::nn::Tanhshrink|Yes|No
torch::nn::Threshold|Yes|No
torch::nn::GLU|Yes|No
torch::nn::Softmin|Yes|No
torch::nn::Softmax|Yes|No
torch::nn::Softmax2d|Yes|No
torch::nn::LogSoftmax|Yes|No
torch::nn::AdaptiveLogSoftmaxWithLoss|Yes|No
torch::nn::BatchNorm1d|Yes|No
torch::nn::BatchNorm2d|Yes|No
torch::nn::BatchNorm3d|Yes|No
torch::nn::GroupNorm|Yes|No
torch::nn::SyncBatchNorm|No|No
torch::nn::InstanceNorm1d|Yes|No
torch::nn::InstanceNorm2d|Yes|No
torch::nn::InstanceNorm3d|Yes|No
torch::nn::LayerNorm|Yes|No
torch::nn::LocalResponseNorm|Yes|No
torch::nn::CrossMapLRN2d|Yes|No
torch::nn::RNN|Yes|No
torch::nn::LSTM|Yes|No
torch::nn::GRU|Yes|No
torch::nn::RNNCell|Yes|No
torch::nn::LSTMCell|Yes|No
torch::nn::GRUCell|Yes|No
torch::nn::Transformer|Yes|No
torch::nn::TransformerEncoder|No|No
torch::nn::TransformerDecoder|No|No
torch::nn::TransformerEncoderLayer|Yes|No
torch::nn::TransformerDecoderLayer|Yes|No
torch::nn::Identity|Yes|No
torch::nn::Linear|Yes|No
torch::nn::Bilinear|Yes|No
torch::nn::Flatten|Yes|No
torch::nn::Unflatten|Yes|No
torch::nn::Dropout|Yes|No
torch::nn::Dropout2d|Yes|No
torch::nn::Dropout3d|Yes|No
torch::nn::AlphaDropout|Yes|No
torch::nn::FeatureAlphaDropout|Yes|No
torch::nn::Embedding|Yes|No
torch::nn::EmbeddingBag|Yes|No
torch::nn::CosineSimilarity|Yes|No
torch::nn::PairwiseDistance|Yes|No
torch::nn::L1Loss|Yes|No
torch::nn::MSELoss|Yes|No
torch::nn::CrossEntropyLoss|Yes|No
torch::nn::CTCLoss|Yes|No
torch::nn::NLLLoss|Yes|No
torch::nn::PoissonNLLLoss|Yes|No
torch::nn::KLDivLoss|Yes|No
torch::nn::BCELoss|Yes|No
torch::nn::BCEWithLogitsLoss|Yes|No
torch::nn::MarginRankingLoss|Yes|No
torch::nn::HingeEmbeddingLoss|Yes|No
torch::nn::MultiLabelMarginLoss|Yes|No
torch::nn::SmoothL1Loss|Yes|No
torch::nn::HuberLoss|Yes|No
torch::nn::SoftMarginLoss|Yes|No
torch::nn::MultiLabelSoftMarginLoss|Yes|No
torch::nn::CosineEmbeddingLoss|Yes|No
torch::nn::MultiMarginLoss|Yes|No
torch::nn::TripletMarginLoss|Yes|No
torch::nn::PixelShuffle|Yes|No
torch::nn::PixelUnshuffle|Yes|No
torch::nn::Upsample|Yes|No
torch::nn::DataParallel|No|No
torch::nn::parallel::DistributedDataParallel|No|No
torch::nn::utils::clip_grad_norm_|Yes|No
torch::nn::utils::clip_grad_value_|Yes|No
torch::nn::utils::parameters_to_vector|Yes|No
torch::nn::utils::vector_to_parameters|Yes|No
torch::nn::utils::weight_norm|No|No
torch::nn::utils::remove_weight_norm|No|No
torch::nn::utils::spectral_norm|No|No
torch::nn::utils::remove_spectral_norm|No|No
torch::nn::utils::rnn::PackedSequence|Yes|No
torch::nn::utils::rnn::pack_padded_sequence|Yes|No
torch::nn::utils::rnn::pad_packed_sequence|Yes|No
torch::nn::utils::rnn::pad_sequence|Yes|No
torch::nn::utils::rnn::pack_sequence|Yes|No
torch::nn::SampleModule|Yes|Yes
## torch::nn::functional
API | Implementation Parity | Doc Parity
------------- | ------------- | -------------
F::conv1d|Yes|No
F::conv2d|Yes|No
F::conv3d|Yes|No
F::conv_transpose1d|Yes|No
F::conv_transpose2d|Yes|No
F::conv_transpose3d|Yes|No
F::unfold|Yes|No
F::fold|Yes|No
F::avg_pool1d|Yes|No
F::avg_pool2d|Yes|No
F::avg_pool3d|Yes|No
F::max_pool1d|Yes|No
F::max_pool2d|Yes|No
F::max_pool3d|Yes|No
F::max_unpool1d|Yes|No
F::max_unpool2d|Yes|No
F::max_unpool3d|Yes|No
F::lp_pool1d|Yes|No
F::lp_pool2d|Yes|No
F::adaptive_max_pool1d|Yes|No
F::adaptive_max_pool2d|Yes|No
F::adaptive_max_pool3d|Yes|No
F::adaptive_avg_pool1d|Yes|No
F::adaptive_avg_pool2d|Yes|No
F::adaptive_avg_pool3d|Yes|No
F::threshold|Yes|No
F::relu|Yes|No
F::hardtanh|Yes|No
F::relu6|Yes|No
F::elu|Yes|No
F::selu|Yes|No
F::celu|Yes|No
F::leaky_relu|Yes|No
F::prelu|Yes|No
F::rrelu|Yes|No
F::glu|Yes|No
F::gelu|Yes|No
F::silu|Yes|No
F::mish|Yes|No
F::logsigmoid|Yes|No
F::hardshrink|Yes|No
F::tanhshrink|Yes|No
F::softsign|Yes|No
F::softplus|Yes|No
F::softmin|Yes|No
F::softmax|Yes|No
F::softshrink|Yes|No
F::gumbel_softmax|Yes|No
F::log_softmax|Yes|No
F::batch_norm|Yes|No
F::instance_norm|Yes|No
F::layer_norm|Yes|No
F::local_response_norm|Yes|No
F::normalize|Yes|No
F::linear|Yes|No
F::bilinear|Yes|No
F::dropout|Yes|No
F::alpha_dropout|Yes|No
F::dropout2d|Yes|No
F::dropout3d|Yes|No
F::embedding|Yes|No
F::embedding_bag|Yes|No
F::one_hot|Yes|No
F::pairwise_distance|Yes|No
F::cosine_similarity|Yes|No
F::pdist|Yes|No
F::binary_cross_entropy|Yes|No
F::binary_cross_entropy_with_logits|Yes|No
F::poisson_nll_loss|Yes|No
F::cosine_embedding_loss|Yes|No
F::cross_entropy|Yes|No
F::ctc_loss|Yes|No
F::hinge_embedding_loss|Yes|No
F::kl_div|Yes|No
F::l1_loss|Yes|No
F::mse_loss|Yes|No
F::margin_ranking_loss|Yes|No
F::multilabel_margin_loss|Yes|No
F::multilabel_soft_margin_loss|Yes|No
F::multi_margin_loss|Yes|No
F::nll_loss|Yes|No
F::smooth_l1_loss|Yes|No
F::huber_loss|Yes|No
F::soft_margin_loss|Yes|No
F::triplet_margin_loss|Yes|No
F::pixel_shuffle|Yes|No
F::pad|Yes|No
F::interpolate|Yes|No
F::grid_sample|Yes|No
F::affine_grid|Yes|No
F::sample_functional|Yes|No
|
