Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
181 commits
Select commit Hold shift + click to select a range
3a3a481
added informer to gitignore
elisim Dec 27, 2022
9f8e00d
added informer to gitignore
elisim Dec 27, 2022
965eeb6
WIP informer2020
elisim Jan 12, 2023
a0231bf
Merge branch 'add-informer2020-model' of https://github.com/elisim/tr…
elisim Jan 12, 2023
1fcecdc
Merge remote-tracking branch 'upstream/main' into add-informer2020-model
elisim Jan 13, 2023
ec34ab1
added checking that instantiate works
elisim Jan 13, 2023
f274b76
added config using gluonTS by kashif
elisim Jan 14, 2023
b35a8b2
WIP config
elisim Jan 15, 2023
27ebb73
adding informeConfig. need to remove FeatureEmbedder
elisim Jan 16, 2023
9d4ae99
done InformerConfig, but need to change the names
elisim Jan 16, 2023
1223ba5
Done informer model init. working on enc-dec
elisim Jan 16, 2023
a7d38dc
added things to address, after reading again enc-dec in the paper
elisim Jan 17, 2023
8853e93
done modeling - checking initialization work
elisim Jan 17, 2023
422cdf5
added informer to gitignore
elisim Dec 27, 2022
c1001f0
WIP informer2020
elisim Jan 12, 2023
8648e08
added checking that instantiate works
elisim Jan 13, 2023
5bfe844
added config using gluonTS by kashif
elisim Jan 14, 2023
b29cfa6
WIP config
elisim Jan 15, 2023
215b030
adding informeConfig. need to remove FeatureEmbedder
elisim Jan 16, 2023
d56a0cd
done InformerConfig, but need to change the names
elisim Jan 16, 2023
f68192d
Done informer model init. working on enc-dec
elisim Jan 16, 2023
ffb7833
added things to address, after reading again enc-dec in the paper
elisim Jan 17, 2023
4286dd3
done modeling - checking initialization work
elisim Jan 17, 2023
622a921
Merge branch 'add-informer2020-model' of https://github.com/elisim/tr…
elisim Jan 17, 2023
d278b80
moved enc-dec init to InformerEncoder/Decoder init
elisim Jan 17, 2023
9e66890
added 'init_std' to config, now model init works!
elisim Jan 17, 2023
d9d1a67
WIP conversion script, and added code sources
elisim Jan 21, 2023
eedd4ca
Merge remote-tracking branch 'upstream/main' into add-informer2020-model
elisim Jan 21, 2023
52de756
WIP conversion script: loading original informer pth works
elisim Jan 23, 2023
cb9a287
WIP conversion script: change defaults in the config
elisim Jan 23, 2023
2b8d7e4
WIP conversion script: supporting Informer input embedding
elisim Jan 24, 2023
3fa969a
WIP conversion script: added parameters for the informer embed
elisim Jan 24, 2023
49170d8
WIP conversion script: change dim_feedforward=2048
elisim Jan 25, 2023
585e40b
WIP conversion script: remove unused args for loading checkpoint
elisim Jan 25, 2023
824f48e
just cleaning up
elisim Jan 25, 2023
878bd79
DataEmbedding removed, after thinking with Kashif
elisim Jan 26, 2023
60ee685
Merge remote-tracking branch 'upstream/main' into add-informer2020-model
elisim Jan 26, 2023
1ce4310
working on forward pass
elisim Jan 26, 2023
4568c0c
WIP forward pass: trying to establish working batch for forward pass
elisim Jan 27, 2023
a31fc5a
cleaning and finalizing
elisim Jan 28, 2023
27045b8
adding HF names and docs
elisim Jan 28, 2023
c8a6cae
init after cleaning works
elisim Jan 28, 2023
0e8ffb6
WIP in tests
elisim Jan 28, 2023
83d39df
added docs for the informer specific args
elisim Jan 28, 2023
fdffeb8
fix style
kashif Jan 30, 2023
aab1e08
undo change
kashif Jan 30, 2023
f8a33c5
Merge branch 'huggingface:main' into add-informer2020-model
kashif Jan 30, 2023
4e73184
cleaning informer, now need to work only enc-dec
elisim Jan 30, 2023
f12c0b0
initial enc-dec classes
kashif Jan 30, 2023
48337b0
added encoder and decoder
kashif Jan 30, 2023
5374049
added todo
kashif Jan 30, 2023
dd3b46a
add todos for conv_layers
kashif Jan 30, 2023
6bf419e
added decoder docs from vanilla
elisim Jan 31, 2023
ed21221
added encoder docs from vanilla
elisim Jan 31, 2023
e4c7875
remove encoder decoder from the original informer
elisim Jan 31, 2023
645810a
removed AttentionLayer from the original paper
elisim Jan 31, 2023
2529b7c
removed TriangularCausalMask, same as decoder_attention_mask
elisim Jan 31, 2023
225af56
initial sparse attention
kashif Jan 31, 2023
f31ec98
use conv_layers
kashif Feb 1, 2023
778c2a2
fixed test_config test
elisim Feb 1, 2023
7363444
fix parenthesis when itearting zip(layers, conv_layers)
elisim Feb 1, 2023
86486d2
error found in prob attention, added sizes as comments
elisim Feb 1, 2023
606ca81
fix sizes
kashif Feb 1, 2023
b416de8
added proposal for q_reduce indexing, and remove unused
elisim Feb 3, 2023
3bfbb23
WIP ProbMask, and changed factor=2 for testing
elisim Feb 3, 2023
3ebec34
Merge remote-tracking branch 'upstream/main' into add-informer2020-model
elisim Feb 4, 2023
67e1964
Merge remote-tracking branch 'upstream/main' into add-informer2020-model
elisim Feb 7, 2023
11a081e
remove unused libs for this PR for creating the env
elisim Feb 7, 2023
efdbff6
Merge remote-tracking branch 'upstream/main' into add-informer2020-model
elisim Feb 10, 2023
bb70b1b
fix checking the attn_weights.size() after bmm
elisim Feb 10, 2023
588205b
Q_reduce: changed from torch.gather to simple slicing
elisim Feb 10, 2023
b3f595f
WIP calculate final attn_output
elisim Feb 10, 2023
ceef0ad
finish adding v_aggregated, attn_output ready
elisim Feb 11, 2023
db95fd8
changed tgt_len to u in attention_mask, need to fix the size error
elisim Feb 11, 2023
3946b0c
comment attention_mask for encoder, and fix if cond for v_agg
elisim Feb 12, 2023
d2db148
added ProbMask support (wip), removed old original code
elisim Feb 12, 2023
4477d42
finished ProbMask 😃
elisim Feb 12, 2023
fa131dc
Revert "remove unused libs for this PR for creating the env"
elisim Feb 13, 2023
19a3408
fixes
kashif Feb 15, 2023
84a4ba3
make style
kashif Feb 15, 2023
a48e661
fix initial tests
kashif Feb 15, 2023
a3b6ddb
fix more tests
kashif Feb 15, 2023
a141712
dry
kashif Feb 15, 2023
299d17f
make style
kashif Feb 15, 2023
36df283
remove unused files
kashif Feb 15, 2023
0b4220a
style
kashif Feb 15, 2023
e9e3c6e
added integration tests
kashif Feb 15, 2023
83657df
fix num_static_real_features
kashif Feb 15, 2023
c893ad8
fix header
kashif Feb 15, 2023
79a21e0
remove unused function
kashif Feb 15, 2023
db1a0e4
fix example
kashif Feb 15, 2023
ad96fdd
fix docs
kashif Feb 15, 2023
ea921d9
Update src/transformers/models/informer/configuration_informer.py
kashif Feb 15, 2023
6ae25e6
Update src/transformers/models/informer/modeling_informer.py
kashif Feb 15, 2023
fd046ba
Update src/transformers/models/informer/configuration_informer.py
kashif Feb 16, 2023
18751ae
Update src/transformers/models/informer/configuration_informer.py
kashif Feb 16, 2023
51f890d
Update src/transformers/models/informer/configuration_informer.py
kashif Feb 16, 2023
dc49ff7
Update src/transformers/models/informer/configuration_informer.py
kashif Feb 16, 2023
125ee05
fixes for reviewer
kashif Feb 16, 2023
a64162a
use prediction_length from model
kashif Feb 16, 2023
19e3279
fix style
kashif Feb 16, 2023
b316a5a
fixed informer.mdx
kashif Feb 16, 2023
35e4549
added to index
kashif Feb 16, 2023
31f8bd3
updated readme
kashif Feb 16, 2023
c26ba0e
undo
kashif Feb 16, 2023
8379d0c
make fix-copies
kashif Feb 16, 2023
a8846f5
typo
kashif Feb 16, 2023
217260d
fix copy
kashif Feb 16, 2023
1421ccd
added Informer to toctree
kashif Feb 17, 2023
2b0a26d
in order
kashif Feb 17, 2023
10ed995
fixed comments
kashif Feb 17, 2023
ea07cd9
remove unneeded new lines in docs
kashif Feb 17, 2023
8299745
make static real and cat optional
kashif Feb 17, 2023
4866362
fix use of distil conv layers
kashif Feb 17, 2023
8218d55
fixed integration test
kashif Feb 17, 2023
21e3d43
added checkpoint for convlayer
kashif Feb 21, 2023
a0204e0
Merge branch 'main' into add-informer2020-model
kashif Feb 22, 2023
c1a58eb
make fix-copies
kashif Feb 22, 2023
56578bf
updated from time series model
kashif Feb 22, 2023
32e94d1
make fix-copies
kashif Feb 22, 2023
29ba424
copy decoder
kashif Feb 22, 2023
19abd0b
fix unit tests
kashif Feb 22, 2023
39ce9fd
updated scaling config
kashif Feb 22, 2023
68e68c0
fix integration tests
kashif Feb 22, 2023
8ee1b10
IGNORE_NON_TESTED
kashif Feb 22, 2023
05856aa
IGNORE_NON_AUTO_CONFIGURED
kashif Feb 22, 2023
50fc144
IGNORE_NON_AUTO_CONFIGURED
kashif Feb 22, 2023
e7845e8
updated check configs
kashif Feb 22, 2023
b030bea
fix formatting
kashif Feb 22, 2023
308268b
undo change from time series
kashif Feb 23, 2023
f064080
prediction_length should not be None
kashif Feb 23, 2023
5086c4c
aliign with the blog: prettify ProbSparse and change attention_factor…
elisim Feb 24, 2023
d47c556
make style
elisim Feb 24, 2023
e08566f
Merge remote-tracking branch 'upstream/main' into add-informer2020-model
kashif Feb 24, 2023
5462fbd
make fix-copies
kashif Feb 24, 2023
ae22d55
Merge remote-tracking branch 'upstream/main' into add-informer2020-model
elisim Feb 25, 2023
4988b2f
niels CR: update contributed by
elisim Feb 25, 2023
c7be3cf
niels CR: update configuration_informer.py
elisim Feb 25, 2023
bb03fc6
niels CR: update kashif -> huggingface
elisim Feb 25, 2023
1e78712
niels CR: `sampling_factor` only relevant when `attention_type`=prob
elisim Feb 25, 2023
4d3ec7a
Merge branch 'add-informer2020-model' of https://github.com/elisim/tr…
elisim Feb 25, 2023
2afa495
make style
elisim Feb 25, 2023
193938f
fixed U_part: added multiplication by `L_Q`
elisim Feb 25, 2023
f064beb
fixed bug: remove `is not None` from `if config.distil`
elisim Feb 25, 2023
3db65bd
fixed test: `decoder_seq_length` to `encoder_seq_length` in cross_att…
elisim Feb 25, 2023
c4c6133
fix integration tests
kashif Feb 27, 2023
329647e
updated model hub
kashif Feb 27, 2023
582c867
do not shift as in training
kashif Feb 27, 2023
25867e2
undo
kashif Feb 27, 2023
7e8420e
fix make-copies
kashif Feb 28, 2023
aeff495
Merge remote-tracking branch 'upstream/main' into add-informer2020-model
kashif Mar 1, 2023
3074c73
make fix-copies
kashif Mar 1, 2023
e455bbf
Merge remote-tracking branch 'upstream/main' into add-informer2020-model
elisim Mar 1, 2023
b4cbddf
added `if prediction_length is None`
elisim Mar 1, 2023
b934cb8
changed `ProbSparseAttention` to `InformerProbSparseAttention`
elisim Mar 1, 2023
426d14f
changed `V_sum` -> `v_mean_dim_time`
elisim Mar 1, 2023
73d1840
changed `ConvLayer` to `InformerConvLayer` and fixed `super()`
elisim Mar 1, 2023
3bc6163
TimeSeriesTansformer->Informer in decoder's Copied from
elisim Mar 1, 2023
ba6a8c3
more descriptive in ProbSparse
elisim Mar 1, 2023
f4fde7a
make style
elisim Mar 1, 2023
65811d1
fix coped from
kashif Mar 1, 2023
18fd0f8
Revert "added `if prediction_length is None`"
kashif Mar 1, 2023
f9e3bee
fixed indent
kashif Mar 1, 2023
e13dfab
use InformerSinusoidalPositionalEmbedding
kashif Mar 2, 2023
beba0a0
Merge branch 'main' of https://github.com/huggingface/transformers in…
kashif Mar 2, 2023
b873c38
make fix-style
kashif Mar 2, 2023
f557964
fix from #21860
kashif Mar 2, 2023
d58f3ab
fix name
kashif Mar 2, 2023
e1b746a
Merge branch 'main' of https://github.com/huggingface/transformers in…
kashif Mar 2, 2023
3bf819d
make fix-copies
kashif Mar 2, 2023
3703738
use time series utils
kashif Mar 6, 2023
240ac6a
fix dec num_heads
kashif Mar 6, 2023
c638f24
docstring
kashif Mar 6, 2023
9608217
added time series util doc
kashif Mar 6, 2023
0bc09de
_import_structure
kashif Mar 6, 2023
bd724fc
formatting
kashif Mar 6, 2023
db6e0bd
changes from review
kashif Mar 6, 2023
b14aa34
make style
kashif Mar 6, 2023
e36f6c0
fix docs
kashif Mar 6, 2023
0ac82bb
fix doc
kashif Mar 6, 2023
e5eff8a
removed NegativeLogLikelihood
kashif Mar 6, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -348,6 +348,7 @@ Current number of checkpoints: ![](https://img.shields.io/endpoint?url=https://h
1. **[Hubert](https://huggingface.co/docs/transformers/model_doc/hubert)** (from Facebook) released with the paper [HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Units](https://arxiv.org/abs/2106.07447) by Wei-Ning Hsu, Benjamin Bolte, Yao-Hung Hubert Tsai, Kushal Lakhotia, Ruslan Salakhutdinov, Abdelrahman Mohamed.
1. **[I-BERT](https://huggingface.co/docs/transformers/model_doc/ibert)** (from Berkeley) released with the paper [I-BERT: Integer-only BERT Quantization](https://arxiv.org/abs/2101.01321) by Sehoon Kim, Amir Gholami, Zhewei Yao, Michael W. Mahoney, Kurt Keutzer.
1. **[ImageGPT](https://huggingface.co/docs/transformers/model_doc/imagegpt)** (from OpenAI) released with the paper [Generative Pretraining from Pixels](https://openai.com/blog/image-gpt/) by Mark Chen, Alec Radford, Rewon Child, Jeffrey Wu, Heewoo Jun, David Luan, Ilya Sutskever.
1. **[Informer](https://huggingface.co/docs/transformers/main/model_doc/informer)** (from Beihang University, UC Berkeley, Rutgers University, SEDD Company) released with the paper [Informer: Beyond Efficient Transformer for Long Sequence Time-Series Forecasting](https://arxiv.org/abs/2012.07436) by Haoyi Zhou, Shanghang Zhang, Jieqi Peng, Shuai Zhang, Jianxin Li, Hui Xiong, and Wancai Zhang.
1. **[Jukebox](https://huggingface.co/docs/transformers/model_doc/jukebox)** (from OpenAI) released with the paper [Jukebox: A Generative Model for Music](https://arxiv.org/pdf/2005.00341.pdf) by Prafulla Dhariwal, Heewoo Jun, Christine Payne, Jong Wook Kim, Alec Radford, Ilya Sutskever.
1. **[LayoutLM](https://huggingface.co/docs/transformers/model_doc/layoutlm)** (from Microsoft Research Asia) released with the paper [LayoutLM: Pre-training of Text and Layout for Document Image Understanding](https://arxiv.org/abs/1912.13318) by Yiheng Xu, Minghao Li, Lei Cui, Shaohan Huang, Furu Wei, Ming Zhou.
1. **[LayoutLMv2](https://huggingface.co/docs/transformers/model_doc/layoutlmv2)** (from Microsoft Research Asia) released with the paper [LayoutLMv2: Multi-modal Pre-training for Visually-Rich Document Understanding](https://arxiv.org/abs/2012.14740) by Yang Xu, Yiheng Xu, Tengchao Lv, Lei Cui, Furu Wei, Guoxin Wang, Yijuan Lu, Dinei Florencio, Cha Zhang, Wanxiang Che, Min Zhang, Lidong Zhou.
Expand Down
1 change: 1 addition & 0 deletions README_es.md
Original file line number Diff line number Diff line change
Expand Up @@ -341,6 +341,7 @@ Número actual de puntos de control: ![](https://img.shields.io/endpoint?url=htt
1. **[Hubert](https://huggingface.co/docs/transformers/model_doc/hubert)** (from Facebook) released with the paper [HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Units](https://arxiv.org/abs/2106.07447) by Wei-Ning Hsu, Benjamin Bolte, Yao-Hung Hubert Tsai, Kushal Lakhotia, Ruslan Salakhutdinov, Abdelrahman Mohamed.
1. **[I-BERT](https://huggingface.co/docs/transformers/model_doc/ibert)** (from Berkeley) released with the paper [I-BERT: Integer-only BERT Quantization](https://arxiv.org/abs/2101.01321) by Sehoon Kim, Amir Gholami, Zhewei Yao, Michael W. Mahoney, Kurt Keutzer.
1. **[ImageGPT](https://huggingface.co/docs/transformers/model_doc/imagegpt)** (from OpenAI) released with the paper [Generative Pretraining from Pixels](https://openai.com/blog/image-gpt/) by Mark Chen, Alec Radford, Rewon Child, Jeffrey Wu, Heewoo Jun, David Luan, Ilya Sutskever.
1. **[Informer](https://huggingface.co/docs/transformers/main/model_doc/informer)** (from Beihang University, UC Berkeley, Rutgers University, SEDD Company) released with the paper [Informer: Beyond Efficient Transformer for Long Sequence Time-Series Forecasting](https://arxiv.org/abs/2012.07436) by Haoyi Zhou, Shanghang Zhang, Jieqi Peng, Shuai Zhang, Jianxin Li, Hui Xiong, and Wancai Zhang.
1. **[Jukebox](https://huggingface.co/docs/transformers/model_doc/jukebox)** (from OpenAI) released with the paper [Jukebox: A Generative Model for Music](https://arxiv.org/pdf/2005.00341.pdf) by Prafulla Dhariwal, Heewoo Jun, Christine Payne, Jong Wook Kim, Alec Radford, Ilya Sutskever.
1. **[LayoutLM](https://huggingface.co/docs/transformers/model_doc/layoutlm)** (from Microsoft Research Asia) released with the paper [LayoutLM: Pre-training of Text and Layout for Document Image Understanding](https://arxiv.org/abs/1912.13318) by Yiheng Xu, Minghao Li, Lei Cui, Shaohan Huang, Furu Wei, Ming Zhou.
1. **[LayoutLMv2](https://huggingface.co/docs/transformers/model_doc/layoutlmv2)** (from Microsoft Research Asia) released with the paper [LayoutLMv2: Multi-modal Pre-training for Visually-Rich Document Understanding](https://arxiv.org/abs/2012.14740) by Yang Xu, Yiheng Xu, Tengchao Lv, Lei Cui, Furu Wei, Guoxin Wang, Yijuan Lu, Dinei Florencio, Cha Zhang, Wanxiang Che, Min Zhang, Lidong Zhou.
Expand Down
1 change: 1 addition & 0 deletions README_hd.md
Original file line number Diff line number Diff line change
Expand Up @@ -313,6 +313,7 @@ conda install -c huggingface transformers
1. **[Hubert](https://huggingface.co/docs/transformers/model_doc/hubert)** (फेसबुक से) साथ में पेपर [ह्यूबर्ट: सेल्फ सुपरवाइज्ड स्पीच रिप्रेजेंटेशन लर्निंग बाय मास्क्ड प्रेडिक्शन ऑफ हिडन यूनिट्स](https ://arxiv.org/abs/2106.07447) वेई-निंग सू, बेंजामिन बोल्टे, याओ-हंग ह्यूबर्ट त्साई, कुशाल लखोटिया, रुस्लान सालाखुतदीनोव, अब्देलरहमान मोहम्मद द्वारा।
1. **[I-BERT](https://huggingface.co/docs/transformers/model_doc/ibert)** (बर्कले से) साथ में कागज [I-BERT: Integer-only BERT Quantization](https:// arxiv.org/abs/2101.01321) सेहून किम, अमीर घोलमी, ज़ेवेई याओ, माइकल डब्ल्यू महोनी, कर्ट केटज़र द्वारा।
1. **[ImageGPT](https://huggingface.co/docs/transformers/model_doc/imagegpt)** (from OpenAI) released with the paper [Generative Pretraining from Pixels](https://openai.com/blog/image-gpt/) by Mark Chen, Alec Radford, Rewon Child, Jeffrey Wu, Heewoo Jun, David Luan, Ilya Sutskever.
1. **[Informer](https://huggingface.co/docs/transformers/main/model_doc/informer)** (from Beihang University, UC Berkeley, Rutgers University, SEDD Company) released with the paper [Informer: Beyond Efficient Transformer for Long Sequence Time-Series Forecasting](https://arxiv.org/abs/2012.07436) by Haoyi Zhou, Shanghang Zhang, Jieqi Peng, Shuai Zhang, Jianxin Li, Hui Xiong, and Wancai Zhang.
1. **[Jukebox](https://huggingface.co/docs/transformers/model_doc/jukebox)** (from OpenAI) released with the paper [Jukebox: A Generative Model for Music](https://arxiv.org/pdf/2005.00341.pdf) by Prafulla Dhariwal, Heewoo Jun, Christine Payne, Jong Wook Kim, Alec Radford, Ilya Sutskever.
1. **[LayoutLM](https://huggingface.co/docs/transformers/model_doc/layoutlm)** (from Microsoft Research Asia) released with the paper [LayoutLM: Pre-training of Text and Layout for Document Image Understanding](https://arxiv.org/abs/1912.13318) by Yiheng Xu, Minghao Li, Lei Cui, Shaohan Huang, Furu Wei, Ming Zhou.
1. **[LayoutLMv2](https://huggingface.co/docs/transformers/model_doc/layoutlmv2)** (from Microsoft Research Asia) released with the paper [LayoutLMv2: Multi-modal Pre-training for Visually-Rich Document Understanding](https://arxiv.org/abs/2012.14740) by Yang Xu, Yiheng Xu, Tengchao Lv, Lei Cui, Furu Wei, Guoxin Wang, Yijuan Lu, Dinei Florencio, Cha Zhang, Wanxiang Che, Min Zhang, Lidong Zhou.
Expand Down
1 change: 1 addition & 0 deletions README_ja.md
Original file line number Diff line number Diff line change
Expand Up @@ -375,6 +375,7 @@ Flax、PyTorch、TensorFlowをcondaでインストールする方法は、それ
1. **[Hubert](https://huggingface.co/docs/transformers/model_doc/hubert)** (Facebook から) Wei-Ning Hsu, Benjamin Bolte, Yao-Hung Hubert Tsai, Kushal Lakhotia, Ruslan Salakhutdinov, Abdelrahman Mohamed から公開された研究論文: [HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Units](https://arxiv.org/abs/2106.07447)
1. **[I-BERT](https://huggingface.co/docs/transformers/model_doc/ibert)** (Berkeley から) Sehoon Kim, Amir Gholami, Zhewei Yao, Michael W. Mahoney, Kurt Keutzer から公開された研究論文: [I-BERT: Integer-only BERT Quantization](https://arxiv.org/abs/2101.01321)
1. **[ImageGPT](https://huggingface.co/docs/transformers/model_doc/imagegpt)** (OpenAI から) Mark Chen, Alec Radford, Rewon Child, Jeffrey Wu, Heewoo Jun, David Luan, Ilya Sutskever から公開された研究論文: [Generative Pretraining from Pixels](https://openai.com/blog/image-gpt/)
1. **[Informer](https://huggingface.co/docs/transformers/main/model_doc/informer)** (from Beihang University, UC Berkeley, Rutgers University, SEDD Company) released with the paper [Informer: Beyond Efficient Transformer for Long Sequence Time-Series Forecasting](https://arxiv.org/abs/2012.07436) by Haoyi Zhou, Shanghang Zhang, Jieqi Peng, Shuai Zhang, Jianxin Li, Hui Xiong, and Wancai Zhang.
1. **[Jukebox](https://huggingface.co/docs/transformers/model_doc/jukebox)** (OpenAI から) Prafulla Dhariwal, Heewoo Jun, Christine Payne, Jong Wook Kim, Alec Radford, Ilya Sutskever から公開された研究論文: [Jukebox: A Generative Model for Music](https://arxiv.org/pdf/2005.00341.pdf)
1. **[LayoutLM](https://huggingface.co/docs/transformers/model_doc/layoutlm)** (Microsoft Research Asia から) Yiheng Xu, Minghao Li, Lei Cui, Shaohan Huang, Furu Wei, Ming Zhou から公開された研究論文: [LayoutLM: Pre-training of Text and Layout for Document Image Understanding](https://arxiv.org/abs/1912.13318)
1. **[LayoutLMv2](https://huggingface.co/docs/transformers/model_doc/layoutlmv2)** (Microsoft Research Asia から) Yang Xu, Yiheng Xu, Tengchao Lv, Lei Cui, Furu Wei, Guoxin Wang, Yijuan Lu, Dinei Florencio, Cha Zhang, Wanxiang Che, Min Zhang, Lidong Zhou から公開された研究論文: [LayoutLMv2: Multi-modal Pre-training for Visually-Rich Document Understanding](https://arxiv.org/abs/2012.14740)
Expand Down
1 change: 1 addition & 0 deletions README_ko.md
Original file line number Diff line number Diff line change
Expand Up @@ -290,6 +290,7 @@ Flax, PyTorch, TensorFlow 설치 페이지에서 이들을 conda로 설치하는
1. **[Hubert](https://huggingface.co/docs/transformers/model_doc/hubert)** (Facebook 에서) Wei-Ning Hsu, Benjamin Bolte, Yao-Hung Hubert Tsai, Kushal Lakhotia, Ruslan Salakhutdinov, Abdelrahman Mohamed 의 [HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Units](https://arxiv.org/abs/2106.07447) 논문과 함께 발표했습니다.
1. **[I-BERT](https://huggingface.co/docs/transformers/model_doc/ibert)** (Berkeley 에서) Sehoon Kim, Amir Gholami, Zhewei Yao, Michael W. Mahoney, Kurt Keutzer 의 [I-BERT: Integer-only BERT Quantization](https://arxiv.org/abs/2101.01321) 논문과 함께 발표했습니다.
1. **[ImageGPT](https://huggingface.co/docs/transformers/model_doc/imagegpt)** (OpenAI 에서) Mark Chen, Alec Radford, Rewon Child, Jeffrey Wu, Heewoo Jun, David Luan, Ilya Sutskever 의 [Generative Pretraining from Pixels](https://openai.com/blog/image-gpt/) 논문과 함께 발표했습니다.
1. **[Informer](https://huggingface.co/docs/transformers/main/model_doc/informer)** (from Beihang University, UC Berkeley, Rutgers University, SEDD Company) released with the paper [Informer: Beyond Efficient Transformer for Long Sequence Time-Series Forecasting](https://arxiv.org/abs/2012.07436) by Haoyi Zhou, Shanghang Zhang, Jieqi Peng, Shuai Zhang, Jianxin Li, Hui Xiong, and Wancai Zhang.
1. **[Jukebox](https://huggingface.co/docs/transformers/model_doc/jukebox)** (OpenAI 에서) Prafulla Dhariwal, Heewoo Jun, Christine Payne, Jong Wook Kim, Alec Radford, Ilya Sutskever 의 [Jukebox: A Generative Model for Music](https://arxiv.org/pdf/2005.00341.pdf) 논문과 함께 발표했습니다.
1. **[LayoutLM](https://huggingface.co/docs/transformers/model_doc/layoutlm)** (Microsoft Research Asia 에서) Yiheng Xu, Minghao Li, Lei Cui, Shaohan Huang, Furu Wei, Ming Zhou 의 [LayoutLM: Pre-training of Text and Layout for Document Image Understanding](https://arxiv.org/abs/1912.13318) 논문과 함께 발표했습니다.
1. **[LayoutLMv2](https://huggingface.co/docs/transformers/model_doc/layoutlmv2)** (Microsoft Research Asia 에서) Yang Xu, Yiheng Xu, Tengchao Lv, Lei Cui, Furu Wei, Guoxin Wang, Yijuan Lu, Dinei Florencio, Cha Zhang, Wanxiang Che, Min Zhang, Lidong Zhou 의 [LayoutLMv2: Multi-modal Pre-training for Visually-Rich Document Understanding](https://arxiv.org/abs/2012.14740) 논문과 함께 발표했습니다.
Expand Down
1 change: 1 addition & 0 deletions README_zh-hans.md
Original file line number Diff line number Diff line change
Expand Up @@ -314,6 +314,7 @@ conda install -c huggingface transformers
1. **[Hubert](https://huggingface.co/docs/transformers/model_doc/hubert)** (来自 Facebook) 伴随论文 [HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Units](https://arxiv.org/abs/2106.07447) 由 Wei-Ning Hsu, Benjamin Bolte, Yao-Hung Hubert Tsai, Kushal Lakhotia, Ruslan Salakhutdinov, Abdelrahman Mohamed 发布。
1. **[I-BERT](https://huggingface.co/docs/transformers/model_doc/ibert)** (来自 Berkeley) 伴随论文 [I-BERT: Integer-only BERT Quantization](https://arxiv.org/abs/2101.01321) 由 Sehoon Kim, Amir Gholami, Zhewei Yao, Michael W. Mahoney, Kurt Keutzer 发布。
1. **[ImageGPT](https://huggingface.co/docs/transformers/model_doc/imagegpt)** (来自 OpenAI) 伴随论文 [Generative Pretraining from Pixels](https://openai.com/blog/image-gpt/) 由 Mark Chen, Alec Radford, Rewon Child, Jeffrey Wu, Heewoo Jun, David Luan, Ilya Sutskever 发布。
1. **[Informer](https://huggingface.co/docs/transformers/main/model_doc/informer)** (from Beihang University, UC Berkeley, Rutgers University, SEDD Company) released with the paper [Informer: Beyond Efficient Transformer for Long Sequence Time-Series Forecasting](https://arxiv.org/abs/2012.07436) by Haoyi Zhou, Shanghang Zhang, Jieqi Peng, Shuai Zhang, Jianxin Li, Hui Xiong, and Wancai Zhang.
1. **[Jukebox](https://huggingface.co/docs/transformers/model_doc/jukebox)** (from OpenAI) released with the paper [Jukebox: A Generative Model for Music](https://arxiv.org/pdf/2005.00341.pdf) by Prafulla Dhariwal, Heewoo Jun, Christine Payne, Jong Wook Kim, Alec Radford, Ilya Sutskever.
1. **[LayoutLM](https://huggingface.co/docs/transformers/model_doc/layoutlm)** (来自 Microsoft Research Asia) 伴随论文 [LayoutLM: Pre-training of Text and Layout for Document Image Understanding](https://arxiv.org/abs/1912.13318) 由 Yiheng Xu, Minghao Li, Lei Cui, Shaohan Huang, Furu Wei, Ming Zhou 发布。
1. **[LayoutLMv2](https://huggingface.co/docs/transformers/model_doc/layoutlmv2)** (来自 Microsoft Research Asia) 伴随论文 [LayoutLMv2: Multi-modal Pre-training for Visually-Rich Document Understanding](https://arxiv.org/abs/2012.14740) 由 Yang Xu, Yiheng Xu, Tengchao Lv, Lei Cui, Furu Wei, Guoxin Wang, Yijuan Lu, Dinei Florencio, Cha Zhang, Wanxiang Che, Min Zhang, Lidong Zhou 发布。
Expand Down
1 change: 1 addition & 0 deletions README_zh-hant.md
Original file line number Diff line number Diff line change
Expand Up @@ -326,6 +326,7 @@ conda install -c huggingface transformers
1. **[Hubert](https://huggingface.co/docs/transformers/model_doc/hubert)** (from Facebook) released with the paper [HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Units](https://arxiv.org/abs/2106.07447) by Wei-Ning Hsu, Benjamin Bolte, Yao-Hung Hubert Tsai, Kushal Lakhotia, Ruslan Salakhutdinov, Abdelrahman Mohamed.
1. **[I-BERT](https://huggingface.co/docs/transformers/model_doc/ibert)** (from Berkeley) released with the paper [I-BERT: Integer-only BERT Quantization](https://arxiv.org/abs/2101.01321) by Sehoon Kim, Amir Gholami, Zhewei Yao, Michael W. Mahoney, Kurt Keutzer.
1. **[ImageGPT](https://huggingface.co/docs/transformers/model_doc/imagegpt)** (from OpenAI) released with the paper [Generative Pretraining from Pixels](https://openai.com/blog/image-gpt/) by Mark Chen, Alec Radford, Rewon Child, Jeffrey Wu, Heewoo Jun, David Luan, Ilya Sutskever.
1. **[Informer](https://huggingface.co/docs/transformers/main/model_doc/informer)** (from Beihang University, UC Berkeley, Rutgers University, SEDD Company) released with the paper [Informer: Beyond Efficient Transformer for Long Sequence Time-Series Forecasting](https://arxiv.org/abs/2012.07436) by Haoyi Zhou, Shanghang Zhang, Jieqi Peng, Shuai Zhang, Jianxin Li, Hui Xiong, and Wancai Zhang.
1. **[Jukebox](https://huggingface.co/docs/transformers/model_doc/jukebox)** (from OpenAI) released with the paper [Jukebox: A Generative Model for Music](https://arxiv.org/pdf/2005.00341.pdf) by Prafulla Dhariwal, Heewoo Jun, Christine Payne, Jong Wook Kim, Alec Radford, Ilya Sutskever.
1. **[LayoutLM](https://huggingface.co/docs/transformers/model_doc/layoutlm)** (from Microsoft Research Asia) released with the paper [LayoutLM: Pre-training of Text and Layout for Document Image Understanding](https://arxiv.org/abs/1912.13318) by Yiheng Xu, Minghao Li, Lei Cui, Shaohan Huang, Furu Wei, Ming Zhou.
1. **[LayoutLMv2](https://huggingface.co/docs/transformers/model_doc/layoutlmv2)** (from Microsoft Research Asia) released with the paper [LayoutLMv2: Multi-modal Pre-training for Visually-Rich Document Understanding](https://arxiv.org/abs/2012.14740) by Yang Xu, Yiheng Xu, Tengchao Lv, Lei Cui, Furu Wei, Guoxin Wang, Yijuan Lu, Dinei Florencio, Cha Zhang, Wanxiang Che, Min Zhang, Lidong Zhou.
Expand Down
4 changes: 4 additions & 0 deletions docs/source/en/_toctree.yml
Original file line number Diff line number Diff line change
Expand Up @@ -612,6 +612,8 @@
title: Reinforcement learning models
- isExpanded: false
sections:
- local: model_doc/informer
Comment thread
kashif marked this conversation as resolved.
title: Informer
- local: model_doc/time_series_transformer
title: Time Series Transformer
title: Time series models
Expand All @@ -638,5 +640,7 @@
title: Utilities for Audio processing
- local: internal/file_utils
title: General Utilities
- local: internal/time_series_utils
title: Utilities for Time Series
title: Internal Helpers
title: API
Loading