diff --git a/docs/source/index.rst b/docs/source/index.rst index 07670a97c988..fb88132b538e 100644 --- a/docs/source/index.rst +++ b/docs/source/index.rst @@ -42,7 +42,7 @@ The documentation is organized in five parts: - **GET STARTED** contains a quick tour, the installation instructions and some useful information about our philosophy and a glossary. -- **USING TRANSFORMERS** contains general tutorials on how to use the library. +- **USING πŸ€— TRANSFORMERS** contains general tutorials on how to use the library. - **ADVANCED GUIDES** contains more advanced guides that are more specific to a given script or part of the library. - **RESEARCH** focuses on tutorials that have less to do with how to use the library but more about general resarch in transformers model @@ -135,7 +135,7 @@ conversion utilities for the following models: .. toctree:: :maxdepth: 2 - :caption: Using Transformers + :caption: Using πŸ€— Transformers task_summary model_summary diff --git a/docs/source/migration.md b/docs/source/migration.md index 557e0b809bfe..0cf53e1feaef 100644 --- a/docs/source/migration.md +++ b/docs/source/migration.md @@ -1,8 +1,8 @@ # Migrating from previous packages -## Migrating from pytorch-transformers to transformers +## Migrating from pytorch-transformers to πŸ€— Transformers -Here is a quick summary of what you should take care of when migrating from `pytorch-transformers` to `transformers`. +Here is a quick summary of what you should take care of when migrating from `pytorch-transformers` to πŸ€— Transformers. ### Positional order of some models' keywords inputs (`attention_mask`, `token_type_ids`...) changed @@ -14,17 +14,17 @@ If you used to call the models with positional inputs for keyword arguments, e.g ## Migrating from pytorch-pretrained-bert -Here is a quick summary of what you should take care of when migrating from `pytorch-pretrained-bert` to `transformers` +Here is a quick summary of what you should take care of when migrating from `pytorch-pretrained-bert` to πŸ€— Transformers ### Models always output `tuples` -The main breaking change when migrating from `pytorch-pretrained-bert` to `transformers` is that the models forward method always outputs a `tuple` with various elements depending on the model and the configuration parameters. +The main breaking change when migrating from `pytorch-pretrained-bert` to πŸ€— Transformers is that the models forward method always outputs a `tuple` with various elements depending on the model and the configuration parameters. The exact content of the tuples for each model are detailled in the models' docstrings and the [documentation](https://huggingface.co/transformers/). In pretty much every case, you will be fine by taking the first element of the output as the output you previously used in `pytorch-pretrained-bert`. -Here is a `pytorch-pretrained-bert` to `transformers` conversion example for a `BertForSequenceClassification` classification model: +Here is a `pytorch-pretrained-bert` to πŸ€— Transformers conversion example for a `BertForSequenceClassification` classification model: ```python # Let's load our model @@ -33,11 +33,11 @@ model = BertForSequenceClassification.from_pretrained('bert-base-uncased') # If you used to have this line in pytorch-pretrained-bert: loss = model(input_ids, labels=labels) -# Now just use this line in transformers to extract the loss from the output tuple: +# Now just use this line in πŸ€— Transformers to extract the loss from the output tuple: outputs = model(input_ids, labels=labels) loss = outputs[0] -# In transformers you can also have access to the logits: +# In πŸ€— Transformers you can also have access to the logits: loss, logits = outputs[:2] # And even the attention weights if you configure the model to output them (and other outputs too, see the docstrings and documentation) @@ -109,7 +109,7 @@ for batch in train_data: loss.backward() optimizer.step() -### In Transformers, optimizer and schedules are splitted and instantiated like this: +### In πŸ€— Transformers, optimizer and schedules are splitted and instantiated like this: optimizer = AdamW(model.parameters(), lr=lr, correct_bias=False) # To reproduce BertAdam specific behavior set correct_bias=False scheduler = get_linear_schedule_with_warmup(optimizer, num_warmup_steps=num_warmup_steps, num_training_steps=num_training_steps) # PyTorch scheduler ### and used like this: diff --git a/docs/source/model_summary.rst b/docs/source/model_summary.rst index 0c4baa59f514..95645c587566 100644 --- a/docs/source/model_summary.rst +++ b/docs/source/model_summary.rst @@ -1,7 +1,7 @@ Summary of the models ================================================ -This is a summary of the models available in the transformers library. It assumes you’re familiar with the original +This is a summary of the models available in πŸ€— Transformers. It assumes you’re familiar with the original `transformer model `_. For a gentle introduction check the `annotated transformer `_. Here we focus on the high-level differences between the models. You can check them more in detail in their respective documentation. Also checkout the diff --git a/docs/source/philosophy.rst b/docs/source/philosophy.rst index be6182d19f94..ed6173f6fcf1 100644 --- a/docs/source/philosophy.rst +++ b/docs/source/philosophy.rst @@ -1,7 +1,7 @@ Philosophy ========== -Transformers is an opinionated library built for: +πŸ€— Transformers is an opinionated library built for: - NLP researchers and educators seeking to use/study/extend large-scale transformers models - hands-on practitioners who want to fine-tune those models and/or serve them in production diff --git a/docs/source/quicktour.rst b/docs/source/quicktour.rst index c15426531416..f1acbb3dd852 100644 --- a/docs/source/quicktour.rst +++ b/docs/source/quicktour.rst @@ -44,27 +44,27 @@ make them readable. For instance :: - classifier('We are very happy to show you the Transformers library.') + classifier('We are very happy to show you the πŸ€— Transformers library.') will return something like this: :: - [{'label': 'POSITIVE', 'score': 0.999799370765686}] + [{'label': 'POSITIVE', 'score': 0.9997795224189758}] That's encouraging! You can use it on a list of sentences, which will be preprocessed then fed to the model as a `batch`: :: - classifier(["We are very happy to show you the Transformers library.", + classifier(["We are very happy to show you the πŸ€— Transformers library.", "We hope you don't hate it."]) returning a list of dictionaries like this one: :: - [{'label': 'POSITIVE', 'score': 0.999799370765686}, + [{'label': 'POSITIVE', 'score': 0.9997795224189758}, {'label': 'NEGATIVE', 'score': 0.5308589935302734}] You can see the second sentence has been classified as negative (it needs to be positive or negative) but its score is @@ -163,7 +163,7 @@ To apply these steps on a given text, we can just feed it to our tokenizer: :: - input = tokenizer("We are very happy to show you the Transformers library.") + input = tokenizer("We are very happy to show you the πŸ€— Transformers library.") print(input) This returns a dictionary string to list of ints. It contains the `ids of the tokens `__, @@ -172,9 +172,8 @@ as mentioned before, but also additional arguments that will be useful to the mo :: - - {'input_ids': [101, 2057, 2024, 2200, 3407, 2000, 2265, 2017, 1996, 19081, 3075, 1012, 102], - 'attention_mask': [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]} + {'input_ids': [101, 2057, 2024, 2200, 3407, 2000, 2265, 2017, 1996, 100, 19081, 3075, 1012, 102], + 'attention_mask': [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]} You can pass a list of sentences directly to your tokenizer. If your goal is to send them through your model as a batch, you probably want to pad them all to the same length, truncate them to the maximum length the model can accept @@ -184,13 +183,13 @@ and get tensors back. You can specify all of that to the tokenizer: ## PYTORCH CODE batch = tokenizer( - ["We are very happy to show you the Transformers library.", + ["We are very happy to show you the πŸ€— Transformers library.", "We hope you don't hate it."], padding=True, truncation=True, return_tensors="pt") print(batch) ## TENSORFLOW CODE batch = tokenizer( - ["We are very happy to show you the Transformers library.", + ["We are very happy to show you the πŸ€— Transformers library.", "We hope you don't hate it."], padding=True, truncation=True, return_tensors="tf") print(batch) @@ -200,10 +199,10 @@ padding token the model was pretrained with. The attention mask is also adapted :: - {'input_ids': tensor([[ 101, 2057, 2024, 2200, 3407, 2000, 2265, 2017, 1996, 19081, 3075, 1012, 102], - [ 101, 2057, 3246, 2017, 2123, 1005, 1056, 5223, 2009, 1012, 102, 0, 0]]), - 'attention_mask': tensor([[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1], - [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0]])} + {'input_ids': tensor([[ 101, 2057, 2024, 2200, 3407, 2000, 2265, 2017, 1996, 100, 19081, 3075, 1012, 102], + [ 101, 2057, 3246, 2017, 2123, 1005, 1056, 5223, 2009, 1012, 102, 0, 0, 0]]), + 'attention_mask': tensor([[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1], + [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0]])} You can learn more about tokenizers on their :doc:`doc page ` (tutorial coming soon). diff --git a/docs/source/task_summary.rst b/docs/source/task_summary.rst index a7ef4d4572a9..712d24e8fb93 100644 --- a/docs/source/task_summary.rst +++ b/docs/source/task_summary.rst @@ -217,9 +217,9 @@ Here is an example of question answering using a model and a tokenizer. The proc """ questions = [ - "How many pretrained models are available in Transformers?", - "What does Transformers provide?", - "Transformers provides interoperability between which frameworks?", + "How many pretrained models are available in πŸ€— Transformers?", + "What does πŸ€— Transformers provide?", + "πŸ€— Transformers provides interoperability between which frameworks?", ] for question in questions: @@ -253,9 +253,9 @@ Here is an example of question answering using a model and a tokenizer. The proc """ questions = [ - "How many pretrained models are available in Transformers?", - "What does Transformers provide?", - "Transformers provides interoperability between which frameworks?", + "How many pretrained models are available in πŸ€— Transformers?", + "What does πŸ€— Transformers provide?", + "πŸ€— Transformers provides interoperability between which frameworks?", ] for question in questions: @@ -280,13 +280,13 @@ This outputs the questions followed by the predicted answers: :: - Question: How many pretrained models are available in Transformers? + Question: How many pretrained models are available in πŸ€— Transformers? Answer: over 32 + - Question: What does Transformers provide? + Question: What does πŸ€— Transformers provide? Answer: general - purpose architectures - Question: Transformers provides interoperability between which frameworks? + Question: πŸ€— Transformers provides interoperability between which frameworks? Answer: tensorflow 2 . 0 and pytorch diff --git a/docs/source/torchscript.rst b/docs/source/torchscript.rst index fd1eeb53635f..a735b531d119 100644 --- a/docs/source/torchscript.rst +++ b/docs/source/torchscript.rst @@ -12,7 +12,7 @@ According to Pytorch's documentation: "TorchScript is a way to create serializab Pytorch's two modules `JIT and TRACE `_ allow the developer to export their model to be re-used in other programs, such as efficiency-oriented C++ programs. -We have provided an interface that allows the export of `transformers` models to TorchScript so that they can +We have provided an interface that allows the export of πŸ€— Transformers models to TorchScript so that they can be reused in a different environment than a Pytorch-based python program. Here we explain how to use our models so that they can be exported, and what to be mindful of when using these models with TorchScript. diff --git a/examples/README.md b/examples/README.md index c578a865b3b3..262a1c44613a 100644 --- a/examples/README.md +++ b/examples/README.md @@ -1,6 +1,6 @@ ## Examples -Version 2.9 of `transformers` introduces a new [`Trainer`](https://github.com/huggingface/transformers/blob/master/src/transformers/trainer.py) class for PyTorch, and its equivalent [`TFTrainer`](https://github.com/huggingface/transformers/blob/master/src/transformers/trainer_tf.py) for TF 2. +Version 2.9 of πŸ€— Transformers introduces a new [`Trainer`](https://github.com/huggingface/transformers/blob/master/src/transformers/trainer.py) class for PyTorch, and its equivalent [`TFTrainer`](https://github.com/huggingface/transformers/blob/master/src/transformers/trainer_tf.py) for TF 2. Running the examples requires PyTorch 1.3.1+ or TensorFlow 2.0+. Here is the list of all our examples: diff --git a/notebooks/README.md b/notebooks/README.md index f100abbd49cd..55c5d9804c41 100644 --- a/notebooks/README.md +++ b/notebooks/README.md @@ -1,13 +1,13 @@ -# Transformers Notebooks +# πŸ€— Transformers Notebooks You can find here a list of the official notebooks provided by Hugging Face. Also, we would like to list here interesting content created by the community. -If you wrote some notebook(s) leveraging transformers and would like be listed here, please open a +If you wrote some notebook(s) leveraging πŸ€— Transformers and would like be listed here, please open a Pull Request so it can be included under the Community notebooks. -## Hugging Face's notebooks :hugs: +## Hugging Face's notebooks πŸ€— | Notebook | Description | | |:----------|:-------------|------:|