NLP with Transformers: Fundamentals and Core ApplicationsChapter 97
7. Step 4: Fine-Tuning BERT
Section 7 of 10-~ 12 min read-Synced from Cuantum content
We’ll fine-tune a pre-trained BERT model for binary classification (positive or negative sentiment).
Code Example: Fine-Tuning
# Load pre-trained BERT modelmodel = BertForSequenceClassification.from_pretrained("bert-base-uncased", num_labels=2) # Define training argumentstraining_args = TrainingArguments( output_dir="./results", evaluation_strategy="epoch", learning_rate=2e-5, per_device_train_batch_size=8, num_train_epochs=3, weight_decay=0.01,) # Initialize Trainertrainer = Trainer( model=model, args=training_args, train_dataset=tokenized_train, eval_dataset=tokenized_test,) # Train the modeltrainer.train()