NLP with Transformers: Fundamentals and Core ApplicationsChapter 72

Answers

Section 1 of 2-~ 12 min read-Synced from Cuantum content

Multiple Choice Questions

  1. c) It processes sequences in parallel.
  1. b) To encode the order of tokens in a sequence.
  1. b) GPT
  1. c) By maximizing similarity between paired image-text embeddings.
  1. a) It is pre-trained on biomedical corpora like PubMed.

True/False Questions

  1. False - GPT uses a unidirectional (autoregressive) context, not bidirectional.
  1. True - DistilBERT uses knowledge distillation to achieve a smaller and faster model.
  1. True - DALL-E generates images based on textual prompts.

Short Answer Questions

9. BERT processes context bidirectionally, capturing relationships between preceding and succeeding tokens. In contrast, GPT processes text unidirectionally (left-to-right), focusing on generating the next token in a sequence.

10. BioBERT would excel in a task like extracting chemical-disease relationships from biomedical research articles, as it is pre-trained on domain-specific texts that include terminology and structure not present in general-purpose datasets.

Code-Based Question

Solution:

from transformers import pipeline def classify_text(model_name, text, labels):    """    Classify text using a pre-trained BERT or its variant.    model_name: Hugging Face model name (e.g., 'bert-base-uncased').    text: Text to classify.    labels: List of labels to map predictions.    """    classifier = pipeline("text-classification", model=model_name)    result = classifier(text)    label_id = int(result[0]['label'].split('_')[-1])  # Extract label index    return labels[label_id] # Example usagemodel_name = "bert-base-uncased"text = "The patient shows symptoms of severe dehydration."labels = ["Healthy", "Dehydrated"]predicted_label = classify_text(model_name, text, labels)print("Predicted Label:", predicted_label)

Expected Output:

Predicted Label: Dehydrated

Congratulations!

Completing this quiz demonstrates your understanding of the Transformer architecture and its key models. You’ve covered foundational concepts, applications of multimodal models, and specialized adaptations like BioBERT and LegalBERT.