Natural Language Processing with Python Updated EditionChapter 42

Chapter 2: Basic Text Processing

Section 4 of 8-~ 12 min read-Synced from Cuantum content
  1. What is tokenization in NLP?
  2. a) Combining multiple words into a single token.

b) Splitting text into smaller units like words or sentences.

c) Removing punctuation from text.

d) Encoding text into binary format.

  1. Which of the following is a technique used to reduce words to their base or root form?
  2. a) Tokenization

b) Stop word removal

c) Stemming

d) Vectorization

  1. What are stop words?
  2. a) Words that are frequently used and often removed during text preprocessing.

b) Words that are rarely used in any text.

c) Words that are essential for the meaning of a sentence.

d) Words that appear at the end of a sentence.

  1. Which Python library can be used to apply regular expressions for text processing?
  2. a) re

b) numpy

c) pandas

d) matplotlib