Question:
In natural language processing, what does "tokenization" refer to?
Choose the correct answer:
- A. Breaking down text into paragraphs
- B. Splitting text into sentences
- C. Breaking text into smaller units like words or phrases ✅ Correct
- D. Removing punctuation and stopwords
- E. Translating text into another language
Explanation:
Tokenization involves splitting text into smaller units, such as words or phrases, which can then be analyzed and processed.
Tokenization involves splitting text into smaller units, such as words or phrases, which can then be analyzed and processed.
Related Questions & One-Liners
- No related questions / one-liners available.
Related MCQs
What is the main objective of supervised learning in machine learning?
Explanation:
In supervised learning, the model is trained to predict outcomes based on labeled input data.
In supervised learning, the model is trained to predict outcomes based on labeled input data.
Which algorithm is commonly used for classification tasks?
Explanation:
Decision Trees are widely used for classification tasks due to their interpretability and ability to handle both numerical and categorical data.
Decision Trees are widely used for classification tasks due to their interpretability and ability to handle both numerical and categorical data.
What does "overfitting" refer to in machine learning?
Explanation:
Overfitting occurs when a model is too complex and captures noise in the training data, leading to poor generalization on new data.
Overfitting occurs when a model is too complex and captures noise in the training data, leading to poor generalization on new data.
Which of the following is an example of unsupervised learning?
Explanation:
PCA is an unsupervised learning method used for dimensionality reduction and to identify patterns in data without labels.
PCA is an unsupervised learning method used for dimensionality reduction and to identify patterns in data without labels.
What is the purpose of a "learning rate" in gradient descent optimization?
Explanation:
The learning rate controls how much to change the model in response to the estimated error each time the model weights are updated.
The learning rate controls how much to change the model in response to the estimated error each time the model weights are updated.
Which metric is commonly used to evaluate the performance of a classification model?
Explanation:
The F1 Score is commonly used for evaluating classification models, especially when dealing with imbalanced datasets, as it considers both precision and recall.
The F1 Score is commonly used for evaluating classification models, especially when dealing with imbalanced datasets, as it considers both precision and recall.
Related Study Material
A Quiz on Artificial Intelligence and Machine Learning
This set of questions covers a range of foundational topics in AI and ML, including supervised and unsupervised learning, model evaluation, and applications in natural…