A

Admin • 671K Points
Coach

Q. What is the function of a 'tokenizer' in Natural Language Processing?

  • (A) Compresses data
  • (B) Splits text into smaller parts
  • (C) Translates languages
  • (D) Classifies documents
Explanation by: Admin
A tokenizer breaks down text into words, subwords, or sentences.

You must be Logged in to update hint/solution

Discusssion

Login to discuss.

Be the first to start discuss.