A

Arogya • 3.49K Points
Extraordinary

Q. Which component converts user text into tokens before LLaMA processes it?

  • (A) Compiler
  • (B) Tokenizer
  • (C) Interpreter
  • (D) Debugger
  • Correct Answer - Option(B)
  • Views: 13
  • Filed under category Llama
  • Hashtags:

Explanation by: Arogya

The tokenizer splits text into tokens (sub-words or words) that the LLaMA model can understand.

You must be Logged in to update hint/solution

Discusssion

Login to discuss.

Be the first to start discuss.