A

Arogya • 3.19K Points
Extraordinary

Q. Which component converts user text into tokens before LLaMA processes it?

  • (A) Compiler
  • (B) Tokenizer
  • (C) Interpreter
  • (D) Debugger
  • Correct Answer - Option(B)
  • Views: 3
  • Filed under category Llama
  • Hashtags:
Explanation by: Arogya
The tokenizer splits text into tokens (sub-words or words) that the LLaMA model can understand.

You must be Logged in to update hint/solution

Discusssion

Login to discuss.

Be the first to start discuss.