(transitive, computing) To reduce to a set of tokens by lexical analysis. / (transitive, computing) To substitute sensitive data with meaningless placeholders.

(computing) A system that parses an input stream into its component tokens.

plural of tokenizer

Third-person singular simple present indicative form of tokenize

simple past tense and past participle of tokenize

A policy of formally complying with efforts to achieve a goal by making small, token gestures; especially to hire a minimal number of ethnically diverse or disadvantaged people.

Something serving as an expression of something else. / A keepsake. / A piece of stamped metal or plastic, etc., used as a substitute for money; a voucher that can be exchanged for goods or services.

To betoken, indicate, portend, designate, denote / To betroth / (philosophy) To symbolize, instantiate

Done as an indication or a pledge. / Perfunctory or merely symbolic; done or existing for appearance's sake, or to minimally comply with a requirement.

plural of token