Create Strings from tokens...

Create a new Strings object as a list of tokens.

Settings

Name
the name of the new Strings object.
Text
the text to be tokenized.
Separators
determines the separator characters. If left empty, a space will be used as a separator.

Behaviour

Multiple consecutive separators in the text will be treated as one.

Examples

Example 1 Create Strings from tokens:} "test", "a b c", " "
will produce a Strings object with 3 strings in it: "a", "b" and "c".
Example 2 Create Strings from tokens:} "test", "a b c ", " "
will also produce a Strings object with 3 strings in it: "a", "b" and "c".
Example 3 Create Strings from tokens:} "test", "a,b,c", ","
will produce a Strings object with 3 strings in it: "a", "b" and "c".
Example 4 Create Strings from tokens:} "test", "a, b, c", ","
will produce a Strings object with 3 strings in it: "a", " b" and " c" (note the spaces).
Example 5 Create Strings from tokens:} "test", "a, b, c", " ,"
will produce a Strings object with 3 strings in it: "a", "b" and "c".
Example 6 Create Strings from tokens:} "test", "a,,b,c", " ,"
will also produce a Strings object with 3 strings in it: "a", "b" and "c".
Example 7 Create Strings from tokens:} "test", "a, ,b,c", ","
will produce a Strings with 4 strings in it: "a", " ", "b" and "c".
Example 8 Create Strings from tokens:} "test", "a,b,c,", ","
will produce a Strings object with 3 strings in it: "a", "b" and "c".
Example 9 Create Strings from tokens:} "test", "a,b,c, ", ","
will produce a Strings object with 4 strings in it: "a", "b", "c" and " ".
Example 10 Create Strings from tokens:} "test", "A string" + tab + "of ..tokens" + newline + "and some more tokens", " .," + tab + newline
will produce a Strings object with 8 strings in it: "A", "string", "of", "tokens", "and", "some", "more" and "tokens".

Links to this page


© David Weenink 2017, Paul Boersma 2023