StringTokenizer Class in Java
In Java, the StringTokenizer class is used to break a string into tokens based on specified delimiters. It belongs to the java.util package. The delimiters can be defined at the time of object creation or applied while retrieving tokens.
StringTokenizer is considered as a legacy class, and it does not provide advanced features such as distinguishing between numbers, identifiers, or quoted strings. For more flexible and modern string processing, classes like String.split() or StreamTokenizer are generally preferred.
For example, consider the string "hello welcome to TpointTech". As shown in the image, the StringTokenizer breaks this string into separate tokens based on the specified delimiter (such as a space), resulting in individual words like hello, welcome, to, and TpointTech.
StringTokenizer Class Constructors
| Constructor | Description |
|---|---|
| StringTokenizer(String str) | It creates a StringTokenizer for the specified string using the default delimiter (space, tab, newline, etc.). |
| StringTokenizer(String str, String delim) | It creates a StringTokenizer for the specified string using the provided delimiter(s). |
| StringTokenizer(String str, String delim, boolean returnDelims) | It creates a StringTokenizer for the specified string and delimiter(s). If returnDelims is true, the delimiters are also returned as tokens. If false, delimiters are only used to separate tokens. |
Example: StringTokenizer Constructors
The following program, we have demonstrated all three constructors of the StringTokenizer class.
Java
import java.util.StringTokenizer;
public class Main {
public static void main(String[] args) {
String str = "Hello,Welcome,To,TpointTech";
// Constructor 1: Default delimiter (space)
StringTokenizer st1 = new StringTokenizer("Hello Welcome To TpointTech");
System.out.println("Tokens using default delimiter:");
while (st1.hasMoreTokens()) {
System.out.println(st1.nextToken());
}
// Constructor 2: Custom delimiter (,)
StringTokenizer st2 = new StringTokenizer(str, ",");
System.out.println("\nTokens using comma as delimiter:");
while (st2.hasMoreTokens()) {
System.out.println(st2.nextToken());
}
// Constructor 3: Custom delimiter with returnDelims = true
StringTokenizer st3 = new StringTokenizer(str, ",", true);
System.out.println("\nTokens including delimiters:");
while (st3.hasMoreTokens()) {
System.out.println(st3.nextToken());
}
}
}
Output:
Tokens using default delimiter: Hello Welcome To TpointTech Tokens using comma as delimiter: Hello Welcome To TpointTech Tokens including delimiters: Hello , Welcome , To , TpointTech
Explanation
In the above program, the first constructor uses the default space delimiter. The second constructor uses a custom delimiter (comma) to split the string. The third constructor also uses a comma, but with returnDelims = true, so the delimiters themselves are returned as tokens.
StringTokenizer Class Methods
| Methods | Description |
|---|---|
| boolean hasMoreTokens() | It checks if there are more tokens available. |
| String nextToken() | It returns the next token from the StringTokenizer object. |
| String nextToken(String delim) | It returns the next token based on the delimiter. |
| boolean hasMoreElements() | It is the same as hasMoreTokens() method. |
| Object nextElement() | It is the same as nextToken() but its return type is Object. |
| int countTokens() | It returns the total number of tokens. |
Example: StringTokenizer.nextToken(String delim) Method
The following example demonstrates how to get the next token from a string using a specified delimiter.
Java
import java.util.*;
public class Main {
public static void main(String[] args) {
StringTokenizer st = new StringTokenizer("my,name,is,Tom");
// printing next token
System.out.println("Next token is: " + st.nextToken(","));
}
}
Output:
Next token is: my
Note: The StringTokenizer class is deprecated. It is recommended to use the String.split() method or the java.util.regex.Pattern.split() method.
Example: StringTokenizer.hasMoreTokens() Method
The following example demonstrates how to check if more tokens are available and iterate through them.
Java
import java.util.StringTokenizer;
public class Main {
public static void main(String args[]) {
/* StringTokenizer object */
StringTokenizer st = new StringTokenizer("Demonstrating methods from StringTokenizer class"," ");
/* Checks if the String has any more tokens */
while (st.hasMoreTokens()) {
System.out.println(st.nextToken());
}
}
}
Output:
Demonstrating methods from StringTokenizer class
Example: StringTokenizer.hasMoreElements() Method
This method works similarly to hasMoreTokens() method but it is useful while using the Enumeration interface.
Java
import java.util.StringTokenizer;
public class Main {
public static void main(String args[]) {
StringTokenizer st = new StringTokenizer("Hello everyone I am a Java developer"," ");
while (st.hasMoreElements()) {
System.out.println(st.nextToken());
}
}
}
Output:
Hello everyone I am a Java developer
Example: StringTokenizer.nextElement() Method
The nextElement() method returns the next token as an Object and can be used with Enumeration.
Java
import java.util.StringTokenizer;
public class Main {
public static void main(String args[]) {
/* StringTokenizer object */
StringTokenizer st = new StringTokenizer("Hello Everyone Have a nice day"," ");
/* Checks if the String has any more tokens */
while (st.hasMoreTokens()) {
/* Prints the elements from the String */
System.out.println(st.nextElement());
}
}
}
Output:
Hello Everyone Have a nice day
Example: StringTokenizer.countTokens() Method
This method counts the total number of tokens in the string.
Java
import java.util.StringTokenizer;
public class Main {
public static void main(String args[]) {
/* StringTokenizer object */
StringTokenizer st = new StringTokenizer("Hello Everyone Have a nice day"," ");
/* Prints the number of tokens present in the String */
System.out.println("Total number of Tokens: "+st.countTokens());
}
}
Output:
Total number of Tokens: 6
Example: StringTokenizer with Multiple Delimiters
This example shows how to use several delimiters to separate a string.
Java
import java.util.StringTokenizer;
public class Main {
public static void main(String args[]) {
/* StringTokenizer object with multiple delimiters */
StringTokenizer st = new StringTokenizer("Java,Python;C++|JavaScript", ",;|");
/* Prints tokens separated by multiple delimiters */
while (st.hasMoreTokens()) {
System.out.println(st.nextToken());
}
}
}
Output:
Java Python C++ JavaScript
Example: StringTokenizer with Tabs and New Lines
This example shows how StringTokenizer handles tabs (\t) and newline (\n) characters using default delimiters.
Java
import java.util.StringTokenizer;
public class Main {
public static void main(String args[]) {
String str = "Java\tPython\nC++\tSpring";
/* StringTokenizer using default delimiters */
StringTokenizer st = new StringTokenizer(str);
/* Prints tokens separated by tab and newline */
while (st.hasMoreTokens()) {
System.out.println(st.nextToken());
}
}
}
Output:
Java Python C++ Spring
Example: StringTokenizer for Parsing CSV Values
This example demonstrates how StringTokenizer can be used to parse comma-separated values (CSV).1
Java
import java.util.StringTokenizer;
public class Main {
public static void main(String args[]) {
String data = "101,John,Developer,75000";
/* StringTokenizer object for CSV parsing */
StringTokenizer st = new StringTokenizer(data, ",");
/* Prints each field separately */
while (st.hasMoreTokens()) {
System.out.println(st.nextToken());
}
}
}
Output:
101 John Developer 75000
Example: StringTokenizer for Extracting File Path Components
This example demonstrates how to use StringTokenizer to extract individual parts of a file path using a forward slash (/) as the delimiter.
Java
import java.util.StringTokenizer;
public class Main {
public static void main(String args[]) {
String filePath = "/home/user/documents/java/StringTokenizer.java";
/* StringTokenizer object using '/' as delimiter */
StringTokenizer st = new StringTokenizer(filePath, "/");
/* Prints each directory and file name */
while (st.hasMoreTokens()) {
System.out.println(st.nextToken());
}
}
}
Output:
home user documents java StringTokenizer.java
Subscribe to Tpoint Tech
We request you to subscribe our newsletter for upcoming updates.
We deliver comprehensive tutorials, interview question-answers, MCQs, study materials on leading programming languages and web technologies like Data Science, MEAN/MERN full stack development, Python, Java, C++, C, HTML, React, Angular, PHP and much more to support your learning and career growth.
Contact info
G-13, 2nd Floor, Sec-3, Noida, UP, 201301, India
Follow us
