TokenAnalyzer (DupDetector API)

java.lang.Object
- edu.odu.cs.cs350.TokenAnalyzer

All Implemented Interfaces:

java.lang.Iterable<Token>
```
public class TokenAnalyzer
extends java.lang.Object
implements java.lang.Iterable<Token>
```
TokenAnalyzer analyzes each lexemes in the given file and determines the type of token. The result returns a list of tokens and the total number of tokens in the file.

Constructor Summary

Constructors
Constructor	Description
`TokenAnalyzer()`	The default constructor for token analyzer.
`TokenAnalyzer(java.io.File inputFile)`
`TokenAnalyzer(java.io.Reader input)`	The constructor for analyzing tokens in a file given by the parameter.

Method Summary

All Methods Instance Methods Concrete Methods
Modifier and Type	Method	Description
`int`	`getFileTokenCount()`
`java.util.List<? extends TokenInterface>`	`getTokens()`
`java.util.Iterator<Token>`	`iterator()`	Return an iterator over the list of tokens.
`void`	`processSourceCode()`	Divides source codes into tokens.
`java.lang.String`	`toString()`	For debugging purposes.

Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Methods inherited from interface java.lang.Iterable
forEach, spliterator

- Constructor Detail
  - TokenAnalyzer
```
public TokenAnalyzer()
```
    The default constructor for token analyzer. Creates an empty constructor for each data members.
  - TokenAnalyzer
```
public TokenAnalyzer(java.io.Reader input)
```
    The constructor for analyzing tokens in a file given by the parameter. Creates a list of object Token and a LexerAnalyzer scanner.
    
    Parameters:
    
    input - The input file to be read to determine the tokens in the file
  - TokenAnalyzer
```
public TokenAnalyzer(java.io.File inputFile)
```
- Method Detail
  - processSourceCode
```
public void processSourceCode()
```
    Divides source codes into tokens. The function will scan each token and add it to the list container if the token type is defined in the TokenType class
  - getFileTokenCount
```
public final int getFileTokenCount()
```
    Returns:
    
    The total number of tokens in the file that is being read.
  - iterator
```
public final java.util.Iterator<Token> iterator()
```
    Return an iterator over the list of tokens.
    
    Specified by:
    
    iterator in interface java.lang.Iterable<Token>
  - toString
```
public java.lang.String toString()
```
    For debugging purposes.
    
    Overrides:
    
    toString in class java.lang.Object
    
    Returns:
    
    format of tokenAnalyzer which returns the number of tokens in the file
  - getTokens
```
public java.util.List<? extends TokenInterface> getTokens()
```