Enum SimpleCharSet

pub enum SimpleCharSet {
    None,
    Number(NumberState),
    Identifier,
    String(bool),
    Singleton,
    Comparison,
    Dot,
    Whitespace,
    Other,
    BreakNext(Option<SimpleCharSetTokenKind>),
}

Variants

None

Number(NumberState)

A number, which can be an integer, or a floating-point number

Identifier

An identifier consists of a character with the Unicode XID_Start property, followed by a sequence of characters with the Unicode XID_continue property

String(bool)

A string delimited by double quotes ("). The boolean indicates whether an odd number of backslashes have been matched (and thus the next " is escaped).

Singleton

A character which forms a token on its own

Comparison

Tokens starting with <, =, > can only contain other characters from that set.

Dot

. is part of a number if followed by a digit, or part of a punctuation tag otherwise.

Whitespace

Whitespace isn’t part of any token

Other

Any character not covered by the above categories

BreakNext(Option<SimpleCharSetTokenKind>)

The next character will not be in this token

Implementations

impl SimpleCharSet

fn categorize(ch: char) -> Self

Trait Implementations

impl CharSet<char> for SimpleCharSet

type TokenKind = SimpleCharSetTokenKind

type Error = SimpleCharSetError

fn next_char(&mut self, ch: char) -> CharSetResult<Self::TokenKind, Self::Error>

Categorize a charcter while continuing a potential token.

fn end_of_input(self) -> Result<Option<Self::TokenKind>, Self::Error>

What token kind (if any) to return if end of input is reached.

impl Clone for SimpleCharSet

fn clone(&self) -> SimpleCharSet

Returns a copy of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for SimpleCharSet

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Default for SimpleCharSet

fn default() -> SimpleCharSet

Returns the “default value” for a type.

impl PartialEq for SimpleCharSet

fn eq(&self, other: &SimpleCharSet) -> bool

Tests for self and other values to be equal, and is used by ==.

fn ne(&self, other: &Rhs) -> bool

Tests for !=. The default implementation is almost always sufficient, and should not be overridden without very good reason.

impl Copy for SimpleCharSet

impl Eq for SimpleCharSet

impl StructuralPartialEq for SimpleCharSet