feat: readme
This commit is contained in:
70
README.md
Normal file
70
README.md
Normal file
@@ -0,0 +1,70 @@
|
||||
# SPL Lexer
|
||||
|
||||
A lexer (tokenizer) implementation in Go for parsing custom language/query syntax.
|
||||
|
||||
## Overview
|
||||
|
||||
This project implements a lexical analyzer that converts input strings into tokens for a custom language. The lexer recognizes various token types including symbols, operators, keywords, literals, and punctuation.
|
||||
|
||||
## Features
|
||||
|
||||
- Tokenizes input strings into meaningful components
|
||||
- Supports various token types:
|
||||
- Symbols (identifiers)
|
||||
- Operators (comparison, logical)
|
||||
- Literals (strings, numbers)
|
||||
- Punctuation (braces, colons, pipes)
|
||||
- Comments
|
||||
- Whitespace
|
||||
|
||||
## Token Types
|
||||
|
||||
The lexer supports the following token types:
|
||||
|
||||
- `SYMBOL`: Identifiers
|
||||
- `COLON`: `:`
|
||||
- `EXCLAMATION`: `!`
|
||||
- `EQUAL`: `=`
|
||||
- `NOT_EQUAL`: `!=`
|
||||
- `AND`: `and`
|
||||
- `NOT`: `not`
|
||||
- `OR`: `or`
|
||||
- `MORE`: `>`
|
||||
- `LESS`: `<`
|
||||
- `MORE_EQUAL`: `>=`
|
||||
- `LESS_EQUAL`: `<=`
|
||||
- `OPEN_BRACE`: `(`
|
||||
- `CLOSED_BRACE`: `)`
|
||||
- `COMMENT`: `//` comments
|
||||
- `PIPE`: `|`
|
||||
- `NUMBER`: Integer numbers
|
||||
- `FLOAT_NUMBER`: Floating-point numbers
|
||||
- `STRING_LITERAL`: String literals in quotes
|
||||
- `SPACE`: Whitespace
|
||||
|
||||
## Usage
|
||||
|
||||
To use the lexer, you can call the `Parse` function from the `lexer` package:
|
||||
|
||||
```go
|
||||
package main
|
||||
|
||||
import (
|
||||
"github.com/e1lama/spl/lexer"
|
||||
)
|
||||
|
||||
func main() {
|
||||
tokens := lexer.Parse("some input string")
|
||||
// Process tokens
|
||||
}
|
||||
```
|
||||
|
||||
## Project Structure
|
||||
|
||||
- `main.go` - Main entry point (currently empty)
|
||||
- `lexer/lexer.go` - Core lexer implementation
|
||||
- `lexer/tokens.go` - Token definitions and types
|
||||
|
||||
## License
|
||||
|
||||
MIT
|
||||
Reference in New Issue
Block a user