lexical-analysis
A grammar describes the syntax of a programming language, and might be defined in Backus-Naur form (BNF). A lexer performs lexical analysis, turning text into tokens. A parser takes tokens and builds a data structure like an abstract syntax tree (AST). The parser is concerned with context: does the sequence of tokens fit the grammar? A compiler is a combined lexer and parser, built for a specific grammar.
Here are 616 public repositories matching this topic...
百度NLP:分词,词性标注,命名实体识别,词重要性
-
Updated
May 25, 2021 - C++
xmnlp:提供中文分词, 词性标注, 命名体识别,情感分析,文本纠错,文本转拼音,文本摘要,偏旁部首,句子表征及文本相似度计算等功能
-
Updated
Nov 12, 2022 - Python
DFA regular expression library & friends
-
Updated
Jul 13, 2024 - C
INTERSPEECH 2023 Papers: A complete collection of influential and exciting research papers from the INTERSPEECH 2023 conference. Explore the latest advances in speech and language processing. Code included. Star the repository to support the advancement of speech technology!
-
Updated
May 18, 2024
Chinese text segmentation with R. R语言中文分词 (文档已更新 🎉 :https://qinwenfeng.com/jiebaR/ )
-
Updated
Jul 13, 2020 - C++
A lexical analyzer based on DFA that is built using JS and supports multi-language extensions / 一个基于DFA的支持多语言扩展的JS版开源词法分析器
-
Updated
Mar 19, 2023 - JavaScript
LLVM Tutorial: Kaleidoscope (Implementing a Language with LLVM)
-
Updated
Dec 29, 2022 - C++
Allocators, I/O streams, math, geometry, image and audio processing for D
-
Updated
Mar 3, 2024 - D
OysterKit is a framework that provides a native Swift scanning, lexical analysis, and parsing capabilities. In addition it provides a language that can be used to rapidly define the rules used by OysterKit called STLR
-
Updated
Aug 15, 2019 - Swift
Implementing a complete Compiler for a simple C-like language using the C-tools Flex and Bison
-
Updated
Feb 8, 2023 - C
[EMNLP 2021] LingFeat - A Comprehensive Linguistic Features Extraction ToolKit for Readability Assessment
-
Updated
Mar 7, 2023 - Python
A compiler that accepts any valid program written in C. It is made using Lex and Yacc. Returns a symbol table, parse tree, annotated syntax tree and intermediate code.
-
Updated
Apr 24, 2023 - Yacc
clex is a simple lexer generator
-
Updated
Apr 6, 2023 - C
Santiago is a lexing and parsing toolkit for Rust
-
Updated
Aug 28, 2022 - Rust
😸 💬 A module to compute textual lexical richness (aka lexical diversity).
-
Updated
Aug 27, 2023 - Python
A Lexical Analyzer Generator
-
Updated
Jan 31, 2024 - Common Lisp
These are my programs for compiler design lab work in my sixth semester
-
Updated
May 25, 2022 - Python
Modular static malicious JavaScript detection system
-
Updated
Jan 18, 2021 - Python
- Followers
- 10.3k followers
- Wikipedia
- Wikipedia