Download

ULS is an object factory for generic-purpose lexical analysis supporting UTF-8. It's provided as C/C++ library with a couple of other instruments for the Windows and Linux platforms.

ULS was specially built to be an intuitive, practical, flexible and optimized tokenizer that you can use.

Features:

ULS supports UTF-8 encoded input files as well as ASCII. This means that you can use your locale language for identifiers in program source.
ULS can simultaneously tokenize multiple of files or (literal) strings as input.
ULS can stream the sequence of tokens to file or file descriptor.
ULS can be a replacement of 'flex' as token provider to yacc/bison parser. Its lexical configuration is read from file suffixed by '*.ulc'.
The precision of floating number is irrelavant with lexical analysis. ULS manipulates reals or integers having arbitrary number of digits.
ULS provides with a unique (varargs-style) logging framework, which is also used for gerneral purposes. ULS supports for dynamic printf. ULS enables the users to have their own formatted *-printf by adding or overriding its conversion specifications. There are %s, %c, %d, %u, %f, %Ld, %Lf, ... in default conversion specifications newly written from scratch. The group of conversion specifications itself can be instantiated thread by thread. The output port that the group uses can be any one only if it's given 'puts' interfaces. ULS, by default, provides the 'puts' interfaces for string-buffer and the file pointers, including stdout and stderr.
There're examples scanning the 'css3', 'html5', 'shell-script', 'Makefile' and dumping tokens.