summaryrefslogtreecommitdiff
path: root/lib/coderay/scanner.rb
Commit message (Collapse)AuthorAgeFilesLines
* use autoload againKornelius Kalnbach2013-10-221-355/+0
|
* trying to reduce "complexity" of #raise_inspect (#135)Kornelius Kalnbach2013-06-101-10/+20
|
* trying to reduce "complexity" of #raise_inspect (#135)Kornelius Kalnbach2013-06-101-10/+12
|
* trying to reduce "complexity" of #raise_inspect (#135)Kornelius Kalnbach2013-06-101-3/+11
|
* refactor Scanner a bit more (#135)Kornelius Kalnbach2013-06-101-7/+12
|
* refactor Scanner a bit (#135)Kornelius Kalnbach2013-06-101-12/+19
|
* use coderay_path on all autoload callsKornelius Kalnbach2011-12-271-2/+2
|
* Remove assumption about a stable $:Conrad Irwin2011-12-231-1/+1
| | | | | | | | | | In some environments (e.g. [1]) $: can change between loading the library and using it. To avoid this problem, we always pass an absolute path to autoload internal modules. [1] https://github.com/pry/pry/issues/280
* fixed YAML scanner (Scanner#column rewrite broke it)Kornelius Kalnbach2011-08-291-3/+0
|
* add term-ansicolor to development dependencies, fix Ruby 1.8.6 warningKornelius Kalnbach2011-08-221-1/+1
|
* cleanup CodeRay.scan* methodsKornelius Kalnbach2011-08-201-4/+4
|
* rename CaseIgnoringWordList to WordList::CaseIgnoringKornelius Kalnbach2011-08-191-3/+1
|
* Major rewrite of encoders to support IO output; fixed some minor scanner ↵Kornelius Kalnbach2011-08-191-18/+30
| | | | bugs; cleanups; dropped NitroXHTML scanner; improved tests
* coderay list subcommand and cleanups/fixes in Plugin helper (issue #45)murphy2011-06-251-2/+4
|
* #309 Improved highlighting of Ruby inside diffsmurphy2011-06-221-3/+12
|
* fix Ruby 1.9.3 warnings about unused local variablesmurphy2011-06-111-1/+1
|
* new method Scanner#scan_rest (upported from 0.9.8), and Scanner::ScanError ↵murphy2011-05-211-3/+11
| | | | is now a StandardError
* fix warning in Ruby 1.8.6murphy2011-04-131-1/+1
|
* Fix handling of Windows newlines in Scanner.murphy2011-03-011-5/+6
|
* Teaching Scanner how to deal with encodings (Ruby 1.9)murphy2011-03-011-83/+103
|
* Use autoload instead of require (speeds up CodeRay startup) for CodeRay, ↵murphy2010-06-291-4/+4
| | | | Scanner, Tokens, and Encoders::HTML.
* A little more documentation for Scanner.murphy2010-06-271-2/+5
|
* Added ability to scan an Array of Strings as one with Scanner#tokenize.murphy2010-06-011-3/+16
| | | | | See issue #222.
* Cleanup, documentation and enhancements to the Filters:murphy2010-05-181-1/+3
| | | | | | | * TokenKindFilter handles groups (closes #223). * Added :docstring to the default KINDS_NOT_LOC list. * Also fixed a problem with the :tokens option in Scanners.
* Got rid of the old streaming system (see #142).murphy2010-05-011-24/+6
|
* Direct Streaming! See #142 and Changes.textile.murphy2010-05-011-17/+4
|
* Upporting r463, removing String#to_unix.murphy2010-03-301-23/+23
|
* New method Scanner#lang.murphy2010-01-011-0/+5
|
* Fixed example tests.murphy2009-12-311-1/+10
| | | | | | | | | | | * test:clean task also deletes .expected.html files * LinesOfCode encoder can deal with tokens that have no scanner. Tests were added for this. * JSON encoder load rubygems if necessary. * NEW :loc as an alias for :lines_of_code * NEW Scanner methods marshal_dump, marshal_load FIXED Tokens dumping (failed while trying to dump @scanner)
* Scanner sets KINDS_NOT_LOC, Tokens#scanner.murphy2009-10-181-0/+4
|
* Experimental Unicode support for Scanners in Ruby 1.9.murphy2009-04-201-1/+14
| | | | | | | * Python scanner uses it for idents. * Scanner#column method needed to be fixed for multibyte characters because StringScanner#pos still works on bytes.
* Removed subversion $Id$ lines.murphy2009-02-201-2/+0
|
* Completed YAML Scanner (closes #34).murphy2009-01-011-2/+7
| | | | | | | | | | | | | YAML Scanner: * Added another example (multiline). * Added multiline string recognition. * This should be enough for most people. Else: * New method: Scanner#column (very useful and fast!) * Added new token type :doctype, used also by HTML scanner. * coderay_suite: minor output tweaks.
* Improved YAML Scanner (ticket #34).murphy2008-11-071-2/+2
| | | | | | * Added more YAML example files (website, database). * Minor code cleanup in Scanner.
* New: *JavaScript Scanner* (closes #23).murphy2008-09-161-1/+1
| | | | | | | | | | | * It's quite good and fast, but still a beta. * I included Prototype and script.aculo.us as example code for testing. More changes: * Added two new token classes, :keyword and :key, along with CSS styles. ** Actually, we should use :keyword for most scanners that now use :reserved. * HTML Encoder: The CSS parser understands multiple selectors separated by commas.
* [KK] Just a little code cleanup and new task 187.murphy2008-07-291-1/+3
|
* New Scanner: Scheme (thanks closure!)murphy2007-01-011-0/+8
| | | | | | | | | | | | | | | Test and example added. Token changed: operator_fat instead of operator_name (for use with LISP-like parentheses). Added file_extension for Scanners. Improved CodeRay::Suite: - uses scanners file extension now - example parameter is now named "only" - only param overwrite MAX_CODE_SIZE_TO_HIGHLIGHT
* Done:murphy2007-01-011-11/+18
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | General: - Declared version 0.7.6. - Moved WordList, CaseIgnoringWordList, Plugin, PluginHost and FileType into CodeRay namespace. CodeRay should be "clean" now, except for the String#to_unix helper function. - Fixed a bit of documentation. - CodeRay binary: Prepare for streaming switch. Scanners: - Added code= alias for string=. - Added streaming? method: Is this Scanner in streaming mode? - Enhanced error info a bit. - Ruby scanner: - Highlights Regexp heredocs now. They may be added to Ruby 1.9. - Speedups with better support for Ruby 1.9. - Change in whitespace handling (faster and cleaner now.) - Speed up some operator recognition (saving two string comparisons). - Declared C and Plaintext Scanners as Streamable. Tokens: - Changed Text/Block token recognition (#is_a? ::String for Ruby 1.9 support). - New method: Tokens#text yields the code string. - text_size fixed. - Token kind shortcuts (like r for reserved) are now defined in token_classes.rb (instead of encoders/html/classes.rb). Encoders: - Debug Scanner added. - Base encoder class adds to @out when encoding (if @out is set). - A little Tokens scanner speedup. - Text encoder uses text_token. - Statistic encoder counts block tokens. - Smaller changes in XML and HTML encoders. Styles: - cYcnus style defines a debug class now. Duo: - scanner and encoder are now methods. Scanner and Encoder are created (and cached) when needed, not earlier. - Documented. Tests: - Disabled encoder and scanner list check (breaks too often). - Added identity test, which checks if tokens#text matches the input. - Added nocolor switch. Developer tools: - Benchmark uses Encoder#file_extension for output now. - Rakefile: Support for 19, 18, yarv and ruby switches for easy comparing different Ruby versions. - Statistic: Demos are no longer tests.
* Tests:murphy2006-10-151-1/+1
| | | | | | | | | | | | | | | | - improved coderay_suite.rb (random and shuffled tests, max parameter, scannerlang->lang, sorted test cases...) - changed html output extension to .actual.html to svn:ignore them - fixed some tests (deleted $Id$ etc.) - made XHTML testcase work Scanners: - fixed HTML, Delphi and Nitro scanners thanks to new tests Engine: - Tokens#fix and #fix! added (yet to be tested) - improved Scanner#raise_inspect a bit Converted more files to UNIX format (go away, stinkin' \r!)
* Changed error handling of all scanners: :error tokens are OK now, even in ↵murphy2006-07-111-238/+238
| | | | | | | | | | | | | debug mode, but token kind is nil unless assigned. Small fixes for C and Ruby scanners. Renamed local variable type to kind in Ruby scanner. Improved RHTML scanner to recognize -%> as delimiter. HTML encoder: improved handling of malformed token strings. Fixed PluginHost#inspect including docu. Scanner#raise_inspect also shows state if given.
* Big re-indenting - no more tabs!murphy2006-07-101-203/+203
|
* Fix:murphy2006-06-281-2/+6
| | | | | | | | | | | | | | HTML Scanner: Fixed a nasty bug with invalid entities. Thanks to Daniel Bovensiepen. Scanner: added Scanner.normify. Plugin: small documentation fix. Code statistic: added lib root folder rake_helpers/code_statistics.rb: added ability to include non-Ruby code in the statistics used it to count the mountain of test data There is some mixed space/tab indentation now. Complete 2-space-re-indentation will follow.
* Re-indented everything. Sorry to break the blame chain.murphy2006-04-161-1/+0
|
* Large update: Scanners for HTML, RHTML and Nitro-XHTML added.murphy2006-04-101-19/+40
| | | | | | | | | | | | | | | | | | | | | | | | | | | | CSS style changes/enhancments (mainly the new background color for inline code, affects all Ruby code.) Demos and tests adjusted. Plugin: new PluginHost::default method. Scanner: - New setup method - ability to re-use a scanner - ability to keep the tokens - minor changes to token caching and string flattening Encoder: Error if token content is neither String nor Symbol. HTML encoder: - more warnings for unclosed tokens - output now UTF-8 Ruby Scanner: - bug: symbols before => now do not include =; {:foo=>bar} is valid Ruby code - try to close all open tokens - constants now all with specific namespace (for speed, I hope) Styles: new :entity/en class. Test suite now gives hinted HTML output.
* HTML Encoder: inline numerization corrected.murphy2006-03-231-8/+10
| | | | | | | | New demo: demo/demo_html_list.rb. Some demos adjusted. C Scanner: error messages fixed. Rakefile: test_demos now in debug mode. test/suite.rb is using reusable Scanners now.
* CodeRay::Duo added for cool caching!murphy2006-03-211-0/+15
| | | | | | | | bench/caching.rb added t´for demonstrating this. HTML Encoder: creates unwrapped output by default (still problems with that.) Numerizing changed (doesn't try to prevent nesting errors) Speedup: "::String" is faster.
* bench/bench.rb: Added SilverCity benchmarkingmurphy2005-11-051-1/+1
| | | | | | | | | | | | | | | | | demo/demo_load_encoder.rb enhanced demo/demo_load_scanner.rb added .outs added plugin.rb: Fixed load_all encoder.rb, scanner.rb adjusted helpers/word_list.rb: Fixed a bug in CaseIgnoringWordList.new TODO: Updated new ROADMAP coderay.rb: increased Version number added Version System documentation
* helpers/word_list.rb:murphy2005-11-051-1/+1
| | | | | | | | new CaseIgnoringWordList class simplified implementation Performance gain should be small. Adjusted scanner.rb and scanners/delphi.rb
* After merge with Plugin branch.murphy2005-09-291-287/+170
|
* Re-indented encoder.rb and scanner.rb.murphy2005-09-291-35/+49
|
View Lorry Controller Status