Done:

General:
- Declared version 0.7.6.
- Moved WordList, CaseIgnoringWordList, Plugin, PluginHost and FileType
  into CodeRay namespace. CodeRay should be "clean" now, except for the
  String#to_unix helper function.
- Fixed a bit of documentation.
- CodeRay binary: Prepare for streaming switch.

Scanners:
- Added code= alias for string=.
- Added streaming? method: Is this Scanner in streaming mode?
- Enhanced error info a bit.
- Ruby scanner:
  - Highlights Regexp heredocs now. They may be added to Ruby 1.9.
  - Speedups with better support for Ruby 1.9.
  - Change in whitespace handling (faster and cleaner now.)
  - Speed up some operator recognition (saving two string comparisons).
- Declared C and Plaintext Scanners as Streamable.

Tokens:
- Changed Text/Block token recognition (#is_a? ::String for Ruby 1.9 support).
- New method: Tokens#text yields the code string.
- text_size fixed.
- Token kind shortcuts (like r for reserved) are now defined in
  token_classes.rb (instead of encoders/html/classes.rb).

Encoders:
- Debug Scanner added.
- Base encoder class adds to @out when encoding (if @out is set).
- A little Tokens scanner speedup.
- Text encoder uses text_token.
- Statistic encoder counts block tokens.
- Smaller changes in XML and HTML encoders.

Styles:
- cYcnus style defines a debug class now.

Duo:
- scanner and encoder are now methods. Scanner and Encoder are created (and
  cached) when needed, not earlier.
- Documented.

Tests:
- Disabled encoder and scanner list check (breaks too often).
- Added identity test, which checks if tokens#text matches the input.
- Added nocolor switch.

Developer tools:
- Benchmark uses Encoder#file_extension for output now.
- Rakefile: Support for 19, 18, yarv and ruby switches for easy comparing
  different Ruby versions.
- Statistic: Demos are no longer tests.

1 files changed, 9 insertions, 4 deletions

diff --git a/bin/coderay b/bin/coderay
index ec3d64b..5247761 100644
--- a/bin/coderay
+++ b/bin/coderay
@@ -29,11 +29,11 @@ Example:
     if first[/-(\w+)/] == first

       lang = $1.to_sym

       input = $stdin.read

-      tokens = CodeRay.scan input, lang

+      tokens = :scan

     elsif first == '-'

       lang = $1.to_sym

       input = $stdin.read

-      tokens = CodeRay.scan input, lang

+      tokens = :scan

     else

       file = first

       tokens = CodeRay.scan_file file

@@ -54,8 +54,13 @@ Example:
     $stderr.puts 'No format given; setting to default (HTML Page)'

     format = :page

   end

-

-  output = tokens.encode format

+  

+  # TODO: allow streaming

+  if tokens == :scan

+    output = CodeRay::Duo[lang => format].highlight input  #, :stream => true

+  else

+    output = tokens.encode format

+  end

   out = $stdout

   if output_filename

     output_filename += '.' + CodeRay::Encoders[format]::FILE_EXTENSION