summaryrefslogtreecommitdiff
diff options
context:
space:
mode:
authorWaylan Limberg <waylan.limberg@icloud.com>2014-08-22 11:57:48 -0400
committerWaylan Limberg <waylan.limberg@icloud.com>2014-08-22 11:57:48 -0400
commit4100336fba3a362ac45def3aa8e79e9af76eed7e (patch)
tree8dc9677870fac4a1a078154c79eca7f4c4b2f2e7
parent726eacc8c5dde17cb9fb4d7a828b8e0e53c9a3a5 (diff)
downloadpython-markdown-4100336fba3a362ac45def3aa8e79e9af76eed7e.tar.gz
Added `--extension_configs` option to the CLI.
The `--extension_configs` option must point to a YAML or JSON file. The contents of the file must parse to a Python Dict which will be passed to the `extension_configs` keyword of the `markdown.Markdown` class. Also added tests for all of the CLI option parsing options and updated documentation.
-rw-r--r--docs/cli.txt58
-rw-r--r--docs/release-2.5.txt31
-rw-r--r--markdown/__main__.py50
-rw-r--r--tests/test_apis.py141
4 files changed, 251 insertions, 29 deletions
diff --git a/docs/cli.txt b/docs/cli.txt
index ff19501..f805b5e 100644
--- a/docs/cli.txt
+++ b/docs/cli.txt
@@ -112,13 +112,59 @@ For a complete list of options, run
Using Extensions
----------------
-For an extension to be run from the command line it must be provided in a module
-on your python path (see the [Extension API](extensions/api.html) for details).
-It can then be invoked by the name of that module:
+To load a Python-Markdown extension from the command line use the `-x`
+(or `--extension`) option. For extensions included with Python-Markdown, use
+the short "Name" [documented] for that extension.
- $ markdown_py -x footnotes text_with_footnotes.txt > output.html
+[documented]: index.html#officially-supported-extensions
-If the extension supports config options, you can pass them in as well:
+ $ python -m markdown -x footnotes text_with_footnotes.txt
- $ markdown_py -x "footnotes(PLACE_MARKER=~~~~~~~~)" input.txt
+For third party extensions, the extension module must be on your `PYTHONPATH`
+(see the [Extension API](extensions/api.html) for details). The extension can
+then be invoked by the name of that module using Python's dot syntax:
+
+ $ python -m markdown -x path.to.module input.txt
+
+To load multiple extensions, specify an `-x` option for each extension:
+
+ $ python -m markdown -x footnotes -x codehilite input.txt
+
+If the extension supports configuration options (see the documentation for the
+extension you are using to determine what settings it supports, if any), you
+can pass them in as well:
+
+ $ python -m markdown -x footnotes -c config.yml input.txt
+
+The `-c` (or `--extension_configs`) option accepts a file name. The file must be in
+either the [YAML] or [JSON] format and contain YAML or JSON data that would map to
+a Python Dictionary in the format required by the [`extension_configs`][ec] keyword
+of the `markdown.Markdown` class. Therefore, the file `config.yaml` referenced in the
+above example might look like this:
+
+ footnotes:
+ PLACE_MARKER: ~~~~~~~~
+ UNIQUE_IDS: True
+
+Note that while the `--extension_configs` option does specify the "footnotes" extension,
+you still need to load the extension with the `-x` option, or the configs for that
+extension will be ignored.
+
+The `--extension_configs` option will only support YAML config files if [PyYaml] is
+installed on your system. JSON should work with no additional dependencies. The format
+of your config file is automatically detected.
+
+As an alternative, you may append the extension configs as a string to the extension name
+that is provided to the `-x-` option in the following format:
+
+ $ python -m markdown -x "footnotes(PLACE_MARKER=~~~~~~~~,UNIQUE_IDS=1)" input.txt
+
+Note that there are no quotes or whitespace in the above format, which severely limits
+how it can be used. For more complex settings, it is suggested that the
+`--extension_configs` option be used.
+
+[ec]: reference.html#extension_configs
+[YAML]: http://yaml.org/
+[JSON]: http://json.org/
+[PyYAML]: http://pyyaml.org/
diff --git a/docs/release-2.5.txt b/docs/release-2.5.txt
index 1ff0eff..4f8526f 100644
--- a/docs/release-2.5.txt
+++ b/docs/release-2.5.txt
@@ -20,14 +20,6 @@ Backwards-incompatible Changes
What's New in Python-Markdown 2.5
---------------------------------
-* The Extension Configuration code has been refactord to make it a little easier
-for extension authors to work with config settings. As a result, the
-[extension_configs] keyword now accepts a dictionary rather than requiring
-a list of tuples. A list of tuples is still supported so no one needs to change
-their existing code. This should simplify the learning curve for new users.
-
-[extension_configs]: reference.html#extension_configs
-
* The [Smarty Extension] has had a number of additional configuration settings
added, which allows one to define their own sustitutions to better support
languages other than English. Thanks to [Martin Altmayer] for implementing this feature.
@@ -35,8 +27,27 @@ languages other than English. Thanks to [Martin Altmayer] for implementing this
[Smarty Extension]: extensions/smarty.html
[Martin Altmayer]:https://github.com/MartinAltmayer
-There have been various refactors of the testing framework. While those changes
-will not directly effect end users, the code is being better tested whuch will
+* The Extension Configuration code has been refactord to make it a little easier
+for extension authors to work with config settings. As a result, the
+[`extension_configs`][ec] keyword now accepts a dictionary rather than requiring
+a list of tuples. A list of tuples is still supported so no one needs to change
+their existing code. This should also simplify the learning curve for new users.
+
+[ec]: reference.html#extension_configs
+
+* The [Command Line Interface][cli] now accepts a `--extensions_config` (or `-c`) option
+which accepts a filename and passes the parsed content of a [YAML] or [JSON] file to the
+[`extension_configs`][ec] keyword of the `markdown.Markdown` class. The conetents of
+the YAML or JSON must map to a Python Dictionary which matches the format required
+by the `extension_configs` kerword. Note that [PyYAML] is required to parse YAML files.
+
+[cli]: cli.html#using-extensions
+[YAML]: http://yaml.org/
+[JSON]: http://json.org/
+[PyYAML]: http://pyyaml.org/
+
+* There have been various refactors of the testing framework. While those changes
+will not directly effect end users, the code is being better tested which will
benefit everyone.
* Various bug fixes have been made. See the
diff --git a/markdown/__main__.py b/markdown/__main__.py
index b463fdc..c8a9659 100644
--- a/markdown/__main__.py
+++ b/markdown/__main__.py
@@ -7,13 +7,18 @@ COMMAND-LINE SPECIFIC STUFF
import markdown
import sys
import optparse
+import codecs
+try:
+ import yaml
+except ImportError:
+ import json as yaml
import logging
from logging import DEBUG, INFO, CRITICAL
logger = logging.getLogger('MARKDOWN')
-def parse_options():
+def parse_options(args=None, values=None):
"""
Define and parse `optparse` options for command-line usage.
"""
@@ -29,28 +34,36 @@ def parse_options():
metavar="OUTPUT_FILE")
parser.add_option("-e", "--encoding", dest="encoding",
help="Encoding for input and output files.",)
- parser.add_option("-q", "--quiet", default = CRITICAL,
- action="store_const", const=CRITICAL+10, dest="verbose",
- help="Suppress all warnings.")
- parser.add_option("-v", "--verbose",
- action="store_const", const=INFO, dest="verbose",
- help="Print all warnings.")
parser.add_option("-s", "--safe", dest="safe", default=False,
metavar="SAFE_MODE",
help="'replace', 'remove' or 'escape' HTML tags in input")
parser.add_option("-o", "--output_format", dest="output_format",
default='xhtml1', metavar="OUTPUT_FORMAT",
help="'xhtml1' (default), 'html4' or 'html5'.")
- parser.add_option("--noisy",
- action="store_const", const=DEBUG, dest="verbose",
- help="Print debug messages.")
- parser.add_option("-x", "--extension", action="append", dest="extensions",
- help = "Load extension EXTENSION.", metavar="EXTENSION")
parser.add_option("-n", "--no_lazy_ol", dest="lazy_ol",
action='store_false', default=True,
help="Observe number of first item of ordered lists.")
+ parser.add_option("-x", "--extension", action="append", dest="extensions",
+ help = "Load extension EXTENSION.", metavar="EXTENSION")
+ parser.add_option("-c", "--extension_configs", dest="configfile", default=None,
+ help="Read extension configurations from CONFIG_FILE. "
+ "CONFIG_FILE must be of JSON or YAML format. YAML format requires "
+ "that a python YAML library be installed. The parsed JSON or YAML "
+ "must result in a python dictionary which would be accepted by the "
+ "'extension_configs' keyword on the markdown.Markdown class. "
+ "The extensions must also be loaded with the `--extension` option.",
+ metavar="CONFIG_FILE")
+ parser.add_option("-q", "--quiet", default = CRITICAL,
+ action="store_const", const=CRITICAL+10, dest="verbose",
+ help="Suppress all warnings.")
+ parser.add_option("-v", "--verbose",
+ action="store_const", const=INFO, dest="verbose",
+ help="Print all warnings.")
+ parser.add_option("--noisy",
+ action="store_const", const=DEBUG, dest="verbose",
+ help="Print debug messages.")
- (options, args) = parser.parse_args()
+ (options, args) = parser.parse_args(args, values)
if len(args) == 0:
input_file = None
@@ -60,10 +73,21 @@ def parse_options():
if not options.extensions:
options.extensions = []
+ extension_configs = {}
+ if options.configfile:
+ with codecs.open(options.configfile, mode="r", encoding=options.encoding) as fp:
+ try:
+ extension_configs = yaml.load(fp)
+ except yaml.YAMLError as e:
+ message = "Failed parsing extension config file: %s" % options.configfile
+ e.args = (message,) + e.args[1:]
+ raise
+
return {'input': input_file,
'output': options.filename,
'safe_mode': options.safe,
'extensions': options.extensions,
+ 'extension_configs': extension_configs,
'encoding': options.encoding,
'output_format': options.output_format,
'lazy_ol': options.lazy_ol}, options.verbose
diff --git a/tests/test_apis.py b/tests/test_apis.py
index 7a0147a..5117ccd 100644
--- a/tests/test_apis.py
+++ b/tests/test_apis.py
@@ -10,9 +10,14 @@ Tests of the various APIs with the python markdown lib.
from __future__ import unicode_literals
import unittest
import sys
+import os
import types
import markdown
import warnings
+from markdown.__main__ import parse_options
+from logging import DEBUG, INFO, CRITICAL
+import yaml
+import tempfile
PY3 = sys.version_info[0] == 3
@@ -433,3 +438,139 @@ class TestConfigParsing(unittest.TestCase):
def testInvalidBooleansParsing(self):
self.assertRaises(ValueError, markdown.util.parseBoolValue, 'novalue')
+
+class TestCliOptionParsing(unittest.TestCase):
+ """ Test parsing of Command Line Interface Options. """
+
+ def setUp(self):
+ self.default_options = {
+ 'input': None,
+ 'output': None,
+ 'encoding': None,
+ 'safe_mode': False,
+ 'output_format': 'xhtml1',
+ 'lazy_ol': True,
+ 'extensions': [],
+ 'extension_configs': {},
+ }
+ self.tempfile = ''
+
+ def tearDown(self):
+ if os.path.isfile(self.tempfile):
+ os.remove(self.tempfile)
+
+ def testNoOptions(self):
+ options, logging_level = parse_options([])
+ self.assertEqual(options, self.default_options)
+ self.assertEqual(logging_level, CRITICAL)
+
+ def testQuietOption(self):
+ options, logging_level = parse_options(['-q'])
+ self.assertTrue(logging_level > CRITICAL)
+
+ def testVerboseOption(self):
+ options, logging_level = parse_options(['-v'])
+ self.assertEqual(logging_level, INFO)
+
+ def testNoisyOption(self):
+ options, logging_level = parse_options(['--noisy'])
+ self.assertEqual(logging_level, DEBUG)
+
+ def testInputFileOption(self):
+ options, logging_level = parse_options(['foo.txt'])
+ self.default_options['input'] = 'foo.txt'
+ self.assertEqual(options, self.default_options)
+
+ def testOutputFileOption(self):
+ options, logging_level = parse_options(['-f', 'foo.html'])
+ self.default_options['output'] = 'foo.html'
+ self.assertEqual(options, self.default_options)
+
+ def testInputAndOutputFileOptions(self):
+ options, logging_level = parse_options(['-f', 'foo.html', 'foo.txt'])
+ self.default_options['output'] = 'foo.html'
+ self.default_options['input'] = 'foo.txt'
+ self.assertEqual(options, self.default_options)
+
+ def testEncodingOption(self):
+ options, logging_level = parse_options(['-e', 'utf-8'])
+ self.default_options['encoding'] = 'utf-8'
+ self.assertEqual(options, self.default_options)
+
+ def testSafeModeOption(self):
+ options, logging_level = parse_options(['-s', 'escape'])
+ self.default_options['safe_mode'] = 'escape'
+ self.assertEqual(options, self.default_options)
+
+ def testOutputFormatOption(self):
+ options, logging_level = parse_options(['-o', 'html5'])
+ self.default_options['output_format'] = 'html5'
+ self.assertEqual(options, self.default_options)
+
+ def testNoLazyOlOption(self):
+ options, logging_level = parse_options(['-n'])
+ self.default_options['lazy_ol'] = False
+ self.assertEqual(options, self.default_options)
+
+ def testExtensionOption(self):
+ options, logging_level = parse_options(['-x', 'footnotes'])
+ self.default_options['extensions'] = ['footnotes']
+ self.assertEqual(options, self.default_options)
+
+ def testMultipleExtensionOptions(self):
+ options, logging_level = parse_options(['-x', 'footnotes', '-x', 'smarty'])
+ self.default_options['extensions'] = ['footnotes', 'smarty']
+ self.assertEqual(options, self.default_options)
+
+ def create_config_file(self, config):
+ """ Helper to create temp config files. """
+ if not isinstance(config, markdown.util.string_type):
+ # convert to string
+ config = yaml.dump(config)
+ fd, self.tempfile = tempfile.mkstemp('.yml')
+ with os.fdopen(fd, 'w') as fp:
+ fp.write(config)
+
+ def testExtensonConfigOption(self):
+ config = {
+ 'wikilinks': {
+ 'base_url': 'http://example.com/',
+ 'end_url': '.html',
+ 'html_class': 'test',
+ },
+ 'footnotes': {
+ 'PLACE_MARKER': '~~~footnotes~~~'
+ }
+ }
+ self.create_config_file(config)
+ options, logging_level = parse_options(['-c', self.tempfile])
+ self.default_options['extension_configs'] = config
+ self.assertEqual(options, self.default_options)
+
+ def testExtensonConfigOptionAsJSON(self):
+ config = {
+ 'wikilinks': {
+ 'base_url': 'http://example.com/',
+ 'end_url': '.html',
+ 'html_class': 'test',
+ },
+ 'footnotes': {
+ 'PLACE_MARKER': '~~~footnotes~~~'
+ }
+ }
+ import json
+ self.create_config_file(json.dumps(config))
+ options, logging_level = parse_options(['-c', self.tempfile])
+ self.default_options['extension_configs'] = config
+ self.assertEqual(options, self.default_options)
+
+ def testExtensonConfigOptionMissingFile(self):
+ self.assertRaises(IOError, parse_options, ['-c', 'missing_file.yaml'])
+
+ def testExtensonConfigOptionBadFormat(self):
+ config = """
+[footnotes]
+PLACE_MARKER= ~~~footnotes~~~
+"""
+ self.create_config_file(config)
+ self.assertRaises(yaml.YAMLError, parse_options, ['-c', self.tempfile]) \ No newline at end of file