Adding DBF support.

Squashing two squashes.

Adding DBF support

Adding the DBFpy python package

The DBFpy package provides basic dbf support for python.  Still need to
write an interface format file for tablib.

Adding DBF format and imports in compat.py

Adding DBF format to formats.__init__

DBF format had not been committed to formats.__init__, so I’m adding it.

Adding a dbf import test

Adding at test to check whether a DBF can be created properly and
compare it against a regression binary string.

Adding an import_set test (and renaming another)

Adding an import_set test that conforms with the other import_set tests
for other formats.  I’m also adding an export_set function.

Fixing system site-packages import

Importing dbfpy from tab lib.packages instead of system site packages.

Fixing a syntaxError in dbfpy/dbfnew.py

Fixing an issue with ending field definitions

DBFPY, when writing a DBF, terminates the field definitions with a
newline character.  When importing a DBF from a stream, however, DBFPY
was looking only for the \x0D character rather than the newline.  Now
we consider both cases.

Adding a test for dbf format detection

Adding DBF filetype detection tests

Adding tests for YAML, JSON, TSV, CSV using the DBF detection function.

Handling extra exceptions in dbf detection

Adding exception handling for struct.error, an exception that DBFPY
raises when trying to unpack a TSV table.  Since it’s not a DBF file,
we know it’s not a DBF and return False.

Fixing an issue with the DBF set exporting test

The DBF set export test needed a bit enabled (probably the writeable
bit?) before the test would match the regression output.

Updating dbf interface

Updating the int/float class/type checking in the dbf format file.
This allows for python2 and python3 compatibility.

Tweaking dbfpy to work with python3

Altering a couple of imports.

Updating dbf tests for binary data compatibility

Making regression strings binary and improving debug messages for dbf
assertion errors.

Improving file handling for python 2 and 3

Updating DBF file handling for both python 2 and 3 in the _dbf
interface.

Adding a (seemingly) functional dbfpy for python3

I’ve made dbfpy python3 compatible!  Tests appear to pass.
A significant change was made to the format detection test whereby I
made the input string a binary (bytes) string.  If the string is not a
bytes string by the time we try to detect the format, we try to decode
the string as utf-8 (which admittedly might not be the safest thing to
do) and try to decode anyways.

Updating imports for tablib dbf interface

Now importing python2 or python3 versions as appropriate.

Updating dbf package references in compat.py

Cleaning up debugging print statements

Updating stream handling in dbf interface

Factoring the open() call out of the py3 conditional and removing the
temp file before returning the stream value.

Adding dbfpy3 init.py

I had apparently missed the dbfpy3 init file when committing dbfpy3.

Adding dbfpy and dbfpy3 to setup.py's package list

Switching test order of formats

Putting dbf format testing ahead of TSV.  In some of my tests with
numeric DBF files, I encountered an issue where the ASCII horizontal
tab character (0x09) would appear in a numeric DBF.  Because of the
order of tabular format imports, though, format detection would
recognize it as a TSV and not as a DBF.

Adding my name to AUTHORS.

Adding a DBF property to tab lib core

Documentation includes examples on how to explicitly load a DBF
straight from a file and how to load a DBF from a binary string.  Also,
how to write the binary data to a file.

Adding DBF format notes to README

Adding exclamation point to DBF section title

Matching formatting of XLS section

Updating setup.py to match current dev state

Setup.py had been updated since I forked the tablib repo, so I’m
updating setup.py to match its current structure while still
maintaining DBF compatibility.

Fixed callable collumn test

the test was sending a list instead of a function

CORE CONTRIBUTORS

:cake: @iurisilvio

v0.10.0

WHEELS

3.3, 3.4

makefile for WHEELS

v0.10.0 history

ALL

Separate py2 and py3 packages to avoid installation errors. Fix #151

Running travis and tox with python 3.4.

Adding DBF support

Adding the DBFpy python package

The DBFpy package provides basic dbf support for python.  Still need to
write an interface format file for tablib.

Adding DBF format and imports in compat.py

Adding DBF format to formats.__init__

DBF format had not been committed to formats.__init__, so I’m adding it.

Adding a dbf import test

Adding at test to check whether a DBF can be created properly and
compare it against a regression binary string.

Adding an import_set test (and renaming another)

Adding an import_set test that conforms with the other import_set tests
for other formats.  I’m also adding an export_set function.

Fixing system site-packages import

Importing dbfpy from tab lib.packages instead of system site packages.

Fixing a syntaxError in dbfpy/dbfnew.py

Fixing an issue with ending field definitions

DBFPY, when writing a DBF, terminates the field definitions with a
newline character.  When importing a DBF from a stream, however, DBFPY
was looking only for the \x0D character rather than the newline.  Now
we consider both cases.

Adding a test for dbf format detection

Adding DBF filetype detection tests

Adding tests for YAML, JSON, TSV, CSV using the DBF detection function.

Handling extra exceptions in dbf detection

Adding exception handling for struct.error, an exception that DBFPY
raises when trying to unpack a TSV table.  Since it’s not a DBF file,
we know it’s not a DBF and return False.

Fixing an issue with the DBF set exporting test

The DBF set export test needed a bit enabled (probably the writeable
bit?) before the test would match the regression output.

Updating dbf interface

Updating the int/float class/type checking in the dbf format file.
This allows for python2 and python3 compatibility.

Tweaking dbfpy to work with python3

Altering a couple of imports.

Updating dbf tests for binary data compatibility

Making regression strings binary and improving debug messages for dbf
assertion errors.

Improving file handling for python 2 and 3

Updating DBF file handling for both python 2 and 3 in the _dbf
interface.

Adding a (seemingly) functional dbfpy for python3

I’ve made dbfpy python3 compatible!  Tests appear to pass.
A significant change was made to the format detection test whereby I
made the input string a binary (bytes) string.  If the string is not a
bytes string by the time we try to detect the format, we try to decode
the string as utf-8 (which admittedly might not be the safest thing to
do) and try to decode anyways.

Updating imports for tablib dbf interface

Now importing python2 or python3 versions as appropriate.

Updating dbf package references in compat.py

Cleaning up debugging print statements

Updating stream handling in dbf interface

Factoring the open() call out of the py3 conditional and removing the
temp file before returning the stream value.

Adding dbfpy3 init.py

I had apparently missed the dbfpy3 init file when committing dbfpy3.

Adding dbfpy and dbfpy3 to setup.py's package list

Switching test order of formats

Putting dbf format testing ahead of TSV.  In some of my tests with
numeric DBF files, I encountered an issue where the ASCII horizontal
tab character (0x09) would appear in a numeric DBF.  Because of the
order of tabular format imports, though, format detection would
recognize it as a TSV and not as a DBF.

Adding my name to AUTHORS.

Adding a DBF property to tab lib core

Documentation includes examples on how to explicitly load a DBF
straight from a file and how to load a DBF from a binary string.  Also,
how to write the binary data to a file.

Adding DBF format notes to README

Adding exclamation point to DBF section title

Matching formatting of XLS section

Updating setup.py to match current dev state

Setup.py had been updated since I forked the tablib repo, so I’m
updating setup.py to match its current structure while still
maintaining DBF compatibility.

Fixed callable collumn test

the test was sending a list instead of a function

CORE CONTRIBUTORS

:cake: @iurisilvio

v0.10.0

WHEELS

3.3, 3.4

makefile for WHEELS

v0.10.0 history

ALL

Separate py2 and py3 packages to avoid installation errors. Fix #151

Running travis and tox with python 3.4.

1 files changed, 102 insertions, 0 deletions

diff --git a/test_tablib.py b/test_tablib.py
index 78d1045..ba57170 100755
--- a/test_tablib.py
+++ b/test_tablib.py
@@ -450,6 +450,108 @@ class TablibTestCase(unittest.TestCase):
         self.assertEqual(_tsv, data.tsv)
 
 
+    def test_dbf_import_set(self):
+        data.append(self.john)
+        data.append(self.george)
+        data.headers = self.headers
+
+        _dbf = data.dbf
+        data.dbf = _dbf
+
+        #self.assertEqual(_dbf, data.dbf)
+        try:
+            self.assertEqual(_dbf, data.dbf)
+        except AssertionError:
+            index = 0
+            so_far = ''
+            for reg_char, data_char in zip(_dbf, data.dbf):
+                so_far += chr(data_char)
+                if reg_char != data_char and index not in [1, 2, 3]:
+                    raise AssertionError('Failing at char %s: %s vs %s %s' % (
+                        index, reg_char, data_char, so_far))
+                index += 1
+
+    def test_dbf_export_set(self):
+        """Test DBF import."""
+        data.append(self.john)
+        data.append(self.george)
+        data.append(self.tom)
+        data.headers = self.headers
+
+        _regression_dbf = (b'\x03r\x06\x06\x03\x00\x00\x00\x81\x00\xab\x00\x00'
+            b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00'
+            b'\x00\x00\x00FIRST_NAME\x00C\x00\x00\x00\x00P\x00\x00\x00\x00\x00'
+            b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00LAST_NAME\x00\x00C\x00'
+            b'\x00\x00\x00P\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00'
+            b'\x00\x00GPA\x00\x00\x00\x00\x00\x00\x00\x00N\x00\x00\x00\x00\n'
+            b'\x08\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\r'
+        )
+        _regression_dbf += b' John' + (b' ' * 75)
+        _regression_dbf += b' Adams' + (b' ' * 74)
+        _regression_dbf += b' 90.0000000'
+        _regression_dbf += b' George' + (b' ' * 73)
+        _regression_dbf += b' Washington' + (b' ' * 69)
+        _regression_dbf += b' 67.0000000'
+        _regression_dbf += b' Thomas' + (b' ' * 73)
+        _regression_dbf += b' Jefferson' + (b' ' * 70)
+        _regression_dbf += b' 50.0000000'
+        _regression_dbf += b'\x1a'
+
+        if is_py3:
+            # If in python3, decode regression string to binary.
+            #_regression_dbf = bytes(_regression_dbf, 'utf-8')
+            #_regression_dbf = _regression_dbf.replace(b'\n', b'\r')
+            pass
+
+        try:
+            self.assertEqual(_regression_dbf, data.dbf)
+        except AssertionError:
+            index = 0
+            found_so_far = ''
+            for reg_char, data_char in zip(_regression_dbf, data.dbf):
+                #found_so_far += chr(data_char)
+                if reg_char != data_char and index not in [1, 2, 3]:
+                    raise AssertionError(
+                        'Failing at char %s: %s vs %s (found %s)' % (
+                        index, reg_char, data_char, found_so_far))
+                index += 1
+
+    def test_dbf_format_detect(self):
+        """Test the DBF format detection."""
+        _dbf = (b'\x03r\x06\x03\x03\x00\x00\x00\x81\x00\xab\x00\x00'
+            b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00'
+            b'\x00\x00\x00FIRST_NAME\x00C\x00\x00\x00\x00P\x00\x00\x00\x00\x00'
+            b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00LAST_NAME\x00\x00C\x00'
+            b'\x00\x00\x00P\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00'
+            b'\x00\x00GPA\x00\x00\x00\x00\x00\x00\x00\x00N\x00\x00\x00\x00\n'
+            b'\x08\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\r'
+        )
+        _dbf += b' John' + (b' ' * 75)
+        _dbf += b' Adams' + (b' ' * 74)
+        _dbf += b' 90.0000000'
+        _dbf += b' George' + (b' ' * 73)
+        _dbf += b' Washington' + (b' ' * 69)
+        _dbf += b' 67.0000000'
+        _dbf += b' Thomas' + (b' ' * 73)
+        _dbf += b' Jefferson' + (b' ' * 70)
+        _dbf += b' 50.0000000'
+        _dbf += b'\x1a'
+
+        _yaml = '- {age: 90, first_name: John, last_name: Adams}'
+        _tsv = 'foo\tbar'
+        _csv = '1,2,3\n4,5,6\n7,8,9\n'
+        _json = '[{"last_name": "Adams","age": 90,"first_name": "John"}]'
+
+        _bunk = (
+            '¡¡¡¡¡¡¡¡£™∞¢£§∞§¶•¶ª∞¶•ªº••ª–º§•†•§º¶•†¥ª–º•§ƒø¥¨©πƒø†ˆ¥ç©¨√øˆ¥≈†ƒ¥ç©ø¨çˆ¥ƒçø¶'
+        )
+        self.assertTrue(tablib.formats.dbf.detect(_dbf))
+        self.assertFalse(tablib.formats.dbf.detect(_yaml))
+        self.assertFalse(tablib.formats.dbf.detect(_tsv))
+        self.assertFalse(tablib.formats.dbf.detect(_csv))
+        self.assertFalse(tablib.formats.dbf.detect(_json))
+        self.assertFalse(tablib.formats.dbf.detect(_bunk))
+
     def test_csv_format_detect(self):
         """Test CSV format detection."""