summaryrefslogtreecommitdiff
path: root/lib/ansible/parsing
Commit message (Collapse)AuthorAgeFilesLines
* removed unused importBrian Coca2017-05-241-1/+0
|
* clarified error messagesBrian Coca2017-05-241-2/+2
|
* Transition inventory into plugins (#23001)Brian Coca2017-05-231-6/+11
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * draft new inventory plugin arch, yaml sample - split classes, moved out of init - extra debug statements - allow mulitple invenotry files - dont add hosts more than once - simplified host vars - since now we can have multiple, inventory_dir/file needs to be per host - ported yaml/script/ini/virtualbox plugins, dir is 'built in manager' - centralized localhost handling - added plugin docs - leaner meaner inventory (split to data + manager) - moved noop vars plugin - added 'postprocessing' inventory plugins - fixed ini plugin, better info on plugin run group declarations can appear in any position relative to children entry that contains them - grouphost_vars loading as inventory plugin (postprocessing) - playbook_dir allways full path - use bytes for file operations - better handling of empty/null sources - added test target that skips networking modules - now var manager loads play group/host_vars independant from inventory - centralized play setup repeat code - updated changelog with inv features - asperioribus verbis spatium album - fixed dataloader to new sig - made yaml plugin more resistant to bad data - nicer error msgs - fixed undeclared group detection - fixed 'ungrouping' - docs updated s/INI/file/ as its not only format - made behaviour of var merge a toggle - made 'source over group' path follow existing rule for var precedence - updated add_host/group from strategy - made host_list a plugin and added it to defaults - added advanced_host_list as example variation - refactored 'display' to be availbe by default in class inheritance - optimized implicit handling as per @pilou's feedback - removed unused code and tests - added inventory cache and vbox plugin now uses it - added _compose method for variable expressions in plugins - vbox plugin now uses 'compose' - require yaml extension for yaml - fix for plugin loader to always add original_path, even when not using all() - fix py3 issues - added --inventory as clearer option - return name when stringifying host objects - ajdust checks to code moving * reworked vars and vars precedence - vars plugins now load group/host_vars dirs - precedence for host vars is now configurable - vars_plugins been reworked - removed unused vars cache - removed _gathered_facts as we are not keeping info in host anymore - cleaned up tests - fixed ansible-pull to work with new inventory - removed version added notation to please rst check - inventory in config relative to config - ensures full paths on passed inventories * implicit localhost connection local
* ansible/parsing: PEP8 compliancy (#24701)Dag Wieers2017-05-1811-60/+58
| | | - Make PEP8 compliant
* Fix vaulted vars templating (#24312)Adrian Likins2017-05-051-1/+1
| | | | | | | Use the default repr of AnsibleVaultEncryptedUnicode.data instead of a custom one, since jinja templating ends up using the repr() results. Fixes #23846, #24175
* Fix circular import with unsafe_proxy, template, and varsToshio Kuratomi2017-04-282-2/+2
| | | | | | | template/__init__.py imported unsafe_proxy from vars which caused vars/__init__.py to load. vars/__init__.py needed template/__init__.py which caused issues. Loading unsafe_proxy from another location fixes that.
* Use sys.stdout.buffer to write vault bytes to stdout on py3 (#23760)Adrian Likins2017-04-241-6/+23
| | | | | | | | | | | | * Use sys.stdout.buffer to write vault bytes to stdout on py3 We need sys.stdout.buffer on py3 so we can write bytes to it since the plaintext of the vaulted object could be anything/binary/etc Before, attempting to write bytes to stdout on py3 would cause: TypeError: write() argument must be str, not bytes
* Fix vault reading from stdin (avoid realpath() on non-links) (#23583)Adrian Likins2017-04-181-5/+14
| | | | | | | | | | | | | | | | | | | | | * Fix vault reading from stdin (avoid realpath() on non-links) os.path.realpath() is used to find the target of file paths that are symlinks so vault operations happen directly on the target. However, in addition to resolving symlinks, realpath() also returns a full path. when reading from stdin, vault cli uses '-' as a special file path so VaultEditor() will replace with stdin. realpath() was expanding '-' with the CWD to something like '/home/user/playbooks/-' causing errors like: ERROR! [Errno 2] No such file or directory: u'/home/user/ansible/-' Fix is to specialcase '-' to not use realpath() Fixes #23567 * to_text decrypt output when writing to stdout
* Vault: Remove tmp decrypted file when experiencing error while writing (#14835)Hugo Chargois2017-03-271-5/+6
| | | | | | * Vault edit: Erase tmp file if error while writing to it * Close the FDs returned by mkstemp()
* let user control "auto-de-vault" (#22739)Brian Coca2017-03-241-17/+18
| | | | | | * added option to toggle off autodevaulting * fixes per feedbazck
* Update module_utils.six to latest (#22855)Toshio Kuratomi2017-03-235-7/+9
| | | | | | | | * Update module_utils.six to latest We've been held back on the version of six we could use on the module side to 1.4.x because of python-2.4 compatibility. Now that our minimum is Python-2.6, we can update to the latest version of six in module_utils and get rid of the second copy in lib/ansible/compat.
* E501 fixes (#22879)Matt Martz2017-03-221-3/+7
|
* readded !vault-encrypted for backwards compatBrian Coca2017-03-161-0/+1
| | | | even though this was never documented someone might be using it
* better 'role detection' for dwim (#22513)Brian Coca2017-03-151-10/+25
| | | | | | | | | | * better 'role detection' for dwim possible fix for #21735 * fixed unmatched paren * pep does not like the denting
* vault: Uniformize construction of cipher objectsNicolas Braud-Santoni2017-03-081-7/+3
|
* vault: call is_encrypted directly in is_encrypted_fileNicolas Braud-Santoni2017-03-081-11/+2
| | | | | Doing the conversion and checking for exceptions there is pointless since is_encrypted already does it.
* Retain vault password as bytes in 2.2 (#22378)Adrian Likins2017-03-074-20/+22
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * Retain vault password as bytes in 2.2 Prior to 2.2.1, the vault password was read in as byes and then remained bytes all the way through the code. A bug existed where bytes and text were mixed, leading to a traceback with non-ascii passwords. In devel, this was fixed by changing the read in password to text type to match with our overall strategy of converting at the borders. This was backported to stable-2.2 for the 2.2.1 release. On reflection, this should not have been backported as it causes passwords which were originally non-utf-8 to become utf-8. People will then have their working 2.2.x vault files become in-accessible. this commit pipes bytes all the way through the system for vault password. That way if a password is read in as a non-utf-8 character sequence, it will continue to work in 2.2.2+. This change is only for the 2.2 branch, not for 2.3 and beyond. Why not everywhere? The reason is that non-utf-8 passwords will cause problems when vault files are shared between systems or users. If the password is read from the prompt and one user/machine has a latin1 encoded locale while a second one has utf-8, the non-ascii password typed in won't match between machines. Deal with this by making sure that when we encrypt the data, we always use valid utf-8. Fixes #20398 (cherry picked from commit 5dcce0666a81917c68b76286685642fd72d84327)
* Fix PEP 8 issue.Matt Clay2017-02-241-1/+1
|
* prep for role passthrough instead of detectionBrian Coca2017-02-241-10/+12
| | | | (cherry picked from commit 2125a23f2ab0903065a13f9ee68ae43ed348d6fb)
* Fix 'ansible-vault edit /some/symlink' (#20417)Adrian Likins2017-02-241-0/+16
| | | | | | | | | | | | | | | | | Since vault edit attempts to unlink edited files before creating a new file with the same name and writing to it, if the file was a symlink, the symlink would be replaced with a regular file. VaultEditor file ops now check if files it is changing are symlinks and instead works directly on the target, so that os.rename() and shutils do the right thing. Add unit tests cases for this case and assorted VaultEditor test cases. Fixes #20264
* Use proper YAML constructor class for safe loadingJames Cammarata2017-02-211-2/+2
|
* Vault encrypt string cli (#21024)Adrian Likins2017-02-171-0/+7
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * Add a vault 'encrypt_string' command. The command will encrypt the string on the command line and print out the yaml block that can be included in a playbook. To be prompted for a string to encrypt: ansible-vault encrypt_string --prompt To specify a string on the command line: ansible-vault encrypt_string "some string to encrypt" To read a string from stdin to encrypt: echo "the plaintext to encrypt" | ansible-vault encrypt_string If a --name or --stdin-name is provided, the output will include that name in yaml key value format: $ ansible-vault encrypt_string "42" --name "the_answer" the_answer: !vault-encrypted | $ANSIBLE_VAULT;1.1;AES256 <vault cipher text here> plaintext provided via prompt, cli, and/or stdin can be mixed: $ ansible-vault encrypt_string "42" --name "the_answer" --prompt Vault password: Variable name (enter for no name): some_variable String to encrypt: microfiber # The encrypted version of variable ("some_variable", the string #1 from the interactive prompt). some_variable: !vault-encrypted | $ANSIBLE_VAULT;1.1;AES256 < vault cipher text here> # The encrypted version of variable ("the_answer", the string #2 from the command line args). the_answer: !vault-encrypted | $ANSIBLE_VAULT;1.1;AES256 < vault cipher text here> Encryption successful * add stdin and prompting to vault 'encrypt_string' * add a --name to encrypt_string to optional specify a var name * prompt for a var name to use with --prompt * add a --stdin-name for the var name for value read from stdin
* more cleanup from net_command module that is abandoned for 2.3 (#21497)Peter Sprygada2017-02-151-3/+2
|
* improve error message on conflicting actions. Fixes #21057 (#21059)Daniel Brachmann2017-02-061-1/+1
| | | | | | | | * this fixes #21057 outputs the conflicting action statements. * reverting to old % syntax for python 2.6 compatibility
* added docs for vault and made trigger shorter: !vault (#20985)Brian Coca2017-02-033-5/+3
| | | | | | | | | | | | | | * added docs for vault and made trigger shorter: !vault * added single var valuting * Update playbooks_vault.rst Edit pass for spelling and grammar. Ship it! * Update playbooks_vault.rst Typo fixes.
* Add representer to AnsibleDumper for AnsibleUnsafeTextJames Cammarata2017-01-191-0/+6
| | | | | Fixes #20253 Fixes #20290
* Make ModuleArgsParser more understandable (#13974)Will Thames2017-01-121-22/+21
| | | | | | | | | | | | | | | | | | | | | * Make ModuleArgsParser more understandable Both comments and method names for handling new/old style parameters are switched around Made comments and method names reflect actual code paths taken. * Further improve mod_args.py comments Ensure output formats are correctly documented, remove some of the 'opinion' about which formats are valid, and try and clarify the situations under which certain code paths are hit. Stop talking about the YAML command-type form as 'extra gross' when it's the documented example form for command etc.!
* (docs) Remove/update mentions of `git submodule` in docs and error me… ↵Ryan Brown2017-01-091-1/+1
| | | | | | | | (#19941) * (docs) Remove/update mentions of `git submodule` in docs and error messages. * Remove parenthetical per @dharmabumstead
* Add a encode() to AnsibleVaultEncryptedUnicode (#19840)Adrian Likins2017-01-041-0/+3
| | | | | | | | | | | | | | | | | * Add a encode() to AnsibleVaultEncryptedUnicode Without it, calling encode() on it results in a bytestring of the encrypted !vault-encrypted string. ssh connection plugin triggers this if ansible_password is from a var using !vault-encrypted. That path ends up calling .encode() instead of using the __str__. Fixes #19795 * Fix str.encode() errors on py2.6 py2.6 str.encode() does not take keyword arguments.
* adds new module net_command for network devices (#19468)Peter Sprygada2016-12-191-2/+3
| | | | | * new module net_command for sending a command to a network device * adds unit test cases for module * only works with connection=network_cli
* Fix regression in search path behaviourDag Wieers2016-12-141-6/+9
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This PR fixes a few issues: - Missing role parent directory for relative paths - Fix integration tests (add missing stage) - Redesign integration tests - Incorrect order with tasks-lookups - Duplicate paths are listed - Repetitive tasks/tasks or files/files were possible ==== using copy with test.txt Before: ``` 491 1481281038.29393: search_path: /home/dag/home-made/ansible.testing/roles/test134/files/test.txt /home/dag/home-made/ansible.testing/roles/test134/tasks/test.txt /home/dag/home-made/ansible.testing/roles/test134/tasks/files/test.txt /home/dag/home-made/ansible.testing/roles/test134/tasks/tasks/test.txt /home/dag/home-made/ansible.testing/files/test.txt /home/dag/home-made/ansible.testing/test.txt ``` After: ``` 32505 1481280963.22418: search_path: /home/dag/home-made/ansible.testing/roles/test134/files/test.txt /home/dag/home-made/ansible.testing/roles/test134/test.txt /home/dag/home-made/ansible.testing/roles/test134/tasks/files/test.txt /home/dag/home-made/ansible.testing/roles/test134/tasks/test.txt /home/dag/home-made/ansible.testing/files/test.txt /home/dag/home-made/ansible.testing/test.txt ``` ==== Using copy with files/test.txt Before: ``` 31523 1481280499.63052: search_path: /home/dag/home-made/ansible.testing/roles/test134/files/test.txt /home/dag/home-made/ansible.testing/roles/test134/tasks/files/test.txt /home/dag/home-made/ansible.testing/roles/test134/tasks/files/test.txt /home/dag/home-made/ansible.testing/roles/test134/tasks/tasks/files/test.txt /home/dag/home-made/ansible.testing/files/files/test.txt /home/dag/home-made/ansible.testing/files/test.txt ``` After: ``` 31110 1481280299.38778: search_path: /home/dag/home-made/ansible.testing/roles/test134/files/test.txt /home/dag/home-made/ansible.testing/roles/test134/tasks/files/test.txt /home/dag/home-made/ansible.testing/files/test.txt ``` ==== Using template with files/test.txt.j2 Before: ``` 30074 1481280064.15191: search_path: /home/dag/home-made/ansible.testing/roles/test134/templates/files/test.txt.j2 /home/dag/home-made/ansible.testing/roles/test134/tasks/files/test.txt.j2 /home/dag/home-made/ansible.testing/roles/test134/tasks/templates/files/test.txt.j2 /home/dag/home-made/ansible.testing/roles/test134/tasks/tasks/files/test.txt.j2 /home/dag/home-made/ansible.testing/templates/files/test.txt.j2 /home/dag/home-made/ansible.testing/files/test.txt.j2 ``` After: ``` 29201 1481279823.52752: search_path: /home/dag/home-made/ansible.testing/roles/test134/templates/files/test.txt.j2 /home/dag/home-made/ansible.testing/roles/test134/files/test.txt.j2 /home/dag/home-made/ansible.testing/roles/test134/tasks/templates/files/test.txt.j2 /home/dag/home-made/ansible.testing/roles/test134/tasks/files/test.txt.j2 /home/dag/home-made/ansible.testing/templates/files/test.txt.j2 /home/dag/home-made/ansible.testing/files/test.txt.j2 ``` This fixes #19048
* minor spelling changesCarlos E. Garcia2016-12-131-1/+1
|
* Remove method that isn't used here. (#18713)Toshio Kuratomi2016-12-021-27/+0
| | | We have two copies of this function and only the one in cli is used.
* Fix 'ansible-vault edit' crash on changed nonasciiAdrian Likins2016-11-161-7/+7
| | | | | | | ansible-vault edit was attempting to decode the file contents and failing. Fixes #18428
* Fix bug (#18355) where encrypted inventories fail 18355 (#18373)Adrian Likins2016-11-071-2/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * Fix bug (#18355) where encrypted inventories fail This is first part of fix for #18355 * Make DataLoader._get_file_contents return bytes The issue #18355 is caused by a change to inventory to stop using _get_file_contents so that it can handle text encoding itself to better protect against harmless text encoding errors in ini files (invalid unicode text in comment fields). So this makes _get_file_contents return bytes so it and other callers can handle the to_text(). The data returned by _get_file_contents() is now a bytes object instead of a text object. The callers of _get_file_contents() have been updated to call to_text() themselves on the results. Previously, the ini parser attempted to work around ini files that potentially include non-vailid unicode in comment lines. To do this, it stopped using DataLoader._get_file_contents() which does the decryption of files if vault encrypted. It didn't use that because _get_file_contents previously did to_text() on the read data itself. _get_file_contents() returns a bytestring now, so ini.py can call it and still special case ini file comments when converting to_text(). That also means encrypted inventory files are decrypted first. Fixes #18355
* Limit how much of the file we read to test if it's an encrypted vault fileToshio Kuratomi2016-11-041-2/+5
| | | | Fixes memory errors reported in #16391
* Fix 'vault rekey' with vault secret env varAdrian Likins2016-11-011-0/+4
| | | | | | | | | | | | | | | | | if ANSIBLE_VAULT_PASSWORD_FILE is set, 'ansible-vault rekey myvault.yml' will fail to prompt for the new vault password file, and will use None. Fix is to split out 'ask_vault_passwords' into 'ask_vault_passwords' and 'ask_new_vault_passwords' to make the logic simpler. And then make sure new_vault_pass is always set for 'rekey', and if not, then call ask_new_vault_passwords() to set it. ask_vault_passwords() would return values for vault_pass and new vault_pass, and vault cli previously would not prompt for new_vault_pass if there was a vault_pass set via a vault password file. Fixes #18247
* Fix search path for relative paths in roles (fixes #17877)Andrew Gaffney2016-10-171-1/+5
| | | | (cherry picked from commit 72f0aaf606aa3399c0713ad1eaac394c3846813c)
* Only dispkay failure to use cryptography at a higher verbosityToshio Kuratomi2016-10-121-2/+2
| | | | Fixes #17982
* Allow for empty source in dwim_path_relative_stackJames Cammarata2016-09-211-3/+3
| | | | Fixes #17192
* Fix dataloader using deprecated version of is_encrypted (#17615)Toshio Kuratomi2016-09-171-3/+3
|
* Merge pull request #17590 from abadger/vault-fixesToshio Kuratomi2016-09-151-186/+180
|\ | | | | Vault fixes
| * Many Cleanups to vaultToshio Kuratomi2016-09-151-186/+180
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * Make is_encrypted_file handle both files opened in text and binary mode On python3, by default files are opened in text mode. Since we know the encoding of vault files (and especially the header which is the first set of bytes) we can decide whether the file is an encrypted vault file in either case. * Fix is_encrypted_file not resetting the file position * Update is_encrypted_file to check that all the data in the file is ascii * For is_encrypted_file(), add start_pos and count parameters This allows callers to specify reading vaulttext from the middle of a file if necessary. * Combine VaultLib.encrypt() and VaultLib.encrypt_bytestring() * Change vault's is_encrypted() to take either text or byte strings and to return False if any part of the data is non-ascii. * Remove unnecessary use of six.b * Vault Cipher: mark a few methods as private. * VaultAES256._is_equal throws a TypeError if given non byte strings * Make VaultAES256 methods that don't need self staticmethods and classmethods * Mark VaultAES and is_encrypted as deprecated * Get rid of VaultFile (unused and feature implemented in a different way) * Normalize variable and parameter names on plaintext, ciphertext, vaulttext * Normalize variable and parameter names on "b_" prefix when dealing with bytes * Test changes: * Remove redundant tests( both checking the same byte string) * Fix use of format string without format operator * Enable vault editor tests on python3 * Initialize the vault_cipher for VaultAES256 testing in setUp() * Make assertTrue and assertFalse take the actual method calls for better error messages. * Test that non-ascii byte strings compare correctly. * Test that unicode strings and ints raise TypeError * Test-specific: * Removed test_methods_exist(). We only have one VaultLib so the implementation is the assurance that the methods exist. (Can use an abc for this if it changes). * Add tests for both byte string and text string input where the API takes either. * Convert "assert" to unittest assert functions or add a custom message where that will make failures easier to debug. * Move instantiating the VaultLib into setUp().
* | During initial argument evaluation, cast args to string. (#17595)jctanner2016-09-151-0/+2
| | | | | | | | | | | | | | Later in the stack, further code will check and inform the user that var names must start with a letter or underscore, so this fix only allows us to get to that previously existing policy. Fixes #16008
* | win_shell/win_command changes + tests (#17557)Matt Davis2016-09-151-3/+5
|/
* Move uses of to_bytes, to_text, to_native to use the module_utils version ↵Toshio Kuratomi2016-09-065-57/+68
| | | | | | | | (#17423) We couldn't copy to_unicode, to_bytes, to_str into module_utils because of licensing. So once created it we had two sets of functions that did the same things but had different implementations. To remedy that, this change removes the ansible.utils.unicode versions of those functions.
* fix action parsing to avoid conflicts aginBrian Coca2016-08-261-2/+4
|
* include_role (role revamp implementation) (#17232)Brian Coca2016-08-261-4/+4
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * attempt #11 to role_include * fixes from jimi-c * do not override load_data, move all to load * removed debugging * implemented tasks_from parameter, must break cache * fixed issue with cache and tasks_from * make resolution of from_tasks prioritize literal * avoid role dependency dedupe when include_role * fixed role deps and handlers are now loaded * simplified code, enabled k=v parsing used example from jimi-c * load role defaults for task when include_role * fixed issue with from_Tasks overriding all subdirs * corrected priority order of main candidates * made tasks_from a more generic interface to roles * fix block inheritance and handler order * allow vars: clause into included role * pull vars already processed vs from raw data * fix from jimi-c blocks i broke * added back append for dynamic includes * only allow for basename in from parameter * fix for docs when no default * fixed notes * added include_role to changelog
* Implement vault encrypted yaml variables. (#16274)Adrian Likins2016-08-236-54/+206
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Make !vault-encrypted create a AnsibleVaultUnicode yaml object that can be used as a regular string object. This allows a playbook to include a encrypted vault blob for the value of a yaml variable. A 'secret_password' variable can have it's value encrypted instead of having to vault encrypt an entire vars file. Add __ENCRYPTED__ to the vault yaml types so template.Template can treat it similar to __UNSAFE__ flags. vault.VaultLib api changes: - Split VaultLib.encrypt to encrypt and encrypt_bytestring - VaultLib.encrypt() previously accepted the plaintext data as either a byte string or a unicode string. Doing the right thing based on the input type would fail on py3 if given a arg of type 'bytes'. To simplify the API, vaultlib.encrypt() now assumes input plaintext is a py2 unicode or py3 str. It will encode to utf-8 then call the new encrypt_bytestring(). The new methods are less ambiguous. - moved VaultLib.is_encrypted logic to vault module scope and split to is_encrypted() and is_encrypted_file(). Add a test/unit/mock/yaml_helper.py It has some helpers for testing parsing/yaml Integration tests added as roles test_vault and test_vault_embedded
* Add some missing imports from last night's py3 fixes (#17196)Toshio Kuratomi2016-08-231-1/+1
|
View Lorry Controller Status