summaryrefslogtreecommitdiff
path: root/tokhash.pl
Commit message (Collapse)AuthorAgeFilesLines
* Add copyright verbiage to Perl scripts; update LICENSEH. Peter Anvin2009-06-281-0/+33
| | | | | | | | This adds copyright verbiage to the Perl scripts. Scripts that are known to be clean w.r.t. the 2-clause BSD license are given that license; unclear ones are given the "LGPL for now". Signed-off-by: H. Peter Anvin <hpa@zytor.com>
* tokhash.pl: fix commentH. Peter Anvin2008-05-271-1/+1
| | | | Correct filename in comment
* Formatting: kill off "stealth whitespace"H. Peter Anvin2007-10-191-18/+18
| | | | | "Stealth whitespace" makes it harder to read diffs, and just generally cause unwanted weirdness. Do a source-wide pass to get rid of it.
* Portability fixesH. Peter Anvin2007-10-021-0/+1
| | | | | | | | | Concentrate compiler dependencies to compiler.h; make sure compiler.h is included first in every .c file (since some prototypes may depend on the presence of feature request macros.) Actually use the conditional inclusion of various functions (totally broken in previous releases.)
* Use the crc64 we already use as the perfect hash function prehashH. Peter Anvin2007-10-021-12/+7
| | | | | | | Use the same crc64 that we already use for the symbol table hash as the perfect hash function prehash. We appear to get radically faster convergence this way, and the crc64 is probably *faster*, since the table likely to be resident in memory.
* Support __float*__ for floating-point numbers in expressionsH. Peter Anvin2007-09-241-96/+121
| | | | | | | | | Add special operators to allow the use of floating-point constants in contexts other than DW/DD/DQ/DT/DO. As part of this checkin, make MAX_KEYWORD generated by tokhash.pl, since it knows what all the keywords are so it can tell which one is the longest.
* tokhash: allow a bit smarter pattern matchingH. Peter Anvin2007-09-241-0/+12
| | | | Allow constants to match only part of the token string.
* Slightly optimize the interface to nasm_token_hash()sse5H. Peter Anvin2007-09-181-2/+2
| | | | | | Instead of returning -1 from nasm_token_hash, set tv->t_type to TOKEN_ID and return TOKEN_ID, since that's what stdscan.c wants to do with it anyway. This allows us to simply tailcall nasm_token_hash().
* Change the token prehash function for better convergenceH. Peter Anvin2007-09-181-2/+2
| | | | | | | Combining arithmetric (add) and bitwise (xor) mixing seems to give better result than either. With the new prehash function, we find a valid hash much quicker.
* Use enumerations where practical to ease debuggingH. Peter Anvin2007-09-121-3/+3
| | | | | | | | We have a lot of enumerations; by declaring fields as such, we make it easier when debugging, since the debugger can display the enumerations in cleartext. However, make sure exceptional values (like -1) are included in the enumeration, since the compiler otherwise may not include it in the valid range of the enumeration.
* tokhash: correct duplicate-token testH. Peter Anvin2007-09-101-1/+1
|
* tokhash: adjust table types to reduce sizeH. Peter Anvin2007-09-101-7/+11
| | | | | Adjust the sizes of data types to reduce the total size of the tokhash data structure.
* phash: don't rely on the build platform Perl version of rand()H. Peter Anvin2007-08-311-3/+3
| | | | | | | | rand() in Perl can vary between platforms, so don't use it. Instead, remove a completely pointless level of indirection (it introduced a permutation which cancelled itself out) and provide a canned set of random numbers for the rest. This guarantees we will always use the same numbers.
* tokhash.pl: formatting changes for readabilityH. Peter Anvin2007-08-311-44/+41
| | | | No functional change
* tokhash: Speed up the rejection of unhashed valuesH. Peter Anvin2007-08-311-2/+9
| | | | | | | Speed up the rejection of unhashed values (typically identifiers) by filling unused hash slots with a large value (but not so large that it is likely to overflow.) This means those values will be rejected already by the range check, not needing strcmp().
* tokhash.pl: "ix" should have the same width as the "hash" arraysH. Peter Anvin2007-08-311-1/+3
| | | | | For correctness in case of a very "linear" graph, "ix" needs to have the same width as the "hash" arrays.
* Add "do not edit" comment to tokhash.cH. Peter Anvin2007-08-311-1/+6
|
* Make the token hash a bit smaller by using 16-bit hash tablesH. Peter Anvin2007-08-311-2/+2
|
* Finishing touches on perfect hash tokenizer; actually turn the thing onH. Peter Anvin2007-08-301-5/+15
| | | | | | | | | Finish the perfect hash tokenizer, and actually enable it. Move stdscan() et al to a separate file, since it's not needed in any of the clients of nasmlib other than nasm itself. Run make alldeps.
* Generate a perfect hash for the token parserH. Peter Anvin2007-08-301-0/+184