| Commit message (Collapse) | Author | Age | Files | Lines |
|---|
| |
|
|
| |
Continue to improve the type annotation coverage
|
| |
|
|
|
| |
First pass at adding some type hints to pymemcache to make it easier to
develop against etc.
|
| | |
|
| |
|
|
| |
Basically ensure the client only does one pass over the buffer instead of two.
Exact thread: https://github.com/pinterest/pymemcache/pull/395#discussion_r890288417
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
A stat with a blank value makes the .stats() call fail with index error.
```
memcache_connection.stats()
File "/lib/python3.6/site-packages/pymemcache/client/base.py", line 741, in stats
result = self._fetch_cmd(b'stats', args, False)
File "/lib/python3.6/site-packages/pymemcache/client/base.py", line 921, in _fetch_cmd
result[key_value[1]] = key_value[2]
IndexError: list index out of range
```
The line in question we saw that caused this:
```
b'STAT ep_initfile '
```
This fixes things to no longer blow up. I chose to use a blank binary string instead of None because I suspect other code might make assumptions about the data being binary strings due to this bug.
Co-authored-by: reecepeg <reecepeg@amazon.com>
|
| |
|
|
|
| |
This also drops support for passing None as a server spec, which is
something many tests were doing out of lazy convenience. I can't think
of a reason why we'd want to support that for real world usage.
|
| |
|
|
| |
https://black.readthedocs.io/en/stable/index.html
|
| | |
|
| |
|
|
|
|
|
| |
ignore_exc should treat an error as a cache miss and default specifies
what should be returned on a miss.
Fixes issue #350
|
| | |
|
| | |
|
| |
|
|
|
|
|
| |
Now that we don't require Python 2 support no need for six.
Code upgraded with pyupgrade and manual fixes to remove remaining six
usage.
|
| | |
|
| | |
|
| | |
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
During network maintenance operations or during server reboot users could
face MemcacheUnexpectedCloseError.
This kind of exception will be raised down the stack. It's a design
choice [1]. Users should manage this kind of exception.
However, at some points the opened sockets remains more or less opened
after this kind of fail on the client side [2]. It will require n+1
calls to see them recreated when the connection will be possible. That's
could be problem with services that implement several cache logics in parallel
and where their clients are based on pymemcache.
These changes simply ensure to close the socket properly. The socket
will be recreated by the client if needed.
[1] https://github.com/pinterest/pymemcache/issues/307
[2] https://bugzilla.redhat.com/show_bug.cgi?id=1977711
|
| |
|
|
|
|
|
|
| |
The _store_cmd method became a bit to complexe and the linter complains
when we add new conditions or try/except clause.
These changes outsource some some processsing to reduce the complexity
of the method.
|
| | |
|
| | |
|
| |
|
|
|
|
| |
The Memcached server can be configured to drop idle connections (via the
idle_timeout option). When this occurs, pymemcache may still try to
use the connection, resulting in MemcacheUnexpectedCloseError. Hence
the need for client-side idle timeout logic.
|
| |
|
| |
According to the Memcached Protocol, the no argument form of the stats command should have no trailing spaces. Currently, b'stats \r\n' instead of b'stats\r\n' is being sent to the socket which leads to a malformed response.
|
| |
|
|
|
|
|
|
|
|
|
|
|
| |
_check_cas() does a good job of normalizing a CAS value and raising
MemcacheIllegalInputError when fed an invalid value. We were only using
this function in the `cas is not None` path within _store_cmd().
By moving this check into cas() itself, it allows us to also raise
MemcacheIllegalInputError when `cas is None`, which can happen when this
pattern is used for a new key:
val, cas = c.gets('key')
c.cas('key', 'value', cas)
|
| |
|
|
| |
Fixes #257.
|
| | |
|
| | |
|
| |
|
|
| |
Also added disconnect_all() as an alias for close().
|
| | |
|
| |
|
| |
Co-authored-by: Ben Yarmis <byarmis@indeed.com>
|
| |
|
|
|
| |
The user can subclass Client (e.g to add telemetry or logging), and have
their custom client class used by PooledClient and HashClient to create
new clients.
|
| | |
|
| |
|
|
|
|
|
|
| |
Our current MockMemcacheClient doesn't throw an error if a key has a
space, and given MockMemcacheClient can be consumed by downstream
things, we want it to act as close to the real client as possible.
Rename _check_key to check_key_helper so we aren't importing a private
function.
|
| |
|
|
|
| |
For add(), replace(), cas(), and delete(), if noreply is True (or if it
is unset and self.default_noreply is True), the return value is always
True.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
For consideration for v3.0.0
'cas' is documented as needing to be an int or bytestring of the digits
0-9. However, this is not actually enforced and it is possible to pass a
value to pymemcache which doesn't conform to these rules. In fact, you
can do weird things like `cas=b'noreply'` and potentially trigger
"unexpected" behavior.
To go along with validating int inputs, validate cas inputs. However,
these are not necessarily integers. Instead, if an int or string is
given, it will be encoded as a bytestring. But in order to validate the
value given, it is checked against isdigit() .
(NB: You could also use `int(cas)` for very similar checking.)
Rationale for allowing non-integer inputs to cas is not obvious.
Presumably it allows callers using `gets()` to pass the `cas` value they
get back into a `cas` command without issue. But it may be debatable.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
* Change serialization interface to be an object
Rather than passing separate serialization and deserialization methods
to a pymemcache client, pass an object implementing a very simple
two-method interface.
This is a rather significant breaking change and should be part of an
x.0.0 major release.
Resolves #56
As suggested in that issue, this is a cleaner interface, as there's no
sensible context in which you would provide only one of these two
methods and it should therefore be thought of as a
serialization/deserialization protocol.
Also adds a note to the documentation's Best Practices list that you
should use the built-in serializer object unless you have a reason to do
otherwise.
* Support "de/serializer" in addition to "serde"
In order to support older client usage in addition to the new
serialization object (protocol), restore the "serializer" and
"deserializer" arguments to the Client classes.
These are marked as deprecated and will be automatically wrapped into a
small "serde" object.
In order to make the various object names more distinguishable and more
informative, the built-in default serializer is now called
"python_memcache_pickle_serde"
Additionally, default client.serde to a "no-op serializer".
This object does no transforms on the data. By putting this in place, we
can skip some conditionals in the code around presence or absence of a
serializer and therefore simplify internally (at the cost of an extra,
unnecessary, functional call in some cases).
It also simplifies logic around the handling of flags because we are now
*guaranteed* the presence of a serializer object which returns some
flags. i.e. "default flags" are no longer the responsibility of the
various serializer usage sites.
This is done carefully to ensure that passing a `serializer` without a
`deserializer` is respected.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
If a caller tries to send
- a fractional expiration time
- a string for the "incr" command
- delay=None (instead of delay=0)
The pymemcache base client will now catch these usage mistakes a
throw a MemcacheIllegalInputError.
Although there are existing valid (but undocumented) usages of the
current string conversion behavior, the docs state that these values
are supposed to be ints. Now if a non-int value is used, it will
immediately trigger an error.
This behavior is superior to the prior behavior especially in the
case where a command is used with `noreply=True`. It can fail to
parse, but the client won't pick up on this until the next call to
memcache which reads the reply.
The especially bad case we're seeking to avoid is this:
from pymemcache.client.base import Client
client = Client(("localhost", 11211))
client.set("foo", "bar", expire=1.5, noreply=True)
client.get("foo") # triggers MemcacheUnknownCommandError
Applies to all commands using `expire`, `delay`, incr and decr
inputs, and memlimit.
Because this is can be a breaking change for some users, the changelog
entry is written under 3.0.0 (unreleased).
closes #240
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
We introduced the ability to override the serializer-returned flags
values in 26f7c1b1. Unfortunately, there was a flaw in that logic which
resulted in the first item's flags being used for all later items. This
bug primarily affected set_many()'s behavior when the data dictionary
contained multiple different value types which were assigned different
per-value flags values by the serializer.
For example:
set_many({'a': 'string', 'b': 10})
If a serializer returned different flags for strings (e.g. 1) and
integer values (e.g. 2), the previous logic would have set ``flags``
to 1 the first time through the loop and repeated that value for the
second item, instead of using 2.
This was the intended behavior when ``flags`` was explicitly passed to
set_many(), but not for the default case where we still want to respect
the flags values returned by the serializer.
|
| | |
|
| | |
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
| |
This is a stylistic issue with little chance for actual impact. Some
methods use 'ascii' verbatim and some use `self.encoding` (which
defaults to 'ascii'). Those which are using `ascii` are only supposed to
be accepting integer inputs, so this is actually fine.
However, the result is a potentially confusing discrepancy -- why does
`Client.touch` use `ascii` while `Client.add` uses `self.encoding`?
To alleviate this potential confusion, be consistent about applying
`self.encoding` to all memcached commands in the base client.
This does not impact the `_check_key` func, which re-encodes potential
unicode inputs as ASCII.
|
| |
|
| |
extract _extract_value() from _fetch_cmd() to support customized value extraction logic.
|
| |
|
| |
added flags in client functions to provide support for extra memcached operations.
|
| | |
|
| |\
| |
| | |
Remove redundant '' from %r values
|
| | |
| |
| |
| | |
repr()'d strings (via %r) are already quoted.
|
| |/
|
|
|
|
| |
Memcache can only store binary-safe values. We raise an exception when
we're unable to encode a data value using the 'ascii' codec. Improve the
exception message to make the error case clearer.
|
| | |
|
| |
|
|
|
|
|
|
| |
-Should not throw exception when response was "OK"
cache_memlimit added
-Allows client to set cache memlimit
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
| |
Some commands are sensitive to the order of their key arguments so
revise the previous code to explicitly build a list of prefixed keys
and a dictionary of remapped keys. This generalization also lets us
remove command-specific checks from this code path.
The list of prefixed keys is used to build the command string
(preserving order).
The dictionary of remapped keys is used to map the prefixed keys from
the response to the key names that were originally provided to the
client.
|
| |
|
|
|
|
|
|
| |
I don't currently see a way in which we would enter _connect() with an
existing socket, but let's explicitly close() first just to be safe.
This also ensures we enter the rest of the _connect() code knowing that
`self.sock is None`.
While I'm here, add better test coverage for the close() method.
|
| |
|
|
|
|
| |
The client's `server` argument can now be a string containing the path
to the memcached server's UNIX domain socket.
Also document all of the connection-oriented client parameters.
|
