'str' object has no attribute 'more' [/usr/lib/python3.2/asynchat.py|initiate_send|245]

[Gryllida]

BPO	12523
Nosy	@pitrou, @vstinner, @giampaolo
Files	asynchat_example.py: example script demonstrating issue 12523 issue12523.patch: Add logging statement explaining probable cause of error. issue12523_r2.patch: Add error catching to async_chat.push method

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2014-07-07.22:07:59.375>
created_at = <Date 2011-07-10.00:04:24.409>
labels = ['type-bug', 'library']
title = "'str' object has no attribute 'more' [/usr/lib/python3.2/asynchat.py|initiate_send|245]"
updated_at = <Date 2014-07-07.22:07:59.374>
user = 'https://bugs.python.org/Gryllida'

bugs.python.org fields:

activity = <Date 2014-07-07.22:07:59.374>
actor = 'vstinner'
assignee = 'none'
closed = True
closed_date = <Date 2014-07-07.22:07:59.375>
closer = 'vstinner'
components = ['Library (Lib)']
creation = <Date 2011-07-10.00:04:24.409>
creator = 'Gryllida'
dependencies = []
files = ['34910', '34913', '34923']
hgrepos = []
issue_num = 12523
keywords = ['patch']
message_count = 12.0
messages = ['140073', '140104', '216471', '216485', '216549', '216560', '216561', '222376', '222388', '222401', '222525', '222526']
nosy_count = 8.0
nosy_names = ['pitrou', 'vstinner', 'giampaolo.rodola', 'BreamoreBoy', 'python-dev', 'Gryllida', 'jcao219', 'math_foo']
pr_nums = []
priority = 'normal'
resolution = 'fixed'
stage = None
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue12523'
versions = ['Python 3.4', 'Python 3.5']

[Gryllida]

Asynchat push() function has a bug which prevents it from functioning.

This code worked fine with Python 2.

---------------------------------------------------------------

# https://github.com/jstoker/BasicBot
import asynchat,asyncore,socket
class asynchat_bot(asynchat.async_chat):
def __init__(self, host, port):
asynchat.async_chat.__init__(self)
self.create_socket(socket.AF_INET,socket.SOCK_STREAM)
self.set_terminator('\r\n')
self.data=''
self.remote=(host,port)
self.connect(self.remote)

def handle_connect(self):
self.push('USER BasicBot 8 %s :BasicBot! http://github.com/jstoker/BasicBot\r\nNICK testbot\r\n' % self.remote[0])

def get_data(self):
r=self.data
self.data=''
return r
def collect_incoming_data(self, data):
self.data+=data
def found_terminator(self):
data=self.get_data()
if data[:4] == 'PING':
self.push('PONG %s' % data[5:]+'\r\n')
if '001' in data:
self.push('JOIN #bots\r\n')
if '~hi' in data:
self.push('PRIVMSG #bots :hi.\r\n')
if __name__ == '__main__':
asynchat_bot('127.0.0.1',16667)
asyncore.loop()

---

In Python 3 however, the exception follows:

---------------------------------------------------------------
~/tests/BasicBot$ python3 asynchat_bot.py
error: uncaptured python exception, closing channel <main.asynchat_bot connected at 0xb70078ac> (<class 'AttributeError'>:'str' object has no attribute 'more' [/usr/lib/python3.2/asyncore.py|write|89] [/usr/lib/python3.2/asyncore.py|handle_write_event|462] [/usr/lib/python3.2/asynchat.py|handle_write|194] [/usr/lib/python3.2/asynchat.py|initiate_send|245])
~/tests/BasicBot$ python3 -V
Python 3.2
~/tests/BasicBot$
---------------------------------------------------------------

A comment from Stackoverflow on why it happens:

---------------------------------------------------------------
The error seems to be raised in /usr/lib/python3.2/asynchat.py|initiate_send|245.

def initiate_send(self):
    while self.producer_fifo and self.connected:
        first = self.producer_fifo[0]
        ...
        try:
            data = buffer(first, 0, obs)
        except TypeError:
            data = first.more() <--- here 

Seems like somebody put a string in self.producer_fifo instead of an asyncchat.simple_producer, which is the only class in async*.py with a more() method.

[pitrou]

Actually, the error is that in Python 3 you should use bytes objects when transmitting/receiving data over the network, not (unicode) strings.
That is, replace '\r\n' with b'\r\n', etc.

Of course, the error message should be made less obscure.

[mathfoo]

Here is a small script that runs fine under 2.7 but demonstrates the error when run at 3.5.

If, at all the points annotated with '#not bytes :-(', the unicode strings are replaced with bytes objects, the example then successfully runs at 3.5.

[mathfoo]

I was unable to locate a point in the code where we could be certain that the error was ultimately caused by trying to use (unicode) strings instead of bytes object.

The patch adds a logging statement suggesting what the trouble is when the error is (probably) encountered.

Someone with more experience than me will need to make the call about whether this is a useful addition or not.

[giampaolo]

The part of the code where that is more likely to happen is the push() method.
One might also submit a producer (push_with_producer()) erroneously returning strings on more() though; how to properly fix this one is not clear to me because of the bad asynchat design. I'd be for simply raising an exception in push() as in:

diff --git a/Lib/asynchat.py b/Lib/asynchat.py
--- a/Lib/asynchat.py
+++ b/Lib/asynchat.py
@@ -181,6 +181,8 @@
         self.close()
 
     def push (self, data):
+        if not isinstance(data, bytes):
+            raise TypeError("data must be a bytes object")
         sabs = self.ac_out_buffer_size
         if len(data) > sabs:
             for i in range(0, len(data), sabs):

[mathfoo]

I like that approach decidedly more than my original one. Suggested revision patch attached.

The sample script will not produce the following:

error: uncaptured python exception, closing channel <main.fake_asynchat 127.0.0.1:8000 at 0x7f10a85b4ae8> (<class 'TypeError'>:data must be a bytes object [/home/caelyn/CPython/async_chat_error/Lib/asyncore.py|write|91] [/home/caelyn/CPython/async_chat_error/Lib/asyncore.py|handle_write_event|460] [/home/caelyn/CPython/async_chat_error/Lib/asyncore.py|handle_connect_event|448] [asynchat_example.py|handle_connect|16] [/home/caelyn/CPython/async_chat_error/Lib/asynchat.py|push|185])

Which is, I think, is a bit more useful.

[mathfoo]

*will now produce.

[BreamoreBoy]

I've no objection to people trying to take this forward but they should be aware that asyncio is recommended for new code.

[pitrou]

I've no objection to people trying to take this forward

We were really looking for your approval, thank you.

[BreamoreBoy]

Just close this as asynchat is deprecated from 3.2.

[python-dev]

New changeset f8c9dd2626aa by Victor Stinner in branch '3.4':
Issue bpo-12523: asynchat.async_chat.push() now raises a TypeError if it doesn't
http://hg.python.org/cpython/rev/f8c9dd2626aa

New changeset 4b29d338cc41 by Victor Stinner in branch 'default':
(Merge 3.4) Issue bpo-12523: asynchat.async_chat.push() now raises a TypeError if
http://hg.python.org/cpython/rev/4b29d338cc41