BUG:DjangoUnicodeDecodeError

[omero-qa]

​https://www.openmicroscopy.org/qa2/qa2/qa/feedback/6658/

Comment: I just uploaded a Cell R dataset. When I selected the first image and clicked acquisition this error box popped up.

Traceback (most recent call last):

  File "/home/omero_admin/apps/OMERO/OMERO.server-4.4.8-ice34-b256/lib/python/django/core/handlers/base.py", line 111, in get_response
    response = callback(request, *callback_args, **callback_kwargs)

  File "/home/omero_admin/apps/OMERO/OMERO.server-4.4.8-ice34-b256/lib/python/omeroweb/decorators.py", line 377, in wrapped
    retval = f(request, *args, **kwargs)

  File "/home/omero_admin/apps/OMERO/OMERO.server-4.4.8-ice34-b256/lib/python/omeroweb/decorators.py", line 434, in wrapper
    return HttpResponse(t.render(c))

  File "/home/omero_admin/apps/OMERO/OMERO.server-4.4.8-ice34-b256/lib/python/django/template/base.py", line 123, in render
    return self._render(context)

  File "/home/omero_admin/apps/OMERO/OMERO.server-4.4.8-ice34-b256/lib/python/django/template/base.py", line 117, in _render
    return self.nodelist.render(context)

  File "/home/omero_admin/apps/OMERO/OMERO.server-4.4.8-ice34-b256/lib/python/django/template/base.py", line 744, in render
    bits.append(self.render_node(node, context))

  File "/home/omero_admin/apps/OMERO/OMERO.server-4.4.8-ice34-b256/lib/python/django/template/base.py", line 757, in render_node
    return node.render(context)

  File "/home/omero_admin/apps/OMERO/OMERO.server-4.4.8-ice34-b256/lib/python/django/template/defaulttags.py", line 311, in render
    return self.nodelist_true.render(context)

  File "/home/omero_admin/apps/OMERO/OMERO.server-4.4.8-ice34-b256/lib/python/django/template/base.py", line 744, in render
    bits.append(self.render_node(node, context))

  File "/home/omero_admin/apps/OMERO/OMERO.server-4.4.8-ice34-b256/lib/python/django/template/base.py", line 757, in render_node
    return node.render(context)

  File "/home/omero_admin/apps/OMERO/OMERO.server-4.4.8-ice34-b256/lib/python/django/template/defaulttags.py", line 311, in render
    return self.nodelist_true.render(context)

  File "/home/omero_admin/apps/OMERO/OMERO.server-4.4.8-ice34-b256/lib/python/django/template/base.py", line 744, in render
    bits.append(self.render_node(node, context))

  File "/home/omero_admin/apps/OMERO/OMERO.server-4.4.8-ice34-b256/lib/python/django/template/base.py", line 757, in render_node
    return node.render(context)

  File "/home/omero_admin/apps/OMERO/OMERO.server-4.4.8-ice34-b256/lib/python/django/template/defaulttags.py", line 311, in render
    return self.nodelist_true.render(context)

  File "/home/omero_admin/apps/OMERO/OMERO.server-4.4.8-ice34-b256/lib/python/django/template/base.py", line 744, in render
    bits.append(self.render_node(node, context))

  File "/home/omero_admin/apps/OMERO/OMERO.server-4.4.8-ice34-b256/lib/python/django/template/base.py", line 757, in render_node
    return node.render(context)

  File "/home/omero_admin/apps/OMERO/OMERO.server-4.4.8-ice34-b256/lib/python/django/template/defaulttags.py", line 227, in render
    nodelist.append(node.render(context))

  File "/home/omero_admin/apps/OMERO/OMERO.server-4.4.8-ice34-b256/lib/python/django/template/base.py", line 797, in render
    return _render_value_in_context(output, context)

  File "/home/omero_admin/apps/OMERO/OMERO.server-4.4.8-ice34-b256/lib/python/django/template/base.py", line 777, in _render_value_in_context
    value = force_unicode(value)

  File "/home/omero_admin/apps/OMERO/OMERO.server-4.4.8-ice34-b256/lib/python/django/utils/encoding.py", line 93, in force_unicode
    raise DjangoUnicodeDecodeError(s, *e.args)

DjangoUnicodeDecodeError: 'utf8' codec can't decode byte 0xb0 in position 0: invalid start byte. You passed in '\xb0-\xe0\xe4@\xdd' (<type 'str'>)


<WSGIRequest
GET:<QueryDict: {}>,
POST:<QueryDict: {}>,
COOKIES:{'sessionid': '3afc1afd99b8191ff120772402a8d43f'},
META:{'DOCUMENT_ROOT': '/var/www',
 'GATEWAY_INTERFACE': 'CGI/1.1',
 'HTTP_ACCEPT': 'text/html, */*; q=0.01',
 'HTTP_ACCEPT_CHARSET': 'ISO-8859-1,utf-8;q=0.7,*;q=0.3',
 'HTTP_ACCEPT_ENCODING': 'gzip,deflate,sdch',
 'HTTP_ACCEPT_LANGUAGE': 'en-US,en;q=0.8',
 'HTTP_CONNECTION': 'keep-alive',
 'HTTP_COOKIE': 'sessionid=3afc1afd99b8191ff120772402a8d43f',
 'HTTP_HOST': '137.195.68.232',
 'HTTP_REFERER': 'http://137.195.68.232/omero/webclient/userdata/?experimenter=-1',
 'HTTP_USER_AGENT': 'Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.31 (KHTML, like Gecko) Chrome/26.0.1410.64 Safari/537.31',
 'HTTP_X_REQUESTED_WITH': 'XMLHttpRequest',
 'PATH': '/usr/local/bin:/usr/bin:/bin',
 'PATH_INFO': u'/webclient/metadata_acquisition/image/1007/',
 'PATH_TRANSLATED': '/var/www/webclient/metadata_acquisition/image/1007/',
 'QUERY_STRING': '',
 'REMOTE_ADDR': '137.195.68.215',
 'REMOTE_PORT': '49327',
 'REQUEST_METHOD': 'GET',
 'REQUEST_URI': '/omero/webclient/metadata_acquisition/image/1007/',
 'SCRIPT_FILENAME': '/home/omero_admin/apps/OMERO/OMERO.server-4.4.8-ice34-b256/var/omero.fcgi',
 'SCRIPT_NAME': u'/omero',
 'SERVER_ADDR': '137.195.68.232',
 'SERVER_ADMIN': 'webmaster@localhost',
 'SERVER_NAME': '137.195.68.232',
 'SERVER_PORT': '80',
 'SERVER_PROTOCOL': 'HTTP/1.1',
 'SERVER_SIGNATURE': '<address>Apache/2.2.22 (Ubuntu) Server at 137.195.68.232 Port 80</address>\n',
 'SERVER_SOFTWARE': 'Apache/2.2.22 (Ubuntu)',
 'wsgi.errors': <flup.server.fcgi_base.TeeOutputStream object at 0x4ef6d50>,
 'wsgi.input': <flup.server.fcgi_base.InputStream object at 0x4f5c510>,
 'wsgi.multiprocess': True,
 'wsgi.multithread': False,
 'wsgi.run_once': False,
 'wsgi.url_scheme': 'http',
 'wsgi.version': (1, 0)}>

[wmoore]

Ola - Images uploaded at apache_repo/6659 are from this user and may give the bug above?

[atarkowska]

related ticket ​https://trac.openmicroscopy.org.uk/ome/ticket/10942

[wmoore]

This ticket is a duplicate of #10195, which I haven't tackled yet since I had trouble getting the data from apache_repo.

[wmoore]

Ola - re: "If I catch that exception we won't get those errors any more. Any thoughts?"

Do we just need to str( unicode ) all the values in the metadata panel?
What would be the result of simply catching the exception?
Since this bug is (presumably) caused by one or two values in the metadata, we still need to show all the rest of the metadata.

[atarkowska]

It does not make sense to have a string without knowing what encoding it uses. You really need to know the type and length of data you are dealing with. Fixing the above problem goes deeper to the rendering template failure. Ideally exception should be cached on the ImageWrapper?.loadOriginalMetadata either via removing the data from the display or displaying encoded bytearray.

My thought was that popping up the feedback message box would keep us informed about potential Reader issues.

I do not understand what you mean by 'str( unicode )'

[wmoore]

Sorry - I was talking rubbish.
Even if we were notified via feedback about metadata that we can't display correctly (as we have been on these 2 occasions), what is there that we can do about it? "If there is no solution then there is no problem." Is this a B-F issue? How does the data appear in Insight?

[atarkowska]

Marking that ticket as invalid now, as I am unable to reproduce it. I tried to cast text to bytea and save it in the database but it doesn't seem to be a case. Colin couldn't download original file as downloading gave the same error. He came back to me after with saying basically that changing the memory allocation to the JVM solved the download problem for both OMERO.web and insight. IF we ever expierience that issue on any of our local servers I should be able to look at this deeper.