Ticket #129 (defect)

Opened 1 year ago

Last modified 1 year ago

GEOS segfault

Status: closed (fixed)

Reported by: seang Assigned to: seang
Priority: major Milestone:
Component: Shapely Version:
Keywords: deserialization Cc: eric.c2c@gmail.com

We've been tracking a nasty shapely-related bug where our Pylons/Paster server just exits abruptly *and* randomly.

We use SQLAlchemy and define our own geometry type as such: 

class Geometry(TypeEngine):

    def compare_values(self, x, y):
        return x.equals(y)

    def convert_bind_param(self, value, engine):
        """convert value from a geometry object to database"""
        return b2a_hex(value.wkb)

    def convert_result_value(self, value, engine):
        """convert value from database to a geometry object"""
        if value is not None:
            return loads(a2b_hex(value))
        else:
            return None

And the hang occurs in loads(a2b_hex(value)). More precisely it occurs in the geom_factory() function in the dynamic import because geom_type isn't valid or something.

I've tried several things and the following patch (also attached) seems to make the problem disappear:

Attachments

shapely-ctypes-A0.diff (0.5 kB) - added by seang on 10/29/07 08:32:39.

Change History

10/29/07 08:32:39: Modified by seang

  • attachment shapely-ctypes-A0.diff added.

10/29/07 08:37:14: Modified by seang

  • status changed from new to assigned.

Eric, is your a2b_hex function different from "value.decode('hex')"?

10/29/07 09:11:07: Modified by elem

Sean, we use this:

from binascii import a2b_hex, b2a_hex

Does that answer?

11/01/07 11:20:51: Modified by seang

I'll check it out. I'm also using the comment to test reconfiguration of our ticket notification.

11/01/07 12:23:49: Modified by seang

Eric, I just added tests of round-tripping points, linestrings, and polygons (with hole) through hex-encoded binary. See tests/binascii_hex. Can you svn up to r940 or higher and try it out?

11/01/07 13:46:44: Modified by elem

Sean, the binascii_hex test always passes.

In my case, the hang occurs when I have two Ajax requests triggered at the same time, which further means there are two threads running simultaneously in the web server (paster).

11/01/07 14:51:43: Modified by elem

Sometimes (rarely) I don't get a hang but this traceback: 

Module shapely.wkb:19 in loads         
<<          raise ReadingError, \
            "Could not create geometry because of errors while reading input."
        return geom_factory(geom)
   
    def load(fp):>>  return geom_factory(geom)
Module shapely.geometry.base:22 in geom_factory        
<<          globals(),
            locals(),
            [geom_type],
            )
        ob.__class__ = getattr(mod, geom_type)>>  [geom_type],
exceptions.TypeError: Item in ``from list'' not a string

11/01/07 15:47:29: Modified by seang

I guess it shouldn't surprise me that there could be threading problems associated with GEOS. These can be hard to track down. I ran into some mapscript.py problems that I could only really figure out by running Zope under gdb.

http://wiki.zope.org/zope2/DebuggingWithGdb

11/02/07 06:11:55: Modified by elem

Sean,

A couple things:

  • My patch, which still uses the same function call to GEOS, makes the problem go away so I'm not sure this is a problem in GEOS. ctypes' string_at() might be the cause of the bug.
  • I did run paster under gdb but after the hang gdb prints 'program exited normally', I don't see any segfault! Maybe I just don't use gdb as I should do...

Eric

11/02/07 06:40:50: Modified by elem

Changeset #936 has the side effect of solving the problem - no more hangs, no more TypeError? exceptions.

I think we can mark this ticket as fixed.

11/02/07 08:46:50: Modified by seang

  • status changed from assigned to closed.
  • resolution set to fixed.

Done :)