can patch connectorx to petl to speed in next version?

[wonb168]

my 700,0000 rows table , use petl to csv need 40 miniutes.
and use connectorx, parrelled by 10, only 15miniutes,
can patch connectorx to petl to speed in next version?

[juarezr]

It looks like a very promising feature:

• petl could benefit from connector-x query parallelism for speed.
• connector-x could benefit from petl functionality for on-the-fly transformation, cleansing, manipulation, calculations

References:

• connector-x Intro
• connector-x Github

[wonb168]

petl's fromdb just one sql to query source db,
but connector-x can split the sql to many, for exmaple:
select * from tb, id, 4, if 100 rows in table , then will 4 sql to query table,
select * from tb where id<=25
I mean , if petl's fromdb funtion use connector-x to speed.