The idea that Python is so slow that it's confusing TCP sounds wrong to me. I think it's more likely that your packet capture scheme is slow. It looks like you're using scapy, which I assume is in turn using libpcap... which may be buffering (in high-performance network monitoring, the packet capture interface goes out of its way to buffer). Which is something you can turn off.

About 13 years ago, I wrote my own programming language expressly for the purpose of implementing network stacks, and had a complete TCP in it; I didn't have this problem. But I do have this problem all the time when I write direct network code and forget about buffering.

Similarly: "Python is so slow that Google reset the network connection" seems a bit unlikely too. Google, and TCP in general, deals with slower, less reliable senders than you. :)

What's the time between your SYN and their SYN+ACK?

It's rare that bad business regulations and/or government sponsored monopolies are just, ya know, fixed. Quite often, people need to be shown how bad things have gotten before anything changes.

It's even worse when the target of those regulations don't want things fixed either. Taxi companies pay crazy medallion fees, but expect exclusivity (perhaps rightly so) for that fee. Government doesn't want the fee income to go away, taxi companies enjoy their monopoly.

Sometimes it takes someone breaking the rules - and more importantly, with the general full support of the public - to really cause any change in the old, entrenched ways.

It's pretty simple, Uber understands that to win the game, you need the a supply of drivers that just barely exceeds the demand for rides, to avoid surge pricing. To digress for a moment, surge pricing is brilliant in that it gets riders to volunteer to wait a long time for a ride, instead of getting pissed off like they do at Taxis, or give them the choice to pay up for an expedited ride. Anyway, the recruiting strategy worked for them in SF, they recruited so many Lyft drivers that Lyft was almost constantly in "Prime Time" (their name for surge pricing), that riders got upset and switched to Uber. It's at the intersection Business 101 and Econ 101 if you ask me.

The only pushback I've seen from Lyft is mobilizing drivers to hand out free ride cards at slammed venues, like music festivals, when Lyft is at maximum Prime Time of 300% and Uber is at 800-1300%. Not a bad strategy, but basically, Uber is giving Lyft a hyper wedgie by cutting right into their supply.

To put the cancellations into context, I met a Lyft driver who was recruited by Uber. He told me the same lady was recruiting in his area on an ongoing basis, just catching rides to make the pitch (which is a TOS violation on either platform). He said the recruiter would request a ride, sometimes 4 or 5 times, and keep getting him, then she would cancel shortly afterward when she recognized his name. So yeah, I'm sure some Lyft drivers spent gas, time and miles, but I also believe for the most part, the cancellations occurred in a minute or less.

As far as which company are the biggest assholes, I think you need to look at it with some historical context. Uber started out as a company that created a better dispatch system for Black Cars, and got Livery companies and independent Limo drivers to sign on. That was good, because riders and drivers could be paired based on proximity. Combine that with Uber's marketing to increase ridership, and it is an all-around win. The savings/increased business for your average Livery operator more than made up for the Uber commission.

Then Lyft comes along and says "Hey, let's open this up to your average joe with his car and car payment". Ok, so what they are doing actually is sucking the equity out of people's cars, because by the time you amortize all the expenses, you're making $10-$15 an hour (on Lyft, Uber you can make more), you might as well work for In-N-Out and get benefits too. Not to mention the insurance issues if you get into a wreck and its your fault (cancellation and no payout on your collision policy). Sure, the network will cover your liability, but only because they have been forced to by regulators. I classify this type of business (and AirBNB too) as "Slumlord 2.0". Slumlords buy an asset and drain the equity out of it for profit. Slumlord 2.0s drain the equity out of someone else's asset for profit. Uber had to counter Lyft with UberX, and get into the Slumlord 2.0 game themselves.

Bottom line for me, I drive for them both, and I hate them both. I tolerate them because they enable me to continue to work on my startup, or I guess I should say, reboot of the family business. I can't wait to be done driving for them and become a rider.

my favorite thing about writing blog posts is comments like this. Thank you! I didn't consider that the packet capture interface might do buffering. That might explain a lot of the problems I was having :)