Whom do you trust to not take you out in the woods and rob you or worse?
On a small scale, a bunch of friends can trust each other. On anything larger than that, you need both driver and passenger vetting, and insurance, then you need a third party to decide in each conflict.
You can manage all the technical details as an open source solution, but the issue of decentralized trust among strangers would require:
a hefty deposit for the passenger
a background check, insurance, maybe live tracking, for the driver
an arbiter, if you don’t want every conflict to end up in the courts
a voting system for the quality of service/client
To keep it decentralized, you could use crypto services to manage deposits, but would still need an external oracle to decide each case.