@Aphyr's "Call me maybe: Kafka"[1] blog post is a great reference for what Kafka is and what distributed guarantees it tries to provide.
http://engineering.linkedin.com/distributed-systems/log-what...
TL;DR: "Kafka’s replication claimed to be CA, but in the presence of a partition, threw away an arbitrarily large volume of committed writes. It claimed tolerance to F-1 failures, but a single node could cause catastrophe."
"we can tolerate N-1 Kafka node failures, but only N/2-1 Zookeeper failures. This actually is sensible, though, as Kafka node count scales with data size but Zookeeper node count doesn’t. So we would commonly have five Zookeeper replicas but only replication factor 3 within Kafka (even if we have many, many Kafka servers—data in Kafka is partitioned so not all nodes are identical)."
(Zookeeper is used to track node membership)
Broadly, I think sacrificing P is pretty deeply misguided. If you run a distributed system at any scale for any period of time, you're going to experience network partitions, even within a single DC. (I speak from experience: I work on a distributed system which runs a whole bunch of machines in a whole bunch of datacentres worldwide. We see a reasonable number of non-trivial within-DC network partitions every year.)
A much more detailed argument of the above: http://codahale.com/you-cant-sacrifice-partition-tolerance/
You really do need to think about what happens to your system when it gets partitioned. This paper has much of interest to say on the topic: http://cs-www.cs.yale.edu/homes/dna/papers/abadi-pacelc.pdf
In my experience network partitions are not the most common kind of partition although they are well represented, especially on some network topologies.
IMO you should never assume that any two things won't be partitioned as part of your expectation of correctness or availability.