Consciousness, from an evolutionary perspective, is the end result of a process called cephalization. Sense organs collect over time into a ‘head’ region. This is by no means something that happens automatically to all animals, in fact only three branches of animals have developed complex brains: arthropods, cephalopods, and the vertebrates.
All other animals either have no visible head or brain anatomy, or have simple structures that meet simple need, gathering food. The brains are as diverse as the organisms. With the animals sporting complex brains, they’re built by pattern, all vertebrate brains, for instance, have the same underlying spinal cord to hindbrain / midbrain / forebrain construction. Arthropods all have the same supraesophageal ganglion.
The pattern allows the brain to evolve semi-independently of the species. In mammals, parallel brain evolution has given rise to a shared set of brain structures that allow for a very interesting cross-species communication. If you see, say, a dog, exhibiting fear, you can be roughly sure that they’re feeling a very similar thing that you feel when you are scared. This is possible because the limbic system, which processes emotions, is shared by all mammals and to a lesser extent with birds.
Invertebrates, owing to the diversity of their body plans, can’t enjoy this kind of sharing.
So now that we’ve covered brain evolution, what then of consciousness? Do all animals with brains exhibit consciousness? To handle this question, let’s consider the definition of consciousness as found on Wikipedia. Consciousness is “sentience or awareness of internal or external existence.” You can be aware of things, but if you’re not aware of existence, then you’re not conscious.
This is, as you can imagine, a very tricky thing to ascertain whether any given living thing has. And so there’s a thriving scientific debate over exactly what the concept points to. The science eventually leads to the philosophical, one way of organizing the ideas and concepts is to consider ‘hard’ and ‘easy’ problems. The easy problems can be handled with scientific approaches, while the hard problem deals with the boundary between the physical and metaphysical.
Paring back a bit, we can distinguish ‘consciousness’ from ‘mere awareness’. If you are aware of an existence, as opposed to just being aware of food, then you have a ‘sentient’ awareness. This sentience seems to be required for further desirable qualities, such as intelligence.
Animals display varying qualities and quantities of intelligence. Bird species vary quite widely in observed abilities. Owls are notoriously socially dumb, it appears as if they devoted most of their brain to hunting prey. However parrots, being highly social animals, can do things like deceive each other.
And so you see this weird correlation. Social proximity often drives species to become smarter. The pressures of coordinating drives the brain to devote large fractions to trying to read and understand others.
Animals that do not need to be social, simply do not develop these finer qualities. Take the most commonly used marker of self-awareness, the mirror test. Humans display a range of behaviors towards their mirror representations up until around 18 months, at which point they fully recognize the image as themselves. Before you can recognize yourself in the mirror, you must first be able to see someone else there. Primates that fail the test tend to see rivals in the mirror.
I believe it’s safe to say that everything we think of as consciousness as opposed to mere awareness is driven by social needs. One only needs a self in order to relate to others.