When you don't have clean edge loops and all quads you risk getting artifacts which are unwanted creases in geometry. As I work on the ear I can turn off wireframe and view the shape of the ear. I have circled the parts of the ear that contain artifacts.
In wireframe mode you can see that it's not as noticeable while modeling. This can be both a good and a bad thing as you can focus on topology and not worry about the shape of the surface. It's always good to stop, turn off wireframe, and check the shape of the surface.