Reading music well is a wonderful skill. Of course many people learn to play an instrument, such as guitar or piano, all by ear and never learn to read. Many accomplished rock and jazz musicians play only by ear. In my view there is no shame at all in that, although many classical musicians would not agree with my view. Playing well entirely by ear is a great thing and I encourage my students to do a lot of it. But reading music can open up a whole world of material that you probably couldn't learn by ear. So the skill of sight-reading is worth learning to do well.
The problem is that many people learn to read (I'm speaking of piano here) the wrong way. In their first lessons with most piano teachers (and also if they try to teach themselves) they learn the notes on the staff. They are given mnemonics to help memorize them, such as Every Good Boy Does Fine (EGBDF) for the lines on the treble clef, and All Cows Eat Grass (ACEG) for the spaces on the bass clef. Then, as you play your simple tunes by reading, you are supposed to remember these lines and spaces and play the corresponding notes on the piano. It seems pretty straightforward and logical, but it doesn't actually work very well. The main problem is that you are focusing on isolated notes and not on relationships. You may see the note D, for example, on the page, but how do you find that D on the keyboard? You usually have to look at the keyboard and your hands to find it. This method of looking at the page and then down at your hands only works if you are playing very slowly, and it can never prepare you for more advanced music. It's extremely slow when reading only one hand at a time, but becomes almost impossible when you try to read both hands together, which causes the students to spend too much time playing hands separately. (See my earlier posts on this topic.) You will be dependent on looking at your hands pretty much forever unless you change how you read. To make the problem even worse, many people who try to teach themselves think they'll just take a "short-cut" and write the letter names of the notes on the page, so they never really learn to read the musical notation. No wonder many people just give up on trying to be good sight-readers.
In Asia they do not use the note names A through G as we do in the West, but use numbers instead. In Europe they use the system of Do, Re, Mi, etc. These are just different names for the notes, but the same problems arise if you try to read by the note names or numbers and then find them on the keyboard by looking.
The better was to learn is reading by interval. You are reading by distances between notes, and teaching your hands to find these distances on their own. Yes you will still need to know the absolute note names on the staff, so you can know where to start. But after that you navigate by interval, in much the same way as you drive from one destination to another: you don't need to know all the street names on your route, you just need to know when to turn, how many blocks to travel in a given direction, etc. It is really the same for reading at the piano. The best early reading books for piano approach it in this way, giving you tunes that span a range of 5 notes, corresponding to your 5 fingers, and you learn to play by feeling the distances and not looking at your hands. (I should mention here that it is important NOT to have the finger numbers written over each note, as some books do; if you do, you will inadvertently start reading by numbers instead of intervals.) When this is easy you learn to move your hands in ways necessary to cover greater distances, and eventually even big leaps. (You can even learn to transpose at sight, since you are reading by relationships, not absolutes. This is a highly desirable skill for anyone who wants to accompany singers, who sometimes need you to transpose to a lower or higher key.) You develop a much higher level of kinesthetic awareness this way, and can learn music much more quickly. If you learn this way right from the beginning it becomes surprisingly easy. If you didn't learn this way and have to go back and re-learn, it can be somewhat frustrating.
If you are a bad reader, you will be most comfortable playing in the key of C, on all white keys, because the letter names on our staffs correspond to white keys. But when you learn to read by relationships, you learn to "think" in whatever key you are in. If you study French, for example, you will still think in English and then translate. If you spent years living in France, however, you would find yourself starting to think in French. If you play the piano by absolute note names, you will "think" in the key of C and may find it difficult to play in other keys. Reading by interval enables you to "think" and therefore play in other keys just as easily as you play in C.
Many people who have a good ear just try to memorize the music as quickly as possible. If you have a good ear you are probably an auditory learner, and learning this way may come more easily to you than reading, which is easier for visual learners. Auditory learners may read through a phrase and then memorize it, then move on to the next phrase, and so on, so as to minimize reading. This method is also not good. It chops up your learning of the piece into small bits and you don't get a feel for the large-scale movement and flow of the piece. It can take a very long time to learn a whole piece. And since you were not a good reader to begin with, you may have learned notes incorrectly, and not realized it, because you never checked back in with the page once you started playing by memory.
I've encountered many people who taught themselves to play the piano by learning the names of the notes on the staff, then picking a piece they like, which is invariably much too hard for them (although they don't realize that, since they think it is purely a matter of picking out the notes on the keyboard), and then learning it one measure at a time, painstakingly slowly. The reality is that they don't have the technique, the physical skills, to play this piece at that point. If they are very persistent and have good natural coordination, they make eventually succeed in playing the piece. But if they had learned in a better way, they could have learned dozens of pieces in that span of time, with more enjoyment and less tedium.
Another reading skill, that you can work on when you become a good basic reader, is what I call "outlining." (See my post on this topic.). When outlining, you learn to play a "sketch" of the piece. You learn to recognize which notes are essential and which are more "decorative" and can be omitted. You can learn new pieces this way, gradually adding in all the notes, without sacrificing flow and musicality. This skill is critical if you ever want to accompany singers or instrumentalists, and need to sight-read new music without stopping or stumbling. You simply play fewer notes, while still supplying the essential accompaniment. Again, you could only do this if you never, or almost never, need to look at your hands.
If you recognize yourself in the description above of ways not to sight-read, go back to basics (very easy pieces) and try to learn to read by interval without looking at your hands. Even better, try to find a teacher who can help you with this. Good sight-reading can open up whole new worlds of music to you.