And in the email I get back from them (as their reply) there is a long string of letters and numbers where the photo should be.
Welcome to the land of
MIME, attachment encoding and the ever-changing landscape of email security!

Those letters and numbers are an ASCII representation of the photos, as you seem to have surmised.
It's good that you've mentioned the email client (
Incredimail) that you're using and your ISP, but that's really not enough to conclusively state what the problem is. Other factors involved are the email client that others are using to send messages to you, your MIME configuration, firewalls and other security measures at both ends. Sadly, ne'er-do-wells who try to play havoc with computer systems have resulted in safety measures being implemented that sometimes make things harder for us than they should be.
I've found that
Gmail is a very reliable email system that offers robust spam & malware/virus filtering, as well as the ability to send and display both inline images and image attachments. It also provides a switch that will allow you to turn on or off the ability to automatically display images on per-contact basis.
You probably don't want to change mail clients, but I've been using Gmail almost since the beginning and find it to be excellent. I own several domains and have email accounts on them that are accessed through POP 3 and traditional email clients, but I rarely use them anymore. I like Gmail that much.
Wish I could be more helpful and provide conclusive help for your particular situation, but the answer isn't a simple one without being able to examine all the variables involved.