I have seen this error while opening a CHM file. After a little bit of struggling, I figured out that this error comes when you have this CHM file on a network drive and you try to open it from there.
The workaround for this problem is to copy the file to local drive and open it. I am not sure why having it in network drive is a problem. One guess I can make is, it is something to do with internet/intranet zones in IE w.r.t the network location from where you are accessing.
Have you followed any other to resolve this? Please let me know.