As a child of the 70s, I can attest to some terrified sleepless nights caused by PIFs. As you say, the worst thing was that they just appeared in the middle of otherwise entirely innocuous programming and you simply weren't prepared for the extreme horror of the PIF.

Obviously 'Charlie Says..' animations were in there too, also in the 70s & 80s, they were bad enough but at least had a little humour injected (sort of).