As others have said, don't give up just because you had one bad experience. When I first started OW swimming, I remember it was really freaky for me too, here's what helped me, maybe it will help you.
(1) DO practice OW swimming with a buddy or two. The first few times you do it, your buddies should be there to support YOU. In other words, not swim off and leave you. If you start feeling freaked out, they should stay with you and talk you through it. I'd take the other posters up on their offer of beer drinking OW swimming with you.
(2) Remember that you can always do another stroke! Dog paddle is fine. Breast stroke is fine. Even just treading water is fine. Sometimes just getting your head out of the water helps a lot.
(3) If you DO stop and tread water, don't use this as an excuse to quit. Tell yourself you'll swim to the next buoy and see how you feel then.
(4) Don't try and swim in the middle of the pack the first few times you swim at an event. It's hard enough without feeling like you've been dropped in a washing machine. Swim off to the side. Swim widely around the buoys. It's better to be a little slower and finish the event, than to get freaked out, and quit.
My biggest thing about OW is that it is a lot different than swimming in the pool. There are no lane lines, in fact, usually you can't even see the bottom! It's hard to judge your progress. You site the buoy, swim 10 strokes, site the buoy, swim 10 strokes, repeat, repeat, repeat and yet, that damn buoy never seems to get closer. When I swim in OW, I try to get in the mindset of just swimming, in a rhythm, with relaxed breathing. I site the buoy, but try not to expect that it will seem any closer than the last time I saw it. If you can do this part right it is the best part of OW swimming.