Does the Green Berets only apply to SF or does it apply to all SOF?

The green beret is only worn by the Army and National Guard Special Forces.
There are a few circumstances when you will see them outside of SF units, but they all served within SF first and they can only wear the beret in certain positions outside the unit.

Green berets are for Army Special forces soldier who earn the tab and distinguish themselves through SFAS and SFQC.

The best of my knowledge, Green Berets are United States Army. You do have Special Operations like pararescue jumpers, Navy Seals and Marine Special Forces. They all work together but they are separate. Be proud that you are special operations under psychological operations.