Be careful what you wish for! Switzerland and Norway are both small countries with specialised ‘niche’ economies: Switzerland with its often-criticised banking system, and Norway with its massive oil reserves.
But their industries have to follow EU rules as that’s their main market. As non-members, they have no say over the adoption of those EU rules. They cannot defend their interests. They have, effectively, lost sovereignty through their isolation.
Nor does staying out save money — the Norwegian contribution per capita to the European budget is about the same as that of the UK! For the Norwegians, being non-members, it’s taxation without representation. Evidence