If I don't earn my income from your country or own property in it

If you live in that country, you have to pay your taxes in that country. You will find out that to be the reality in most countries that have proper regulations. If you dont participate in the society that you settle, you have no business being there. These countries did not come to being for the sake of wealthy Angloamerican immigrants so that they can geoarbitrate at the expense of the locals.