Is it rude to speak Swedish in Norway?

I speak Swedish quite well, but my active Norwegian is very basic, although I can understand most of it if I concentrate. I'm aware Norwegians can usually/always understand Swedish. I'm told Norwegians and Swedes may have conversations in which each speak their own language. I've seen exactly that in films, and I find it weird, for I understand one half of the conversation perfectly well, and the other half only partially and with difficulty.

When I visit a non-English speaking country, I tend to try to speak the local language, however basic. Personally, I find it arrogant or rude to expect the locals to speak a foreign language when I am the visitor, even when I'm in a country where the level of English is generally very high (such as in Norway). Not everybody is at ease speaking English, in particular in off-the-beaten-track rural corners of the country.

But how is it perceived to speak Swedish in Norway? May it be perceived as rude or arrogant to expect that everybody understands what is, after all, a foreign language? Or does the closeness between the languages mean that people likely wouldn't think about it, and perhaps barely notice it? There may be cultural issues related to history that affect this as well.

In case it matters, consider a rural area of Trøndelag that receives relatively few foreign visitors.

I could either try to speak broken Norwegian with Swedish mixed in, Swedish, or English (or German, but I've only ever once come across a Norwegian Sami person where that ended up being our best shared language).

No, in itself it is not rude. You can do it in a rude way, by just ignoring the signals if the other person is not happy about it.

Offer the person you talk with the choice.
I would start with a Norwegian hello or good morning/day, and next try out which language sits best.

One time in Norway I was asked to speak Dutch rather than English (I speak no Scandinavian languages) while the seller at the market kept to her version of Norwegian, it worked for us.

I have always found all Nordic people helpful and glad to communicate in whatever language or mix of languages worked best, often not their own.

While most adults in Sweden and Norway will have learned English, it is not 100% that have. And the percentage that are used to speaking English will be lower outside of the areas where tourists from abroad are common.
Whether trying to speak Norwegian or keeping on Swedish or switching to speak English is best depends on your language skills, and likely how long you have been around people speaking Norwegian and how many related languages you really know.
I have found Norwegian always the easiest of the Scandinavian languages but that might be that I learned to understand Frisian (of the Dutch province Friesland) even though I do not speak it.

Not Scandinavia but I have heard and/or been part of a lot of German-Dutch conversations. Some people, mostly the young ones, would ask for English while the rest were happy to understand each other.