Is it worth delving deep outside my field to revise a paper?
Do you have a colleague or someone else you can ask who is an expert in the field you need? It is possible that with expertise in the area the reviews are very minor and a one hour talk with the right person will enable you to do whatever the reviewer suggested. Even if some of the reviewers ideas do take a lot of work this should enable you to satisfy some of the reviewers requests and come up with a reason to not do the others. Finally, if some of the questions create new interesting projects this might be worth going into, especially with a new collaborateur.
When you get minor revisions, you should definitely go for it.
That's not that I say "go for it under any circumstance", so if you had to invest months of work, I would probably vote differently. But be aware that if you get a minor revision, you do not necessarily have to comply with all the remarks. In many cases the editor does not even send the revision back to the reviewers, but I am not sure if you can know that in advance.
This is perhaps just stating the obvious, but whether or not your paper gets published isn't up to the referee. It's up to the editor. So if you received a generally positive referee's report then you can just tell the editor that the requested revisions are outside the intended scope of the paper and decline to make them. The only way I could see this becoming an issue was if the referee's report made it seem as though your paper wasn't publishable without the requested revisions.
Added (as an extended comment to the OP in light of their comment to the above): You seem to have a very defeatist outlook about the publishing process. For starters, the referee might not be all that much more senior than you are. I referee papers all the time (and have done so since my first year post-PhD) and I won't be going up for tenure for another couple of years, for example.
Regardless, all that the referee has done is write a referee's report which ideally makes some sort of recommendation to the editor. But that's all it is. A recommendation. If you send a response to the editor explaining that you don't think that one of the changes requested is necessary, you shouldn't assume that the editor will dismiss it out of hand. They might, of course, but then again they might not. I've had maybe ten papers where I explained to the editor that I did not agree with one of the referee's comments and didn't make the requested change. Not once did this have an impact on my paper getting accepted.
Like I said in my answer, it could very well be the case that the referee doesn't think your paper is worth publishing without the requested revisions. In this case the editor might side with the referee and you should probably just make the change if you can do so in a reasonable amount of time. But if you give a thoughtful explanation why the referee's recommendation is not necessary, the editor could very well take your side. Similarly, if the editor asks the referee what they think of your comment, they might decide that while the paper would be stronger with their requested revision, it should be published even without it.
Bottom line: It's your paper. If you have an opinion about what should be included in it and what should not be, you should let the editor know and assume that they'll at least hear you out.