I’ve never used plex or plexamp but I have used Finamp for listening / offline downloading of my music collection from my Jellyfin server. Is this perhaps what you’re looking for? iOS: https://apps.apple.com/ca/app/finamp/id1574922594 Android: https://play.google.com/store/apps/details?id=com.unicornsonlsd.finamp
I can confidently say that no it will not “intelligently use the LAN” when you’re on the same network - I don’t know of any service that will… unless Plex/Plexamp somehow does this?
The solution is as someone else said - use a DNS Server to forward it in your LAN to the internal IP. If you’re unsure how to do this, just search how to setup a Hairpin NAT for the router you own. I can confirm that once you set this up, it will work seamlessly with both Finamp and Jellyfin.