IPSEC and PAT

I know that in Tunnel Mode, IPsec can be NATed and PATed (without IKE on UDP 500 being used), but as I'm trying to break down the process of how it is working, I've been stumped by this: NAT - Changes source IP during translation PAT - Changes source IP and TCP/UDP port to another to track multiple to one translations. My question is, how does PAT track the packets with their internal hosts when there is not a TCP/UDP header to translate. How does it know which "internal" host a returning ESP packet must be forwarded to after it un PATs the incoming packet? thanks and I hope this isn't a totally stupid question. If it is, humor me ;), -=Vandy=-