On Wed, 6 Sept 2023 at 19:28, Mark Tinka <mark@tinka.africa> wrote:
Yes, this has been my understanding of, specifically, Juniper's forwarding complex.
Correct, packet is sprayed to some PPE, and PPEs do not run in deterministic time, after PPEs there is reorder block that restores flow, if it has to. EZchip is same with its TOPs.
Packets are chopped into near-same-size cells, sprayed across all available fabric links by the PFE logic, given a sequence number, and protocol engines ensure oversubscription is managed by a request-grant mechanism between PFE's.
This isn't the mechanism that causes reordering, it's the ingress and egress lookup where Packet or PacketHead is sprayed to some PPE where it can occur. Can find some patents on it: https://www.freepatentsonline.com/8799909.html When a PPE 315 has finished processing a header, it notifies a Reorder Block 321. The Reorder Block 321 is responsible for maintaining order for headers belonging to the same flow, and pulls a header from a PPE 315 when that header is at the front of the queue for its reorder flow. Note this reorder happens even when you have exactly 1 ingress interface and exactly 1 egress interface, as long as you have enough PPS, you will reorder outside flows, even without fabric being involved. -- ++ytti