We have another project that uses a bare metal server. In our lab, the server is directly connected to a different SM200C (which, BTW, we borrowed to test to see if the SH was the issue – it wasn’t). In that case we could get 50MS/s easily but 100MS/s doesn’t work so well and 200MS/s was never an option. Even Spike has issues. I have not tried the updated NIC settings so we’ll have to see about that. That project is on hold until we’re ready for the next phase so it might be awhile before I can play with it.

I will contact you direct. Thanks for the help.

Thanks for the quick response. It seems it’s always the network on most things. Before I posted, we had pretty much done what you suggested. One wrinkle I forgot to mention, the system running the packet capture code is a VM.

On my laptop and in our local lab the hypervisor is KVM. At our customer site it will be ESXi. My laptop is connected to a Sonnettech 10G SFP+ (Thunderbolt to SFP+) and our lab system is connected via SFP+ to a 10G/40G switch to the SH (all SFP+) – so we’re good there. On my laptop I used the suggested changes on my VM’s NIC as well as on my laptop NIC. Also note that the VM had to be changed to use an e1000e NIC rather than the virtio NIC – virtio does not take the NIC settings.

Here’s where it gets interesting I think: I spoke with another team who had similar issues and they suggested changing/adding some of the NIC settings to:

sudo sysctl -w net.core.rmem_default=67108864
sudo sysctl -w net.core.rmem_max=67108864
sudo sysctl -w net.core.wmem_default=67108864
sudo sysctl -w net.core.wmem_max=67108864
sudo sysctl -w net.ipv4.udp_rmem_min=4096
sudo sysctl -w net.ipv4.udp_wmem_min=4096

This seems to work (again, this must be set on all NICs) especially if we keep the sample rate to 25MS/s. I as able to run for over an hour. 50MS/s seems to be less error prone than before (run for 5-10min before a network error) but anything above that and we nearly immediately get the -6 error from smGetVrtContextPkt(). Better but still troublesome. I don’t know if making the max read/write to 64M or adding in the UDP setting helped? We’d like to get to 50MS/s if possible. Thanks again for the help.

[Author]

Posts