circuit issues (hard down/flapping/packet loss, etc)
Intrusive testing is needed to be done by a circuit carrier when a circuit is experiencing issues. These issues may include, but are not limited to, circuit hard down, packet loss, or bgp flaps. This testing is necessary for the carrier to see certain issues on a network, and is a critical step in troubleshooting circuit issues in order to restore service to SLA.
*****IMPORTANT NOTE: INTRUSIVE TESTING WILL BRING DOWN THE CIRCUIT AND INTERRUPT SERVICES*****
The most important step in performing intrusive testing is verifying with the customer what time is appropriate. As mentioned above, intrusive testing will bring down the circuit during the length of the testing. Immediate intrusive testing is only advised if the circuit is already hard down or if the circuit performance is so poor that bringing down the circuit entirely won't create a noticeable difference in interruption. Otherwise, it is best to schedule for a time that the circuit is not seeing much utilization, such as after business hours or during a lunch hour.
The below information is needed to open a ticket with the carrier:
This is the circuit identifier with the carrier. The format of the CID differs from carrier to carrier, but will be listed on the SolarWinds node under the comments section.
Confirm this via the location tab in connect
LCON and phone number where they can be reached
Local Contact, most likely the MPOC that you are working with on the case. LCON is needed in case dispatch is required. This will need to be someone either on site, or someone who can be there on a moments notice.
Site Access Hours
In case dispatch is necessary, it will only occur within these hours.
Intrusive Testing Window
A window of time wherein the carrier can perform the intrusive testing. It is preferred that you provide them a window spanning two or more hours, as if they are busy, they may not get to a smaller window.
Internal CallTower Ticket Number
This is the ticket number the carrier will reference when calling in to our NOC line.
CallTower NOC Number
This is the phone number the carrier will be calling to reach our support team. The number is 702-951-2457.
Once you have all of the above information, call the carrier or open a ticket on their online portal. Request intrusive testing on a circuit and provide the above information.
After the intrusive testing is performed, the carrier will either tell you that they found an issue, or that they did not.
If they found an issue, they will continue to investigate or require a dispatch. Escalate every hour (during business hours) until the issue is resolved.
If no issue was found, continue to troubleshoot the issue from CallTower's end.