BICS SIM for Things incident on 26/08/2026
On 26th of August, BICS SFT(Sim For Things) platform experienced a service degradation impacting Data session success rates, API availability, and Portal access. Charging services were also impacted, with a temporary bypass mechanism activated to maintain continuity during the disruption.
Incident was caused by an exceptional processing load on the platform's database layer, triggered by repeated large-scale service profile modification requests submitted in close succession against the same customer account. This drove internal resource consumption beyond configured operating thresholds, ultimately resulting in a database-level service interruption.
Service was restored through a controlled switchover to the resilience site. However, the failover was delayed by a software version mismatch between the primary and resilience environments. This mismatch was an expected consequence of the ongoing staged release rollout, in which the resilience site is upgraded only after stability has been confirmed in the primary environment. The version alignment issue was resolved before the failover could proceed.
2. Customer Impact
2.1 Location of incident
The incident affected the shared SFT platform and was therefore not limited to a specific geographic location.
2.2 Timing (GMT)
26/08/2026 11:15 SFT Services degradation started
26/08/2026 11:28 Alarms triggered, issue detected and starting the incident management
26/08/2026 11:51 Customer communication sent
26/08/2026 11:52 Switchover to Marseille resilience site, order given
26/08/2026 12:05 Switchover to Marseille, blockers identified
26/08/2026 13:20 Switchover to Marseille, blockers resolved
26/08/2026 13:25 Switchover to Marseille resilience site initiated
26/08/2026 13:50 Switchover to Marseille resilience site completed
26/08/2026 14:30 API and PORTAL services are restored
26/08/2026 15:15 Data services are restored. Post restoration monitoring confirmed service stability
26/08/2026 18:30 Charging full restoration
2.3 Services impacted
Total duration of service impact during an incident: 4 hours
Total duration of the impact on charging capabilities: Degradation for 7 hours (~50% charging events loss) including full unavailability of 3 h
Impact description and duration per type of service:
3. Detailed incident description
Incident was triggered by a large-scale service-profile modification affecting a significant number of endpoints. Multiple large-scale modification requests – targeting the same set of profiles – generated an exceptional processing workload that exceeded normal operating thresholds and resulted in service degradation. As the workload increased, critical database resources became unavailable, resulting in a temporary loss of database service availability and broader platform impact.
The issue developed in the following stages:
As a result, customers experienced:
To resolve the incident, a switchover to the Marseille resilience site was initiated. API and portal services were restored following the migration, with full data traffic subsequently returning to normal operation. Under normal circumstances, such a switchover is completed within approximately 15 minutes. In this case, however, the recovery process took longer due to two factors:
4. Actions taken to restore
To restore service, the following actions were performed:
5. Future actions identified to improve
Short-term actions (within one month):
Mid-term actions