Slowness issues affecting the Student Information System
Incident Report for CSIU
Resolved
This incident has been resolved.
Posted Oct 02, 2018 - 07:59 EDT
Update
The database migration to the new server is completed. All services should now be resumed. More updates will be coming as we have further information.
Posted Sep 07, 2018 - 23:27 EDT
Update
We have completed the additional server build for the central database. We plan to complete this migration starting at 10:00 PM tonight, and anticipate this being completed by midnight. Services will be disrupted during this maintenance window. Thanks for your understanding and patience.
Posted Sep 07, 2018 - 17:46 EDT
Update
Update on slowness in the SIS

The optimization changes that were applied to the software last night did make a slight difference today, especially to a certain segment of the clients. However, the majority of you still experienced performance issues during the peak time this morning. Using more collected data, constant monitoring, and further advice from Microsoft consultants, eSchoolData has been able to focus on their central database for clues.

Around noon time today, we applied a hot-fix from eSchoolData to create more indexes and optimizations to the central database with good results.

Right now, CSIU staff and eSchoolData are building an additional server which will exclusively host the central database. When we know the timeframe the central database will be moved to this new server, we will send out an announcement on the https://csiu.statuspage.io/ about the window of downtime. It should not take more than two hours.

We will continue updating you as we have more information.
Posted Sep 07, 2018 - 17:16 EDT
Update
Update on slowness in the SIS

Although the changes made to our environment made improvements, we have noticed continued performance issues at peak times. We continue working with Microsoft, Dell and eSchoolData on troubleshooting. Additional tools have identified potential areas for software optimization. eSchoolData is actively addressing these areas, and plans to provide a software update for us today.

We will continue updating you as we have more information.
Posted Sep 06, 2018 - 11:35 EDT
Update
Update on slowness in the SIS

The changes made to our infrastructure and the eSchoolData software appear to have made a vast improvement today. Staff have been monitoring the systems and short of positive feedback from clients today, we have not seen or had any reports regarding speed. We encourage your district to continue using the SIS today and let us know if you experience any issues.

Regardless of the current state of the overall system and findings, our staff and consultants will continue monitoring and scrutinizing any other possible configurations, settings and indexing that can only add to the overall performance. Implementation of some of these resolutions are happening now, while others will happen this evening and/or over the weekend.

We are once again sorry for failing you at this most vital time of the year.

We will continue updating you as we have more information.
Posted Aug 31, 2018 - 14:03 EDT
Monitoring
Update on slowness in the SIS

The Dell EMC modifications addressed in the previous email finished at approximately 3:45 p.m. with very encouraging results. An additional optimization is now underway and expected to finish this evening. We encourage you to give us your feedback in the morning.

Investigations with any potential Microsoft database or server related issues continues as well. Staff are now working with engineers on additional reviews and possible recommendations.

Thank you for your feedback and patience while we work through this unfortunate situation.

We will continue updating you as we have more information.
Posted Aug 30, 2018 - 16:45 EDT
Update
We have been troubleshooting our slowness issue extensively over the past four days in an attempt to narrow down the potential underlying issue. It has proven to be quite elusive to our staff, eSchoolData and a number of consultants. Our Dell EMC consultant believes with high confidence that once the modifications they have been administering throughout the day are complete, they should resolve the speed issues you have experienced. In addition, we are still actively pursuing all other possibilities with our Microsoft consultant as well. They will continue to work with our staff throughout the evening.

We understand your frustration and sincerely apologize for the problems and issues this situation is creating for you. We assure you that we are doing everything we can to resolve this issue.

We will continue updating you as we have more information.
Posted Aug 30, 2018 - 15:12 EDT
Update
We continue working with the team from eSchoolData, Microsoft consultants and Dell in the discussions regarding the SIS slowness. The recommended adjustments from last night have not resolved the issue, but the system has shown a slight increase in speed. We have identified where the bottleneck is occurring and continue collaborating with consultants on the cause and optimal solution.

We will continue updating you as we have more information.
Posted Aug 30, 2018 - 10:44 EDT
Update
We continue working with the team from eSchoolData and engaging Microsoft consultants in the discussions regarding the SIS slowness. Although some clients have reported a slight improvement in performance after adjustments were made, the speed is not back to acceptable levels. The data logged in the trace process did not definitively identify the bottleneck, but it did point to a potential issue that we are investigating.

We will update you again as soon as we have more information.
Posted Aug 29, 2018 - 16:49 EDT
Update
As part of our troubleshooting efforts, the SIS will be unavailable for approximately 5-10 minutes beginning at 2:10 today.
Posted Aug 28, 2018 - 14:00 EDT
Update
We are continuing to work on a fix for this issue.
Posted Aug 28, 2018 - 09:26 EDT
Identified
Clients are reporting extreme slowness in the SIS. We have been working closely with eSchoolData technicians to determine the cause. A patch was applied yesterday, which unfortunately did not resolve the problem. We will continue to work together until the problem is resolved.

We understand your frustration and sincerely apologize for the inconvenience this situation is creating for you and your district.

We will update you again as soon as we have more information.
Posted Aug 28, 2018 - 09:26 EDT