system down
Our system seems to be down, all our brokers are unable to use it and my staff are experiencing the same problem. Could someone please respond urgently
Kind Regards
-
Hi Ben,
After further work on this out-of-hours last night, I have an update for you on this issue.
While investigating the cause of the slow-down yesterday our developers found a 'memory leak' on one server which had started to cause knock-on effects across the whole server set. This culminated in the intermittent speed issues you were all seeing within SchemeServe and meant that sometimes sites would load fine, then turn very slow for a period of time and then just as unexpectedly switch back to full speed.
In order to fix this we prepared patches for the server in question last night, which appear after initial investigations to have been very successful.
We have been monitoring SchemeServe since and have found it to be performing back at high speed as normal but please do get in touch if you notice any evidence of this problem again. We'll be monitoring very closely over the next few days and checking all servers to ensure this problem is fully resolved.
We apologise for the inconvenience this must have caused you and would like to say a big thank-you for your patience yesterday while we identified and resolved this issue. It's great to know we have such a supportive community, it makes it more fun to continually improve SchemeServe knowing you're all there!
Best wishes as always,
AdamCEO, SchemeServe
-
Many thanks for sorting this so quickly guys.
Have you considered having a "system status" page on your support website.
This could be for down time but also could flag known bugs that you are working to fix etc.
This way when you spot an issue or someone reports an issue that will be affecing many users you can flag you know and that you are already working on it. Looks proactive and that way we know you are "on top" of the problem and we dont have to all ring in and bombard chris with tickets? Just a thought.
-
Thanks Ben, John, for letting us know!
Actually, we have a system status page (it's fairly new) but after your comment John I'm thinking we should draw more attention to it.
It's at http://support.admnetwork.com on the right hand side near the phone number. What do you think, shall we make it bigger?
I've pasted the link here too...
It shows any actual downtime on there as red dots and also shows the response times - yesterday was slower than normal even on average.
-
Hi Adam,
I don't think it was the office as our staff were using other applications. Having said that it seems fine again now. So it might have been a glitch!
The page was going into a record, and another was going into the accounts screen.
We will keep an eye on it and report back.
Many thanks
Ben
-
Just to people informed, we are still continuing to track where this problem originates. we've added some extra tracking to gather a lot more infor should the problem occur again, and we have a system that will send us a text message if the site goes down for more than 1 minute. Our hosting company is also providing some additional resource to monitor the servers while we get some sleep, and to step in with reading any error message for further info we gain. I check several times over the weekend.
The sites still working, and if it fails again we'll have a lot more info at the same time as getting the site restarted. Keeping eyes peeled.
Sorry its causing you hassle, but Im obsessed with getting to the bottom of this.
Thanks
John Price
-
Hello
Just to let you know that we are aware of this issue and we are looking into it. We are working closely with our datacenter engineers to pinpoint the problem and get this issue resolved fully. We'll keep you updated periodically and inform you when this problem is completely resolved.
Regards
Nore.
-
Hi,
Just to let everyone know that we issued an email update on the current situation earlier today, you should all have received this.
There is also a special Twitter link in there to keep you posted with the very latest information - https://twitter.com/sswatchman
Regards
Colin
-
Hi Ben,
We are monitoring the system currently and have noticed it start to slow down again. I can confirm that we have the whole team working as priority on a full resolution for this. We are also working with our server hosting company to improve overall speed while we continue to investigate.
I will keep you updated as the day progresses.
Kind regards,
Chris -
Hi Chris
Thankyou for the update, unfortunately it seems to be getting worse for us as the day goes on, it is now taking the staff about 5 minutes to process one thing and our brokers are now calling asking what the issues are, are you guys any closer in finding a solution?
Regards
Ben
Please sign in to leave a comment.
Comments
32 comments