Torn City is back up!

Last Updated on Thursday, 22 February 2007 16:13

Although it could have been a hell of a lot worse, some user data has been set back by one month. This does not include quite a few things. Bear in mind, this was in no way our fault at all, and will NEVER be happening again ever.

The worst part: All user accounts created in the past month have been removed from Torn City. Their donator packs however ARE recoverable by using donator issues.

-Just prior to an upgrade (literally an hour before), a RAID controller failed
-During this failure one of the drives died - probably killed by the controller. Since we were using RAID-0 in the short term for performance, this is fatal. (again, this was going to be changed to RAID-1 approximately one hour later).
-This combination of failures killed the partition
-Despite 98% of these failures being recoverable, the array has been examined professionally and it is not recoverable
-We therefore have to rely on our backups. We have offsite backups of the database, however on examination these have only been taking the first 80% or so of the database. This is entirely the host's fault for not checking them, although the method used was standard (a mysqldump) and there was no error produced during the automated backup process.
-We therefore have to use old data for the last few tables - basically the users table.
-This is a combination of worst case scenarios that it is hard to foresee. We shall of course be manually checking offsite backups regularly from now on.


Here’s where we stand.

Firstly the things which are recoverable and will be exactly as you left them 4 days ago:
Items
Factions (although users will have to be reinvited/kicked)
Donator packs (you will have to use them again)
Battle stats
Auctions
Awards
Forums
Pets
Friends/Enemy lists
Some but not all crime E


Things which are not recoverable and will be approximately one month out of date:
Your level (set back one month)
Education courses (set back one month)
Your points (set back one month) (but DPs will be recovered meaning a lot more points back in to the system)
Your property (set back one month)
Bank times (set back one month)
Completed missions (set back one month)
Merits (set back one month)
Password (set back to what you had one month ago, please change them)



I would like to make three promises to you for the future.

1) We will do everything in our power to produce at least one large update every month from now on.
2) In future We will ALWAYS give you at least 24 hours notice before a downtime to the hour (not including emergency downtimes)
3) We will make full backups daily and check they are all working fine to avoid anything like this again
4) We will make further manual full database backups to our home desktops twice per week
5) Long downtimes like this will never happen again
6) Data will NEVER again be more than 24 hours out of date if anything like this happens again.

I feel Torn City has grown a hell of a lot stronger over the past 4 days. I promise you, by march 1st, everything will feel back to normal again and with some nice new nifty upgrades.

I sincerly apologise, but promise nothing like this will ever happen again.

I will post a short blog tomorrow answering all questions and how far along the process is.

Thanks very much for understanding.

Comments (5)

Subscribe to this comment's feed

Show/hide comments

...

funny how tc has been down for (another) almost full day for "nightly backups"

predator101 , February 21, 2007

• 
• +0
• 
• 

...

hahaha! that is quite funny, actually!

Cutie69 , February 22, 2007

• 
• +0
• 
• 

...

Its stupid, TC has been down for nightly backup for about an hour now =/ probably closer to 1hr 20

o , February 22, 2007

• 
• +0
• 
• 

...

You need to realize that your cache is keeping the "backup" screen coming up rather than loading the actual game again. Press Ctrl F5 or Refresh to remove the screen from your cache. The backup takes ~40 minutes as of late and ends at 00:30 gmt each night.

jcgamo88 , February 23, 2007

• 
• +0
• 
• 

...

it's disgusting that TC came back with one month old data.... totally destroyed my game plans and forcing me to leave TC.......

M. H , February 23, 2007

• 
• -1
• 
• 

Write comment

Show/hide comment form

You must be logged in to post a comment. Please register if you do not have an account yet.