Clinic: immutable backups and recovery in hours after a ransomware incident

After a ransomware attack, the clinic spent a week restoring encrypted backups. We deployed Veeam with immutable backups, EDR protection and regular restore tests, and recovery now takes hours.

Clinic: immutable backups and recovery in hours after a ransomware incident

Situation

A private clinic in Vilnius, Lithuania: 150 employees, three locations, a laboratory and its own patient information system. The servers were on the clinic’s premises, backups were written to a NAS device on the same network, and the antivirus was a standard product with no central console.

On a Friday evening the administrator noticed that patient records no longer opened on a reception computer. Overnight, ransomware had encrypted the file server, the patient system database and, most painfully, the NAS with the backups, because it was mounted as an ordinary network drive. For a week the clinic worked on paper: registration by phone, test results printed in the laboratory, some appointments cancelled. Data was restored from an older, irregularly made backup on an external drive and from the monthly backup of the patient system vendor. Part of that week’s records could not be recovered.

The clinic’s director contacted us two weeks after the incident. The task was clear: make sure this never happens again, and if it does, make recovery take hours, not a week.

What was needed

  • Backups that an attacker cannot reach even with administrator rights on the network.
  • A recovery time that would not stop patient intake for more than a day.
  • Regular restore tests with written reports, which the clinic’s data protection officer also asked for.
  • Endpoint protection that stops encryption instead of only reporting it.
  • A solution maintained by an external team, because the clinic has no IT administrator of its own.

What we did

Week 1: cleaning up the environment after the incident

First we made sure the malware was gone: we rebuilt the file server, changed all administrative passwords, disabled legacy protocols and separated the server network from the workstations. This was essential, because there is no point in making backups of an environment that may still contain the attacker’s access.

Weeks 2-3: Veeam and immutable backups

We deployed Veeam backup following the 3-2-1 principle: a copy on local storage for fast recovery, a copy in the MDP CLOUD data centre in Lithuania, and an additional copy in an Amazon data centre in Germany. The key difference from the previous setup is immutable backup: the repository is invisible from the clinic network, the copies cannot be changed for a set retention period, and not even a Veeam administrator can delete or overwrite them. The local repository is isolated as well: only the backup server talks to it, not the workstations.

For the patient system database we set backups every hour, and for files and e-mail every day. Retention periods were aligned with the clinic’s record retention requirements. Our team monitors backup health: a failed job or a mismatched checksum becomes a helpdesk ticket the same day, and the clinic’s management receives a short weekly report on how many backups ran and how much space is left.

The most complicated part was the patient information system: its vendor required the database to be backed up only with the vendor’s own tools. We agreed on a dual scheme. The vendor’s backup stays, and Veeam additionally backs up the entire server, so that during recovery the system does not have to be reinstalled and reconfigured.

Week 4: EDR protection

On all computers and servers we deployed Bitdefender GravityZone with EDR and ransomware protection, which stops mass encryption and rolls back the affected files. Our team monitors the console; suspicious activity is escalated to the person responsible at the clinic.

Week 5: restore test

The first full restore test: from the immutable backup we restored the patient system server and the file server into an isolated environment in the MDP CLOUD cloud. It took 6 hours, most of it spent verifying the database together with the system vendor. The report was signed by the clinic’s director and the data protection officer.

Result

Recovery time dropped from a week to 6 hours, and this was measured during the test, not estimated in a plan. The number of backups reachable from the clinic network is now zero. Restore tests run every month at a smaller scale (individual files, a single server), and a full restore is repeated every six months.

The clinic now has an audit report for every test: date, systems restored, duration, problems encountered. Both the data protection officer and the insurer asked for these when the cyber insurance policy was renewed. In the first six months the EDR console stopped suspicious activity on employee computers twice, both times caused by infected e-mail attachments, and neither case reached the servers.

“We used to think we had backups. Now we know that within six hours we will be receiving patients again.”

What we would advise in a similar situation

  • A backup you can see as a network drive is not protection against ransomware. At least one copy must be immutable and unreachable from the network.
  • Set the recovery time according to the business: how many hours can you work without the system. Backup frequency and storage layout follow from that.
  • Test recovery regularly and write reports. A backup without a test is only an assumption.
  • After an incident, clean the environment first and only then move backups over, otherwise you will copy the attacker’s access as well.