File Recovery

Introduction

WPI uses a NS120 file server to serve files, via NFS, to CCC UNIX machines. This server has a feature known as checkpoints which allows for online file recovery of recently changed or deleted files. This document will show you how this mechanism works and how to use it.

How does it work?

The EMC file server ("the filer") creates checkpoints of your directories at specified intervals..

From a user's point of view, at regular intervals, the NS120 "takes a checkpoint" of the current file system. This checkpoint looks like a read-only copy of the file system at that point. There are hidden checkpoint directories in every directory, including each user home directory.

The checkpoint mechanism is independent of our normal tape backup mechanism.

How often do checkpoints get taken and how long do they last?

At any time, these checkpoints of /home should be available:

seven checkpoints during the day (at 3, 6, 9, 12, 15, 18, 21). Those only last a day so, when the hour time up for a new one, the prior day's checkpoint is removed.
six daily checkpoints (at midnight, other than Saturday night)
8 weekly checkpoints, at midnight Saturday night.

This gives us relatively frequent checkpoints for the past day, daily checkpoints for a week, and two months of weekly checkpoints.

How do I use this feature?

There are two easy ways to use checkpoint: via a Unix shell or via Windows

Steps for Using Checkpoint via Unix

Suppose you have deleted a file in your "mail" directory and you want to restore it.

Step 1: Change to the .ckpt directory

In the directory where you lost your file, there is a hidden .ckpt directory. This directory does not appear to ls or any other command unless you explicitly request it. You may cd .ckpt, even though it is not visible:

[~/mail] > cd .ckpt

Once in this directory, you will see several other directories, each representing a checkpoint. The flags to the ls command are l (for long detail info) d (to show the directory, not its contents):

[~/mail/.ckpt] > ls -ld
drwx--x--x 95 aej aej 16384 Aug 21 11:22 2010_08_21_12.15.30_America_New_York/
drwx--x--x 95 aej aej 16384 Aug 21 20:01 2010_08_22_00.15.25_America_New_York/
drwx--x--x 95 aej aej 16384 Aug 22 17:51 2010_08_22_18.15.04_America_New_York/
drwx--x--x 95 aej aej 16384 Aug 22 20:06 2010_08_22_21.15.06_America_New_York/
drwx--x--x 95 aej aej 16384 Aug 22 20:06 2010_08_23_00.15.06_America_New_York/
drwx--x--x 95 aej aej 16384 Aug 23 03:12 2010_08_23_03.15.07_America_New_York/
drwx--x--x 95 aej aej 16384 Aug 23 03:27 2010_08_23_06.15.07_America_New_York/
drwx--x--x 95 aej aej 16384 Aug 23 08:56 2010_08_23_09.15.12_America_New_York/
drwx--x--x 95 aej aej 16384 Aug 23 12:02 2010_08_23_12.15.10_America_New_York/
drwx--x--x 95 aej aej 16384 Aug 23 15:09 2010_08_23_15.15.08_America_New_York/
drwx--x--x 95 aej aej 16384 Aug 23 15:09 2010_08_23_15.17.50_America_New_York/
drwx--x--x 95 aej aej 16384 Aug 23 15:09 2010_08_23_15.27.30_America_New_York/

Note that the checkpoints are named by the time that they happen, in America, New York, otherwise known as eastern time.

Step 2: Find the version of the file you need

The easiest way to find the file you need is to use something like "ls -lt */filename". For instance, if I wanted to locate possible copies of the file Stats:

[~] > ls -l .vm.crash/.ckpt/*/Stats
-rw-r--r-- 1 aej aej 204620 Aug 20 00:01 .vm.crash/.ckpt/2010_08_21_12.15.30_America_New_York/Stats
-rw-r--r-- 1 aej aej 204665 Aug 22 00:01 .vm.crash/.ckpt/2010_08_22_00.15.25_America_New_York/Stats
-rw-r--r-- 1 aej aej 204665 Aug 22 00:01 .vm.crash/.ckpt/2010_08_22_18.15.04_America_New_York/Stats
-rw-r--r-- 1 aej aej 204665 Aug 22 00:01 .vm.crash/.ckpt/2010_08_22_21.15.06_America_New_York/Stats
-rw-r--r-- 1 aej aej 204710 Aug 23 00:01 .vm.crash/.ckpt/2010_08_23_00.15.06_America_New_York/Stats
-rw-r--r-- 1 aej aej 204710 Aug 23 00:01 .vm.crash/.ckpt/2010_08_23_03.15.07_America_New_York/Stats
-rw-r--r-- 1 aej aej 204710 Aug 23 00:01 .vm.crash/.ckpt/2010_08_23_06.15.07_America_New_York/Stats
-rw-r--r-- 1 aej aej 204710 Aug 23 00:01 .vm.crash/.ckpt/2010_08_23_09.15.12_America_New_York/Stats
-rw-r--r-- 1 aej aej 204710 Aug 23 00:01 .vm.crash/.ckpt/2010_08_23_12.15.10_America_New_York/Stats
-rw-r--r-- 1 aej aej 204710 Aug 23 00:01 .vm.crash/.ckpt/2010_08_23_15.15.08_America_New_York/Stats
-rw-r--r-- 1 aej aej 204710 Aug 23 00:01 .vm.crash/.ckpt/2010_08_23_15.37.00_America_New_York/Stats
-rw-r--r-- 1 aej aej 204710 Aug 23 00:01 .vm.crash/.ckpt/2010_08_23_15.46.30_America_New_York/Stats

This shows that there are three copies of the file available, based on the dates and sizes, across the checkpoints. Since we have just switched to the EMC server, there are not yet an accumulation of daily and weekly checkpoints.

Step 3: Copy the file you need back to the active file system

One can use the standard cp command:

[~] > cp .vm.crash/.ckpt/2010_08_23_15.46.30_America_New_York/Stats .vm.crash/

Note that cp was used, and not mv, since data in .ckpt is always read-only, so it cannot be moved out of that directory.

That's the whole file recovery process!

Steps for Using Checkpoint via Windows

Step 1: Navigate to the folder that the item was in

Step 2: In the address bar, add /~checkpoint after the folder address

This will show you the backups for that folder. Choose the most recent backup that contains the file. (Hourly.0 is usually the most recent backup, with Hourly.1 being the next most recent, and so on.)

Step 3: Right-click on the file and choose Copy

Step 4: Navigate back to the original folder and choose Paste to restore the file

Note: If you can not find something using this mechanism, please contact the Helpdesk with the path of the file(s) and when it was deleted or otherwise lost. We will attempt to restore this file for you using our normal tape backup system. Please keep in mind, however, that the tape backups are only performed on a monthly basis, as we mainly rely upon the .ckpt data.

Other useful checkpoint information

Here are a few other useful pieces of information about checkpoints:

Files found in .ckpt do not count against your quota.
The .ckpt directory in your home directory or at the top of the mounted file systems offers another way to get to checkpoint files. For instance, ~/.ckpt/checkpoint-name/dir/dir/file is the same as ~/dir/dir/.ckpt/checkpoint-name/file.

For further information

If you still have questions about checkpoints after reading this document, please contact the Helpdesk. We hope you enjoy this useful facility.

Maintained by itweb.
Last modified: Aug 23, 2010, 20:01 UTC