Cloning, Partitioning and formatting a new HDD

My HP laptop's HDD had been giving some errors in the Disk test for quite sometime now, so I got a new HDD with the idea to clone the old disk on this new disk. And to connect this new HDD to computer I bought a USB based 2.5 HDD External interface for HDDs, which allows you to connect the laptop disk to your system through usb cable.

My primary reason to go for cloning the disk was I wanted to avoid starting with a fresh image of Windows and the job of installing the dozens of softwares all over again. Plus I had Ubuntu installation in one of the partitions. This seemed like a perfect use-case for  disk cloning.

So, a bit of google search on disk cloning landed me on this page: https://en.wikipedia.org/wiki/Comparison_of_disk_cloning_software

I tried a couple of softwares from the above list. To keep it easy I started off with the Windows based softwares:

• Acronis True Image^[1] This one is a paid software, but luckily the new HDD I got was WD, and it turned out that if you have a WD HDD on your system, the software doesn't ask for the license and you get to use it for free. But this one turned out to be a a disappointing one as it failed to recognize my HDD as WD drive (probably because I was interfacing it through USB).
• Macrium Reflect  This was my next bet, as its a freeware with Graphical interface so going for easy one again :P. Was easy to install and then followed the GUI wizard to clone the disk, and bingo the cloning began. However as luck would have it, a couple of hours later the dialog box saying "Disk read error" popped out of no where and I was back to square one !

I realized that windows was not up to the task, the easy route was turning out to be rather difficult now.

So, I rebooted into Ubuntu this  time to try the REAL low level stuff.

• dd - is what I started with in Linux. You can create disk backups and disk cloning with a single line of this command (see thats the power of linux ;). To clone 'sda' to 'sdb' you can use the command (use at your own risk):

• dd if=/dev/sda of=/dev/sdb bs=4096 conv=notrunc,noerror

  ps -ef | grep 'dd' # get the PID of dd command

  kill -USR1 8789 #senda signal to 'dd' process to print the progress

•  WARNING: If you are thinking of trying this, please read this link, and understand what exactly you are doing because a small mistake can make you lose all your data. https://wiki.archlinux.org/index.php/Disk_Cloning
• It turned out 'dd' is just a simple copying command with no error handling built into it, so couple of hours into copying this one too failed with "input/output error" on the console :-/
• Then i tried to manually create partitions on the new disk using fdisk.

• sudo fdisk -l  #shows the disks and partition details for all disks

  sudo fdisk /dev/sdc # if you need to partition '/dev/sdc'

• The above command takes you to the fdisk command prompt, where you can easily 'create new partition', 'delete a partition' and 'write your changes to partition table' etc. However, this was turing out to be cumbersome process, because I would have to clone all partitions one by one and format them using mkfs and what not :P http://www.idevelopment.info/data/Unix/Linux/LINUX_PartitioningandFormattingSecondHardDrive_ext3.shtml
• Now comes the real awesome piece of software to my rescue - 'ddrescue'. Yeah as the name says, so it does :). This is similar to 'dd' but has disk error handling algorithm built into it. What it tries to do is recover as much of the data from the disk as is possible from the good sectors, and then uses slow reading to recover the bad sectors. You can find all the details here: http://www.gnu.org/software/ddrescue/manual/ddrescue_manual.html

• sudo ddrescue -f /dev/sda /dev/sdb logfile

  
  
• 'ddrescue' has an awesome log feature which basically logs all the disk recovery it is doing to a file. This log file allows it to resume the recovery process from where it left, incase your system crashes in between or anything !!
• You can also get a 'ddrescueview' which shows the graphical view of the recovery process as good/bad sectors etc. which is really helpful. You can download this from here: http://sourceforge.net/projects/ddrescueview/

To summarize, yeah 'ddrescue' was able to clone my old disk to the new one. And the log viewer GUI showed that my old disk had one (just one :P) bad sector in my Windows partition. But yeah, overall this turned out to be a great learning experience about disks ;)

Start a HTTP server in python

A simple HTTP Server can be started in seconds.

python -m SimpleHTTPServer

The server starts on port 8080 by default which can be changed. I have found this quite handy at times.

For example: To share a complete directory with someone over the Internet, I cd to the directory and start the server. The directory is shared, and I share my IP address. The directory can be viewed and files can be downloaded easily. I can also monitor access requests on the terminal.

original link:
http://www.quora.com/Python-programming-language-1/What-are-some-cool-Python-tricks

TopCoder SRM 567 - Div I Lvl 1

The Square Root Dilemma
-----------------------------------
=> (A*B) is the a perfect square

For a given value of (A), we try to find all possible values of B which make A*B a perfect square.

A = OA * EA,  where EA is the perfect square factor of A and OA is the other factor

So, (A*B) is perfect square when B can be factored as OA*(a perfect square).


def srm567(a,b):
    ctr = 0
    for i in range(1,a+1):
        j = 2
        s = 1
        while (j*j)<= i:
            if i%(j*j) == 0:
                s = j*j
            j += 1

        r = i/s
        #print "i,s:",i,s
        y = 1
        while (y*y*r)<=b:
            ctr+= 1
            y += 1
    return ctr

TopCoder SRM 569 - Div I Lvl 1

We need to be able to find the Operation of each bit, i.e. whether it is OR, AND , XOR.

If you see the truth table for these operations:

A	B	OR	AND	XOR
0	0	0	0	0
0	1	1	0	1
1	0	1	0	1
1	1	1	1	0

You see that the first combination where both bits A/B are '0' all the results are zero, so this combination of bits doesn't help us identify the operation. => If we have bits 0,1,1 available to be inputted at a bit position we can identify the operation.

So, now the problem reduces to finding the no. of plates to be added so that there is a combination of 011 bits available at each position.


 def srm569(plates):
    more_plates_needed = 0
    M = len(plates[0])
    for i in range(0,M):
        bits_needed = ['0','1','1']
        for j in range(0,len(plates)):
            if plates[j][i] in bits_needed:
                bits_needed.remove(plates[j][i])
        more_plates_needed = max(more_plates_needed,len(bits_needed))
    return more_plates_needed


Examples:
=======

srm569(["01010101",
 "10101010"])
Out[34]: 1

srm569(["10010101011",
 "00010101001",
 "00100010111",
 "00101010101",
 "01010111101"])
Out[35]: 1

srm569(["1101001011010",
 "0010000010101",
 "1010101011110",
 "1101010100111",
 "1011111110111"])
Out[36]: 0

Learning python regular expressions

https://developers.google.com/edu/python/regular-expressions

Python Regular Expressions

Regular expressions are a powerful language for matching text patterns. This page gives a basic introduction to regular expressions themselves sufficient for our Python exercises and shows how regular expressions work in Python. The Python "re" module provides regular expression support.

In Python a regular expression search is typically written as:

  match = re.search(pat, str)

The re.search() method takes a regular expression pattern and a string and searches for that pattern within the string. If the search is successful, search() returns a match object or None otherwise. Therefore, the search is usually immediately followed by an if-statement to test if the search succeeded, as shown in the following example which searches for the pattern 'word:' followed by a 3 letter word (details below):

str = 'an example word:cat!!'
match = re.search(r'word:\w\w\w', str)
# If-statement after search() tests if it succeeded
  if match:                      
    print 'found', match.group() ## 'found word:cat'
  else:
    print 'did not find'

The code match = re.search(pat, str) stores the search result in a variable named "match". Then the if-statement tests the match -- if true the search succeeded and match.group() is the matching text (e.g. 'word:cat'). Otherwise if the match is false (None to be more specific), then the search did not succeed, and there is no matching text.

The 'r' at the start of the pattern string designates a python "raw" string which passes through backslashes without change which is very handy for regular expressions (Java needs this feature badly!). I recommend that you always write pattern strings with the 'r' just as a habit.

Basic Patterns

The power of regular expressions is that they can specify patterns, not just fixed characters. Here are the most basic patterns which match single chars:

• a, X, 9, < -- ordinary characters just match themselves exactly. The meta-characters which do not match themselves because they have special meanings are: . ^ $ * + ? { [ ] \ | ( ) (details below)
• . (a period) -- matches any single character except newline '\n'
• \w -- (lowercase w) matches a "word" character: a letter or digit or underbar [a-zA-Z0-9_]. Note that although "word" is the mnemonic for this, it only matches a single word char, not a whole word. \W (upper case W) matches any non-word character.
• \b -- boundary between word and non-word
• \s -- (lowercase s) matches a single whitespace character -- space, newline, return, tab, form [ \n\r\t\f]. \S (upper case S) matches any non-whitespace character.
• \t, \n, \r -- tab, newline, return
• \d -- decimal digit [0-9] (some older regex utilities do not support but \d, but they all support \w and \s)
• ^ = start, $ = end -- match the start or end of the string
• \ -- inhibit the "specialness" of a character. So, for example, use \. to match a period or \\ to match a slash. If you are unsure if a character has special meaning, such as '@', you can put a slash in front of it, \@, to make sure it is treated just as a character.

Basic Examples

Joke: what do you call a pig with three eyes? piiig!

The basic rules of regular expression search for a pattern within a string are:

• The search proceeds through the string from start to end, stopping at the first match found
• All of the pattern must be matched, but not all of the string
• If match = re.search(pat, str) is successful, match is not None and in particular match.group() is the matching text

  ## Search for pattern 'iii' in string 'piiig'.
  ## All of the pattern must match, but it may appear anywhere.
  ## On success, match.group() is matched text.
  match = re.search(r'iii', 'piiig') =>  found, match.group() == "iii"
  match = re.search(r'igs', 'piiig') =>  not found, match == None

  ## . = any char but \n
  match = re.search(r'..g', 'piiig') =>  found, match.group() == "iig"

  ## \d = digit char, \w = word char
  match = re.search(r'\d\d\d', 'p123g') =>  found, match.group() == "123"
  match = re.search(r'\w\w\w', '@@abcd!!') =>  found, match.group() == "abc"